Seatunnel在Dolphinscheduler调度中报错
原创
文章标签 java spark unix 文章分类 Hadoop 大数据
©著作权归作者所有：来自51CTO博客作者江南独孤客的原创作品，请联系作者获取转载授权，否则将追究法律责任
[INFO] 2022-07-06 13:12:16.714 - [taskAppId=TASK-393-81727-908774]:[238] - process has exited, execute path:/data/dolphinscheduler/exec/process/9/393/81727/908774, processId:25215 ,exitStatusCode:1 ,processWaitForStatus:true ,processExitValue:1[INFO] 2022-07-06 13:12:17.201 - [taskAppId=TASK-393-81727-908774]:[138] - -> tools size = 2813343 4574389 5445379 4238956 4253580 /data/dolphinscheduler/exec/process/9/393/81727/908774/393_81727_908774_node.sh: line 7: [: too many arguments dbos-bigdata-clickhouse001 8000 0 2 0 dbos-bigdata-clickhouse003 8000 0 1 0 dbos-bigdata-clickhouse002 8000 0 0 0 SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Exception in thread "main" java.nio.file.AccessDeniedException: plugins.tar.gz at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at org.apache.seatunnel.core.base.utils.CompressionUtils.tarGzip(CompressionUtils.java:62) at org.apache.seatunnel.core.spark.SparkStarter$ClusterModeSparkStarter.buildCommands(SparkStarter.java:390) at org.apache.seatunnel.core.spark.SparkStarter.main(SparkStarter.java:103)
方案：
查看脚本中的源码，需要代码中cd /opt/apache-seatunnel-incubating-2.1.2/
#!/bin/bashhive_table_name='dim_dbos_mac_userid_relation_his'hdfs_size=hdfs dfs -du oss://dbbigdatapro/group/user/tools/meta/hive-temp-table/${hive_table_name}/ | awk '{print $1}' whoami "size = ${hdfs_size}" if [ $hdfs_size -eq 0 ]then echo "没有数据不需要导出"else clickhouse-client -h 172.18.2.111 --port 8000 -u default -d default -m -q "TRUNCATE TABLE dmp.dim_dbos_mac_userid_relation_his on CLUSTER cluster_clickhouse" cd /opt/apache-seatunnel-incubating-2.1.2 /opt/apache-seatunnel-incubating-2.1.2/bin/start-seatunnel-spark.sh --master yarn --deploy-mode cluster --config /home/tools/bigsource/etl/DMP/new/clickhouse_ddl/${hive_table_name}.conffi
报错解除：
- 打赏
- 赞
- 收藏
- 评论
- *举报
下一篇：Yarn开启ACL用户认证之后无法查看Yarn历史任务日志解决办法
Original: https://blog.51cto.com/wang/5447424
Author: 江南独孤客
Title: Seatunnel在Dolphinscheduler调度中报错
原创文章受到原创版权保护。转载请注明出处：https://www.johngo689.com/516706/
转载文章受原作者版权保护。转载请注明原作者出处！