rait-winter opened a new issue, #10172: URL: https://github.com/apache/seatunnel/issues/10172
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened ### 问题描述 在使用 SeaTunnel 将数据从 PostgreSQL 写入开启 Kerberos 认证的 HDFS 时，任务启动失败，抛出 `java.lang.NoSuchMethodError: org.apache.hadoop.fs.FsTracer.get` 异常。Kerberos 认证本身已通过，但初始化 HDFS 客户端时出现 Hadoop API 不兼容错误。 ### 环境信息 * **SeaTunnel 版本**: 2.3.8 * **运行模式**: SeaTunnel Engine (Local) * **Java 版本**: 1.8.0_291 * **源端数据库**: PostgreSQL * **目标端 HDFS 版本**: Hadoop 3.1.1 (启用 Kerberos) * **客户端 Hadoop 环境**: 未明确指定，疑似与 SeaTunnel 内置或环境变量版本冲突。 ### 任务配置文件 (`config/pg-to-hdfs.conf`) ```hocon env { job.mode = "BATCH" execution.parallelism = 1 } source { Jdbc { url = "jdbc:postgresql://157.3.xx.xx:5432/sqjw?gssEncMode=disable&sslmode=disable" driver = "org.postgresql.Driver" user = "root" password = "******" query = "SELECT * from tb limit 100" fetch_size = 1000 result_table_name = "tb_data" } } sink { HdfsFile { path = "/db/hive/warehouse/YBSS/test-seatunnel" file_format_type = "orc" fs.defaultFS = "hdfs://hacluster" dfs.client.use.datanode.hostname = "true" hadoop.security.authentication = "kerberos" kerberos_principal = "[email protected]" krb5_path = "/opt/krb5.conf" kerberos_keytab_path = "/opt/user.keytab" # 也曾尝试使用 hadoopConfig {} 块进行配置，问题相同。 } } ### SeaTunnel Version 2.3.8 ### SeaTunnel Config ```conf env { job.mode = "BATCH" execution.parallelism = 1 } source { Jdbc { url = "jdbc:postgresql://157.3.xx.xx:5432/sqjw?gssEncMode=disable&sslmode=disable" driver = "org.postgresql.Driver" user = "root" password = "******" query = "SELECT * from tb limit 100" fetch_size = 1000 result_table_name = "tb_data" } } sink { HdfsFile { path = "/db/hive/warehouse/YBSS/test-seatunnel" file_format_type = "orc" fs.defaultFS = "hdfs://hacluster" dfs.client.use.datanode.hostname = "true" hadoop.security.authentication = "kerberos" kerberos_principal = "[email protected]" krb5_path = "/opt/krb5.conf" kerberos_keytab_path = "/opt/user.keytab" # 也曾尝试使用 hadoopConfig {} 块进行配置，问题相同。 } } ``` ### Running Command ```shell bin/seatunnel.sh -c config/pg-to-hdfs.conf -m local ``` ### Error Exception ```log Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:213) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.NoSuchMethodError: org.apache.hadoop.fs.FsTracer.get(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/tracing/Tracer; at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:323) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:308) at org.apache.hadoop.hdfs.DistributedFileSystem.initDFSClient(DistributedFileSystem.java:204) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:189) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3354) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:474) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226) at org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopFileSystemProxy.lambda$initializeWithKerberosLogin$0(HadoopFileSystemProxy.java:256) at org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopLoginFactory.lambda$loginWithKerberos$0(HadoopLoginFactory.java:52) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopLoginFactory.loginWithKerberos(HadoopLoginFactory.java:50) at org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopFileSystemProxy.initializeWithKerberosLogin(HadoopFileSystemProxy.java:249) at org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopFileSystemProxy.initialize(HadoopFileSystemProxy.java:215) at org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopFileSystemProxy.getFileSystem(HadoopFileSystemProxy.java:171) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.getOrCreateWriter(OrcWriteStrategy.java:124) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.write(OrcWriteStrategy.java:74) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:134) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:249) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:66) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:75) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:50) at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:51) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:693) at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:1018) at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:39) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:205) ``` ### Zeta or Flink or Spark Version zeta ### Java or Scala Version java版本 1.8.0_291 ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
