Dylan1312 opened a new issue #8338: Missing hadoop client dependency URL: https://github.com/apache/incubator-druid/issues/8338 ### Affected Version Noticed in version 0.14.2, looks like it affects version 0.14.0 through to 0.15.1 ### Description Upgrading out test clusters to 0.14.2 resulted in Hadoop Ingestion Tasks failing with the below stack trace. It looks like since 0.14.0 the dependency hadoop-mapreduce-client-core-{$version}.jar is no longer present in the extension druid-hdfs-storage. This dependency is present in the hadoopDependencies folder, however if my understanding of the classloader code is right, we purposefully do not include these jars in the mapreduce job's classpath. Looks related to #6828 and #6967 `java.lang.NoSuchMethodError: org.apache.hadoop.mapreduce.jobhistory.EventWriter.<init>(Lorg/apache/hadoop/fs/FSDataOutputStream;)V at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.createEventWriter(JobHistoryEventHandler.java:420) ~[hadoop-mapreduce-client-app-2.7.3.jar:?] at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.setupEventWriter(JobHistoryEventHandler.java:459) ~[hadoop-mapreduce-client-app-2.7.3.jar:?] at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:549) ~[hadoop-mapreduce-client-app-2.7.3.jar:?] at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:318) ~[hadoop-mapreduce-client-app-2.7.3.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]`
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
