mmiklavc commented on issue #1523: METRON-2232 Upgrade to Hadoop 3.1.1 URL: https://github.com/apache/metron/pull/1523#issuecomment-537660369 Turns out this classpath issue is due to a combination of factors. In https://github.com/apache/metron/pull/1358/files#diff-1a76680a25447c0c0b3a245ce767b3f8L45 we removed the libjars option because of some classpath issues. This wasn't an issue at the time as metron-data-management was a fully self-contained uber jar. This changed in https://github.com/apache/metron/pull/1436/files#diff-1a76680a25447c0c0b3a245ce767b3f8R39 where stellar-common was pulled out into a separate dependency jar. Unfortunately, HADOOP_CLASSPATH only sets the classpath for the local client application, not the map and reduce task jvm classpaths. For that you need to use the libjars option to pass in the extra stellar jar. This was missed in testing and is a current regression in master.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
