zuston opened a new issue, #160: URL: https://github.com/apache/incubator-uniffle/issues/160
Currently, uniffle use the default Hadoop version of 2.8.5. When using the `./build_distribution.sh --spark2-profile 'spark2' --spark3-mvn '-Dspark.version=2.4.3' --spark3-profile 'spark3' --spark3-mvn '-Dspark.version=3.1.1' -Dhadoop.version=2.6.0`, it will throw exceptions due to some methods and vars not supported in Hadoop 2.6.0. Some non-compatible params and methods as follows 1. `CallContext`, introduced by >= 2.8.0. 2. `MRJobConfig.DEFAULT_SHUFFLE_MERGE_PERCENT` introduced by 2.8.0. [ticket link](https://issues.apache.org/jira/browse/MAPREDUCE-6105) 3. `MRApps.getSystemPropertiesToLog` introduced by 2.8.0 [ticket link](https://issues.apache.org/jira/browse/MAPREDUCE-5232) I think we could use the reflection to be compatible with lower hadoop version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
