abstractdog commented on PR #464: URL: https://github.com/apache/tez/pull/464#issuecomment-4236413959
> Hi @abstractdog thanks for taking a look. RE: testing, yes that is ongoing and I will follow-up with an end-to-end example which exercises individual Nutch MapReduce jobs running as dual-vertex Tez DAG's. > > My test environment (I'll link the docker compose project soon) utilizes the [official Apache Hadoop 3.4.3 Docker image](https://hub.docker.com/layers/apache/hadoop/3.4.3/images/sha256-127774dadab40ce84df7ac668a7a8c99945688b3fe336f1388f4477ca33e1529) which is both interesting and challenging for a few reasons. > > 1. Hadoop 3.4.3 cluster Java 11 runtime > 2. Tez 1.0.0-SNAPSHOT Java 21 compile-time dependency (with Java 11 runtime flag), this literally required reintroeucing some Java 11 logic into the codebase... I will go into more detail with a follow-up patch > 3. Nutch 1.23-SNAPHOT which was also compiled with Java 21 with Java 11 runtime flag. > > This is somewhat messy but it will be cleared up soon when Hadoop 3.5.0 is released and we can upgrade in both Tez and Nutch and drop Java 11 support. thanks for the details @lewismc! as I don't want this valuable patch to be forgotten/abandoned, I'm fine with any proof (even screenshots) to get this committed to master: from the code point of view, it looks good -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
