JiaLiangC commented on PR #315: URL: https://github.com/apache/tez/pull/315#issuecomment-1857129442
@abstractdog @ayushtkn I apologize for not providing a detailed explanation earlier due to being too busy: The command mvn clean package -Dtar -Dhadoop.version=${HADOOP_VERSION} -Phadoop28 -DskipTests compiles without errors. However, parallel compilation fails, and the failure error is as posted by "ayushtkn" in last comment. The reason for the failure is that parallel compilation attempts to compile all modules in a single pass, and the compilation order is determined as a tree structure based on dependencies. In the normal sequential compilation, tez-dist is the last module to compile, so before compiling this module, all other modules and their dependencies are already compiled. However, in parallel compilation, tez-dist is compiled concurrently with other modules. Therefore, it is necessary to ensure that tez-dist includes dependencies from all other modules. This ensures that tez-dist is compiled only after all other modules have been compiled. To resolve the issue using the command mvn -T2C clean package -Dtar -Dhadoop.version=${HADOOP_VERSION} -Phadoop28 -DskipTests, the error reported is due to tez-dist missing the tez-job-analyzer dependency. Adding this dependency will prevent errors during parallel compilation. I also recommend using mvn dependency:tree and maven-to-plantuml to troubleshoot dependency issues causing the inability to compile in parallel. To investigate the dependency relationships between project modules, you can use the following tools: mvn dependency:tree: This Maven command generates a tree structure of project dependencies. It can help you visualize the dependencies between modules. maven-to-plantuml: This is a project for analyzing and visualizing Maven project dependencies. You can use it to generate a PlantUML diagram from the dependency tree generated by mvn dependency:tree. Here are the steps: Download maven-to-plantuml: wget https://github.com/phxql/maven-to-plantuml/releases/download/v1.0/maven-to-plantuml-1.0.jar Generate the dependency tree: mvn dependency:tree > dep.txt Generate a PlantUML diagram from the dependency tree: java -jar maven-to-plantuml.jar --input dep.txt --output dep.puml These tools will help you analyze the dependencies between modules in your project and identify any issues that may be causing problems during parallel compilation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@tez.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org