JiaLiangC opened a new pull request, #1212:
URL: https://github.com/apache/bigtop/pull/1212
…dditional Components
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'BIGTOP-3638: Your PR title ...'.
-->
### Description of PR
Background:
Within the components maintained by Bigtop, a significant portion is built
using Java and relies on Maven as the build tool.
Rationale:
Compiling components that consist of numerous modules can be a
time-consuming process. For instance, some components contain hundreds of
modules, and compiling them one by one consumes a substantial amount of time.
Even when all dependencies are pre-downloaded for a second compilation, the
process remains slow due to the sequential nature of compilation. Additionally,
compiling all components together still results in sequential compilation,
making it challenging to fully leverage CPU resources and reduce compilation
time significantly. Consequently, repetitive compilation and testing phases
impose prolonged waiting periods.
Proposal:
I propose the introduction of a new parameter that allows users to toggle
parallel compilation for components built using Maven, thus empowering them to
align compilation practices with their specific needs.
Related Pull Requests (PRs):
This discussion can be divided into two main parts:
The first part entails adding parallel compilation functionality and
enabling it for components that have undergone testing without encountering
additional issues related to parallel compilation. These components may include
Hive, HBase, Flink, ZooKeeper, among others.
The second part involves enabling parallel compilation for components that
face challenges with parallel compilation and necessitate additional patches to
address Maven's parallel compilation capabilities. These components may include
Ranger, Tez, Hadoop, Spark, and more.
### How was this patch tested?
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'BIGTOP-3638. Your PR title ...')?
- [ ] Make sure that newly added files do not have any licensing issues.
When in doubt refer to https://www.apache.org/licenses/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
