Hi, I did try to add the mentioned extension to Hudi pom. Here are the results:
Clean with cache extension disabled mvn clean package -DskipTests -Dspark3.3 -Dscala-2.12 -Dmaven.build.cache.enabled=false [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 10:57 min With cache after changing HoodieSpark33CatalogUtils mvn package -DskipTests -Dspark3.3 -Dscala-2.12 [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:52 min With cache no changes mvn package -DskipTests -Dspark3.3 -Dscala-2.12 [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3.485 s If anyone would like to try it out: https://github.com/apache/hudi/pull/7935 BR, Daniel pt., 10 lut 2023 o 16:59 Daniel Kaźmirski <d.kazmir...@gmail.com> napisał(a): > Hi all, > > Going back to this topic, Maven 3.9.0 has been released recently along > with a new build cache extension that provides incremental builds: > https://maven.apache.org/extensions/maven-build-cache-extension/ > Might be worth considering. > > pon., 24 paź 2022 o 19:59 Shiyan Xu <xu.shiyan.raym...@gmail.com> > napisał(a): > >> Thank you all for the valuable inputs! I think we can close this topic for >> now, given the majority is leaning towards continuing with maven. >> >> On Mon, Oct 17, 2022 at 8:48 PM zhaojing yu <maidou...@gmail.com> wrote: >> >> > I have experienced some gradle development projects and want to share >> some >> > thoughts. >> > >> > The flexibility and faster speed of gradle itself can certainly bring >> some >> > advantages, but it will also greatly increase the troubleshooting time >> due >> > to the bugs of gradle itself, and gradle DSL is very different from >> that of >> > maven. There are also many learning costs for developers in the >> community. >> > >> > I think it does consume too much time on code release, but users or >> > developers usually only compile part of the module. >> > >> > So I think, a certain advantage in build time alone is not enough to >> cover >> > so much cost. >> > >> > Best, >> > Zhaojing >> > >> > Gary Li <gar...@apache.org> 于2022年10月17日周一 19:22写道： >> > >> > > Hi folks, >> > > >> > > I'd share my thoughts as well. I personally won't build the whole >> project >> > > too often, only before push to the remote branch or make big changes >> in >> > > different modules. If I just make some changes and run a test, the IDE >> > will >> > > only build the necessary modules I believe. In addition, each time I >> deal >> > > with dependency issues, the years of maven experience does help me >> locate >> > > the issue quickly, especially when the dependency tree is pretty >> > > complicated. The learning effort and the new environment setup effort >> are >> > > considerable as well. >> > > >> > > Happy to learn if there are other benefits gradle or bazel could >> bring to >> > > us, but if the only benefit is the xx% faster build time, I am a bit >> > > unconvinced to make this change. >> > > >> > > Best, >> > > Gary >> > > >> > > On Mon, Oct 17, 2022 at 2:58 PM Danny Chan <danny0...@apache.org> >> wrote: >> > > >> > > > I have a full experience with how Apache Calcite switches from Maven >> > > > to Gradle, and I want to share some thoughts. >> > > > >> > > > The gradle build is fast, but it relies heavily on its local cache, >> > > > usually it needs too much time to download these cache jars because >> > > > gradle upgrade itself very frequently. >> > > > >> > > > The gradle is very flexive for building, but it also has many bugs, >> > > > you may need more time to debug its bug compared with building with >> > > > maven. >> > > > >> > > > The gradle DSL for building is a must to learn for all the >> developers. >> > > > >> > > > For all above cases, I don't think switching to gradle is a right >> > > > decision for Apache Calcite. Julian Hyde which is the creator of >> > > > Calcite may have more words to say here. >> > > > >> > > > So I would not suggest we do that for Hudi. >> > > > >> > > > >> > > > Best, >> > > > Danny Chan >> > > > >> > > > Shiyan Xu <xu.shiyan.raym...@gmail.com> 于2022年10月1日周六 13:48写道： >> > > > > >> > > > > Hi all, >> > > > > >> > > > > I'd like to raise a discussion around the build tool for Hudi. >> > > > > >> > > > > Maven has been a mature yet slow (10min to build on 2021 macbook >> pro) >> > > > build >> > > > > tool compared to modern ones like gradle or bazel. We all want >> faster >> > > > > builds, however, we also need to consider the efforts and risks to >> > > > upgrade, >> > > > > and the developers' feedback on usability. >> > > > > >> > > > > What do you all think about upgrading to gradle or bazel? Please >> > share >> > > > your >> > > > > thoughts. Thanks. >> > > > > >> > > > > -- >> > > > > Best, >> > > > > Shiyan >> > > > >> > > >> > >> >> >> -- >> Best, >> Shiyan >> > > > -- > Daniel Kaźmirski > -- Daniel Kaźmirski
