This is cool! :) On Mon, Feb 13, 2023 at 2:02 PM Daniel Kaźmirski <d.kazmir...@gmail.com> wrote:
> Hi, > > I did try to add the mentioned extension to Hudi pom. Here are the results: > > Clean with cache extension disabled > mvn clean package -DskipTests -Dspark3.3 -Dscala-2.12 > -Dmaven.build.cache.enabled=false > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 10:57 min > > With cache after changing HoodieSpark33CatalogUtils > mvn package -DskipTests -Dspark3.3 -Dscala-2.12 > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 03:52 min > > With cache no changes > mvn package -DskipTests -Dspark3.3 -Dscala-2.12 > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 3.485 s > > If anyone would like to try it out: > https://github.com/apache/hudi/pull/7935 > > BR, > Daniel > > pt., 10 lut 2023 o 16:59 Daniel Kaźmirski <d.kazmir...@gmail.com> > napisał(a): > > > Hi all, > > > > Going back to this topic, Maven 3.9.0 has been released recently along > > with a new build cache extension that provides incremental builds: > > https://maven.apache.org/extensions/maven-build-cache-extension/ > > Might be worth considering. > > > > pon., 24 paź 2022 o 19:59 Shiyan Xu <xu.shiyan.raym...@gmail.com> > > napisał(a): > > > >> Thank you all for the valuable inputs! I think we can close this topic > for > >> now, given the majority is leaning towards continuing with maven. > >> > >> On Mon, Oct 17, 2022 at 8:48 PM zhaojing yu <maidou...@gmail.com> > wrote: > >> > >> > I have experienced some gradle development projects and want to share > >> some > >> > thoughts. > >> > > >> > The flexibility and faster speed of gradle itself can certainly bring > >> some > >> > advantages, but it will also greatly increase the troubleshooting time > >> due > >> > to the bugs of gradle itself, and gradle DSL is very different from > >> that of > >> > maven. There are also many learning costs for developers in the > >> community. > >> > > >> > I think it does consume too much time on code release, but users or > >> > developers usually only compile part of the module. > >> > > >> > So I think, a certain advantage in build time alone is not enough to > >> cover > >> > so much cost. > >> > > >> > Best, > >> > Zhaojing > >> > > >> > Gary Li <gar...@apache.org> 于2022年10月17日周一 19:22写道： > >> > > >> > > Hi folks, > >> > > > >> > > I'd share my thoughts as well. I personally won't build the whole > >> project > >> > > too often, only before push to the remote branch or make big changes > >> in > >> > > different modules. If I just make some changes and run a test, the > IDE > >> > will > >> > > only build the necessary modules I believe. In addition, each time I > >> deal > >> > > with dependency issues, the years of maven experience does help me > >> locate > >> > > the issue quickly, especially when the dependency tree is pretty > >> > > complicated. The learning effort and the new environment setup > effort > >> are > >> > > considerable as well. > >> > > > >> > > Happy to learn if there are other benefits gradle or bazel could > >> bring to > >> > > us, but if the only benefit is the xx% faster build time, I am a bit > >> > > unconvinced to make this change. > >> > > > >> > > Best, > >> > > Gary > >> > > > >> > > On Mon, Oct 17, 2022 at 2:58 PM Danny Chan <danny0...@apache.org> > >> wrote: > >> > > > >> > > > I have a full experience with how Apache Calcite switches from > Maven > >> > > > to Gradle, and I want to share some thoughts. > >> > > > > >> > > > The gradle build is fast, but it relies heavily on its local > cache, > >> > > > usually it needs too much time to download these cache jars > because > >> > > > gradle upgrade itself very frequently. > >> > > > > >> > > > The gradle is very flexive for building, but it also has many > bugs, > >> > > > you may need more time to debug its bug compared with building > with > >> > > > maven. > >> > > > > >> > > > The gradle DSL for building is a must to learn for all the > >> developers. > >> > > > > >> > > > For all above cases, I don't think switching to gradle is a right > >> > > > decision for Apache Calcite. Julian Hyde which is the creator of > >> > > > Calcite may have more words to say here. > >> > > > > >> > > > So I would not suggest we do that for Hudi. > >> > > > > >> > > > > >> > > > Best, > >> > > > Danny Chan > >> > > > > >> > > > Shiyan Xu <xu.shiyan.raym...@gmail.com> 于2022年10月1日周六 13:48写道： > >> > > > > > >> > > > > Hi all, > >> > > > > > >> > > > > I'd like to raise a discussion around the build tool for Hudi. > >> > > > > > >> > > > > Maven has been a mature yet slow (10min to build on 2021 macbook > >> pro) > >> > > > build > >> > > > > tool compared to modern ones like gradle or bazel. We all want > >> faster > >> > > > > builds, however, we also need to consider the efforts and risks > to > >> > > > upgrade, > >> > > > > and the developers' feedback on usability. > >> > > > > > >> > > > > What do you all think about upgrading to gradle or bazel? Please > >> > share > >> > > > your > >> > > > > thoughts. Thanks. > >> > > > > > >> > > > > -- > >> > > > > Best, > >> > > > > Shiyan > >> > > > > >> > > > >> > > >> > >> > >> -- > >> Best, > >> Shiyan > >> > > > > > > -- > > Daniel Kaźmirski > > > > > -- > Daniel Kaźmirski >
