Sounds great, thanks for chasing this down! - Dan
On Wed, Oct 3, 2018 at 7:20 PM Grant Henke <ghe...@cloudera.com.invalid> wrote: > Hi Kudu dev community, > > I wanted to provide a progress and plan update to switching over to Gradle > as the primary build tool. > > First, the following steps have already been completed: > > - Fixed outstanding Gradle build issues > - Added dist-test support to Gradle > - Started using Gradle as the default build in Jenkins pre-commit > - Changed the Java README to reference Gradle instead of Maven > - Changed the releasing documentation to reference Gradle instead of > Maven > > Next, the plan is to release Kudu 1.8 using Gradle and remove the maven > build shortly after branching. > > There are two motivations for removing the Maven build now. The primary > reason is to reduce the workload of maintaining two builds. The secondary > reason is to simplify some of the test dependencies and break the test > utilities out into their own module. This is much more straightforward in > Gradle than it is in Maven. > > If there are any concerns or issues with this plan please let me know. > > Thank you, > Grant > > On Fri, Apr 27, 2018 at 5:07 PM Adar Lieber-Dembo <a...@cloudera.com> > wrote: > > > I am also in favor of this switch. I have nothing to add that you're > > not already aware of (i.e. need to implement some sort of failed test > > retry to make the pre-commit change). > > > > On Fri, Apr 27, 2018 at 10:05 AM, Grant Henke <ghe...@cloudera.com> > wrote: > > > A recent patch has evaluated and addressed any remaining Artifact > issues. > > > See the commit message here for details: > > > > > > https://github.com/apache/kudu/commit/7f04873975d73bb335e09ee2a9bbbb9fd4b0cd89 > > > > > > That effort improved both the Maven and the Gradle artifacts. > > > > > > I wouldn't say the two are exactly the same. I would say that the > Gradle > > > artifacts are better. > > > > > > - The manifest information in the jars is better. > > > - Including an accurate NOTICE.txt and LICENSE.txt > > > - The pom's are more simplified. > > > - They only include what a user needs to know, not all the > complex > > > build details. > > > > > > I will walkthrough the full release process as a part of #3 in my > > proposal > > > well before the 1.8 release to verify there are no other issues. > > > > > > Thank you, > > > Grant > > > > > > On Fri, Apr 27, 2018 at 11:56 AM, Todd Lipcon <t...@cloudera.com> > wrote: > > > > > >> I'm on board with this. > > >> > > >> Have we already tested the artifact publishing process with gradle? > Does > > >> the resulting pom look identical to the pom that we publish with > maven? > > I > > >> assume so, but want to make sure we don't miss this part of the > process > > >> which we don't do except for at release time. > > >> > > >> -Todd > > >> > > >> On Fri, Apr 27, 2018 at 9:31 AM, Grant Henke <ghe...@cloudera.com> > > wrote: > > >> > > >> > Hi Kudu dev community, > > >> > > > >> > For some time now the Java build has had an experimental Gradle > build > > ( > > >> > KUDU-2066 <https://issues.apache.org/jira/browse/KUDU-2241>) along > > side > > >> > the > > >> > primary Maven build. Since then the Gradle build has been improved > to > > >> > support all of the necessary functionality of the Maven build and > > more. > > >> > > > >> > There are many benefits to using Gradle as listed here: > > >> > https://gradle.org/maven-vs-gradle/ > > >> > > > >> > Some of the benefits specific to Kudu have been: > > >> > > > >> > - Shading Control > > >> > - We have had many issues in Maven because modules can not > > depend > > >> on > > >> > the shaded artifacts of other modules. This resulted in the > > >> > classpath at > > >> > compile and test time being different from the classpath in > our > > >> > deployed > > >> > artifacts. (KUDU-2241 > > >> > <https://issues.apache.org/jira/browse/KUDU-2241>, KUDU-1780 > > >> > <https://issues.apache.org/jira/browse/KUDU-1780>) > > >> > - Reliable Incremental Builds > > >> > - Maven can pull module dependencies from the local repository > > >> during > > >> > incremental builds. Because of some of the shading issues > above, > > >> > this could > > >> > result in different resulting artifacts depending on > > how/when/where > > >> > the > > >> > build was run. (KUDU-2043 > > >> > <https://issues.apache.org/jira/browse/KUDU-2043>) > > >> > - Mini-Cluster discovery: Because Maven can use locally > > installed > > >> > module jars in incremental builds, modules other than > > kudu-client > > >> > can not > > >> > discover the kudu binaries. This happens because the search is > > >> > started from > > >> > the ~/.m2 directory where the installed kudu-client jar is > > instead > > >> > of the > > >> > project directory. See here > > >> > <https://github.com/apache/kudu/blob/master/java/kudu- > > >> > client/src/test/java/org/apache/kudu/client/TestUtils.java#L93-L98> > > >> > for details. > > >> > - Gradle Wrapper > > >> > - We can upgrade our gradle version without breaking any build > > >> > environments. This prevent's us from being stuck on a lower > > build > > >> > version > > >> > for compatibility purposes. > > >> > - Dependency updates > > >> > - The Gradle build can indicate when newer versions of > libraries > > >> are > > >> > available. This has been used to keep Kudu dependencies > current. > > >> > - Distributed testing > > >> > - A WIP patch for running dist-test on the java tests is > posted > > in > > >> > Gerrit here <https://gerrit.cloudera.org/#/c/9932/>. > > >> > - Future Benefits > > >> > - The flexibility of Gradle will allow for faster more > reliable > > >> > builds in the future. > > >> > - Example: Precise testing control so we can run parallel > > local > > >> > tests. > > >> > - Gradle is being improved upon a lot with regular feature > > >> releases. > > >> > > > >> > > > >> > I would like to propose the following plan to make Gradle the > primary > > >> build > > >> > for Kudu: > > >> > > > >> > 1. Fix any issues or gaps that remain with the Gradle build > > >> > - Please experiment with the Gradle build and open Jiras about > > any > > >> > issues you find or improvements that should be made. > > >> > - See the README > > >> > <https://github.com/apache/kudu/tree/master/java# > > >> > building-with-gradle> > > >> > for some tips on using Gradle (I will update documentation to > be > > >> more > > >> > robust). > > >> > 2. Use Gradle as the default build in Jenkins pre-commit > > >> > - I am testing this now and fixing any issues. > > >> > 3. Change documentation referencing Maven to use Gradle > instead > > >> > - This includes releasing documentation. > > >> > 4. Release Kudu 1.8 using Gradle in place of Maven > > >> > 5. Remove the Maven build from the project > > >> > - This shouldn't be immediately but maintaining two builds is > a > > >> pain, > > >> > so the Maven build should be removed at some point. > > >> > > > >> > Please let me know your thoughts on the above proposal and plan. > > >> > > > >> > Thank you, > > >> > Grant > > >> > -- > > >> > Grant Henke > > >> > Software Engineer | Cloudera > > >> > gr...@cloudera.com | twitter.com/gchenke | > linkedin.com/in/granthenke > > >> > > > >> > > >> > > >> > > >> -- > > >> Todd Lipcon > > >> Software Engineer, Cloudera > > >> > > > > > > > > > > > > -- > > > Grant Henke > > > Software Engineer | Cloudera > > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > > > > > -- > Grant Henke > Software Engineer | Cloudera > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke >
