+1, thanks Grant! Mike
On Thu, Oct 4, 2018 at 12:07 PM Dan Burkert <danburk...@apache.org> wrote: > Sounds great, thanks for chasing this down! > > - Dan > > On Wed, Oct 3, 2018 at 7:20 PM Grant Henke <ghe...@cloudera.com.invalid> > wrote: > > > Hi Kudu dev community, > > > > I wanted to provide a progress and plan update to switching over to > Gradle > > as the primary build tool. > > > > First, the following steps have already been completed: > > > > - Fixed outstanding Gradle build issues > > - Added dist-test support to Gradle > > - Started using Gradle as the default build in Jenkins pre-commit > > - Changed the Java README to reference Gradle instead of Maven > > - Changed the releasing documentation to reference Gradle instead of > > Maven > > > > Next, the plan is to release Kudu 1.8 using Gradle and remove the maven > > build shortly after branching. > > > > There are two motivations for removing the Maven build now. The primary > > reason is to reduce the workload of maintaining two builds. The secondary > > reason is to simplify some of the test dependencies and break the test > > utilities out into their own module. This is much more straightforward in > > Gradle than it is in Maven. > > > > If there are any concerns or issues with this plan please let me know. > > > > Thank you, > > Grant > > > > On Fri, Apr 27, 2018 at 5:07 PM Adar Lieber-Dembo <a...@cloudera.com> > > wrote: > > > > > I am also in favor of this switch. I have nothing to add that you're > > > not already aware of (i.e. need to implement some sort of failed test > > > retry to make the pre-commit change). > > > > > > On Fri, Apr 27, 2018 at 10:05 AM, Grant Henke <ghe...@cloudera.com> > > wrote: > > > > A recent patch has evaluated and addressed any remaining Artifact > > issues. > > > > See the commit message here for details: > > > > > > > > > > https://github.com/apache/kudu/commit/7f04873975d73bb335e09ee2a9bbbb9fd4b0cd89 > > > > > > > > That effort improved both the Maven and the Gradle artifacts. > > > > > > > > I wouldn't say the two are exactly the same. I would say that the > > Gradle > > > > artifacts are better. > > > > > > > > - The manifest information in the jars is better. > > > > - Including an accurate NOTICE.txt and LICENSE.txt > > > > - The pom's are more simplified. > > > > - They only include what a user needs to know, not all the > > complex > > > > build details. > > > > > > > > I will walkthrough the full release process as a part of #3 in my > > > proposal > > > > well before the 1.8 release to verify there are no other issues. > > > > > > > > Thank you, > > > > Grant > > > > > > > > On Fri, Apr 27, 2018 at 11:56 AM, Todd Lipcon <t...@cloudera.com> > > wrote: > > > > > > > >> I'm on board with this. > > > >> > > > >> Have we already tested the artifact publishing process with gradle? > > Does > > > >> the resulting pom look identical to the pom that we publish with > > maven? > > > I > > > >> assume so, but want to make sure we don't miss this part of the > > process > > > >> which we don't do except for at release time. > > > >> > > > >> -Todd > > > >> > > > >> On Fri, Apr 27, 2018 at 9:31 AM, Grant Henke <ghe...@cloudera.com> > > > wrote: > > > >> > > > >> > Hi Kudu dev community, > > > >> > > > > >> > For some time now the Java build has had an experimental Gradle > > build > > > ( > > > >> > KUDU-2066 <https://issues.apache.org/jira/browse/KUDU-2241>) > along > > > side > > > >> > the > > > >> > primary Maven build. Since then the Gradle build has been improved > > to > > > >> > support all of the necessary functionality of the Maven build and > > > more. > > > >> > > > > >> > There are many benefits to using Gradle as listed here: > > > >> > https://gradle.org/maven-vs-gradle/ > > > >> > > > > >> > Some of the benefits specific to Kudu have been: > > > >> > > > > >> > - Shading Control > > > >> > - We have had many issues in Maven because modules can not > > > depend > > > >> on > > > >> > the shaded artifacts of other modules. This resulted in the > > > >> > classpath at > > > >> > compile and test time being different from the classpath in > > our > > > >> > deployed > > > >> > artifacts. (KUDU-2241 > > > >> > <https://issues.apache.org/jira/browse/KUDU-2241>, > KUDU-1780 > > > >> > <https://issues.apache.org/jira/browse/KUDU-1780>) > > > >> > - Reliable Incremental Builds > > > >> > - Maven can pull module dependencies from the local > repository > > > >> during > > > >> > incremental builds. Because of some of the shading issues > > above, > > > >> > this could > > > >> > result in different resulting artifacts depending on > > > how/when/where > > > >> > the > > > >> > build was run. (KUDU-2043 > > > >> > <https://issues.apache.org/jira/browse/KUDU-2043>) > > > >> > - Mini-Cluster discovery: Because Maven can use locally > > > installed > > > >> > module jars in incremental builds, modules other than > > > kudu-client > > > >> > can not > > > >> > discover the kudu binaries. This happens because the search > is > > > >> > started from > > > >> > the ~/.m2 directory where the installed kudu-client jar is > > > instead > > > >> > of the > > > >> > project directory. See here > > > >> > <https://github.com/apache/kudu/blob/master/java/kudu- > > > >> > > client/src/test/java/org/apache/kudu/client/TestUtils.java#L93-L98> > > > >> > for details. > > > >> > - Gradle Wrapper > > > >> > - We can upgrade our gradle version without breaking any > build > > > >> > environments. This prevent's us from being stuck on a lower > > > build > > > >> > version > > > >> > for compatibility purposes. > > > >> > - Dependency updates > > > >> > - The Gradle build can indicate when newer versions of > > libraries > > > >> are > > > >> > available. This has been used to keep Kudu dependencies > > current. > > > >> > - Distributed testing > > > >> > - A WIP patch for running dist-test on the java tests is > > posted > > > in > > > >> > Gerrit here <https://gerrit.cloudera.org/#/c/9932/>. > > > >> > - Future Benefits > > > >> > - The flexibility of Gradle will allow for faster more > > reliable > > > >> > builds in the future. > > > >> > - Example: Precise testing control so we can run parallel > > > local > > > >> > tests. > > > >> > - Gradle is being improved upon a lot with regular feature > > > >> releases. > > > >> > > > > >> > > > > >> > I would like to propose the following plan to make Gradle the > > primary > > > >> build > > > >> > for Kudu: > > > >> > > > > >> > 1. Fix any issues or gaps that remain with the Gradle build > > > >> > - Please experiment with the Gradle build and open Jiras > about > > > any > > > >> > issues you find or improvements that should be made. > > > >> > - See the README > > > >> > <https://github.com/apache/kudu/tree/master/java# > > > >> > building-with-gradle> > > > >> > for some tips on using Gradle (I will update documentation > to > > be > > > >> more > > > >> > robust). > > > >> > 2. Use Gradle as the default build in Jenkins pre-commit > > > >> > - I am testing this now and fixing any issues. > > > >> > 3. Change documentation referencing Maven to use Gradle > > instead > > > >> > - This includes releasing documentation. > > > >> > 4. Release Kudu 1.8 using Gradle in place of Maven > > > >> > 5. Remove the Maven build from the project > > > >> > - This shouldn't be immediately but maintaining two builds > is > > a > > > >> pain, > > > >> > so the Maven build should be removed at some point. > > > >> > > > > >> > Please let me know your thoughts on the above proposal and plan. > > > >> > > > > >> > Thank you, > > > >> > Grant > > > >> > -- > > > >> > Grant Henke > > > >> > Software Engineer | Cloudera > > > >> > gr...@cloudera.com | twitter.com/gchenke | > > linkedin.com/in/granthenke > > > >> > > > > >> > > > >> > > > >> > > > >> -- > > > >> Todd Lipcon > > > >> Software Engineer, Cloudera > > > >> > > > > > > > > > > > > > > > > -- > > > > Grant Henke > > > > Software Engineer | Cloudera > > > > gr...@cloudera.com | twitter.com/gchenke | > linkedin.com/in/granthenke > > > > > > > > > -- > > Grant Henke > > Software Engineer | Cloudera > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > > >
