Thanks Yifan, this is great! My unsolicited feedback: - Could it warn against dependencies that did not get updates for a long time? For python there were examples of a dependency being abandoned by its own developers and it took us a while to figure it out and switch to maintained one. (Currently googledatastore is a dependency like that.)
I will second the Scott's question for what should I do with this report? Is it possible to add a link to quickly create a JIRA issue for a given dependency? Or is it possible to link to already open issues for the identified dependencies? Ahmet On Wed, Jun 13, 2018 at 10:13 AM, Scott Wegner <[email protected]> wrote: > Nifty. Here's some unsolicited feedback: > > * The report gives a nice view of the data and leaves it as an exercise to > the reader to do the math on each row (v0.25.0 to v1.3.0 = 1 major version > behind, 2017-06-26 to 2018-06-08 = 1 year behind). I would find the report > more digestable if these details were already included. > * The next question after reading this report is "What should I do with > this?" I recommend embedding details or links to answer that. For example: > > "High Priority Dependency Updates are defined as XYZ. In Beam, we make a > best-effort attempt at keeping all dependencies up-to-date according to > ABC. In the future, issues will be filed and tracked for these > automatically, but in the meantime you can search for existing issues or > open a new one <here>. Read more about our dependency update policy <here>." > > On Wed, Jun 13, 2018 at 9:02 AM Pablo Estrada <[email protected]> wrote: > >> Ahh very nice... thanks Yifan & Cham! >> Lots of old dependencies eh... very interesting. >> Best >> -P. >> >> On Wed, Jun 13, 2018 at 7:45 AM Yifan Zou <[email protected]> wrote: >> >>> Hi, >>> >>> >>> I want to follow up and explain this email. >>> >>> >>> This is a sample email that reports the results of Beam SDK dependency >>> check, which was proposed here >>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp>. >>> The goal is finding updates for all Beam Python & Java SDKs' dependencies >>> and prioritize them. The job will be auto triggered in Jenkins once a week >>> and generate a report. The report lists the high priority updates base on >>> the following criteria: >>> >>> >>> The dependency update is high priority if: >>> >>> 1. It has major versions update available; >>> >>> e.g. org.assertj:assertj-core 2.5.0 -> 3.10.0 >>> >>> 2. or, it is over 3 minor versions behind the latest version; >>> >>> e.g. org.tukaani:xz 1.5 -> 1.8 >>> >>> 3. or, the current version is behind the later version for over 180 days. >>> >>> >>> e.g. com.google.auto.service:auto-service 2014-10-24 -> 2017-12-11 >>> >>> >>> This job helps Beam contributors to determine the dependency which is >>> far behind the latest released version. The next step would be automating >>> filing JIRA bugs for dep updates, group dependencies and identify owners to >>> take care of the upgrades follow Chamikara's proposal >>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit> >>> . >>> >>> >>> For more readings: >>> >>> [Proposal] Beam dependency check automation >>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp> >>> by Yifan Zou >>> >>> [Proposal] Beam dependency update policy >>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit> >>> by *Chamikara Jayalath* >>> >>> Thank you. >>> >>> Yifan Zou >>> >>> On Wed, Jun 13, 2018 at 7:41 AM Apache Jenkins Server < >>> [email protected]> wrote: >>> >>>> High Priority Dependency Updates Of Beam Python SDK: >>>> *Dependency Name* *Current Version* *Later Version* *Current Version >>>> Release Date* *Later Version Release Date* >>>> google-cloud-bigquery 0.25.0 1.3.0 2017-06-26 2018-06-08 >>>> httplib2 0.9.2 0.11.3 2015-09-28 2018-03-30 High Priority Dependency >>>> Updates Of Beam Java SDK: >>>> *Dependency Name* *Current Version* *Later Version* *Current Version >>>> Release Date* *Later Version Release Date* >>>> org.assertj:assertj-core 2.5.0 3.10.0 2016-07-03 2018-05-11 >>>> com.google.auto.service:auto-service 1.0-rc2 1.0-rc4 2014-10-24 >>>> 2017-12-11 >>>> biz.aQute:bndlib 1.43.0 2.0.0.20130123-133441 2011-04-01 2013-02-27 >>>> org.apache.cassandra:cassandra-all 3.9 3.11.2 2016-09-26 2018-02-14 >>>> commons-cli:commons-cli 1.2 1.4 2009-03-19 2017-03-09 >>>> commons-codec:commons-codec 1.9 1.11 2013-12-20 2017-10-17 >>>> org.apache.commons:commons-dbcp2 2.1.1 2.3.0 2015-08-02 2018-05-08 >>>> com.typesafe:config 1.3.0 1.3.3 2015-05-08 2018-02-21 >>>> de.flapdoodle.embed:de.flapdoodle.embed.mongo 1.50.1 2.0.3 2015-12-11 >>>> 2018-02-14 >>>> de.flapdoodle.embed:de.flapdoodle.embed.process 1.50.1 2.0.3 2015-12-11 >>>> 2018-02-14 >>>> org.apache.derby:derby 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03 >>>> org.apache.derby:derbyclient 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03 >>>> org.apache.derby:derbynet 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03 >>>> org.elasticsearch:elasticsearch 5.6.3 6.2.4 2017-10-06 2018-04-12 >>>> org.elasticsearch:elasticsearch-hadoop 5.0.0 6.2.4 2016-10-26 >>>> 2018-04-12 >>>> org.elasticsearch.client:elasticsearch-rest-client 5.6.3 6.2.4 >>>> 2017-10-06 2018-04-12 >>>> com.alibaba:fastjson 1.2.12 1.2.47 2016-05-21 2018-03-15 >>>> org.elasticsearch.test:framework 5.6.3 6.2.4 2017-10-06 2018-04-12 >>>> org.freemarker:freemarker 2.3.25-incubating 2.3.28 2016-06-14 >>>> 2018-03-30 >>>> org.codehaus.groovy:groovy-all 2.4.13 3.0.0-alpha-2 2017-11-22 >>>> 2018-04-16 >>>> org.apache.hbase:hbase-common 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>> 2018-05-31 >>>> org.apache.hbase:hbase-hadoop-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>> 2018-05-31 >>>> org.apache.hbase:hbase-hadoop2-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>> 2018-05-31 >>>> org.apache.hbase:hbase-server 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>> 2018-05-31 >>>> org.apache.hbase:hbase-shaded-client 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>> 2018-05-31 >>>> org.apache.hbase:hbase-shaded-server 1.2.6 2.0.0-alpha2 2017-05-29 >>>> 2018-05-31 >>>> org.apache.hive:hive-cli 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21 >>>> org.apache.hive:hive-common 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21 >>>> org.apache.hive:hive-exec 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21 >>>> org.apache.hive.hcatalog:hive-hcatalog-core 2.1.0 3.0.0.3.0.0.3-2 >>>> 2016-06-16 2018-05-21 >>>> org.apache.httpcomponents:httpasyncclient 4.1.2 4.1.3 2016-06-18 >>>> 2017-02-05 >>>> org.apache.httpcomponents:httpclient 4.5.2 4.5.5 2016-02-21 2018-01-18 >>>> org.apache.httpcomponents:httpcore 4.4.5 4.4.9 2016-06-08 2018-01-11 >>>> net.java.dev.javacc:javacc 4.0 7.0.3 2018-06-08 2017-11-06 >>>> jline:jline 2.14.6 3.0.0.M1 2018-03-26 2018-06-08 >>>> net.java.dev.jna:jna 4.1.0 4.5.1 2014-03-06 2017-12-27 >>>> com.esotericsoftware.kryo:kryo 2.21 2.24.0 2013-02-27 2014-05-04 >>>> io.dropwizard.metrics:metrics-core 3.1.2 4.1.0-rc2 2015-04-25 >>>> 2018-05-03 >>>> org.mongodb:mongo-java-driver 3.2.2 3.8.0-beta3 2016-02-15 2018-05-29 >>>> io.netty:netty-all 4.1.17.Final 5.0.0.Alpha2 2017-11-08 2018-06-06 >>>> io.grpc:protoc-gen-grpc-java 1.2.0 1.12.0 2017-03-15 2018-05-07 >>>> org.apache.qpid:proton-j 0.13.1 0.27.1 2016-07-01 2018-04-25 >>>> com.carrotsearch.randomizedtesting:randomizedtesting-runner 2.5.0 2.6.3 >>>> 2017-01-23 2018-06-11 >>>> org.scala-lang:scala-library 2.11.8 2.13.0-M4 2017-03-08 2018-05-14 >>>> org.slf4j:slf4j-api 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21 >>>> org.slf4j:slf4j-jdk14 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21 >>>> org.apache.solr:solr-core 5.5.4 7.3.1 2017-10-20 2018-05-17 >>>> org.apache.solr:solr-solrj 5.5.4 7.3.1 2017-10-20 2018-05-17 >>>> org.apache.solr:solr-test-framework 5.5.4 7.3.1 2017-10-20 2018-05-17 >>>> org.springframework:spring-expression 4.3.5.RELEASE 5.0.7.RELEASE >>>> 2017-01-25 2018-06-12 >>>> sqlline:sqlline 1.3.0 1.4.0 2017-05-30 2018-05-30 >>>> com.clearspring.analytics:stream 2.9.5 2.9.6 2016-08-10 2018-01-10 >>>> org.elasticsearch.client:transport 5.0.0 6.2.4 2016-10-25 2018-04-12 >>>> org.elasticsearch.plugin:transport-netty4-client 5.6.3 6.2.4 2017-11-06 >>>> 2018-04-12 >>>> org.tukaani:xz 1.5 1.8 2014-03-08 2018-01-04 >>>> >>> -- >> Got feedback? go/pabloem-feedback >> <https://goto.google.com/pabloem-feedback> >> >
