Wow, this is extremely readable and actionable. Are these all direct dependencies, or also transitive?
Kenn On Wed, Jun 13, 2018 at 10:45 AM Chamikara Jayalath <[email protected]> wrote: > > Thanks Yifan. > > On Wed, Jun 13, 2018 at 10:21 AM Ahmet Altay <[email protected]> wrote: > >> Thanks Yifan, this is great! >> >> My unsolicited feedback: >> - Could it warn against dependencies that did not get updates for a long >> time? For python there were examples of a dependency being abandoned by its >> own developers and it took us a while to figure it out and switch to >> maintained one. (Currently googledatastore is a dependency like that.) >> >> I will second the Scott's question for what should I do with this report? >> Is it possible to add a link to quickly create a JIRA issue for a given >> dependency? Or is it possible to link to already open issues for the >> identified dependencies? >> > > We ultimately want to create JIRAs automatically and assign them to owners > if defined according to the policy document. We can also add a link to the > JIRA to the report. But given the number of outdated dependencies listed > here, looks like we'll have to bootstrap the process by manually updating > many of these dependencies. > > - Cham > > >> Ahmet >> >> On Wed, Jun 13, 2018 at 10:13 AM, Scott Wegner <[email protected]> >> wrote: >> >>> Nifty. Here's some unsolicited feedback: >>> >>> * The report gives a nice view of the data and leaves it as an exercise >>> to the reader to do the math on each row (v0.25.0 to v1.3.0 = 1 major >>> version behind, 2017-06-26 to 2018-06-08 = 1 year behind). I would find the >>> report more digestable if these details were already included. >>> * The next question after reading this report is "What should I do with >>> this?" I recommend embedding details or links to answer that. For example: >>> >>> "High Priority Dependency Updates are defined as XYZ. In Beam, we make >>> a best-effort attempt at keeping all dependencies up-to-date according to >>> ABC. In the future, issues will be filed and tracked for these >>> automatically, but in the meantime you can search for existing issues or >>> open a new one <here>. Read more about our dependency update policy <here>." >>> >>> On Wed, Jun 13, 2018 at 9:02 AM Pablo Estrada <[email protected]> >>> wrote: >>> >>>> Ahh very nice... thanks Yifan & Cham! >>>> Lots of old dependencies eh... very interesting. >>>> Best >>>> -P. >>>> >>>> On Wed, Jun 13, 2018 at 7:45 AM Yifan Zou <[email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> >>>>> I want to follow up and explain this email. >>>>> >>>>> >>>>> This is a sample email that reports the results of Beam SDK dependency >>>>> check, which was proposed here >>>>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp>. >>>>> The goal is finding updates for all Beam Python & Java SDKs' dependencies >>>>> and prioritize them. The job will be auto triggered in Jenkins once a week >>>>> and generate a report. The report lists the high priority updates base on >>>>> the following criteria: >>>>> >>>>> >>>>> The dependency update is high priority if: >>>>> >>>>> 1. It has major versions update available; >>>>> >>>>> e.g. org.assertj:assertj-core 2.5.0 -> 3.10.0 >>>>> >>>>> 2. or, it is over 3 minor versions behind the latest version; >>>>> >>>>> e.g. org.tukaani:xz 1.5 -> 1.8 >>>>> >>>>> 3. or, the current version is behind the later version for over 180 >>>>> days. >>>>> >>>>> e.g. com.google.auto.service:auto-service 2014-10-24 -> >>>>> 2017-12-11 >>>>> >>>>> >>>>> This job helps Beam contributors to determine the dependency which is >>>>> far behind the latest released version. The next step would be automating >>>>> filing JIRA bugs for dep updates, group dependencies and identify owners >>>>> to >>>>> take care of the upgrades follow Chamikara's proposal >>>>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit> >>>>> . >>>>> >>>>> >>>>> For more readings: >>>>> >>>>> [Proposal] Beam dependency check automation >>>>> <https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit#heading=h.u75g8bk11ngp> >>>>> by Yifan Zou >>>>> >>>>> [Proposal] Beam dependency update policy >>>>> <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit> >>>>> by *Chamikara Jayalath* >>>>> >>>>> Thank you. >>>>> >>>>> Yifan Zou >>>>> >>>>> On Wed, Jun 13, 2018 at 7:41 AM Apache Jenkins Server < >>>>> [email protected]> wrote: >>>>> >>>>>> High Priority Dependency Updates Of Beam Python SDK: >>>>>> *Dependency Name* *Current Version* *Later Version* *Current Version >>>>>> Release Date* *Later Version Release Date* >>>>>> google-cloud-bigquery 0.25.0 1.3.0 2017-06-26 2018-06-08 >>>>>> httplib2 0.9.2 0.11.3 2015-09-28 2018-03-30 High Priority Dependency >>>>>> Updates Of Beam Java SDK: >>>>>> *Dependency Name* *Current Version* *Later Version* *Current Version >>>>>> Release Date* *Later Version Release Date* >>>>>> org.assertj:assertj-core 2.5.0 3.10.0 2016-07-03 2018-05-11 >>>>>> com.google.auto.service:auto-service 1.0-rc2 1.0-rc4 2014-10-24 >>>>>> 2017-12-11 >>>>>> biz.aQute:bndlib 1.43.0 2.0.0.20130123-133441 2011-04-01 2013-02-27 >>>>>> org.apache.cassandra:cassandra-all 3.9 3.11.2 2016-09-26 2018-02-14 >>>>>> commons-cli:commons-cli 1.2 1.4 2009-03-19 2017-03-09 >>>>>> commons-codec:commons-codec 1.9 1.11 2013-12-20 2017-10-17 >>>>>> org.apache.commons:commons-dbcp2 2.1.1 2.3.0 2015-08-02 2018-05-08 >>>>>> com.typesafe:config 1.3.0 1.3.3 2015-05-08 2018-02-21 >>>>>> de.flapdoodle.embed:de.flapdoodle.embed.mongo 1.50.1 2.0.3 2015-12-11 >>>>>> 2018-02-14 >>>>>> de.flapdoodle.embed:de.flapdoodle.embed.process 1.50.1 2.0.3 >>>>>> 2015-12-11 2018-02-14 >>>>>> org.apache.derby:derby 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03 >>>>>> org.apache.derby:derbyclient 10.12.1.1 10.14.2.0 2015-10-10 >>>>>> 2018-05-03 >>>>>> org.apache.derby:derbynet 10.12.1.1 10.14.2.0 2015-10-10 2018-05-03 >>>>>> org.elasticsearch:elasticsearch 5.6.3 6.2.4 2017-10-06 2018-04-12 >>>>>> org.elasticsearch:elasticsearch-hadoop 5.0.0 6.2.4 2016-10-26 >>>>>> 2018-04-12 >>>>>> org.elasticsearch.client:elasticsearch-rest-client 5.6.3 6.2.4 >>>>>> 2017-10-06 2018-04-12 >>>>>> com.alibaba:fastjson 1.2.12 1.2.47 2016-05-21 2018-03-15 >>>>>> org.elasticsearch.test:framework 5.6.3 6.2.4 2017-10-06 2018-04-12 >>>>>> org.freemarker:freemarker 2.3.25-incubating 2.3.28 2016-06-14 >>>>>> 2018-03-30 >>>>>> org.codehaus.groovy:groovy-all 2.4.13 3.0.0-alpha-2 2017-11-22 >>>>>> 2018-04-16 >>>>>> org.apache.hbase:hbase-common 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>>>> 2018-05-31 >>>>>> org.apache.hbase:hbase-hadoop-compat 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>>>> 2018-05-31 >>>>>> org.apache.hbase:hbase-hadoop2-compat 1.2.6 2.0.0.3.0.0.3-2 >>>>>> 2017-05-29 2018-05-31 >>>>>> org.apache.hbase:hbase-server 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>>>> 2018-05-31 >>>>>> org.apache.hbase:hbase-shaded-client 1.2.6 2.0.0.3.0.0.3-2 2017-05-29 >>>>>> 2018-05-31 >>>>>> org.apache.hbase:hbase-shaded-server 1.2.6 2.0.0-alpha2 2017-05-29 >>>>>> 2018-05-31 >>>>>> org.apache.hive:hive-cli 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21 >>>>>> org.apache.hive:hive-common 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 >>>>>> 2018-05-21 >>>>>> org.apache.hive:hive-exec 2.1.0 3.0.0.3.0.0.3-2 2016-06-16 2018-05-21 >>>>>> org.apache.hive.hcatalog:hive-hcatalog-core 2.1.0 3.0.0.3.0.0.3-2 >>>>>> 2016-06-16 2018-05-21 >>>>>> org.apache.httpcomponents:httpasyncclient 4.1.2 4.1.3 2016-06-18 >>>>>> 2017-02-05 >>>>>> org.apache.httpcomponents:httpclient 4.5.2 4.5.5 2016-02-21 >>>>>> 2018-01-18 >>>>>> org.apache.httpcomponents:httpcore 4.4.5 4.4.9 2016-06-08 2018-01-11 >>>>>> net.java.dev.javacc:javacc 4.0 7.0.3 2018-06-08 2017-11-06 >>>>>> jline:jline 2.14.6 3.0.0.M1 2018-03-26 2018-06-08 >>>>>> net.java.dev.jna:jna 4.1.0 4.5.1 2014-03-06 2017-12-27 >>>>>> com.esotericsoftware.kryo:kryo 2.21 2.24.0 2013-02-27 2014-05-04 >>>>>> io.dropwizard.metrics:metrics-core 3.1.2 4.1.0-rc2 2015-04-25 >>>>>> 2018-05-03 >>>>>> org.mongodb:mongo-java-driver 3.2.2 3.8.0-beta3 2016-02-15 2018-05-29 >>>>>> io.netty:netty-all 4.1.17.Final 5.0.0.Alpha2 2017-11-08 2018-06-06 >>>>>> io.grpc:protoc-gen-grpc-java 1.2.0 1.12.0 2017-03-15 2018-05-07 >>>>>> org.apache.qpid:proton-j 0.13.1 0.27.1 2016-07-01 2018-04-25 >>>>>> com.carrotsearch.randomizedtesting:randomizedtesting-runner 2.5.0 >>>>>> 2.6.3 2017-01-23 2018-06-11 >>>>>> org.scala-lang:scala-library 2.11.8 2.13.0-M4 2017-03-08 2018-05-14 >>>>>> org.slf4j:slf4j-api 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21 >>>>>> org.slf4j:slf4j-jdk14 1.7.25 1.8.0-beta2 2017-03-16 2018-03-21 >>>>>> org.apache.solr:solr-core 5.5.4 7.3.1 2017-10-20 2018-05-17 >>>>>> org.apache.solr:solr-solrj 5.5.4 7.3.1 2017-10-20 2018-05-17 >>>>>> org.apache.solr:solr-test-framework 5.5.4 7.3.1 2017-10-20 2018-05-17 >>>>>> org.springframework:spring-expression 4.3.5.RELEASE 5.0.7.RELEASE >>>>>> 2017-01-25 2018-06-12 >>>>>> sqlline:sqlline 1.3.0 1.4.0 2017-05-30 2018-05-30 >>>>>> com.clearspring.analytics:stream 2.9.5 2.9.6 2016-08-10 2018-01-10 >>>>>> org.elasticsearch.client:transport 5.0.0 6.2.4 2016-10-25 2018-04-12 >>>>>> org.elasticsearch.plugin:transport-netty4-client 5.6.3 6.2.4 >>>>>> 2017-11-06 2018-04-12 >>>>>> org.tukaani:xz 1.5 1.8 2014-03-08 2018-01-04 >>>>>> >>>>> -- >>>> Got feedback? go/pabloem-feedback >>>> <https://goto.google.com/pabloem-feedback> >>>> >>> >>
