Makes sense. Then finding a good home on the web site is the way to go. Kenn
On Mon, Jun 11, 2018 at 10:35 PM Bashir Sadjad <bas...@google.com> wrote: > FWIW, I also think that this has relevance for users. I am a user of Beam > not a contributor and only monitor this list at a high level. But I think > the dependency issue is something that many users have to deal with. It has > bitten us at least twice over the last few months due to the fact that we > depend on other libraries too and sometimes we get version conflicts (which > is one of the issues highlighted in the doc > <https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit> > Cham shared). I usually go through file histories on GitHub to try to > figure out why a certain version requirement is there. It would be nice if > the reasons are maintained at a higher level easier to consume by users. > > Cheers > > -B > > On Tue, Jun 12, 2018 at 12:19 AM Ahmet Altay <al...@google.com> wrote: > >> I think this is relevant for users. It makes sense for users to know >> about how Beam work with its dependencies and understand how conflicts will >> be addressed and when dependencies will be upgraded. >> >> On Mon, Jun 11, 2018 at 9:09 PM, Kenneth Knowles <k...@google.com> wrote: >> >>> Do you think this has relevance for users? >>> >>> If not, it might be a good use of the new Confluence space. I'm not too >>> familiar with the way permission work, but perhaps we can have a more >>> locked down area that is for policy decisions like this. >>> >>> Kenn >>> >>> On Mon, Jun 11, 2018 at 3:58 PM Chamikara Jayalath <chamik...@google.com> >>> wrote: >>> >>>> Hi All, >>>> >>>> Based on the vote (3 PMC +1s and no -1s) and based on the discussions >>>> in the doc (seems to be mostly positive), I think we can go ahead and >>>> implement some of the policies discussed so far. >>>> >>>> I have given some of the potential action items below. >>>> >>>> * Automatically generate human readable reports on status of Beam >>>> dependencies weekly and share in dev list. >>>> * Create JIRAs for significantly outdated dependencies based on above >>>> reports. >>>> * Copy some of the component level dependency version declarations to >>>> top level. >>>> * Try to identify owners for dependencies and specify owners in >>>> comments close to dependency declarations. >>>> * Vendor any dependencies that can cause issues if leaked to other >>>> components. >>>> * Add policies discussed so far to the Web site along with reasoning >>>> (from doc). >>>> >>>> Of course, I'm happy to refine or add to these polices as needed. >>>> >>>> Thanks, >>>> Cham >>>> >>>> >>>> On Thu, Jun 7, 2018 at 9:40 AM Lukasz Cwik <lc...@google.com> wrote: >>>> >>>>> +1 >>>>> >>>>> On Thu, Jun 7, 2018 at 5:18 AM Kenneth Knowles <k...@google.com> wrote: >>>>> >>>>>> +1 to these. Thanks for clarifying! >>>>>> >>>>>> Kenn >>>>>> >>>>>> On Wed, Jun 6, 2018 at 10:40 PM Chamikara Jayalath < >>>>>> chamik...@google.com> wrote: >>>>>> >>>>>>> Hi Kenn, >>>>>>> >>>>>>> On Wed, Jun 6, 2018 at 8:14 PM Kenneth Knowles <k...@google.com> >>>>>>> wrote: >>>>>>> >>>>>>>> +0.5 >>>>>>>> >>>>>>>> I like the spirit of these policies. I think they need a little >>>>>>>> wording work. Comments inline. >>>>>>>> >>>>>>>> On Wed, Jun 6, 2018 at 4:53 PM, Chamikara Jayalath < >>>>>>>>> chamik...@google.com> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> (1) Human readable reports on status of Beam dependencies are >>>>>>>>>> generated weekly and shared with the Beam community through the dev >>>>>>>>>> list. >>>>>>>>>> >>>>>>>>> >>>>>>>> Who is responsible for generating these? The mechanism or >>>>>>>> responsibility should be made clear. >>>>>>>> >>>>>>>> I clicked through a doc -> thread -> doc to find even some details. >>>>>>>> It looks like manual run of a gradle command was adopted. So the >>>>>>>> responsibility needs an owner, even if it is "unspecified volunteer on >>>>>>>> dev@ >>>>>>>> and feel free to complain or do it yourself if you don't see it" >>>>>>>> >>>>>>> >>>>>>> This is described in following doc (referenced by my doc). >>>>>>> >>>>>>> https://docs.google.com/document/d/1rqr_8a9NYZCgeiXpTIwWLCL7X8amPAVfRXsO72BpBwA/edit# >>>>>>> >>>>>>> Proposal is to run an automated Jenkins job that is run weekly, so >>>>>>> no need for someone to manually generate these reports. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> (2) Beam components should define dependencies and their versions >>>>>>>>>> at the top level. >>>>>>>>>> >>>>>>>>> >>>>>>>> I think the big "should" works better with some guidance about when >>>>>>>> something might be an exception, or at least explicit mention that >>>>>>>> there >>>>>>>> can be rare exceptions. Unless you think that is never the case. If >>>>>>>> there >>>>>>>> are no exceptions, then say "must" and if we hit a roadblock we can >>>>>>>> revisit >>>>>>>> the policy. >>>>>>>> >>>>>>> >>>>>>> The idea was to allow exceptions. Added more details to the doc. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> (3) A significantly outdated dependency (identified manually or >>>>>>>>>> through tooling) should result in a JIRA that is a blocker for the >>>>>>>>>> next >>>>>>>>>> release. Release manager may choose to push the blocker to the >>>>>>>>>> subsequent >>>>>>>>>> release or downgrade from a blocker. >>>>>>>>>> >>>>>>>>> >>>>>>>> How is "significantly outdated" defined? By dev@ discussion? Seems >>>>>>>> like the right way. Anyhow that's what will happen in practice as >>>>>>>> people >>>>>>>> debate the blocker bug. >>>>>>>> >>>>>>> >>>>>>> This will be either through the automated Jenkins job (see the doc >>>>>>> above, where the proposal is to flag new major versions and new minor >>>>>>> versions that are more than six months old) or manually (for any >>>>>>> critical >>>>>>> updates that will not be captured by the Jenkins job) (more details in >>>>>>> the >>>>>>> doc). Manually identified critical dependency updates may involve a >>>>>>> discussion in the dev list. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> (4) Dependency declarations may identify owners that are >>>>>>>>>> responsible for upgrading the respective dependencies. >>>>>>>>>> >>>>>>>>>> (5) Dependencies of Java SDK components that may cause issues to >>>>>>>>>> other components if leaked should be shaded. >>>>>>>>>> >>>>>>>>> >>>>>>>> We previously agreed upon our intent to migrate to "pre-shaded" aka >>>>>>>> "vendored" packages: >>>>>>>> https://lists.apache.org/thread.html/12383d2e5d70026427df43294e30d6524334e16f03d86c9a5860792f@%3Cdev.beam.apache.org%3E >>>>>>>> >>>>>>>> With Maven, this involved a lot of boilerplate so I never did it. >>>>>>>> With Gradle, we can easily build a re-usable rule to create such a >>>>>>>> package >>>>>>>> in a couple of lines. I just opened the first WIP PR here: >>>>>>>> https://github.com/apache/beam/pull/5570 it is blocked by deleting >>>>>>>> the poms anyhow so by then we should have a configuration that works to >>>>>>>> vendor our currently shaded artifacts. >>>>>>>> >>>>>>>> So I think this should be rephrased to "should be vendored" so we >>>>>>>> don't have to revise the policy. >>>>>>>> >>>>>>> >>>>>>> Thanks for the pointer. I agree that vendoring is a good approach. >>>>>>> >>>>>>> Here are the updated policies (and more details added to doc). I >>>>>>> agree with Ahmet's point that votes should be converted to web sites >>>>>>> where >>>>>>> we can give more details and examples. >>>>>>> >>>>>>> (1.a) Human readable reports on status of Beam dependencies are >>>>>>> generated weekly by an automated Jenkins job and shared with the Beam >>>>>>> community through the dev list. >>>>>>> >>>>>>> (2.a) Beam components should define dependencies and their versions >>>>>>> at the top level. There can be rare exceptions, but they should come >>>>>>> with >>>>>>> explanations. >>>>>>> >>>>>>> (3.a) A significantly outdated dependency (identified manually or >>>>>>> through the automated Jenkins job) should result in a JIRA that is a >>>>>>> blocker for the next release. Release manager may choose to push the >>>>>>> blocker to the subsequent release or downgrade from a blocker. >>>>>>> >>>>>>> (4.a) Dependency declarations may identify owners that are >>>>>>> responsible for upgrading the respective dependencies. >>>>>>> >>>>>>> (5.a) Dependencies of Java SDK components that may cause issues to >>>>>>> other components if leaked should be vendored. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Cham >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Kenn >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Please vote: >>>>>>>>>> [ ] +1, Approve that we adapt these policies >>>>>>>>>> [ ] -1, Do not approve (please provide specific comments) >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Cham >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://lists.apache.org/thread.html/8738c13ad7e576bc2fef158d2cc6f809e1c238ab8d5164c78484bf54@%3Cdev.beam.apache.org%3E >>>>>>>>>> [2] >>>>>>>>>> https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit?usp=sharing >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>
