OK. Bringing an important update on licensing to this thread for consideration. Discussion on https://issues.apache.org/jira/browse/LEGAL-601 has concluded with key takeaways. These are things that were already true and people who are good at this stuff already may know, but I'm just going to say them again as I understand them:
- We can dual license MIT-0 and ASL2, which means "we" gives "users" the permissions of both licenses - they can take their pick so they can treat it as MIT-0 licensed. - BUT the copyright holders are the contributors to the project. They must agree that their contributions can be licensed like this. The ASF ICLA only agrees to ASL2 so we need to let them know. I suggest a CONTRIBUTING.md that mentions it and maybe a PULL_REQUEST_TEMPLATE.md with a checkbox*. - If we want, we can include a README that explains this and tells users they can delete the bits related to ASL2/ASF and CONTRIBUTING.md if they want to change it however they want. So I guess now the decision is whether all of the above is complicated enough for users that it outweighs the benefit. I'm not really sure. *Exactly how formal we need to get here is a matter of some debate and risk tolerance. For these repos I think there is very little risk. One could even argue the contents are so unoriginal as to be uncopyrightable., but the bar in the US for i.p. is comically low so that's not a good argument to depend on. Kenn On Tue, Mar 1, 2022 at 10:28 AM David Cavazos <dcava...@google.com> wrote: > Friendly ping on this :) > > On Fri, Feb 25, 2022 at 12:52 PM David Cavazos <dcava...@google.com> > wrote: > >> Can we create an empty file on each directory so I can fork the repo? It >> doesn't look like there is a workaround to cloning empty repos in GitHub. >> Then I can send a pull request. >> >> On Fri, Feb 18, 2022 at 10:40 AM David Cavazos <dcava...@google.com> >> wrote: >> >>> Got it, thank you! I'll go ahead and add the NOTICE file. >>> >>> I was trying to create a PR to merge the starter project contents, but I >>> can't fork the repo because it's empty. Can I either get permissions to >>> directly push or bother you with creating an empty README or some other >>> file so I can fork it and open a PR? Thanks! >>> >>> [image: image.png] >>> >>> On Fri, Feb 18, 2022 at 8:32 AM Kenneth Knowles <k...@apache.org> wrote: >>> >>>> I always get mixed up myself. The policies are at >>>> https://www.apache.org/legal/src-headers.html#notice and there's some >>>> step by step at https://infra.apache.org/licensing-howto.html >>>> >>>> TL;DR the contents should be like so: >>>> >>>> Apache Beam >>>> Copyright [2022-] The Apache Software Foundation >>>> >>>> This product includes software developed at >>>> The Apache Software Foundation (http://www.apache.org/). >>>> >>>> Kenn >>>> >>>> On Thu, Feb 17, 2022 at 2:28 PM David Cavazos <dcava...@google.com> >>>> wrote: >>>> >>>>> I found this example NOTICE >>>>> <https://infra.apache.org/licensing-howto.html#example-notice> file, >>>>> but it doesn't look like it does what we want. It looks like it has to be >>>>> written in a formal legal language and I don't feel comfortable writing >>>>> it. >>>>> Can I ask for help on writing out the contents of the NOTICE file? >>>>> >>>>> On Thu, Feb 17, 2022 at 11:00 AM David Cavazos <dcava...@google.com> >>>>> wrote: >>>>> >>>>>> Can someone point me to an example on how the NOTICE file should look >>>>>> like? I'm not familiar with it and would like to get it right. >>>>>> >>>>>> On Thu, Feb 17, 2022 at 10:53 AM David Cavazos <dcava...@google.com> >>>>>> wrote: >>>>>> >>>>>>> +1 >>>>>>> For the starter projects I like them being "clone and go", but I'd >>>>>>> like to keep them as minimal as possible. We could have another repo >>>>>>> like >>>>>>> `beam-working-examples` for more complete examples where each >>>>>>> subdirectory >>>>>>> is a self-contained example with all its build files and everything. >>>>>>> >>>>>>> On Wed, Feb 16, 2022 at 5:59 AM Kenneth Knowles <k...@apache.org> >>>>>>> wrote: >>>>>>> >>>>>>>> I like the goal: for things where the build has extra setup, have >>>>>>>> an example that is fully functional on its own. There is of course the >>>>>>>> problem of "where does it end?" since this is infinity things. >>>>>>>> >>>>>>>> The other piece is that a user wanting to know some of these bits >>>>>>>> may be past the "clone and go" stage of their project. They probably >>>>>>>> already have a project and now they need a working example to read and >>>>>>>> learn from. So it could be just one additional repo >>>>>>>> `beam-working-examples` >>>>>>>> where each subdirectory is an independent working setup. I do like >>>>>>>> having >>>>>>>> it a separate repo to avoid the temptation to leverage anything from >>>>>>>> the >>>>>>>> Beam build. And each subdirectory should be entirely independent and we >>>>>>>> also have to avoid the temptation to share configuration across them, >>>>>>>> or it >>>>>>>> would defeat the purpose. >>>>>>>> >>>>>>>> Kenn >>>>>>>> >>>>>>>> On Tue, Feb 15, 2022 at 9:28 PM Reza Ardeshir Rokni < >>>>>>>> raro...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> This is great! >>>>>>>>> >>>>>>>>> What do folks think about also having a less minimal set of >>>>>>>>> starters? For Java I am thinking about protobuf / autovalue. For >>>>>>>>> Python >>>>>>>>> maybe an opinionated setup with tox etc... Again this would just >>>>>>>>> contain >>>>>>>>> 'hello' world samples to get folks going. >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> Reza >>>>>>>>> >>>>>>>>> On Wed, 9 Feb 2022 at 13:56, Robert Burke <r...@google.com> wrote: >>>>>>>>> >>>>>>>>>> SGTM. >>>>>>>>>> >>>>>>>>>> On Wed, Feb 9, 2022 at 1:09 PM Kenneth Knowles <k...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Based on discussion on >>>>>>>>>>> https://issues.apache.org/jira/browse/LEGAL-601 I think it will >>>>>>>>>>> be simplest to license it under ASL2 and include a NOTICE file. The >>>>>>>>>>> user >>>>>>>>>>> will be free to "clone and go". >>>>>>>>>>> >>>>>>>>>>> I would bring these points back to the dev list: >>>>>>>>>>> >>>>>>>>>>> - ASL2 is what people expect from an ASF project, so it is >>>>>>>>>>> "least surprise" >>>>>>>>>>> - Dual-licensing is possible (but I think not worthwhile due to >>>>>>>>>>> its impact on contributor license agreements) >>>>>>>>>>> - ASL2 says "You must cause any modified files to carry >>>>>>>>>>> prominent notices stating that You changed the files" which won't >>>>>>>>>>> apply to >>>>>>>>>>> the user's code and I would guess they simply won't bother with for >>>>>>>>>>> files >>>>>>>>>>> in the template. Or maybe there is a clever way to phrase the >>>>>>>>>>> header so it >>>>>>>>>>> is already good to go. >>>>>>>>>>> - ASL2 says if the work includes a NOTICE file, you have to >>>>>>>>>>> includes the attributions from it. The NOTICE file is required by >>>>>>>>>>> ASF >>>>>>>>>>> policy. We can easily set it up to be a noop for the user. >>>>>>>>>>> >>>>>>>>>>> So my overall take is that we should go ahead with ASL2 and a >>>>>>>>>>> simple NOTICE file. Check the Jira for details. >>>>>>>>>>> >>>>>>>>>>> Kenn >>>>>>>>>>> >>>>>>>>>>> On Mon, Feb 7, 2022 at 10:47 AM Kenneth Knowles <k...@apache.org> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> And I've created the repos just now. >>>>>>>>>>>> >>>>>>>>>>>> Kenn >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Feb 7, 2022 at 10:39 AM Kenneth Knowles < >>>>>>>>>>>> k...@apache.org> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Legal question asked at >>>>>>>>>>>>> https://issues.apache.org/jira/browse/LEGAL-601 >>>>>>>>>>>>> >>>>>>>>>>>>> Kenn >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Feb 4, 2022 at 7:58 AM Danny McCormick < >>>>>>>>>>>>> dannymccorm...@google.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Sure - I'm happy to help out with the Actions setup (and/or >>>>>>>>>>>>>> with the Go template). I will say though, the Actions config >>>>>>>>>>>>>> should be >>>>>>>>>>>>>> pretty darn simple for these examples - >>>>>>>>>>>>>> https://github.com/davidcavazos/beam-java/blob/main/.github/workflows/test.yaml >>>>>>>>>>>>>> seems right, for each language configuration we're targeting we >>>>>>>>>>>>>> basically >>>>>>>>>>>>>> just want a job with: >>>>>>>>>>>>>> >>>>>>>>>>>>>> - checkout >>>>>>>>>>>>>> - setup-<language> >>>>>>>>>>>>>> - inlined script to run tests >>>>>>>>>>>>>> >>>>>>>>>>>>>> Always happy to help with or consult on any actions issues 🙂 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Danny >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Feb 4, 2022 at 10:21 AM Kerry Donny-Clark < >>>>>>>>>>>>>> kerr...@google.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Danny has extensive experience with GitHub actions, and may >>>>>>>>>>>>>>> be able to help out. >>>>>>>>>>>>>>> Kerry >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Thu, Feb 3, 2022, 11:47 PM Kenneth Knowles < >>>>>>>>>>>>>>> k...@apache.org> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm convinced on all points. My main motivation was to keep >>>>>>>>>>>>>>>> it simple. But of course we should keep it simple for users, >>>>>>>>>>>>>>>> not us :-) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I can take on the task of asking about MIT license and >>>>>>>>>>>>>>>> requesting the repos be created. Not sure if it needs my level >>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>> privileges but I'm happy to do it anyhow. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Kenn >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, Feb 2, 2022 at 10:30 AM Robert Bradshaw < >>>>>>>>>>>>>>>> rober...@google.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Wed, Feb 2, 2022 at 10:12 AM David Cavazos < >>>>>>>>>>>>>>>>> dcava...@google.com> wrote: >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > MIT is much more permissive, but I also don't have any >>>>>>>>>>>>>>>>> problems changing it to Apache license. In any case, how >>>>>>>>>>>>>>>>> about we create >>>>>>>>>>>>>>>>> the following repos? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> For these starter projects, we don't want to encumber any >>>>>>>>>>>>>>>>> users of >>>>>>>>>>>>>>>>> these templates with any particular licensing requirements >>>>>>>>>>>>>>>>> (right?) >>>>>>>>>>>>>>>>> and we don't even care about attribution. We want these to >>>>>>>>>>>>>>>>> be pretty >>>>>>>>>>>>>>>>> much as close to public domain as possible. That's not >>>>>>>>>>>>>>>>> what the Apache >>>>>>>>>>>>>>>>> licence does. (If it's even relevant, a good argument >>>>>>>>>>>>>>>>> could likely be >>>>>>>>>>>>>>>>> made for de minis or fair use, but I think it's best to be >>>>>>>>>>>>>>>>> explicit >>>>>>>>>>>>>>>>> about this. Perhaps this'd be a good question for apache >>>>>>>>>>>>>>>>> legal? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> > apache/beam-starter-java >>>>>>>>>>>>>>>>> > apache/beam-starter-python >>>>>>>>>>>>>>>>> > apache/beam-starter-go >>>>>>>>>>>>>>>>> > apache/beam-starter-kotlin >>>>>>>>>>>>>>>>> > apache/beam-starter-scala >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > We'll start by populating the Java one which is the most >>>>>>>>>>>>>>>>> pressing one and the one that is ready, but the rest should >>>>>>>>>>>>>>>>> be simpler. >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > +David Huntsperger, tldr; these are minimal starter >>>>>>>>>>>>>>>>> projects for every language. Once we have Java, Python and >>>>>>>>>>>>>>>>> Go, it might be >>>>>>>>>>>>>>>>> a good idea to change the quickstarts to use these instead of >>>>>>>>>>>>>>>>> the word >>>>>>>>>>>>>>>>> count. There is already a dedicated word count walkthrough so >>>>>>>>>>>>>>>>> I think that >>>>>>>>>>>>>>>>> is already covered. >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > If we all agree on the repo names, who can help us >>>>>>>>>>>>>>>>> create them? >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > On Thu, Jan 27, 2022 at 12:58 PM Robert Bradshaw < >>>>>>>>>>>>>>>>> rober...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> On Tue, Jan 18, 2022 at 6:17 AM Kenneth Knowles < >>>>>>>>>>>>>>>>> k...@apache.org> wrote: >>>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>>> >> > Agree with Luke here. "Just git clone and go" is a >>>>>>>>>>>>>>>>> big part of it. >>>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>>> >> > But also the answer to "I simply don't know what one >>>>>>>>>>>>>>>>> would put in a Python repo than, other than a bare setup.py >>>>>>>>>>>>>>>>> that lists a >>>>>>>>>>>>>>>>> dependency on apache_beam" is answered by David's initial >>>>>>>>>>>>>>>>> email and his >>>>>>>>>>>>>>>>> repo, namely: >>>>>>>>>>>>>>>>> >> > >>>>>>>>>>>>>>>>> >> > - GitHub Actions configuration >>>>>>>>>>>>>>>>> >> > - README.md >>>>>>>>>>>>>>>>> >> > - example that already runs >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> OK, fair enough. >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> > - LICENSE (notably you've got it as MIT but to be >>>>>>>>>>>>>>>>> part of Apache software it needs to be ASL2) >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> On the topic of licence, it's a bit tricky because one >>>>>>>>>>>>>>>>> doesn't want to >>>>>>>>>>>>>>>>> >> bind the users of such a template as being a derivative >>>>>>>>>>>>>>>>> work of a >>>>>>>>>>>>>>>>> >> too-restrictive licence. The licence of the template >>>>>>>>>>>>>>>>> itself should >>>>>>>>>>>>>>>>> >> generally be very permissive. >>>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>>> >> > On Fri, Jan 14, 2022 at 2:34 PM Luke Cwik < >>>>>>>>>>>>>>>>> lc...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>>> >> >> I think for consistency it makes sense to users to >>>>>>>>>>>>>>>>> be told to checkout this git repo for the language of your >>>>>>>>>>>>>>>>> choice and run. >>>>>>>>>>>>>>>>> Some repos will have more/less than others when it comes to >>>>>>>>>>>>>>>>> setup necessary. >>>>>>>>>>>>>>>>> >> >> >>>>>>>>>>>>>>>>> >> >> On Fri, Jan 14, 2022 at 2:26 PM Robert Bradshaw < >>>>>>>>>>>>>>>>> rober...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>>>>> >> >>> +1 for doing this for Java, as setting up a project >>>>>>>>>>>>>>>>> there is quite >>>>>>>>>>>>>>>>> >> >>> complicated. I simply don't know what one would put >>>>>>>>>>>>>>>>> in a Python repo >>>>>>>>>>>>>>>>> >> >>> than, other than a bare setup.py that lists a >>>>>>>>>>>>>>>>> dependency on >>>>>>>>>>>>>>>>> >> >>> apache_beam. We don't have recommendations on file >>>>>>>>>>>>>>>>> layout, etc. more >>>>>>>>>>>>>>>>> >> >>> than that (though there's plenty of generic advice >>>>>>>>>>>>>>>>> to be found out >>>>>>>>>>>>>>>>> >> >>> there on the topic). I have a hunch go is similar, >>>>>>>>>>>>>>>>> and javascript >>>>>>>>>>>>>>>>> >> >>> would be as well (npm install apache-beam and your >>>>>>>>>>>>>>>>> package.json file >>>>>>>>>>>>>>>>> >> >>> gets updated). >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>>>>> >> >>> On Fri, Jan 14, 2022 at 2:17 PM Luke Cwik < >>>>>>>>>>>>>>>>> lc...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> > >>>>>>>>>>>>>>>>> >> >>> > There are several examples already within the >>>>>>>>>>>>>>>>> Beam repo found in: >>>>>>>>>>>>>>>>> >> >>> > >>>>>>>>>>>>>>>>> https://github.com/apache/beam/tree/master/examples >>>>>>>>>>>>>>>>> >> >>> > >>>>>>>>>>>>>>>>> https://github.com/apache/beam/tree/master/sdks/go/examples >>>>>>>>>>>>>>>>> >> >>> > >>>>>>>>>>>>>>>>> https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples >>>>>>>>>>>>>>>>> >> >>> > >>>>>>>>>>>>>>>>> >> >>> > >>>>>>>>>>>>>>>>> >> >>> > On Fri, Jan 14, 2022 at 11:07 AM Sachin Agarwal < >>>>>>>>>>>>>>>>> sachi...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >> >>>>>>>>>>>>>>>>> >> >>> >> I'd love to do something other than Wordcount >>>>>>>>>>>>>>>>> just for novelty/freshness but agreed with the suggestion >>>>>>>>>>>>>>>>> that having an >>>>>>>>>>>>>>>>> example in each quickstart would be ideal. >>>>>>>>>>>>>>>>> >> >>> >> >>>>>>>>>>>>>>>>> >> >>> >> On Fri, Jan 14, 2022 at 11:06 AM David >>>>>>>>>>>>>>>>> Huntsperger <dhuntsper...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>> >>>>>>>>>>>>>>>>> >> >>> >>> + 1 to a separate repo for each language. >>>>>>>>>>>>>>>>> >> >>> >>> >>>>>>>>>>>>>>>>> >> >>> >>> Would it make sense to include the Wordcount >>>>>>>>>>>>>>>>> example in each repo? I know that makes the repos less >>>>>>>>>>>>>>>>> minimal, but we >>>>>>>>>>>>>>>>> could rewrite the quickstarts around these repos instead of >>>>>>>>>>>>>>>>> the current >>>>>>>>>>>>>>>>> Wordcount examples. Or maybe we don't need to use the >>>>>>>>>>>>>>>>> Wordcount example in >>>>>>>>>>>>>>>>> the quickstarts... >>>>>>>>>>>>>>>>> >> >>> >>> >>>>>>>>>>>>>>>>> >> >>> >>> On Wed, Jan 12, 2022 at 1:54 PM David Cavazos < >>>>>>>>>>>>>>>>> dcava...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>> >>>>>>>>>>>>>>>>> >> >>> >>>> I agree with dropping the archetypes. Less >>>>>>>>>>>>>>>>> maintenance is preferable, and the github repos are more >>>>>>>>>>>>>>>>> flexible and >>>>>>>>>>>>>>>>> maintainable. >>>>>>>>>>>>>>>>> >> >>> >>>> >>>>>>>>>>>>>>>>> >> >>> >>>> How about we create: >>>>>>>>>>>>>>>>> >> >>> >>>> >>>>>>>>>>>>>>>>> >> >>> >>>> apache/beam-starter-java >>>>>>>>>>>>>>>>> >> >>> >>>> apache/beam-starter-python >>>>>>>>>>>>>>>>> >> >>> >>>> apache/beam-starter-go >>>>>>>>>>>>>>>>> >> >>> >>>> >>>>>>>>>>>>>>>>> >> >>> >>>> During our OKR planning, +Keith Malvetti would >>>>>>>>>>>>>>>>> prefer having repos for all languages. It makes sense for >>>>>>>>>>>>>>>>> consistency as >>>>>>>>>>>>>>>>> well. >>>>>>>>>>>>>>>>> >> >>> >>>> >>>>>>>>>>>>>>>>> >> >>> >>>> On Mon, Jan 10, 2022 at 5:14 PM Luke Cwik < >>>>>>>>>>>>>>>>> lc...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>> As long as we have tags so that people can >>>>>>>>>>>>>>>>> pull out a specific version of the examples that coincides >>>>>>>>>>>>>>>>> with a specific >>>>>>>>>>>>>>>>> SDK version then we could drop the archetypes. >>>>>>>>>>>>>>>>> >> >>> >>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>> On Mon, Jan 10, 2022 at 4:09 PM Brian Hulette >>>>>>>>>>>>>>>>> <bhule...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>> > Being such minimal examples, I don't >>>>>>>>>>>>>>>>> expect them to break commonly, but I think it would be good >>>>>>>>>>>>>>>>> to make sure >>>>>>>>>>>>>>>>> tests aren't failing when a release is published. >>>>>>>>>>>>>>>>> >> >>> >>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>> Yeah it would be very unfortunate if we >>>>>>>>>>>>>>>>> discovered a breakage after the release. Agree we should >>>>>>>>>>>>>>>>> verify RCs >>>>>>>>>>>>>>>>> (document as part of the release process), or even better, >>>>>>>>>>>>>>>>> add automation >>>>>>>>>>>>>>>>> to verify the repo against snapshots. The automation could be >>>>>>>>>>>>>>>>> nice to have >>>>>>>>>>>>>>>>> anyway since it provides an example for users to follow if >>>>>>>>>>>>>>>>> they want to >>>>>>>>>>>>>>>>> test against snapshots and report issues to us sooner. >>>>>>>>>>>>>>>>> >> >>> >>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>> If we move forward with this can we drop the >>>>>>>>>>>>>>>>> archetype? >>>>>>>>>>>>>>>>> >> >>> >>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>> On Fri, Jan 7, 2022 at 3:54 PM Luke Cwik < >>>>>>>>>>>>>>>>> lc...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>> Sounds reasonable. >>>>>>>>>>>>>>>>> >> >>> >>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>> On Wed, Jan 5, 2022 at 12:47 PM David >>>>>>>>>>>>>>>>> Cavazos <dcava...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> I personally like the idea of a separate >>>>>>>>>>>>>>>>> repo since we can see how a true minimal project looks like. >>>>>>>>>>>>>>>>> Having it in >>>>>>>>>>>>>>>>> the main repo would inherit build file configurations and >>>>>>>>>>>>>>>>> other settings >>>>>>>>>>>>>>>>> that would be different from a clean project, so it could be >>>>>>>>>>>>>>>>> non-trivial to >>>>>>>>>>>>>>>>> adapt. Also as its own repo, it's easier to clone and modify, >>>>>>>>>>>>>>>>> or create an >>>>>>>>>>>>>>>>> instance of the template. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> Dependabot can take care of updating the >>>>>>>>>>>>>>>>> Beam version and other dependencies automatically. Testing is >>>>>>>>>>>>>>>>> already set >>>>>>>>>>>>>>>>> up via GitHub actions for every pull request, so it would >>>>>>>>>>>>>>>>> automatically be >>>>>>>>>>>>>>>>> tested as soon as there is a new dependency version available. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> Being such minimal examples, I don't >>>>>>>>>>>>>>>>> expect them to break commonly, but I think it would be good >>>>>>>>>>>>>>>>> to make sure >>>>>>>>>>>>>>>>> tests aren't failing when a release is published. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> I'm okay with having one repo per >>>>>>>>>>>>>>>>> language, and having all the build systems we want to support >>>>>>>>>>>>>>>>> for them. As >>>>>>>>>>>>>>>>> long as we document which files are for which build system. >>>>>>>>>>>>>>>>> That way there >>>>>>>>>>>>>>>>> are less repos to maintain. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>> On Mon, Dec 13, 2021 at 9:25 AM Luke Cwik < >>>>>>>>>>>>>>>>> lc...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> The github repo is definitely more >>>>>>>>>>>>>>>>> flexible then the archetypes but the archetypes have a few >>>>>>>>>>>>>>>>> conveniences >>>>>>>>>>>>>>>>> since they are integrated with apache/beam repo. For example, >>>>>>>>>>>>>>>>> updates/testing are done at the same time a corresponding >>>>>>>>>>>>>>>>> change to the >>>>>>>>>>>>>>>>> main repo is done (like library version updates), they are >>>>>>>>>>>>>>>>> released when >>>>>>>>>>>>>>>>> the SDK is released. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> Should these be part of the main repo, or >>>>>>>>>>>>>>>>> a single starter repo containing all the starters or one per >>>>>>>>>>>>>>>>> language or >>>>>>>>>>>>>>>>> one per build system? >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> When should updates to the starter happen? >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> How as a community do we get them to >>>>>>>>>>>>>>>>> happen (e.g. release manager owns it)? >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>> On Sun, Dec 12, 2021 at 4:06 PM David >>>>>>>>>>>>>>>>> Cavazos <dcava...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>> We could do the Maven archetype, but >>>>>>>>>>>>>>>>> that wouldn't work very well for Gradle and SBT users. I >>>>>>>>>>>>>>>>> think a GitHub >>>>>>>>>>>>>>>>> template might be the more flexible option, and we could have >>>>>>>>>>>>>>>>> something >>>>>>>>>>>>>>>>> similar for other languages as well. Having said that, we >>>>>>>>>>>>>>>>> could still >>>>>>>>>>>>>>>>> create a Maven archetype. If someone is familiar with that >>>>>>>>>>>>>>>>> process, please >>>>>>>>>>>>>>>>> let me know since I'm not too familiar with Maven and its >>>>>>>>>>>>>>>>> ecosystem. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>> @Ahmet Altay I think right now we only >>>>>>>>>>>>>>>>> need to pin down the name of the repo, create it, and move >>>>>>>>>>>>>>>>> the code there. >>>>>>>>>>>>>>>>> I was thinking either `apache/beam-java-template` or >>>>>>>>>>>>>>>>> `apache/beam-java-starter`. What do you think? >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>> What would be the next steps on creating >>>>>>>>>>>>>>>>> the repo? >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>> On Thu, Dec 9, 2021 at 11:09 AM Ahmet >>>>>>>>>>>>>>>>> Altay <al...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>> This is great David. Was there any >>>>>>>>>>>>>>>>> progress on this? Do you need help? >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>> On Wed, Dec 1, 2021 at 3:54 PM Brian >>>>>>>>>>>>>>>>> Hulette <bhule...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> This is cool, thanks! >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> We do have a template in apache/beam >>>>>>>>>>>>>>>>> already, built with Maven Archetype [1]. It's what powers the >>>>>>>>>>>>>>>>> Java >>>>>>>>>>>>>>>>> quickstart [2]. Could we de-dupe these (e.g. reference the >>>>>>>>>>>>>>>>> GitHub template >>>>>>>>>>>>>>>>> in the quickstart, or co-locate the archetype with the GitHub >>>>>>>>>>>>>>>>> template)? >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> As far as creating an Apache repo, >>>>>>>>>>>>>>>>> would we put this somewhere like apache/beam-java-template? I >>>>>>>>>>>>>>>>> think apache >>>>>>>>>>>>>>>>> repositories like beam-* are allowed. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> Brian >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>> https://maven.apache.org/archetype/index.html >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>> https://beam.apache.org/get-started/quickstart-java/#get-the-example-code >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>> On Wed, Dec 1, 2021 at 11:30 AM David >>>>>>>>>>>>>>>>> Cavazos <dcava...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> +Ahmet Altay >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> +Valentyn Tymofieiev >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> +Kenneth Knowles >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> Please feel free to include anyone >>>>>>>>>>>>>>>>> else! >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>> On Mon, Oct 25, 2021 at 11:31 AM >>>>>>>>>>>>>>>>> David Cavazos <dcava...@google.com> wrote: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Hi Beam community! >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> To make it easier to create a new >>>>>>>>>>>>>>>>> Beam Java project, I've been working on a GitHub template >>>>>>>>>>>>>>>>> containing a >>>>>>>>>>>>>>>>> minimal Beam Java pipeline for people to start with. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Link to the GitHub template: >>>>>>>>>>>>>>>>> https://github.com/davidcavazos/beam-java >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> So far, here's what the template >>>>>>>>>>>>>>>>> contains: >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Minimal "Hello World" Beam pipeline >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Minimal test file >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Build files for Gradle, sbt, and >>>>>>>>>>>>>>>>> Maven (Direct runner) >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Continuous integration via GitHub >>>>>>>>>>>>>>>>> actions (around 1-2 minutes to run) >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> README with instructions on how to >>>>>>>>>>>>>>>>> build, run, test, and add other runners >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> It's easy to create a new GitHub >>>>>>>>>>>>>>>>> repo from a template. >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Next steps >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Some reviewers to make sure everyone >>>>>>>>>>>>>>>>> is happy with it 🙂 >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Right now it lives in my personal >>>>>>>>>>>>>>>>> GitHub account, so we need to create an Apache repo to host it >>>>>>>>>>>>>>>>> >> >>> >>>>>>>>>>>>>> Update/create docs with instructions >>>>>>>>>>>>>>>>> on how to create a new Beam Java pipeline >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>
