I'm convinced on all points. My main motivation was to keep it simple. But of course we should keep it simple for users, not us :-)
I can take on the task of asking about MIT license and requesting the repos be created. Not sure if it needs my level of privileges but I'm happy to do it anyhow. Kenn On Wed, Feb 2, 2022 at 10:30 AM Robert Bradshaw <[email protected]> wrote: > On Wed, Feb 2, 2022 at 10:12 AM David Cavazos <[email protected]> wrote: > > > > MIT is much more permissive, but I also don't have any problems changing > it to Apache license. In any case, how about we create the following repos? > > For these starter projects, we don't want to encumber any users of > these templates with any particular licensing requirements (right?) > and we don't even care about attribution. We want these to be pretty > much as close to public domain as possible. That's not what the Apache > licence does. (If it's even relevant, a good argument could likely be > made for de minis or fair use, but I think it's best to be explicit > about this. Perhaps this'd be a good question for apache legal? > > > apache/beam-starter-java > > apache/beam-starter-python > > apache/beam-starter-go > > apache/beam-starter-kotlin > > apache/beam-starter-scala > > > > We'll start by populating the Java one which is the most pressing one > and the one that is ready, but the rest should be simpler. > > > > +David Huntsperger, tldr; these are minimal starter projects for every > language. Once we have Java, Python and Go, it might be a good idea to > change the quickstarts to use these instead of the word count. There is > already a dedicated word count walkthrough so I think that is already > covered. > > > > If we all agree on the repo names, who can help us create them? > > > > On Thu, Jan 27, 2022 at 12:58 PM Robert Bradshaw <[email protected]> > wrote: > >> > >> On Tue, Jan 18, 2022 at 6:17 AM Kenneth Knowles <[email protected]> > wrote: > >> > > >> > Agree with Luke here. "Just git clone and go" is a big part of it. > >> > > >> > But also the answer to "I simply don't know what one would put in a > Python repo than, other than a bare setup.py that lists a dependency on > apache_beam" is answered by David's initial email and his repo, namely: > >> > > >> > - GitHub Actions configuration > >> > - README.md > >> > - example that already runs > >> > >> OK, fair enough. > >> > >> > - LICENSE (notably you've got it as MIT but to be part of Apache > software it needs to be ASL2) > >> > >> On the topic of licence, it's a bit tricky because one doesn't want to > >> bind the users of such a template as being a derivative work of a > >> too-restrictive licence. The licence of the template itself should > >> generally be very permissive. > >> > >> > On Fri, Jan 14, 2022 at 2:34 PM Luke Cwik <[email protected]> wrote: > >> >> > >> >> I think for consistency it makes sense to users to be told to > checkout this git repo for the language of your choice and run. Some repos > will have more/less than others when it comes to setup necessary. > >> >> > >> >> On Fri, Jan 14, 2022 at 2:26 PM Robert Bradshaw <[email protected]> > wrote: > >> >>> > >> >>> +1 for doing this for Java, as setting up a project there is quite > >> >>> complicated. I simply don't know what one would put in a Python repo > >> >>> than, other than a bare setup.py that lists a dependency on > >> >>> apache_beam. We don't have recommendations on file layout, etc. more > >> >>> than that (though there's plenty of generic advice to be found out > >> >>> there on the topic). I have a hunch go is similar, and javascript > >> >>> would be as well (npm install apache-beam and your package.json file > >> >>> gets updated). > >> >>> > >> >>> On Fri, Jan 14, 2022 at 2:17 PM Luke Cwik <[email protected]> wrote: > >> >>> > > >> >>> > There are several examples already within the Beam repo found in: > >> >>> > https://github.com/apache/beam/tree/master/examples > >> >>> > https://github.com/apache/beam/tree/master/sdks/go/examples > >> >>> > > https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples > >> >>> > > >> >>> > > >> >>> > On Fri, Jan 14, 2022 at 11:07 AM Sachin Agarwal < > [email protected]> wrote: > >> >>> >> > >> >>> >> I'd love to do something other than Wordcount just for > novelty/freshness but agreed with the suggestion that having an example in > each quickstart would be ideal. > >> >>> >> > >> >>> >> On Fri, Jan 14, 2022 at 11:06 AM David Huntsperger < > [email protected]> wrote: > >> >>> >>> > >> >>> >>> + 1 to a separate repo for each language. > >> >>> >>> > >> >>> >>> Would it make sense to include the Wordcount example in each > repo? I know that makes the repos less minimal, but we could rewrite the > quickstarts around these repos instead of the current Wordcount examples. > Or maybe we don't need to use the Wordcount example in the quickstarts... > >> >>> >>> > >> >>> >>> On Wed, Jan 12, 2022 at 1:54 PM David Cavazos < > [email protected]> wrote: > >> >>> >>>> > >> >>> >>>> I agree with dropping the archetypes. Less maintenance is > preferable, and the github repos are more flexible and maintainable. > >> >>> >>>> > >> >>> >>>> How about we create: > >> >>> >>>> > >> >>> >>>> apache/beam-starter-java > >> >>> >>>> apache/beam-starter-python > >> >>> >>>> apache/beam-starter-go > >> >>> >>>> > >> >>> >>>> During our OKR planning, +Keith Malvetti would prefer having > repos for all languages. It makes sense for consistency as well. > >> >>> >>>> > >> >>> >>>> On Mon, Jan 10, 2022 at 5:14 PM Luke Cwik <[email protected]> > wrote: > >> >>> >>>>> > >> >>> >>>>> As long as we have tags so that people can pull out a > specific version of the examples that coincides with a specific SDK version > then we could drop the archetypes. > >> >>> >>>>> > >> >>> >>>>> On Mon, Jan 10, 2022 at 4:09 PM Brian Hulette < > [email protected]> wrote: > >> >>> >>>>>> > >> >>> >>>>>> > Being such minimal examples, I don't expect them to break > commonly, but I think it would be good to make sure tests aren't failing > when a release is published. > >> >>> >>>>>> > >> >>> >>>>>> Yeah it would be very unfortunate if we discovered a > breakage after the release. Agree we should verify RCs (document as part of > the release process), or even better, add automation to verify the repo > against snapshots. The automation could be nice to have anyway since it > provides an example for users to follow if they want to test against > snapshots and report issues to us sooner. > >> >>> >>>>>> > >> >>> >>>>>> > >> >>> >>>>>> If we move forward with this can we drop the archetype? > >> >>> >>>>>> > >> >>> >>>>>> On Fri, Jan 7, 2022 at 3:54 PM Luke Cwik <[email protected]> > wrote: > >> >>> >>>>>>> > >> >>> >>>>>>> Sounds reasonable. > >> >>> >>>>>>> > >> >>> >>>>>>> On Wed, Jan 5, 2022 at 12:47 PM David Cavazos < > [email protected]> wrote: > >> >>> >>>>>>>> > >> >>> >>>>>>>> I personally like the idea of a separate repo since we can > see how a true minimal project looks like. Having it in the main repo would > inherit build file configurations and other settings that would be > different from a clean project, so it could be non-trivial to adapt. Also > as its own repo, it's easier to clone and modify, or create an instance of > the template. > >> >>> >>>>>>>> > >> >>> >>>>>>>> Dependabot can take care of updating the Beam version and > other dependencies automatically. Testing is already set up via GitHub > actions for every pull request, so it would automatically be tested as soon > as there is a new dependency version available. > >> >>> >>>>>>>> > >> >>> >>>>>>>> Being such minimal examples, I don't expect them to break > commonly, but I think it would be good to make sure tests aren't failing > when a release is published. > >> >>> >>>>>>>> > >> >>> >>>>>>>> I'm okay with having one repo per language, and having all > the build systems we want to support for them. As long as we document which > files are for which build system. That way there are less repos to maintain. > >> >>> >>>>>>>> > >> >>> >>>>>>>> On Mon, Dec 13, 2021 at 9:25 AM Luke Cwik < > [email protected]> wrote: > >> >>> >>>>>>>>> > >> >>> >>>>>>>>> The github repo is definitely more flexible then the > archetypes but the archetypes have a few conveniences since they are > integrated with apache/beam repo. For example, updates/testing are done at > the same time a corresponding change to the main repo is done (like library > version updates), they are released when the SDK is released. > >> >>> >>>>>>>>> > >> >>> >>>>>>>>> Should these be part of the main repo, or a single > starter repo containing all the starters or one per language or one per > build system? > >> >>> >>>>>>>>> > >> >>> >>>>>>>>> When should updates to the starter happen? > >> >>> >>>>>>>>> How as a community do we get them to happen (e.g. release > manager owns it)? > >> >>> >>>>>>>>> > >> >>> >>>>>>>>> > >> >>> >>>>>>>>> On Sun, Dec 12, 2021 at 4:06 PM David Cavazos < > [email protected]> wrote: > >> >>> >>>>>>>>>> > >> >>> >>>>>>>>>> We could do the Maven archetype, but that wouldn't work > very well for Gradle and SBT users. I think a GitHub template might be the > more flexible option, and we could have something similar for other > languages as well. Having said that, we could still create a Maven > archetype. If someone is familiar with that process, please let me know > since I'm not too familiar with Maven and its ecosystem. > >> >>> >>>>>>>>>> > >> >>> >>>>>>>>>> @Ahmet Altay I think right now we only need to pin down > the name of the repo, create it, and move the code there. I was thinking > either `apache/beam-java-template` or `apache/beam-java-starter`. What do > you think? > >> >>> >>>>>>>>>> > >> >>> >>>>>>>>>> What would be the next steps on creating the repo? > >> >>> >>>>>>>>>> > >> >>> >>>>>>>>>> On Thu, Dec 9, 2021 at 11:09 AM Ahmet Altay < > [email protected]> wrote: > >> >>> >>>>>>>>>>> > >> >>> >>>>>>>>>>> This is great David. Was there any progress on this? Do > you need help? > >> >>> >>>>>>>>>>> > >> >>> >>>>>>>>>>> On Wed, Dec 1, 2021 at 3:54 PM Brian Hulette < > [email protected]> wrote: > >> >>> >>>>>>>>>>>> > >> >>> >>>>>>>>>>>> This is cool, thanks! > >> >>> >>>>>>>>>>>> > >> >>> >>>>>>>>>>>> We do have a template in apache/beam already, built > with Maven Archetype [1]. It's what powers the Java quickstart [2]. Could > we de-dupe these (e.g. reference the GitHub template in the quickstart, or > co-locate the archetype with the GitHub template)? > >> >>> >>>>>>>>>>>> > >> >>> >>>>>>>>>>>> As far as creating an Apache repo, would we put this > somewhere like apache/beam-java-template? I think apache repositories like > beam-* are allowed. > >> >>> >>>>>>>>>>>> > >> >>> >>>>>>>>>>>> Brian > >> >>> >>>>>>>>>>>> > >> >>> >>>>>>>>>>>> [1] https://maven.apache.org/archetype/index.html > >> >>> >>>>>>>>>>>> [2] > https://beam.apache.org/get-started/quickstart-java/#get-the-example-code > >> >>> >>>>>>>>>>>> > >> >>> >>>>>>>>>>>> On Wed, Dec 1, 2021 at 11:30 AM David Cavazos < > [email protected]> wrote: > >> >>> >>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>> +Ahmet Altay > >> >>> >>>>>>>>>>>>> +Valentyn Tymofieiev > >> >>> >>>>>>>>>>>>> +Kenneth Knowles > >> >>> >>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>> Please feel free to include anyone else! > >> >>> >>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>> On Mon, Oct 25, 2021 at 11:31 AM David Cavazos < > [email protected]> wrote: > >> >>> >>>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>>> Hi Beam community! > >> >>> >>>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>>> To make it easier to create a new Beam Java project, > I've been working on a GitHub template containing a minimal Beam Java > pipeline for people to start with. > >> >>> >>>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>>> Link to the GitHub template: > https://github.com/davidcavazos/beam-java > >> >>> >>>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>>> So far, here's what the template contains: > >> >>> >>>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>>> Minimal "Hello World" Beam pipeline > >> >>> >>>>>>>>>>>>>> Minimal test file > >> >>> >>>>>>>>>>>>>> Build files for Gradle, sbt, and Maven (Direct > runner) > >> >>> >>>>>>>>>>>>>> Continuous integration via GitHub actions (around > 1-2 minutes to run) > >> >>> >>>>>>>>>>>>>> README with instructions on how to build, run, test, > and add other runners > >> >>> >>>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>>> It's easy to create a new GitHub repo from a > template. > >> >>> >>>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>>> Next steps > >> >>> >>>>>>>>>>>>>> > >> >>> >>>>>>>>>>>>>> Some reviewers to make sure everyone is happy with > it 🙂 > >> >>> >>>>>>>>>>>>>> Right now it lives in my personal GitHub account, so > we need to create an Apache repo to host it > >> >>> >>>>>>>>>>>>>> Update/create docs with instructions on how to > create a new Beam Java pipeline >
