Agree with Luke here. "Just git clone and go" is a big part of it.
But also the answer to "I simply don't know what one would put in a Python repo than, other than a bare setup.py that lists a dependency on apache_beam" is answered by David's initial email and his repo, namely: - GitHub Actions configuration - README.md - example that already runs - LICENSE (notably you've got it as MIT but to be part of Apache software it needs to be ASL2) Kenn On Fri, Jan 14, 2022 at 2:34 PM Luke Cwik <lc...@google.com> wrote: > I think for consistency it makes sense to users to be told to checkout > this git repo for the language of your choice and run. Some repos will have > more/less than others when it comes to setup necessary. > > On Fri, Jan 14, 2022 at 2:26 PM Robert Bradshaw <rober...@google.com> > wrote: > >> +1 for doing this for Java, as setting up a project there is quite >> complicated. I simply don't know what one would put in a Python repo >> than, other than a bare setup.py that lists a dependency on >> apache_beam. We don't have recommendations on file layout, etc. more >> than that (though there's plenty of generic advice to be found out >> there on the topic). I have a hunch go is similar, and javascript >> would be as well (npm install apache-beam and your package.json file >> gets updated). >> >> On Fri, Jan 14, 2022 at 2:17 PM Luke Cwik <lc...@google.com> wrote: >> > >> > There are several examples already within the Beam repo found in: >> > https://github.com/apache/beam/tree/master/examples >> > https://github.com/apache/beam/tree/master/sdks/go/examples >> > >> https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples >> > >> > >> > On Fri, Jan 14, 2022 at 11:07 AM Sachin Agarwal <sachi...@google.com> >> wrote: >> >> >> >> I'd love to do something other than Wordcount just for >> novelty/freshness but agreed with the suggestion that having an example in >> each quickstart would be ideal. >> >> >> >> On Fri, Jan 14, 2022 at 11:06 AM David Huntsperger < >> dhuntsper...@google.com> wrote: >> >>> >> >>> + 1 to a separate repo for each language. >> >>> >> >>> Would it make sense to include the Wordcount example in each repo? I >> know that makes the repos less minimal, but we could rewrite the >> quickstarts around these repos instead of the current Wordcount examples. >> Or maybe we don't need to use the Wordcount example in the quickstarts... >> >>> >> >>> On Wed, Jan 12, 2022 at 1:54 PM David Cavazos <dcava...@google.com> >> wrote: >> >>>> >> >>>> I agree with dropping the archetypes. Less maintenance is >> preferable, and the github repos are more flexible and maintainable. >> >>>> >> >>>> How about we create: >> >>>> >> >>>> apache/beam-starter-java >> >>>> apache/beam-starter-python >> >>>> apache/beam-starter-go >> >>>> >> >>>> During our OKR planning, +Keith Malvetti would prefer having repos >> for all languages. It makes sense for consistency as well. >> >>>> >> >>>> On Mon, Jan 10, 2022 at 5:14 PM Luke Cwik <lc...@google.com> wrote: >> >>>>> >> >>>>> As long as we have tags so that people can pull out a specific >> version of the examples that coincides with a specific SDK version then we >> could drop the archetypes. >> >>>>> >> >>>>> On Mon, Jan 10, 2022 at 4:09 PM Brian Hulette <bhule...@google.com> >> wrote: >> >>>>>> >> >>>>>> > Being such minimal examples, I don't expect them to break >> commonly, but I think it would be good to make sure tests aren't failing >> when a release is published. >> >>>>>> >> >>>>>> Yeah it would be very unfortunate if we discovered a breakage >> after the release. Agree we should verify RCs (document as part of the >> release process), or even better, add automation to verify the repo against >> snapshots. The automation could be nice to have anyway since it provides an >> example for users to follow if they want to test against snapshots and >> report issues to us sooner. >> >>>>>> >> >>>>>> >> >>>>>> If we move forward with this can we drop the archetype? >> >>>>>> >> >>>>>> On Fri, Jan 7, 2022 at 3:54 PM Luke Cwik <lc...@google.com> wrote: >> >>>>>>> >> >>>>>>> Sounds reasonable. >> >>>>>>> >> >>>>>>> On Wed, Jan 5, 2022 at 12:47 PM David Cavazos < >> dcava...@google.com> wrote: >> >>>>>>>> >> >>>>>>>> I personally like the idea of a separate repo since we can see >> how a true minimal project looks like. Having it in the main repo would >> inherit build file configurations and other settings that would be >> different from a clean project, so it could be non-trivial to adapt. Also >> as its own repo, it's easier to clone and modify, or create an instance of >> the template. >> >>>>>>>> >> >>>>>>>> Dependabot can take care of updating the Beam version and other >> dependencies automatically. Testing is already set up via GitHub actions >> for every pull request, so it would automatically be tested as soon as >> there is a new dependency version available. >> >>>>>>>> >> >>>>>>>> Being such minimal examples, I don't expect them to break >> commonly, but I think it would be good to make sure tests aren't failing >> when a release is published. >> >>>>>>>> >> >>>>>>>> I'm okay with having one repo per language, and having all the >> build systems we want to support for them. As long as we document which >> files are for which build system. That way there are less repos to maintain. >> >>>>>>>> >> >>>>>>>> On Mon, Dec 13, 2021 at 9:25 AM Luke Cwik <lc...@google.com> >> wrote: >> >>>>>>>>> >> >>>>>>>>> The github repo is definitely more flexible then the archetypes >> but the archetypes have a few conveniences since they are integrated with >> apache/beam repo. For example, updates/testing are done at the same time a >> corresponding change to the main repo is done (like library version >> updates), they are released when the SDK is released. >> >>>>>>>>> >> >>>>>>>>> Should these be part of the main repo, or a single starter repo >> containing all the starters or one per language or one per build system? >> >>>>>>>>> >> >>>>>>>>> When should updates to the starter happen? >> >>>>>>>>> How as a community do we get them to happen (e.g. release >> manager owns it)? >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> On Sun, Dec 12, 2021 at 4:06 PM David Cavazos < >> dcava...@google.com> wrote: >> >>>>>>>>>> >> >>>>>>>>>> We could do the Maven archetype, but that wouldn't work very >> well for Gradle and SBT users. I think a GitHub template might be the more >> flexible option, and we could have something similar for other languages as >> well. Having said that, we could still create a Maven archetype. If someone >> is familiar with that process, please let me know since I'm not too >> familiar with Maven and its ecosystem. >> >>>>>>>>>> >> >>>>>>>>>> @Ahmet Altay I think right now we only need to pin down the >> name of the repo, create it, and move the code there. I was thinking either >> `apache/beam-java-template` or `apache/beam-java-starter`. What do you >> think? >> >>>>>>>>>> >> >>>>>>>>>> What would be the next steps on creating the repo? >> >>>>>>>>>> >> >>>>>>>>>> On Thu, Dec 9, 2021 at 11:09 AM Ahmet Altay <al...@google.com> >> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>> This is great David. Was there any progress on this? Do you >> need help? >> >>>>>>>>>>> >> >>>>>>>>>>> On Wed, Dec 1, 2021 at 3:54 PM Brian Hulette < >> bhule...@google.com> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>> This is cool, thanks! >> >>>>>>>>>>>> >> >>>>>>>>>>>> We do have a template in apache/beam already, built with >> Maven Archetype [1]. It's what powers the Java quickstart [2]. Could we >> de-dupe these (e.g. reference the GitHub template in the quickstart, or >> co-locate the archetype with the GitHub template)? >> >>>>>>>>>>>> >> >>>>>>>>>>>> As far as creating an Apache repo, would we put this >> somewhere like apache/beam-java-template? I think apache repositories like >> beam-* are allowed. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Brian >> >>>>>>>>>>>> >> >>>>>>>>>>>> [1] https://maven.apache.org/archetype/index.html >> >>>>>>>>>>>> [2] >> https://beam.apache.org/get-started/quickstart-java/#get-the-example-code >> >>>>>>>>>>>> >> >>>>>>>>>>>> On Wed, Dec 1, 2021 at 11:30 AM David Cavazos < >> dcava...@google.com> wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> +Ahmet Altay >> >>>>>>>>>>>>> +Valentyn Tymofieiev >> >>>>>>>>>>>>> +Kenneth Knowles >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Please feel free to include anyone else! >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On Mon, Oct 25, 2021 at 11:31 AM David Cavazos < >> dcava...@google.com> wrote: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Hi Beam community! >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> To make it easier to create a new Beam Java project, I've >> been working on a GitHub template containing a minimal Beam Java pipeline >> for people to start with. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Link to the GitHub template: >> https://github.com/davidcavazos/beam-java >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> So far, here's what the template contains: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Minimal "Hello World" Beam pipeline >> >>>>>>>>>>>>>> Minimal test file >> >>>>>>>>>>>>>> Build files for Gradle, sbt, and Maven (Direct runner) >> >>>>>>>>>>>>>> Continuous integration via GitHub actions (around 1-2 >> minutes to run) >> >>>>>>>>>>>>>> README with instructions on how to build, run, test, and >> add other runners >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> It's easy to create a new GitHub repo from a template. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Next steps >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Some reviewers to make sure everyone is happy with it 🙂 >> >>>>>>>>>>>>>> Right now it lives in my personal GitHub account, so we >> need to create an Apache repo to host it >> >>>>>>>>>>>>>> Update/create docs with instructions on how to create a >> new Beam Java pipeline >> >
