Re: Improve container support

2019-11-19 Thread Robert Bradshaw
Good to know. I added an icon and link. Good pointer about trying to see what we can do to make these official. On Tue, Nov 19, 2019 at 11:16 AM Kyle Weaver wrote: > > We do link from https://beam.apache.org/documentation/runtime/environments/. > > On Tue, Nov 19, 2019 at 10:59 AM Robert

Re: Improve container support

2019-11-19 Thread Kyle Weaver
We do link from https://beam.apache.org/documentation/runtime/environments/. On Tue, Nov 19, 2019 at 10:59 AM Robert Bradshaw wrote: > We should probably add a link to these from our site as well, for > visibility. > > On Tue, Nov 19, 2019 at 10:56 AM Kyle Weaver wrote: > > > > +1 Thanks for

Re: Improve container support

2019-11-19 Thread Robert Bradshaw
We should probably add a link to these from our site as well, for visibility. On Tue, Nov 19, 2019 at 10:56 AM Kyle Weaver wrote: > > +1 Thanks for bringing that up Chad, I had the same problem locating the > docker images on Docker hub (searching "apachebeam" is the only way that > seems to

Re: Improve container support

2019-11-19 Thread Kyle Weaver
+1 Thanks for bringing that up Chad, I had the same problem locating the docker images on Docker hub (searching "apachebeam" is the only way that seems to work). Another little thing I noticed is that https://hub.docker.com/u/apachebeam is mostly empty. It'd be good to at least add the Beam logo

Re: Improve container support

2019-11-19 Thread Chad Dombrova
Hi all, I think it might be good to update the description of the beam docker images and add some descriptive tags, because searching for "apache beam" in docker hub does not turn up anything: https://hub.docker.com/search?q=apache%20beam=image. I clicked through 10 pages worth and couldn't find

Re: Improve container support

2019-09-06 Thread Hannah Jiang
Hi team I haven't received any objections, so will proceed with settings mentioned in a previous email. A reminder to PMC members, please let me know your docker hub id if you want to be an admin. Thanks, Hannah On Thu, Sep 5, 2019 at 5:02 PM Ankur Goenka wrote: > Please ignore the previous

Re: Improve container support

2019-09-05 Thread Ankur Goenka
Please ignore the previous email. I was looking at the older document in the mail thread. On Thu, Sep 5, 2019 at 4:58 PM Ankur Goenka wrote: > I think sdk in the name is obsolete as they are all under sdks name space. > > On Thu, Sep 5, 2019 at 3:26 PM Hannah Jiang > wrote: > >> Hi Team >> >>

Re: Improve container support

2019-09-05 Thread Ankur Goenka
I think sdk in the name is obsolete as they are all under sdks name space. On Thu, Sep 5, 2019 at 3:26 PM Hannah Jiang wrote: > Hi Team > > Thanks for all the comments about beam containers. > After considering various opinions and investigating gcr and docker hub, > we decided to push images

Re: Improve container support

2019-09-05 Thread Hannah Jiang
Hi Team Thanks for all the comments about beam containers. After considering various opinions and investigating gcr and docker hub, we decided to push images to docker hub. Each image will have two tags, {version}_rc and {version}. {version} tag will be added after the release candidate image is

Re: Improve container support

2019-09-04 Thread Thomas Weise
This will greatly simplify trying out portable runners: https://beam.apache.org/documentation/runners/flink/#executing-a-beam-pipeline-on-a-flink-cluster Can't wait for following to disappear from the instructions page: ./gradlew :sdks:python:container:docker On Wed, Sep 4, 2019 at 3:35 PM

Re: Improve container support

2019-09-04 Thread Thomas Weise
Awesome, thank you! On Wed, Sep 4, 2019 at 3:22 PM Hannah Jiang wrote: > Hi Thomas > > I created snapshot images from head as of around 2PM today. > You can pull images from gcr.io/apache-beam-testing/beam/sdks/snapshot. > > Thanks, > Hannah > > On Wed, Sep 4, 2019 at 1:41 PM Thomas Weise

Re: Improve container support

2019-09-04 Thread Hannah Jiang
Hi Thomas I created snapshot images from head as of around 2PM today. You can pull images from gcr.io/apache-beam-testing/beam/sdks/snapshot. Thanks, Hannah On Wed, Sep 4, 2019 at 1:41 PM Thomas Weise wrote: > Hi Hannah, > > Thank you, I know how to build the containers locally, but not how

Re: Improve container support

2019-09-04 Thread Thomas Weise
Hi Hannah, Thank you, I know how to build the containers locally, but not how to publish them! The cwiki says "Publishing images to gcr.io/beam requires permissions in apache-beam-testing project." Can I get access to the testing project (at least temporarily) and what would I need to setup to

Re: Improve container support

2019-09-04 Thread Hannah Jiang
Hi Thomas I haven't uploaded any snapshot images yet. Here is how you can create one from head. > cd [...]/beam/ # For Python > ./gradlew :sdks:python:container:py{version}:docker *where version is {2,35,36,37}* # For Java > ./gradlew -p sdks/java/container docker # For Go > ./gradlew -p

Re: Improve container support

2019-09-04 Thread Thomas Weise
I actually found something in [1], but it is 2.15 unfortunately. [1] https://console.cloud.google.com/gcr/images/apache-beam-testing/GLOBAL/beam/sdks/release/python2.7?gcrImageListsize=30 On Wed, Sep 4, 2019 at 10:35 AM Thomas Weise wrote: > Thanks for working on this. Do you happen to have

Re: Improve container support

2019-09-04 Thread Thomas Weise
Thanks for working on this. Do you happen to have publicly accessible snapshots published for your testing currently (even when the final location isn't sorted out)? I would like to use a 2.16 based Python SDK image for working on my downstream project, but could not find anything in

Re: Improve container support

2019-08-30 Thread Hannah Jiang
>>> I have a question. Do you know what would it take to create a new >>>>>>> gcp project similar to apache-beam-testing for purposes of distributing >>>>>>> gcr >>>>>>> packages? We can use the same billing account. >>>>&g

Re: Improve container support

2019-08-28 Thread Sam Bourne
I have a question. Do you know what would it take to create a new gcp >>>>>> project similar to apache-beam-testing for purposes of distributing gcr >>>>>> packages? We can use the same billing account. >>>>>> >>>>>> Hannah, Robert, depending o

Re: Improve container support

2019-08-28 Thread Ahmet Altay
now what would it take to create a new gcp >>>>>> project similar to apache-beam-testing for purposes of distributing gcr >>>>>> packages? We can use the same billing account. >>>>>> >>>>>> Hannah, Robert, depending on the complexity of creating

Re: Improve container support

2019-08-28 Thread Pablo Estrada
apache-beam-testing for purposes of distributing gcr >>>>> packages? We can use the same billing account. >>>>> >>>>> Hannah, Robert, depending on the complexity of creating another gcp >>>>> project we can go with that, or simply

Re: Improve container support

2019-08-28 Thread Lukasz Cwik
gt;>> project we can go with that, or simply create a new bintray account. Either >>>> way would give us a clean new project to publish artifacts. >>>> >>>> Ahmet >>>> >>>> -- Forwarded message - >>>&

Re: Improve container support

2019-08-28 Thread Gleb Kanterov
ve us a clean new project to publish artifacts. >>> >>> Ahmet >>> >>> -- Forwarded message - >>> From: Robert Bradshaw >>> Date: Tue, Aug 27, 2019 at 6:48 PM >>> Subject: Re: Improve container support >>

Re: Improve container support

2019-08-27 Thread Hannah Jiang
y create a new bintray account. Either >> way would give us a clean new project to publish artifacts. >> >> Ahmet >> >> ------ Forwarded message - >> From: Robert Bradshaw >> Date: Tue, Aug 27, 2019 at 6:48 PM >> Subject: Re: Improve c

Re: Improve container support

2019-08-27 Thread Ahmet Altay
On Tue, Aug 27, 2019 at 5:50 PM Robert Bradshaw wrote: > On Tue, Aug 27, 2019 at 3:35 PM Hannah Jiang > wrote: > > > > Hi team > > > > I am working on improving docker container support for Beam. We would > like to publish prebuilt containers for each release version and daily > snapshot.