Re: Python 3.7 docker container fails to build

2020-04-29 Thread Hannah Jiang
Max, I found a link from your PR and noticed below errors. This would be the true error. *07:57:03* >* Task :sdks:python:container:py37:docker**07:57:03* [91mERROR: Could not install packages due to an EnvironmentError: [Errno 28] No space left on device*07:57:03* *07:57:03* [0m*07:57:03* >* Task

Re: possible bug in AvroUtils

2020-04-29 Thread Reuven Lax
I've copied this failing test into my client, and it passes for me. I can't reproduce the failure. On Wed, Apr 29, 2020 at 6:34 PM Luke Cwik wrote: > +dev +Brian Hulette +Reuven > Lax > > On Wed, Apr 29, 2020 at 4:21 AM Paolo Tomeo wrote: > >> Hi all, >> >> I think the method AvroUtils.toBea

Re: contributor permission for Beam Jira tickets

2020-04-29 Thread Darshan Jani
Thanks Luke. Thanks for pointing that out. I am new to Beam contribution community. I would appreciate if you can point me or tag someone of the contributing members who are working on Beam SQL feature, so that we can have a discussion. My view is BEAM-9825 is more about set operations, which work

Re: possible bug in AvroUtils

2020-04-29 Thread Luke Cwik
+dev +Brian Hulette +Reuven Lax On Wed, Apr 29, 2020 at 4:21 AM Paolo Tomeo wrote: > Hi all, > > I think the method AvroUtils.toBeamSchema has a not expected side effect. > I found out that, if you invoke it and then you run a pipeline of > GenericRecords containing a timestamp (l tried with

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

2020-04-29 Thread Hannah Jiang
Yes, that's the plan for release images. I had a PR[1] merged several days ago, and need to make change again as the default settings changed. I will add validation tasks with details to release guide as well. The other question is, how easy would it be for a release manager to > accidentally push

Re: Python 3.7 docker container fails to build

2020-04-29 Thread Hannah Jiang
There is a PythonDocker Precommit test running for PRs with Python changes. It seems running well.[1] Max, can you please give me a link so I can check more details? Do other images with different Python versions fail as well? 1. https://builds.apache.org/job/beam_PreCommit_PythonDocker_Commit/

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Robert Bradshaw
On Wed, Apr 29, 2020 at 5:08 PM Ahmet Altay wrote: > Nam, this looks better. At least links are working, and the website > visually looks similar and generally in good shape. I think there are still > issues. For example, I do not see any of the images (e.g. the beam logo on > top left is missing

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Ahmet Altay
Nam, this looks better. At least links are working, and the website visually looks similar and generally in good shape. I think there are still issues. For example, I do not see any of the images (e.g. the beam logo on top left is missing.) On Wed, Apr 29, 2020 at 3:11 PM Brian Hulette wrote: >

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

2020-04-29 Thread Ahmet Altay
On Wed, Apr 29, 2020 at 3:49 PM Valentyn Tymofieiev wrote: > > > On Tue, Apr 28, 2020 at 5:03 PM Robert Bradshaw > wrote: > >> On Tue, Apr 28, 2020 at 4:46 PM Hannah Jiang >> wrote: >> >>> I guess I assumed there was some reason we needed "lightweight images" in our tests (because licenses

Re: sdks:java:container:generateThirdPartyLicenses effect on build time / stability

2020-04-29 Thread Valentyn Tymofieiev
On Tue, Apr 28, 2020 at 5:03 PM Robert Bradshaw wrote: > On Tue, Apr 28, 2020 at 4:46 PM Hannah Jiang > wrote: > >> I guess I assumed there was some reason we needed "lightweight images" in >>> our tests (because licenses take up a lot of space IIRC), but maybe not. >>> Can you elaborate on the

Re: Greetings from Tyson

2020-04-29 Thread Alan Myrvold
Welcome, Tyson! On Wed, Apr 29, 2020 at 3:15 PM Rui Wang wrote: > Welcome! > > -Rui > > On Wed, Apr 29, 2020, 3:13 PM Brian Hulette wrote: > >> Welcome Tyson! >> >> On Wed, Apr 29, 2020 at 2:54 PM Ahmet Altay wrote: >> >>> Welcome! >>> >>> On Tue, Apr 28, 2020 at 3:06 PM Hannah Jiang >>> wrot

Re: Jira PR links not being generated?

2020-04-29 Thread Kyle Weaver
Added a comment on https://issues.apache.org/jira/browse/INFRA-19967 On Wed, Apr 29, 2020 at 5:57 PM Ahmet Altay wrote: > Would it be worth filing this as an infra ticket? > > On Mon, Apr 27, 2020 at 4:29 PM Kyle Weaver wrote: > >> Slight correction.. Jira to Github links are back. Github to Ji

Re: Greetings from Tyson

2020-04-29 Thread Rui Wang
Welcome! -Rui On Wed, Apr 29, 2020, 3:13 PM Brian Hulette wrote: > Welcome Tyson! > > On Wed, Apr 29, 2020 at 2:54 PM Ahmet Altay wrote: > >> Welcome! >> >> On Tue, Apr 28, 2020 at 3:06 PM Hannah Jiang >> wrote: >> >>> Welcome to the community! >>> >>> >>> On Tue, Apr 28, 2020 at 2:45 PM Tyso

Re: Greetings from Tyson

2020-04-29 Thread Brian Hulette
Welcome Tyson! On Wed, Apr 29, 2020 at 2:54 PM Ahmet Altay wrote: > Welcome! > > On Tue, Apr 28, 2020 at 3:06 PM Hannah Jiang > wrote: > >> Welcome to the community! >> >> >> On Tue, Apr 28, 2020 at 2:45 PM Tyson Hamilton >> wrote: >> >>> Hello Beam Community, >>> >>> This is just a simple 'He

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Brian Hulette
I left a comment on the PR [1]. I think the reason all of the website content is not being tracked as file renames is because there was a series of commits that created files in the new directory, and then one commit that deleted the old directory. If there were a single commit with all of the dele

Re: Jira PR links not being generated?

2020-04-29 Thread Ahmet Altay
Would it be worth filing this as an infra ticket? On Mon, Apr 27, 2020 at 4:29 PM Kyle Weaver wrote: > Slight correction.. Jira to Github links are back. Github to Jira links > (which were only recently added) are not being added. > > On Mon, Apr 27, 2020 at 7:20 PM Kyle Weaver wrote: > >> Well

Re: Greetings from Tyson

2020-04-29 Thread Ahmet Altay
Welcome! On Tue, Apr 28, 2020 at 3:06 PM Hannah Jiang wrote: > Welcome to the community! > > > On Tue, Apr 28, 2020 at 2:45 PM Tyson Hamilton wrote: > >> Hello Beam Community, >> >> This is just a simple 'Hello' to introduce myself. I'm a Software >> Engineer at Google and have worked with data

Re: Automation for Jira

2020-04-29 Thread Ahmet Altay
+1 for the automations. I agree with concerns related to #4. Auto closing issues is not a good experience. A person goes through the work of reporting an issue. This might very well be their first contribution. Automatically closing these issues with no human comments might make the reporter feel i

Re: Python 3.7 docker container fails to build

2020-04-29 Thread Ahmet Altay
+Valentyn Tymofieiev +Hannah Jiang -- in case they have relevant information. On Wed, Apr 29, 2020 at 12:35 PM Maximilian Michels wrote: > Hi, > > has anyone noticed the Python 3.7 Docker container fails to build? I > haven't been able to build the Python 3.7 container, neither locally nor > o

Re: Rethinking Python's PortableRunner default job server

2020-04-29 Thread Robert Bradshaw
+1, I was actually thinking about this just the other day. PortableRunner should require job_endpoint to be set, and we can have a nice error message directing the explicit use of FlinkRunner for the old behavior. On Wed, Apr 29, 2020 at 11:50 AM Kyle Weaver wrote: > > Could the error message su

Python 3.7 docker container fails to build

2020-04-29 Thread Maximilian Michels
Hi, has anyone noticed the Python 3.7 Docker container fails to build? I haven't been able to build the Python 3.7 container, neither locally nor on Jenkins. I get: 17:48:10 > Task :sdks:python:container:py37:docker 17:49:36 The command '/bin/sh -c pip install -r /tmp/base_image_requirements.txt

Re: Rethinking Python's PortableRunner default job server

2020-04-29 Thread Kyle Weaver
> Could the error message suggest switching to FlinkRunner (and/or other runners that start a job server for you)? Then it seems like the breakage would only be a minor annoyance. Definitely. On Wed, Apr 29, 2020 at 2:49 PM Brian Hulette wrote: > Could the error message suggest switching to Fli

Re: Rethinking Python's PortableRunner default job server

2020-04-29 Thread Brian Hulette
Could the error message suggest switching to FlinkRunner (and/or other runners that start a job server for you)? Then it seems like the breakage would only be a minor annoyance. Brian On Wed, Apr 29, 2020 at 11:32 AM Kyle Weaver wrote: > Hi all, > > Currently, when running a pipeline that has t

Re: Automation for Jira

2020-04-29 Thread Tyson Hamilton
+1 for automation. Regarding #4, what about adding the constraint that this rule only applies to issues that are incomplete and require more information from the reporter? Unfortunately it would require a human to triage issues to determine this and apply an appropriate label. Triage should hap

Rethinking Python's PortableRunner default job server

2020-04-29 Thread Kyle Weaver
Hi all, Currently, when running a pipeline that has the options runner=PortableRunner and job_endpoint unset, the Python SDK spins up a Dockerized Flink job server [1]. This is problematic because the PortableRunner can be used by any portable runner. So for example, a Spark runner user was recent

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Nam Bui
Hi guys, I'm Nam - from the responsible team of Apache Beam website migration. I am pleased to answer some of the questions here. @aizhamal: Thanks for informing to the community. :) @altay, @robertwb: Yes. there is a problem with the staged version at the moment. We didn't expect some behaviours

Re: Automation for Jira

2020-04-29 Thread Brian Hulette
Agree I think this all sounds good except for 4. I like the idea of using automation to help tame the backlog of jiras, but I worry that 4 could lead to a bad experience for users. Say they file a jira and maybe get it assigned, and then watch as it bounces all the way down to closed as obsolete b

Re: Automation for Jira

2020-04-29 Thread Robert Bradshaw
+1 to more automation. I'm in favor of all but 4, I think it's quite common for issues to be noticed but not worked on for 60+ days. Most of the time when a developer files an issue they either (1) are working on it right now or (2) are filing it away because it's something they're not working on,

Automation for Jira

2020-04-29 Thread Kenneth Knowles
Hi all, A while ago [1], we discussed using "Automation for Jira" to improve triage and backlog processing (I spend a lot of my time on this). Due to some friction [2] [3] back then, I did not finish it. Now, I just happened to check and I do have the ability to create rules directly. That's conv