[Reminder] Beam 2.16.0 Release Soon

2019-09-05 Thread Mark Liu
Hi all, Reminder, the plan to cut the release branch is on 09/11 (less than a week from now)[1]. Please mark all release blocking issues with fix version 2.16[2]. [1] https://calendar.google.com/calendar/embed?src=0p73sl034k80oob7seouanigd0%40group.calendar.google.com [2]

Re: Improve container support

2019-09-05 Thread Ankur Goenka
Please ignore the previous email. I was looking at the older document in the mail thread. On Thu, Sep 5, 2019 at 4:58 PM Ankur Goenka wrote: > I think sdk in the name is obsolete as they are all under sdks name space. > > On Thu, Sep 5, 2019 at 3:26 PM Hannah Jiang > wrote: > >> Hi Team >> >>

Re: Possible Python SDK performance regression

2019-09-05 Thread Ahmet Altay
On Thu, Sep 5, 2019 at 4:15 PM Thomas Weise wrote: > The workload is quite different. What I have is streaming with state and > timers. > > > > On Thu, Sep 5, 2019 at 3:47 PM Pablo Estrada wrote: > >> We only recently started running Chicago Taxi Example. +Michał Walenia >> I don't see it in

Re: Improve container support

2019-09-05 Thread Ankur Goenka
I think sdk in the name is obsolete as they are all under sdks name space. On Thu, Sep 5, 2019 at 3:26 PM Hannah Jiang wrote: > Hi Team > > Thanks for all the comments about beam containers. > After considering various opinions and investigating gcr and docker hub, > we decided to push images

Re: Possible Python SDK performance regression

2019-09-05 Thread Thomas Weise
The workload is quite different. What I have is streaming with state and timers. On Thu, Sep 5, 2019 at 3:47 PM Pablo Estrada wrote: > We only recently started running Chicago Taxi Example. +Michał Walenia > I don't see it in the dashboards. Do you > know if it's possible to see any trends

Re: Possible Python SDK performance regression

2019-09-05 Thread Pablo Estrada
We only recently started running Chicago Taxi Example. +Michał Walenia I don't see it in the dashboards. Do you know if it's possible to see any trends in the data? We have a few tests running now: - Combine tests:

Re: Possible Python SDK performance regression

2019-09-05 Thread Thomas Weise
It probably won't be practical to do a bisect due to the high cost of each iteration with our fork/deploy setup. Perhaps it is time to setup something with the synthetic source that works just with Beam as dependency. On Thu, Sep 5, 2019 at 3:23 PM Ahmet Altay wrote: > There are a few in this

Re: Improve container support

2019-09-05 Thread Hannah Jiang
Hi Team Thanks for all the comments about beam containers. After considering various opinions and investigating gcr and docker hub, we decided to push images to docker hub. Each image will have two tags, {version}_rc and {version}. {version} tag will be added after the release candidate image is

Re: Possible Python SDK performance regression

2019-09-05 Thread Ahmet Altay
There are a few in this dashboard [1], but not very useful in this case because they do not go back more than a month and not very comprehensive. I do not see a jump there. Thomas, would it be possible to bisect to find what commit caused the regression? +Pablo Estrada do we have any python on

Possible Python SDK performance regression

2019-09-05 Thread Thomas Weise
Hi, Are there any performance tests run for the Python SDK as part of release verification (or otherwise as well)? I see what appears to be a regression in master (compared to 2.14) with our in-house application (~ 25% jump in cpu utilization and corresponds drop in throughput). I wanted to see

Re: Hackathon @BeamSummit @ApacheCon

2019-09-05 Thread Chad Dombrova
Has a date and time been picked for this? I'll be there for part of the week and would love to join. On Tue, Sep 3, 2019 at 11:31 AM Brian Hulette wrote: > I will be around all week as well and would love to help with a Beam > hackathon in any way :) > > On Thu, Aug 29, 2019 at 9:46 AM

installing Apache Beam on Pycharm with Python 3.7

2019-09-05 Thread Priti Badami
Hi Dev Team, I am trying to install Apache Beam. I have pip 19.2.3 but I am facing issues while installing Beam. please advice, Thanks, Priti Badami

Re: [VOTE] Vendored Dependencies Release

2019-09-05 Thread Lukasz Cwik
LGTM On Wed, Sep 4, 2019 at 4:24 PM Rui Wang wrote: > Thanks Pablo for jumping in for help. > > Now the sources are moved to [1]. Please let me know if it is ok. > > [1]: > https://dist.apache.org/repos/dist/release/beam/vendor/calcite/1_20_0/ > > -Rui > > On Wed, Sep 4, 2019 at 4:15 PM Pablo

Re: [PROPOSAL] Storing, displaying and detecting anomalies in test results

2019-09-05 Thread Kamil Wasilewski
We've just started working on this and here comes the first PR: https://github.com/apache/beam/pull/9482. Feel free to share your thoughts and ideas. Kamil On Tue, Aug 27, 2019 at 1:19 AM Pablo Estrada wrote: > Thanks Kamil for bringing this! > +Manisha Bhardwaj +Mark Liu have > worked on

Re: Jira - Contributor

2019-09-05 Thread Jean-Baptiste Onofré
Done: you are in the contributor list and the Jira is assigned to you. Regards JB On 05/09/2019 10:31, Matthew Darwin wrote: > Hi, > > Could I please be added as a contributor on Jira so I can assign > https://issues.apache.org/jira/browse/BEAM-8153 to me? > > Kind regards > > Matthew --

Jira - Contributor

2019-09-05 Thread Matthew Darwin
Hi, Could I please be added as a contributor on Jira so I can assign https://issues.apache.org/jira/browse/BEAM-8153 to me? Kind regards Matthew Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and