Re: Graduation!
Congrats, guys! On Tue, Jan 10, 2017 at 12:07 PM, Davor Bonaci wrote: > The ASF has publicly announced our graduation! > > > https://blogs.apache.org/foundation/entry/the-apache- > software-foundation-announces > > https://beam.apache.org/blog/2017/01/10/beam-graduates.html > > Graduation is a recognition of the community that we have built together. I > am humbled to be part of this group and this project, and so excited for > what we can accomplish together going forward. > > Davor > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 e: sergio.fernan...@redlink.co w: http://redlink.co
Re: [DISCUSS] Python SDK status and next steps
stribution of the Beam Python SDK had a total of 23K > downloads in the past 6 months [6]. Some of those users are already engaged > with the community (e.g. [7]). There might be an increased amount > engagement from the rest of them after the merge. > Python 3 support is something we definitively need to look ahead. I'd try to make the codebase compatible with both 2.7.x and 3.6.x, rather than using other solutions like 2to3. Looking forward to hearing your thoughts and comments on “graduating” > python-sdk to the master. > > Thank you, > Ahmet > > (*) Python SDK branch currently has a diverse group of contributors. > Regular contributors include Charles Chen, Chamikara Jayalath, María García > Herrero, Mark Liu, Pablo Estrada, Robert Bradshaw (Apache Beam PMC), > Sourabh Bajaj, and Vikas Kedigehalli. We have also had contributions from > Abdullah Bashir, Marco Buccini, Sergio Fernández, Seunghyun Lee, and > Younghee Kwon. > > [1] https://github.com/apache/beam/tree/python-sdk/sdks/python > [2] https://beam.apache.org/documentation/programming-guide/ > [3] https://issues.apache.org/jira/browse/BEAM-1265 > [4] > https://issues.apache.org/jira/issues/?jql=status%20%3D%20Op > en%20AND%20labels%20%3D%20sdk-consistency > [5] https://issues.apache.org/jira/browse/BEAM-1218 > [6] https://pypi.python.org/pypi/google-cloud-dataflow/json > [7] https://issues.apache.org/jira/browse/BEAM-1251 > Great summary, Ahmet. Thanks. Cheers, -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 e: sergio.fernan...@redlink.co w: http://redlink.co
Re: [VOTE] Merge Python SDK to the master branch
+1 On Fri, Jan 20, 2017 at 9:24 PM, Robert Bradshaw < rober...@google.com.invalid> wrote: > On Fri, Jan 20, 2017 at 9:03 AM, Ahmet Altay > wrote: > > Hi all, > > > > > > Please review the earlier discussion on the status of the Python SDK [1] > > and vote on merging the python-sdk branch to the master branch, as > follows: > > [X] +1, Merge python-sdk branch to master after the 0.5.0 release, and > release it in the subsequent minor release. > > - Robert > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 e: sergio.fernan...@redlink.co w: http://redlink.co
Re: [VOTE] Apache Beam, version 0.5.0, release candidate #1
+1 (non-binding) So far I've successfully checked: * signatures and digests * source releases file layouts * matched git tags and commit ids * incubator suffix and disclaimer * NOTICE and LICENSE files * license headers * clean build (Java 1.8.0_91, Maven 3.3.9, Debian amd64) Two minor comments that do not block the release: * Usually I like to see the commit id referencing the rc, since git tags can be changed. * Just a formality, "PPMC" is not committee that plays a role anymore, you're a PMC now ;-) On Fri, Jan 27, 2017 at 9:55 PM, Jean-Baptiste Onofré wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version 0.5.0 > as follows: > > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > The complete staging area is available for your review, which includes: > > * JIRA release notes [1], > * the official Apache source release to be deployed to dist.apache.org > [2], which is signed with the key with fingerprint C8282E76 [3], > * all artifacts to be deployed to the Maven Central Repository [4], > * source code tag "v0.5.0-RC1" [5], > * website pull request listing the release and publishing the API > reference manual [6]. > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PPMC affirmative votes. > > Thanks, > JB > > [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje > ctId=12319527&version=12338859 > [2] https://dist.apache.org/repos/dist/dev/beam/0.5.0/ > [3] https://dist.apache.org/repos/dist/release/beam/KEYS > [4] https://repository.apache.org/content/repositories/orgapachebeam-1010/ > [5] https://git-wip-us.apache.org/repos/asf?p=beam.git;a=tag;h=r > efs/tags/v0.5.0-RC1 > [6] https://github.com/apache/beam-site/pull/132 > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 e: sergio.fernan...@redlink.co w: http://redlink.co
Re: [DISCUSS] Python SDK status and next steps
great! On Tue, Jan 31, 2017 at 8:10 AM, Ahmet Altay wrote: > Hi all, > > This merge is completed. Python SDK is now officially part of the master > branch! Thank you all for the support. Please open an issue, if you notice > a reference to the now obsolete python-sdk branch in the documentation. > > There will not be any more merges to the python-sdk branch. Going forward > please use the master branch for Python SDK development. There are a few > existing open PRs to the python-sdk [1]. If you are the author of one of > those PRs, please rebase them on top of master. > > Thank you, > Ahmet > > [1] https://github.com/pulls?utf8=✓&q=is%3Aopen+is%3Apr+base% > <https://github.com/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+base%25> > 3Apython-sdk+repo%3Aapache%2Fbeam+ > <https://github.com/pulls?utf8=%E2%9C%93&q=is%3Aopen+is% > 3Apr+base%3Apython-sdk+repo%3Aapache%2Fbeam+> > > On Fri, Jan 20, 2017 at 10:06 AM, Kenneth Knowles > wrote: > > > To clarify the implied criteria of that last exchange, it is "An SDK > should > > have at least one runner that can execute the complete model (may be a > > direct runner)" > > > > I want to highlight this, because whether an _SDK_ supports unbounded > data > > is not particularly well-defined, and will evolve: > > > > - With the Runner API, an SDK will need to support building a graph with > > unbounded constructs, as today with probably minimal changes. > > > > - With the Fn API, if any part of the Fn API is specific to unbounded > > data, the SDK will need to implement it. I think right now there is no > such > > thing, and we don't want such a thing, so SDKs implementing the Fn API > > automatically support unbounded data. > > > > - There will also likely be an SDK-specific shim just as there is today, > > to leverage idiomatic deserialized representations. The richness of this > > shim will decrease so that it will need to "support" unbounded data but > > that will be a ~one liner. > > > > Getting the Python SDK on master will accelerate our progress towards the > > Fn API - partly technical, partly community - which is the best path > > towards support for unbounded data across multiple runners. I think the > > criteria are written with the completed portability framework in mind. So > > this exchange makes me actually more convinced we should merge python-sdk > > to master. > > > > On Fri, Jan 20, 2017 at 9:53 AM, Robert Bradshaw < > > rober...@google.com.invalid> wrote: > > > > > On Thu, Jan 19, 2017 at 11:56 PM, Dan Halperin > > > wrote: > > > > I do not think that Python SDK yet meets the bar [1] for implementing > > the > > > > Beam model -- supporting Unbounded data is very important. That said, > > > given > > > > the committed and sustained set of contributors, it generally makes > > sense > > > > to me to make an exception in anticipation of these features being > > > fleshed > > > > out soon; including potentially new users/contributors that would > > arrive > > > > once in master. > > > > > > > > [1] https://lists.apache.org/thread.html/CAAzyFAxcmexUQnbF=Y > > > > k0plmm3f5e5bqwjz4+c5doruclnxo...@mail.gmail.com > > > > > > That is a valid point. The Python SDK supports all the unbounded parts > > > of the model except for unbounded sources, which was deferred while > > > seeing how https://s.apache.org/splittable-do-fn played out. I've been > > > working with the team and merging/reviewing most of their code, and > > > have full confidence this will be coming (and on that note can vouch > > > for a healthy community and support which are much harder to add > > > later). > > > > > > In short, I think it has the required maturity, and I'm in favor of > > > merging soonish. > > > > > > > On Wed, Jan 18, 2017 at 12:24 AM, Ahmet Altay > > > > > > > wrote: > > > > > > > >> Thank you all for the comments so far. I would follow the process as > > > >> suggested by Davor and others in this thread. > > > >> > > > >> Ahmet > > > >> > > > >> On Tue, Jan 17, 2017 at 11:47 PM, Sergio Fernández < > wik...@apache.org > > > > > > >> wrote: > > > >> > > > >> > Hi > > > >> > > > > >> > On Tue, Jan 17, 2017 at 5:22 PM, Ahmet Altay > > > > >
Re: [DISCUSS] Python SDK status and next steps
ticipation of these features being >>>>>> >>>>> fleshed >>>>> >>>>>> out soon; including potentially new users/contributors that would >>>>>> >>>>> arrive >>>> >>>>> once in master. >>>>>> >>>>>> [1] https://lists.apache.org/thread.html/CAAzyFAxcmexUQnbF=Y >>>>>> k0plmm3f5e5bqwjz4+c5doruclnxo...@mail.gmail.com >>>>>> >>>>> >>>>> That is a valid point. The Python SDK supports all the unbounded parts >>>>> of the model except for unbounded sources, which was deferred while >>>>> seeing how https://s.apache.org/splittable-do-fn played out. I've been >>>>> working with the team and merging/reviewing most of their code, and >>>>> have full confidence this will be coming (and on that note can vouch >>>>> for a healthy community and support which are much harder to add >>>>> later). >>>>> >>>>> In short, I think it has the required maturity, and I'm in favor of >>>>> merging soonish. >>>>> >>>>> On Wed, Jan 18, 2017 at 12:24 AM, Ahmet Altay >>>>>> >>>>> >>>>> >>>>> wrote: >>>>>> >>>>>> Thank you all for the comments so far. I would follow the process as >>>>>>> suggested by Davor and others in this thread. >>>>>>> >>>>>>> Ahmet >>>>>>> >>>>>>> On Tue, Jan 17, 2017 at 11:47 PM, Sergio Fernández < >>>>>>> wik...@apache.org >>>>>>> >>>>>> >>>>> wrote: >>>>>>> >>>>>>> Hi >>>>>>>> >>>>>>>> On Tue, Jan 17, 2017 at 5:22 PM, Ahmet Altay >>>>>>>> >>>>>>> >>> >>>>> >>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> tl;dr: I would like to start a discussion about merging python-sdk >>>>>>>>> >>>>>>>> branch >>>>>>> >>>>>>>> to master branch. Python SDK is mature enough and merging it to >>>>>>>>> >>>>>>>> master >>>>> >>>>>> will >>>>>>>> >>>>>>>>> accelerate its development and adoption. >>>>>>>>> >>>>>>>>> >>>>>>>> Good point, Ahmet! >>>>>>>> >>>>>>>> I've following closed the development since it was imported in June. >>>>>>>> >>>>>>> For >>>>> >>>>>> the prototypes I've implemented so far it works quite well; I guess >>>>>>>> >>>>>>> we'd >>>>> >>>>>> just need to focus the next months in bringing more runners support. >>>>>>>> >>>>>>>> With a great effort from a lot of contributors(*), Python SDK [1] is >>>>>>>> >>>>>>> now >>>>> >>>>>> a >>>>>>> >>>>>>>> mostly complete, tested, performant Python implementation of the >>>>>>>>> >>>>>>>> Beam >>>>> >>>>>> model. Since June, when we first started with Python SDK in Apache >>>>>>>>> >>>>>>>> Beam >>>>> >>>>>> we >>>>>>>> >>>>>>>>> have been continuously improving it. >>>>>>>>> >>>>>>>>> >>>>>>>> I wouldn't merge during the preparation of 0.5.0 release, but after >>>>>>>> >>>>>>> that >>>>> >>>>>> could be a good time to merge back into master. >>>>>>>> >>>>>>>> >>>>>>>> ** Python SDK currently supports: >>>>>>>> >>>>>>>>> >>>>>>>>> * Model: All main concepts are present (ParDo, GroupByKey, >>>>>>>>> >>>>>>>
Re: [DISCUSS] Python SDK status and next steps
PR #1879 provides the basics: https://github.com/apache/beam/pull/1879 On Tue, Jan 31, 2017 at 1:33 PM, Jean-Baptiste Onofré wrote: > No, that's fine as soon as we clearly document the prerequisite for the > build. IMHO, we should provide quick BUILDING instructions in the README.md. > > Regards > JB > > > On 01/31/2017 01:24 PM, Sergio Fernández wrote: > >> Originally we integrate the build in Maven with the default profile. >> Do you feel like it'd be better to have it under a separated profile or >> so? >> >> On Tue, Jan 31, 2017 at 11:07 AM, Jean-Baptiste Onofré >> wrote: >> >> Just to be clear, the prerequisite to be able to build the Python SDK are: >>> >>> apt-get install python-setuptools >>> apt-get install python-pip >>> >>> It's also required by the default "regular" build. >>> >>> Regards >>> JB >>> >>> >>> On 01/31/2017 11:02 AM, Jean-Baptiste Onofré wrote: >>> >>> Just one thing I noticed (and can be helpful for others): to build Beam >>>> we now need python setuptools installed. >>>> >>>> For instance, on Ubuntu, you have to do: >>>> >>>> apt-get install python-setuptools >>>> >>>> Same for the pip distribution. >>>> >>>> I guess (if not already done), we have to update README/Building >>>> instructions. >>>> >>>> Correct ? >>>> >>>> Regards >>>> JB >>>> >>>> On 01/31/2017 08:10 AM, Ahmet Altay wrote: >>>> >>>> Hi all, >>>>> >>>>> This merge is completed. Python SDK is now officially part of the >>>>> master >>>>> branch! Thank you all for the support. Please open an issue, if you >>>>> notice >>>>> a reference to the now obsolete python-sdk branch in the documentation. >>>>> >>>>> There will not be any more merges to the python-sdk branch. Going >>>>> forward >>>>> please use the master branch for Python SDK development. There are a >>>>> few >>>>> existing open PRs to the python-sdk [1]. If you are the author of one >>>>> of >>>>> those PRs, please rebase them on top of master. >>>>> >>>>> Thank you, >>>>> Ahmet >>>>> >>>>> [1] https://github.com/pulls?utf8=✓&q=is%3Aopen+is%3Apr+base% >>>>> <https://github.com/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+base%25> >>>>> <https://github.com/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+base%25> >>>>> 3Apython-sdk+repo%3Aapache%2Fbeam+ >>>>> <https://github.com/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr >>>>> +base%3Apython-sdk+repo%3Aapache%2Fbeam+> >>>>> >>>>> >>>>> On Fri, Jan 20, 2017 at 10:06 AM, Kenneth Knowles >>>>> >>>>> wrote: >>>>> >>>>> To clarify the implied criteria of that last exchange, it is "An SDK >>>>> >>>>>> should >>>>>> have at least one runner that can execute the complete model (may be a >>>>>> direct runner)" >>>>>> >>>>>> I want to highlight this, because whether an _SDK_ supports unbounded >>>>>> data >>>>>> is not particularly well-defined, and will evolve: >>>>>> >>>>>> - With the Runner API, an SDK will need to support building a graph >>>>>> with >>>>>> unbounded constructs, as today with probably minimal changes. >>>>>> >>>>>> - With the Fn API, if any part of the Fn API is specific to unbounded >>>>>> data, the SDK will need to implement it. I think right now there is >>>>>> no such >>>>>> thing, and we don't want such a thing, so SDKs implementing the Fn API >>>>>> automatically support unbounded data. >>>>>> >>>>>> - There will also likely be an SDK-specific shim just as there is >>>>>> today, >>>>>> to leverage idiomatic deserialized representations. The richness of >>>>>> this >>>>>> shim will decrease so that it will need to "support" unbounded data >>>>>> but >>>>>> that will be a ~one liner. &g
Re: [VOTE] Apache Beam, version 0.5.0, release candidate #2
+1 (non-binding) So far I've successfully checked: * signatures and digests * source releases file layouts * no binaries included in the source release * matched git tag * NOTICE and LICENSE files * license headers * clean build (Java 1.8.0_91, Maven 3.3.9, Debian amd64) As I already commented on RC1, formally it's better to include commit id referencing the release, since git tags can be changed. Just take that into account for upcoming releases. On Fri, Feb 3, 2017 at 1:27 AM, Davor Bonaci wrote: > Hi everyone, > With JB leaving for his vacation, I'll try to push the 0.5.0 release across > the finish line. Please review and vote on the release candidate #2 for the > version 0.5.0, as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > The complete staging area is available for your review, which includes: > * JIRA release notes [1], > * the official Apache source release to be deployed to dist.apache.org > [2], > which is signed with the key with fingerprint 8F0D334F [3], > * all artifacts to be deployed to the Maven Central Repository [4], > * source code tag "v0.5.0-RC2" [5], > * website pull request listing the release and publishing the API reference > manual [6]. > > Compared to release candidate #1, this candidate contains pull requests > #1903 [7] and #1908 [8]; see the discussion for reasoning. > > A passing suite of Jenkins jobs: > * PreCommit_Java_MavenInstall [9], > * PostCommit_Java_MavenInstall [10], > * PostCommit_Java_RunnableOnService_Apex [11], > * PostCommit_Java_RunnableOnService_Flink [12], > * PostCommit_Java_RunnableOnService_Spark [13], > * PostCommit_Java_RunnableOnService_Dataflow [14]. > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PMC affirmative votes. > > Thanks, > Davor > > [1] > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje > ctId=12319527&version=12338859 > [2] https://dist.apache.org/repos/dist/dev/beam/0.5.0/RC2/ > [3] https://dist.apache.org/repos/dist/release/beam/KEYS > [4] https://repository.apache.org/content/repositories/orgapachebeam-1011/ > [5] https://github.com/apache/beam/tree/v0.5.0-RC2 > [6] https://github.com/apache/beam-site/pull/132 > [7] https://github.com/apache/beam/pull/1903 > [8] https://github.com/apache/beam/pull/1908 > [9] https://builds.apache.org/job/beam_PreCommit_Java_MavenInstall/7028/ > [10] https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/2514/ > [11] > https://builds.apache.org/job/beam_PostCommit_Java_RunnableO > nService_Apex/386/ > [12] > https://builds.apache.org/job/beam_PostCommit_Java_RunnableO > nService_Flink/1521/ > [13] > https://builds.apache.org/job/beam_PostCommit_Java_RunnableO > nService_Spark/830/ > [14] > https://builds.apache.org/job/beam_PostCommit_Java_RunnableO > nService_Dataflow/2180/ > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 <+43%20660%202747925> e: sergio.fernan...@redlink.co w: http://redlink.co
Re: We've hit 2000 PRs!
An example what community development means :-) Thanks for sharing, Dan! On Feb 16, 2017 5:05 PM, "Dan Halperin" wrote: > Checking my previous claims: > > PR #1: Feb 26, 2016 > PR #1000: Sep 24, 2016 (211 days later) > PR #2000: Feb 13, 2016 (142 days later) Yep -- much quicker! > > I'm excited to see this community growing and innovating as we march > towards the true Beam Technical Vision, a first major release, and really > empowering users to build portable, long-lived, fast data processing > pipelines. > > Thanks everyone for making this community and keeping this project really > fun :) > > Dan > > On Mon, Sep 26, 2016 at 2:47 PM, Dan Halperin wrote: > > > Hey folks! > > > > Just wanted to send out a note -- we've hit 1000 PRs in GitHub as of > > Saturday! That's a tremendous amount of work for the 7 months since PR#1. > > > > I bet we hit 2000 in much fewer than 7 months ;) > > > > Dan > > >
Re: Interest in a (virtual) contributor meeting?
I think it's a really great idea! On Wed, Feb 22, 2017 at 4:18 AM, Davor Bonaci wrote: > In the early days of the project, we have held a few meetings for the > initial community to get to know each other. Since then, the community has > grown a huge amount, but we haven't organized any get-togethers. > > I wanted to gauge interest in a potential video conference call in the near > future. No specific agenda -- simply a chance for everyone to meet others > and see the faces of people we share a common passion with. Of course, an > open discussion on any topic of interest to the contributor community is > welcome. This would be strictly informal -- any decisions are reserved for > the mailing list discussions. > > If you'd be interested in attending, please reply back. If there's > sufficient interest, I'd be happy to try to organize something in the near > future. > > Thanks! > > Davor > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 e: sergio.fernan...@redlink.co w: http://redlink.co
Re: Release 0.6.0
Sounds good! Ahmet, notice ASF has not current infrastructure to stage Python Release Candidates. Anyway we left unmanaged the Maven deploy lifecycle for the Python SDK, but it should be discussed at some point. On Mon, Feb 27, 2017 at 11:01 PM, Ahmet Altay wrote: > Hi all, > > It's been about a month since the last release. I would like propose > starting the next release. There are no releasing blocking bugs in JIRA > [1]. Are there any release blocking issues I am missing? > > Unless there is an objection I will volunteer to manage this release. This > will be the first release with Python content. In case there are issues > with that it might be easier for me to resolve and document those as part > of the release process. > > Thank you, > Ahmet > > [1] > https://issues.apache.org/jira/issues/?jql=project%20% > 3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND% > 20fixVersion%20%3D%200.6.0%20ORDER%20BY%20due%20ASC%2C% > 20priority%20DESC%2C%20created%20ASC > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 e: sergio.fernan...@redlink.co w: http://redlink.co
Re: Apache Beam (virtual) contributor meeting @ Tue Mar 7, 2017
Thanks for organizing, Davor! BTW, I can; t find the Hangout link within the calendar invitation. Can you posted it here again? Thanks. Talk tomorrow! On Mon, Mar 6, 2017 at 6:23 PM, Davor Bonaci wrote: > Just a remainder that this is happening in about ~22 hours from now. Hope > to see all of you there. > > On Thu, Mar 2, 2017 at 4:22 PM, Davor Bonaci wrote: > > > I'd prefer not to record the video; just to keep things informal. We'll, > > however, keep the notes and share anything that may be relevant. > > > > On Thu, Mar 2, 2017 at 2:24 PM, Amit Sela wrote: > > > >> I'll be there! > >> > >> On Thu, Mar 2, 2017 at 1:06 PM Aljoscha Krettek > >> wrote: > >> > >> > Shoot, I can't because I already have another meeting scheduled. Don't > >> mind > >> > me, though. Will you also maybe produce a video of the meeting? > >> > > >> > On Wed, 1 Mar 2017 at 21:50 Davor Bonaci wrote: > >> > > >> > > Hi everyone, > >> > > Based on the high demand [1], let's try to organize a virtual > >> contributor > >> > > meeting on Tuesday, March 7, 2017 at 15:00 UTC. For convenience, > >> calendar > >> > > link [2] and an .ics file are attached. > >> > > > >> > > I tried to accommodate as many time zones as possible, but I know it > >> > might > >> > > be hard for some of us at 7 AM on the US west coast or 11 PM in > China. > >> > > Sorry about that. > >> > > > >> > > Let's use Google Hangouts as the video conferencing technology. I > >> think > >> > we > >> > > may be limited to something like 30 participants, so I'd encourage > any > >> > > co-located contributors to consider joining together (if > appropriate). > >> > > Joining the meeting should be straightforward -- please find the > link > >> > > within. No special requirements that I'm aware of. > >> > > > >> > > Just to re-state the expectations: > >> > > * This is totally optional and informal. > >> > > * It is simply a chance for everyone to meet others and see the > faces > >> of > >> > > people we share a common passion with. > >> > > * No specific agenda. > >> > > * An open discussion on any topic of interest to the contributor > >> > community > >> > > is > >> > > welcome -- please feel free to bring up any topics you care about. > >> > > * No formal discussion or decisions should to be made. > >> > > * We'll keep notes and share them on the mailing list shortly after > >> the > >> > > meeting. > >> > > > >> > > Thanks -- and hope to see all of you there! > >> > > > >> > > Davor > >> > > > >> > > [1] > >> > > > >> > https://lists.apache.org/thread.html/baf057b81c5f6d4127abada > >> c165d923a224d34438fe67b71d73743ad@%3Cdev.beam.apache.org%3E > >> > > [2] > >> > > > >> > https://calendar.google.com/calendar/event?action=TEMPLATE&; > >> tmeid=a3A2MzdhaWdhdjByNWRibzZrN2ZnOG1kMTAgZGF2b3JAZ29vZ2xlLm > >> NvbQ&tmsrc=davor%40google.com > >> > > > >> > > >> > > > > > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 e: sergio.fernan...@redlink.co w: http://redlink.co
Re: Apache Beam (virtual) contributor meeting @ Tue Mar 7, 2017
Thanks, Davor! On Tue, Mar 7, 2017 at 3:20 AM, Davor Bonaci wrote: > Link: https://hangouts.google.com/hangouts/_/google.com/beam-dev-mtg > > I'll try to be available on Slack shortly before the meeting, just in case > someone has trouble connecting. > > On Mon, Mar 6, 2017 at 9:27 AM, Amit Sela wrote: > > > PayPal team will be there joined together. > > > > On Mon, Mar 6, 2017 at 7:23 PM Davor Bonaci wrote: > > > > > Just a remainder that this is happening in about ~22 hours from now. > Hope > > > to see all of you there. > > > > > > On Thu, Mar 2, 2017 at 4:22 PM, Davor Bonaci wrote: > > > > > > > I'd prefer not to record the video; just to keep things informal. > > We'll, > > > > however, keep the notes and share anything that may be relevant. > > > > > > > > On Thu, Mar 2, 2017 at 2:24 PM, Amit Sela > > wrote: > > > > > > > >> I'll be there! > > > >> > > > >> On Thu, Mar 2, 2017 at 1:06 PM Aljoscha Krettek < > aljos...@apache.org> > > > >> wrote: > > > >> > > > >> > Shoot, I can't because I already have another meeting scheduled. > > Don't > > > >> mind > > > >> > me, though. Will you also maybe produce a video of the meeting? > > > >> > > > > >> > On Wed, 1 Mar 2017 at 21:50 Davor Bonaci > wrote: > > > >> > > > > >> > > Hi everyone, > > > >> > > Based on the high demand [1], let's try to organize a virtual > > > >> contributor > > > >> > > meeting on Tuesday, March 7, 2017 at 15:00 UTC. For convenience, > > > >> calendar > > > >> > > link [2] and an .ics file are attached. > > > >> > > > > > >> > > I tried to accommodate as many time zones as possible, but I > know > > it > > > >> > might > > > >> > > be hard for some of us at 7 AM on the US west coast or 11 PM in > > > China. > > > >> > > Sorry about that. > > > >> > > > > > >> > > Let's use Google Hangouts as the video conferencing technology. > I > > > >> think > > > >> > we > > > >> > > may be limited to something like 30 participants, so I'd > encourage > > > any > > > >> > > co-located contributors to consider joining together (if > > > appropriate). > > > >> > > Joining the meeting should be straightforward -- please find the > > > link > > > >> > > within. No special requirements that I'm aware of. > > > >> > > > > > >> > > Just to re-state the expectations: > > > >> > > * This is totally optional and informal. > > > >> > > * It is simply a chance for everyone to meet others and see the > > > faces > > > >> of > > > >> > > people we share a common passion with. > > > >> > > * No specific agenda. > > > >> > > * An open discussion on any topic of interest to the contributor > > > >> > community > > > >> > > is > > > >> > > welcome -- please feel free to bring up any topics you care > about. > > > >> > > * No formal discussion or decisions should to be made. > > > >> > > * We'll keep notes and share them on the mailing list shortly > > after > > > >> the > > > >> > > meeting. > > > >> > > > > > >> > > Thanks -- and hope to see all of you there! > > > >> > > > > > >> > > Davor > > > >> > > > > > >> > > [1] > > > >> > > > > > >> > https://lists.apache.org/thread.html/baf057b81c5f6d4127abada > > > >> c165d923a224d34438fe67b71d73743ad@%3Cdev.beam.apache.org%3E > > > >> > > [2] > > > >> > > > > > >> > https://calendar.google.com/calendar/event?action=TEMPLATE&; > > > >> tmeid=a3A2MzdhaWdhdjByNWRibzZrN2ZnOG1kMTAgZGF2b3JAZ29vZ2xlLm > > > >> NvbQ&tmsrc=davor%40google.com > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 e: sergio.fernan...@redlink.co w: http://redlink.co
Re: [VOTE] Release 0.6.0, release candidate #1
+1 (non-binding) So far I've successfully checked: * signatures and digests * source releases file layouts * no binaries included in the source release * matched git tag * NOTICE and LICENSE files * license headers * clean build (Java 1.8.0_91, Maven 3.3.9, Python 2.7.12+, Debian amd64) BTW, some minor things: * As I already commented on past release votes, formally it's better to include commit id referencing the release, since git tags can be changed. Just take that into account for upcoming releases. * I've noticed the build take much more time than previous releases. * Also I got this minor warning on the Maven build: [WARNING] Some problems were encountered while building the effective model for org.apache.beam:beam-runners-google-cloud-dataflow-java:jar:0.6.0 [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: org.apache.beam:beam-runners-core-construction-java:jar -> duplicate declaration of version (?) @ org.apache.beam:beam-runners-g oogle-cloud-dataflow-java:[unknown-version], /home/wikier/tmp/beam/apache-b eam-0.6.0/runners/google-cloud-dataflow-java/pom.xml, line 360, column 19 * Notice the current Python SDK may require sudo permissions in some environments Cheers, On Fri, Mar 10, 2017 at 1:07 AM, Ahmet Altay wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version 0.6.0, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > The complete staging area is available for your review, which includes: > * JIRA release notes [1], > * the official Apache source release to be deployed to dist.apache.org > [2], > which is signed with the key with fingerprint 6096FA00 [3], > * all artifacts to be deployed to the Maven Central Repository [4], > * source code tag "v0.6.0-RC1" [5], > * website pull request listing the release and publishing the API reference > manual [6]. > * python artifacts are deployed along with the source release to to > dist.apache.org [2]. > > A suite of Jenkins jobs: > * PreCommit_Java_MavenInstall [7], > * PostCommit_Java_MavenInstall [8], > * PostCommit_Java_RunnableOnService_Apex [9], > * PostCommit_Java_RunnableOnService_Flink [10], -> ran into a known issue > [14] > * PostCommit_Java_RunnableOnService_Spark [11], > * PostCommit_Java_RunnableOnService_Dataflow [12] -> timed out at 100 > minutes, the logs are good up to that point [15] is for increasing this > timeout. > * PostCommit_Python_Verify [13] > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PMC affirmative votes. > > Thanks, > Ahmet > > [1] > https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje > ctId=12319527&version=12339256 > [2] https://dist.apache.org/repos/dist/dev/beam/0.6.0/ > [3] https://dist.apache.org/repos/dist/dev/beam/KEYS > [4] https://repository.apache.org/content/repositories/orgapachebeam-1012/ > [5] > https://git-wip-us.apache.org/repos/asf?p=beam.git;a=tag;h=r > efs/tags/v0.6.0-RC1 > [6] https://github.com/apache/beam-site/pull/175 > [7] > https://builds.apache.org/view/Beam/job/beam_PreCommit_Java_ > MavenInstall/8281/ > [8] > https://builds.apache.org/view/Beam/job/beam_PostCommit_Java > _MavenInstall/2858/ > [9] > https://builds.apache.org/view/Beam/job/beam_PostCommit_Java > _RunnableOnService_Apex/717/ > [10] > https://builds.apache.org/view/Beam/job/beam_PostCommit_Java > _RunnableOnService_Flink/1874/ > [11] > https://builds.apache.org/view/Beam/job/beam_PostCommit_Java > _RunnableOnService_Spark/1184/ > [12] > https://builds.apache.org/view/Beam/job/beam_PostCommit_Java > _RunnableOnService_Dataflow/2511/ > [13] > https://builds.apache.org/view/Beam/job/beam_PostCommit_Pyth > on_Verify/1466/ > [14] https://issues.apache.org/jira/browse/BEAM-1674 > [15] https://github.com/apache/beam/pull/2197 > -- Sergio Fernández Partner Technology Manager Redlink GmbH m: +43 6602747925 <+43%20660%202747925> e: sergio.fernan...@redlink.co w: http://redlink.co
Re: Python build artifacts seem to be misconfigured
Maybe we have to push-up the configurations the Maven build does https://github.com/apache/beam/blob/master/sdks/python/pom.xml#L35 to the main Python build, so it stays the same whatever is invoked. On Sat, Apr 15, 2017 at 3:28 AM, Ahmet Altay wrote: > Robert, would passing `build_dir='target/build'` to `cythonize` accomplish > what Davor is saying? > > Using build_dir seems to be working but I could not find documentation of > it. > > Ahmet > > On Wed, Apr 12, 2017 at 7:11 PM, Davor Bonaci wrote: > > > I think it would be great if it can be configured that all (or, as many > as > > possible) build-generated files use one specific directory -- "target/". > > Likely, all problems would just go away. > > > > On Wed, Apr 12, 2017 at 3:24 PM, Ahmet Altay > > wrote: > > > > > This is also root cause for the flakiness in test_using_slow_impl very > > > flaky locally tests (https://issues.apache.org/jira/browse/BEAM-1910). > > > > > > Kenn, have you found anything that might explain why tox is not > deleting > > > them? > > > > > > Ahmet > > > > > > On Tue, Apr 11, 2017 at 11:50 AM, Robert Bradshaw < > > > rober...@google.com.invalid> wrote: > > > > > > > We should also ignore them: https://github.com/apache/beam/pull/2494 > > > > > > > > On Thu, Apr 6, 2017 at 6:45 PM, Kenneth Knowles > > > > > > > wrote: > > > > > Thanks for the pointer. I'll dig in to tox docs to see why this > isn't > > > > > happening. Probably something to do with unclean shutdowns. > > > > > > > > > > On Thu, Apr 6, 2017 at 6:10 PM, Vikas RK > wrote: > > > > > > > > > >> Those are cython generated files that should be deleted according > to > > > > >> https://github.com/apache/beam/blob/master/sdks/python/ > tox.ini#L54 > > > > >> > > > > >> > > > > >> > > > > >> On 6 April 2017 at 17:58, Kenneth Knowles > > > > > wrote: > > > > >> > > > > >> > Hi all, > > > > >> > > > > > >> > It appears that the Python build process creates quite a few > files > > > > that > > > > >> are > > > > >> > not accounted for in our .gitignore and that also trip the RAT > > check > > > > next > > > > >> > time around. These should be set up so that RAT and git both > > ignore > > > > the > > > > >> > files. > > > > >> > > > > > >> > It is possible that others have defaults that differ from mine, > > but > > > > >> > droppings from a recent `mvn verify` include: > > > > >> > > > > > >> > sdks/python/apache_beam/coders/coder_impl.c > > > > >> > sdks/python/apache_beam/coders/coder_impl.so > > > > >> > sdks/python/apache_beam/coders/stream.c > > > > >> > sdks/python/apache_beam/coders/stream.so > > > > >> > sdks/python/apache_beam/metrics/execution.c > > > > >> > sdks/python/apache_beam/metrics/execution.so > > > > >> > sdks/python/apache_beam/runners/common.c > > > > >> > sdks/python/apache_beam/runners/common.so > > > > >> > sdks/python/apache_beam/transforms/cy_combiners.c > > > > >> > sdks/python/apache_beam/transforms/cy_combiners.so > > > > >> > sdks/python/apache_beam/utils/counters.c > > > > >> > sdks/python/apache_beam/utils/counters.so > > > > >> > sdks/python/apache_beam/utils/windowed_value.c > > > > >> > sdks/python/apache_beam/utils/windowed_value.so > > > > >> > sdks/python/nose-1.3.7-py2.7.egg/ > > > > >> > > > > > >> > Can someone who knows the Python SDK build process rectify? > > > > >> > > > > > >> > Kenn > > > > >> > > > > > >> > > > > > > > > > >
