Re: [VOTE] Release Apache Beam, version 0.2.0-incubating
+1 (binding) - built from source - “incubating” in file name - NOTICE and LICENSE look good - license headers present - no wayward binaries - signatures check out -Taylor > On Aug 1, 2016, at 12:46 PM, Dan Halperin wrote: > > Hey folks! > > Here's the vote for the second release of Apache Beam: version > 0.2.0-incubating. > > The complete staging area is available for your review, which includes: > * the official Apache source release to be deployed to dist.apache.org [1], > and > * all artifacts to be deployed to the Maven Central Repository [2]. > > This corresponds to the tag "v0.2.0-incubating-RC2" in source control, [3]. > > New for this release: > * Release notes are available in JIRA [4]. > * We made sure to address all the issues that the Apache Incubator PMC > raised in the previous release [5]. > > The Apache Beam community has unanimously approved this release: [6], [7]. > > Please vote as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > Thanks, > Dan > > As customary, the vote will be open for at least 72 hours. It is adopted by > a majority approval with at least three PMC affirmative votes. If approved, > we will proceed with the release. > > [1]https://dist.apache.org/repos/dist/dev/incubator/beam/0.2.0-incubating/RC2/ > [2] https://repository.apache.org/content/repositories/orgapachebeam-1004/ > [3] https://github.com/apache/incubator-beam/tree/v0.2.0-incubating-RC2 > [4]https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12335766 > [5] Thread > afterhttp://mail-archives.apache.org/mod_mbox/incubator-general/201606.mbox/%3CCAMdX748VZg5-p%3D5x63se-iBZU0e32n20aRyVsDPWhWZaoq7SoA%40mail.gmail.com%3E > [6]http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201607.mbox/%3CCAA8k_FJeyg%2BGWUBMeSPFQhnaPN3V4MrenJtrDbiyXyKJkzH7ZA%40mail.gmail.com%3E > [7] > http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201607.mbox/browser signature.asc Description: Message signed with OpenPGP using GPGMail
Utility for Serializing PipelineOptions
Looking at the runner implementations for Spark, Flink, and Gearpump, they all have the same requirement of being able to serialize/deserialize PipelineOptions, and they each seem to use slightly different approaches. In working on a Beam runner for Storm I have the same requirement, and imagine other runner implementations might as well. Would it make sense to add this functionality in the core SDK? -Taylor signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [PROPOSAL] IRC or slack channel for Apache Beam
Can I get an invite to the slack channel? I’m in the early stages of implementing a Beam runner for Apache Storm and have some (probably stupid ;) ) questions. Also, would it make sense to document the process on the Beam website so new users/devs can find out about it and how to join? -Taylor > On May 24, 2016, at 11:39 AM, Jean-Baptiste Onofré wrote: > > Hi, > > We already discussed about that during the latest Beam developer meetup. > > Basically, as I said: Slack, IRC, Hangout, whatever are very convenient to > discuss ideas, fixes, etc. > > However, all discussions happening there have to be summarized and shared on > the mailing list. > > Good idea to have such reminder in the welcome message (I'm doing it). > > Thanks, > Regards > JB > > On 05/24/2016 05:33 PM, Ganelin, Ilya wrote: >> Hi, all - I'm a big fan of Slack and would love an invite to the room as >> well. >> >> With that said, from an early discussion on another Apache project, we were >> reminded that a key component of the Apache way is for all substantive >> discussion to be publicly visible, archived, and searchable. >> >> Slack may have the unintended consequence of promoting direct and invisible >> interaction, which while useful to the participants, may hinder the success >> of the project overall. >> >> As a relatively new member of Apache, I would defer to those wiser and more >> experienced in its ways (e.g our mentors) for further guidance but I think >> this is a point worth reminding folks of. >> >> Perhaps the answer is as simple as ensuring the welcome message on Slack >> reminds people of this. >> >> /my2c >> >> >> >> Sent with Good (www.good.com) >> >> From: Jean-Baptiste Onofré >> Sent: Tuesday, May 24, 2016 11:13:17 AM >> To: dev@beam.incubator.apache.org >> Subject: Re: [PROPOSAL] IRC or slack channel for Apache Beam >> >> Done >> >> Regards >> JB >> >> On 05/24/2016 04:46 PM, Jesse Anderson wrote: >>> Me too >>> >>> On Tue, May 24, 2016, 7:37 AM Jean-Baptiste Onofré >>> wrote: >>> Done Regards JB On 05/24/2016 04:31 PM, Simone Robutti wrote: > I would like to join, if it's possible. Thanks :) > > 2016-05-24 14:55 GMT+02:00 Jean-Baptiste Onofré : > >> Good idea ! >> >> Thanks ! >> Regards >> JB >> >> >> On 05/24/2016 02:53 PM, Maximilian Michels wrote: >> >>> Thanks. I've also invited Aljoscha Krettek and Kostas Kloudas. >>> >>> On Tue, May 24, 2016 at 2:15 PM, Jean-Baptiste Onofré >>> wrote: >>> Hi Max, I just invited you. Regards JB On 05/24/2016 02:12 PM, Maximilian Michels wrote: > > +1 for Slack. > > @James Could you invite me? > > On Thu, May 19, 2016 at 9:24 PM, James Malone > wrote: > >> >> Hi all, >> >> It sounds like Slack is the clear winner here. So, I am happy to say >> that >> we now have our own Slack Team, open to all! >> >> http://apachebeam.slack.com >> >> Once I created the Slack team, it rejected the large blanket list of >> "acceptable email domains" I wanted to use (so signup is painless.) >> Instead, it looks like we'll have to use an invite system. I've already >> modified the team so anyone can invite anyone else (to make it easy to >> grow >> the Beam community.) But, we will need to manually invite some people >> to >> get this process started. >> >> If you'd like an invite today, can you please email me - >> jamesmal...@apache.org and I will invite you ASAP. >> >> Best, >> >> James >> >> On Thu, May 19, 2016 at 9:36 AM, Milindu Sanoj Kumarage < >> agentmili...@gmail.com> wrote: >> >> +1 for Slack. >>> On 19 May 2016 5:43 p.m., "GANESH RAJU" wrote: >>> >>> +1 on slack Ganesh Raju Sent from my iPhone On May 18, 2016, at 3:41 AM, Jean-Baptiste Onofré < j...@nanthrax.net> > wrote: > > > Good point Robert. > > I will be on the channel for sure (I'm already on bunch of Apache > IRC > channels ;)). > > > Regards > JB > > On 05/18/2016 10:26 AM, Robert Bradshaw wrote: >> The value in such a channel is highly dependent on people regularly >> being there--do we have a critical mass of developers that would
Re: 0.1.0-incubating release
Thanks for the clarification JB. In the projects I’ve been involved with, I’ve not seen that practice. As long as the resulting release ends up on dist.a.o I don’t think it’s a problem. -Taylor > On Jun 8, 2016, at 12:49 AM, Jean-Baptiste Onofré wrote: > > Hi Taylor, > > Just to be clearn, in most other projects, we stage the distributions on > repository. We upload the distro and signatures to dist.apache.org only when > the vote passed. > > Basically, the release process I talked with Davor (and that I will document) > is: > - Tag and stage using mvn release:prepare release:perform > - Close repo > - Start vote > - If passed, forward vote to incubator > - If passed, close repo > - Upload distro to dist > - Announce the release (mailing lists, website) > > It's based on what I do in Karaf, ServiceMix, etc. > > Regards > JB > > On 06/08/2016 02:39 AM, P. Taylor Goetz wrote: >> Out of curiosity, is there a reason for distributing the release on >> repository.a.o vs. dist.a.o? >> >> In my experience repository.a.o has traditionally been used for maven >> artifacts, and dist.a.o has been for release artifacts (source archives and >> convenience binaries). >> >> I'd be happy to help with documenting the process. >> >> I ask because this might come up during an IPMC release vote. >> >> -Taylor >> >>> On Jun 1, 2016, at 9:46 PM, Davor Bonaci wrote: >>> >>> Hi everyone! >>> We've started the release process for our first release, 0.1.0-incubating. >>> >>> To recap previous discussions, we don't have particular functional goals >>> for this release. Instead, we'd like to make available what's currently in >>> the repository, as well as work through the release process. >>> >>> With this in mind, we've: >>> * branched off the release branch [1] at master's commit 8485272, >>> * updated master to prepare for the second release, 0.2.0-incubating, >>> * built the first release candidate, RC1, and deployed it to a staging >>> repository [2]. >>> >>> We are not ready to start a vote just yet -- we've already identified a few >>> issues worth fixing. That said, I'd like to invite everybody to take a peek >>> and comment. I'm hoping we can address as many issues as possible before we >>> start the voting process. >>> >>> Please let us know if you see any issues. >>> >>> Thanks, >>> Davor >>> >>> [1] https://github.com/apache/incubator-beam/tree/release-0.1.0-incubating >>> [2] https://repository.apache.org/content/repositories/orgapachebeam-1000/ > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com signature.asc Description: Message signed with OpenPGP using GPGMail
Re: 0.1.0-incubating release
Out of curiosity, is there a reason for distributing the release on repository.a.o vs. dist.a.o? In my experience repository.a.o has traditionally been used for maven artifacts, and dist.a.o has been for release artifacts (source archives and convenience binaries). I'd be happy to help with documenting the process. I ask because this might come up during an IPMC release vote. -Taylor > On Jun 1, 2016, at 9:46 PM, Davor Bonaci wrote: > > Hi everyone! > We've started the release process for our first release, 0.1.0-incubating. > > To recap previous discussions, we don't have particular functional goals > for this release. Instead, we'd like to make available what's currently in > the repository, as well as work through the release process. > > With this in mind, we've: > * branched off the release branch [1] at master's commit 8485272, > * updated master to prepare for the second release, 0.2.0-incubating, > * built the first release candidate, RC1, and deployed it to a staging > repository [2]. > > We are not ready to start a vote just yet -- we've already identified a few > issues worth fixing. That said, I'd like to invite everybody to take a peek > and comment. I'm hoping we can address as many issues as possible before we > start the voting process. > > Please let us know if you see any issues. > > Thanks, > Davor > > [1] https://github.com/apache/incubator-beam/tree/release-0.1.0-incubating > [2] https://repository.apache.org/content/repositories/orgapachebeam-1000/
Re: Permission problems
Any of your mentors ought to be able to take care of this for you. When you need help from mentors, it sometimes helps to prepend “[MENTORS]” to the subject to get their attention. -Taylor > On Mar 4, 2016, at 4:17 PM, Davor Bonaci wrote: > > This is something that JB took care for us. I believe only a member of the > Incubator PMC can fix it [1]. > > I did a quick check for all committers -- Max, I think you are the only > committer without proper access right now. JB, is this something you could > take care of? > > [1] http://www.apache.org/dev/pmc.html#newcommitter > > On Fri, Mar 4, 2016 at 9:56 AM, Maximilian Michels wrote: > >> Hi Beamers, >> >> While working on migrating the Beam web site to Git, I came across >> problems with the LDAP permissions. According to Infra, I'm not part >> of the incubator group [1]. >> >> Now that we want to merge the Flink Runner [2], I think I'll be unable >> to merge any valuable changes to the repository. If we want to move >> the main development of the runner to the Beam repository, we need to >> fix that. >> >> I know we're just getting started but it would be great to setup >> permissions properly. >> >> Thanks, >> Max >> >> [1] >> https://issues.apache.org/jira/browse/INFRA-11318?focusedCommentId=15174230&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15174230 >> >> [2] https://github.com/apache/incubator-beam/pull/12 >> signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Apache Storm Runner for Beam?
Good question, and not clear from the proposal. Probably a good first thing (among many) for the podling community to figure out... -Taylor > On Feb 10, 2016, at 8:11 PM, Henry Saputra wrote: > > Hi Frances, > > Just want to make sure, the plan is to make existing Runners to be brought > in into Bean source code repository as internal modules, right? > > Thanks, > > Henry > > On Wed, Feb 10, 2016 at 4:59 PM, Frances Perry > wrote: > >> One of the initial things we are focusing on as we move things to Beam is >> cleaning up the API that new runners will implement. (The current version >> is neither particularly stable nor clean.) But as long as you are >> comfortable with a little churn, feel free to start poking around in the >> code that will be seeding the repository (after some in-progress >> refactoring): >> >> * Dataflow Java SDK: >> https://github.com/GoogleCloudPlatform/DataflowJavaSDK (includes >> the user-level API, DirectRunner, and DataflowRunner) >> * Flink runner: https://github.com/dataArtisans/flink-dataflow >> * Spark runner: https://github.com/cloudera/spark-dataflow >> >> Gathering together folks that are interested in coordinating on various >> runners is a great idea. As we get the repo structured -- we'll make sure >> that that's easy to do. >> >> Frances >> >>> On Wed, Feb 10, 2016 at 4:50 PM, Yang, Connie wrote: >>> >>> Hi Taylor, >>> >>> Good news indeed that the upcoming Storm 1.0 release will include sliding >>> and tumble windowing. >>> >>> Subash and I would definitely interested in joining Storm running design >>> and implementation. Is there an ongoing design discussion that we can be >>> part of? Is there a development branch we can take a look? >>> >>> Thanks >>> Connie >>> >>>> On 2/10/16, 2:55 PM, "P. Taylor Goetz" wrote: >>>> >>>> Hi Connie, >>>> >>>> I've definitely considered it (no implementation yet, just thoughts), >> and >>>> would like to start coordinating such an effort. That's one of the >>>> reasons I wanted to get involved with Beam. >>>> >>>> A Storm runner would likely/definitely depend on features in the >> upcoming >>>> 1.0 release (namely MillWheel style windowing). That release is >> currently >>>> stabilizing, but should be out soon. >>>> >>>> The question I have for the Beam community would be where/how to host >> the >>>> collaboration effort: Storm, Beam, or a combination? >>>> >>>> I'm happy to facilitate with the Storm community/PMC. >>>> >>>> -Taylor >>>> >>>>> On Feb 9, 2016, at 2:07 PM, Yang, Connie wrote: >>>>> >>>>> Hi Beamers, >>>>> >>>>> We¹re considering adding an Apache Storm Runner to Apache Beam. >>>>> >>>>> * Has anyone in the community considered this and started working >> on >>>>> this? >>>>> * If so, what¹s the current state? >>>>> >>>>> We would like to join the design discussion and development of it if >>>>> it¹s in flight. >>>>> >>>>> Thanks >>>>> Connie >>
Re: Apache Storm Runner for Beam?
Hi Connie, I've definitely considered it (no implementation yet, just thoughts), and would like to start coordinating such an effort. That's one of the reasons I wanted to get involved with Beam. A Storm runner would likely/definitely depend on features in the upcoming 1.0 release (namely MillWheel style windowing). That release is currently stabilizing, but should be out soon. The question I have for the Beam community would be where/how to host the collaboration effort: Storm, Beam, or a combination? I'm happy to facilitate with the Storm community/PMC. -Taylor > On Feb 9, 2016, at 2:07 PM, Yang, Connie wrote: > > Hi Beamers, > > We’re considering adding an Apache Storm Runner to Apache Beam. > > * Has anyone in the community considered this and started working on this? > * If so, what’s the current state? > > We would like to join the design discussion and development of it if it’s in > flight. > > Thanks > Connie
