Re: Configuring IntelliJ to enforce checkstyle rules

[Seetharam Venkatesh]

You can also export idea settings and interested developers can import it
instead.

Adding check style slows down idea significantly in my experience but ymmv.

On Wed, Aug 24, 2016 at 8:29 AM Kenneth Knowles 
wrote:

> Nice step-by-step.
>
> +1 to adding tips for particular IDEs in the contribution guide.
>
> On Wed, Aug 24, 2016 at 7:48 AM, Jean-Baptiste Onofré 
> wrote:
>
> > Hi Stas,
> >
> > Thanks for sharing !
> >
> > As discussed with Amit on Hangout (and indirectly with you ;)), it's what
> > I'm using in my config.
> >
> > Some stuff that I added on IntelliJ:
> > - in Editor -> Code Style -> Java -> Tabs and Indents, I disabled "Use
> tab
> > character", and defined 2 for "Tab Size" and "Indent", and 4 for
> > "Continuation indent".
> > - in Editor -> Code Style -> Java -> Imports, I changed the layout to
> > match the checkstyle definition (static first, ,
> > org.apache.beam.*, , com.google.*, ...)
> > - in Editor -> Code Style -> Java -> Wrapping and Braces, I enabled
> > "Ensure right margin is not exceeded"
> >
> > I also enabled checkstyle in code inspection, and the checkstyle button
> > (next to the terminal button in the button bar).
> >
> > Related to the discussion about checkstyle, I think it makes sense to add
> > a section about "Configuring IDE" in the contribution guide.
> >
> > WDYT ?
> >
> > Regards
> > JB
> >
> >
> > On 08/24/2016 04:35 PM, Stas Levin wrote:
> >
> >> Hi guys,
> >>
> >> Having IntelliJ enforce Beam's Checkstyle rules turned out to be very
> >> useful for me, so I figured I'd share the steps just in case.
> >>
> >>1. Install the Checkstyle plugin
> >>   1. Select the *Plugins* menu from the *Preferences* (Cmd+",")
> >>   2. Click "*Browse Repositories*"
> >>   3. Type "*Checkstyle-IDEA*" in the search box and select the
> search
> >>   result
> >>   4. Click "*install*"
> >>   5. *Restart* Intellij
> >>2. Activate Beam's checkstyle rules
> >>   1. Select the newly added "*Checkstyle*" menu from the Preferences
> >>   (Cmd+",") menu
> >>   2. Hit the "*+*" icon to add a checkstyle file, *select Beam's
> >>   checkstyle.xm*l
> >>   "..incubator-beam/sdks/java/build-tools/src/main/resources/
> >> beam/checkstyle.xml"
> >>   3. Check the "*Active*" checkbox to activate
> >>3. Done!
> >>
> >> Hope you find this useful.
> >>
> >> Regards,
> >> Stas
> >>
> >>
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>

Re: 0.1.0-incubating release

[Seetharam Venkatesh]

Aljoscha, if you want to be sure and want only one RC like me, I'd suggest
you search general incubator mail archive and look for comments from Justin
& Sebb - they are very thorough and will give you a headstart instead of
iterating multiple times.

On Tue, Jun 7, 2016 at 12:59 PM Jean-Baptiste Onofré 
wrote:

> Hi Aljoscha
>
> It's basically here:
> http://incubator.apache.org/guides/releasemanagement.html
>
> The checklist is interesting to check the release content:
>
> http://incubator.apache.org/guides/releasemanagement.html#check-list
>
> Regards
> JB
>
> On 06/07/2016 09:56 PM, Aljoscha Krettek wrote:
> > By the way, is there any document where we keep track of what checks to
> run
> > for a release? Maybe I missed something, there.
> >
> > On Tue, 7 Jun 2016 at 21:29 Jean-Baptiste Onofré 
> wrote:
> >
> >> Just submitted: https://github.com/apache/incubator-beam/pull/428
> >>
> >> to fix the src distribution content.
> >>
> >> Regards
> >> JB
> >>
> >> On 06/07/2016 09:26 PM, Jean-Baptiste Onofré wrote:
> >>> I have to revert my vote to -1:
> >>>
> >>> the source distribution zip file is empty.
> >>>
> >>> I gonna submit a new PR to fix that.
> >>>
> >>> Sorry about that.
> >>>
> >>> Regards
> >>> JB
> >>>
> >>> On 06/07/2016 09:12 PM, Jean-Baptiste Onofré wrote:
>  +1
> 
>  it looks good to me:
> 
>  - all files have incubating
>  - signatures check out (asc, md5, sha1) (and KEYS there)
>  - disclaimer exists
>  - LICENSE and NOTICE good
>  - No unexpected binary in source
>  - All ASF licensed files have ASF headers
>  - Source distribution is available, with a correct name, and correct
>  content:
> 
> 
> >>
> https://repository.apache.org/content/repositories/orgapachebeam-1001/org/apache/beam/apache-beam/0.1.0-incubating/apache-beam-0.1.0-incubating-src.zip
> 
> 
>  I'm more comfortable to move forward on a formal release vote with
> this
>  staging, and forward to the IPMC review.
> 
>  Thanks all and especially to Davor (to support me when I bother him
>  bunch of times a day ;)).
> 
>  Regards
>  JB
> 
>  On 06/07/2016 08:58 PM, Davor Bonaci wrote:
> > The second release candidate is available for everyone's review [1].
> >
> > We plan to call for a vote shortly; please comment if there's any
> > additional feedback.
> >
> > [1]
> >
> https://repository.apache.org/content/repositories/orgapachebeam-1001
> >
> > On Tue, Jun 7, 2016 at 9:33 AM, Kenneth Knowles
>  >>>
> > wrote:
> >
> >> +1
> >>
> >> Lovely. Very readable.
> >>
> >> The "-parent" artifacts are just leaked implementation details of
> our
> >> build
> >> configuration that no one should ever actually reference, right?
> >>
> >> Kenn
> >>
> >> On Tue, Jun 7, 2016 at 8:54 AM, Dan Halperin
> >> 
> >> wrote:
> >>
> >>> +2! This seems most concordant with other Apache products and the
> >> most
> >>> future-proof.
> >>>
> >>> On Mon, Jun 6, 2016 at 9:35 PM, Jean-Baptiste Onofré <
> >> j...@nanthrax.net>
> >>> wrote:
> >>>
>  +1
> 
>  Regards
>  JB
> 
> 
>  On 06/07/2016 02:49 AM, Davor Bonaci wrote:
> 
> > After a few rounds of discussions and examining patterns of other
> > projects,
> > I think we are converging towards:
> >
> > * A flat group structure, where all artifacts belong to the
> > org.apache.beam
> > group.
> > * Prefix all artifact ids with "beam-".
> > * Name artifacts according to the existing directory/module
> layout:
> > beam-sdks-java-core, beam-runners-google-cloud-dataflow-java,
> etc.
> > * Suffix all parents with "-parent", e.g., "beam-parent",
> > "sdks-java-parent", etc.
> > * Create a "distributions" module, for the purpose of packaging
> the
> >>> source
> > code for the ASF release.
> >
> > I believe this approach takes into account everybody's feedback
> so
> >> far,
> > and
> > I think opposing positions have been retracted.
> >
> > Please comment if that's not the case, or if there are any
> > additional
> > points that we may have missed. If not, this is implemented in
> > pending
> > pull
> > requests #420 and #423.
> >
> > Thanks!
> >
> > On Fri, Jun 3, 2016 at 9:59 AM, Thomas Weise
> > 
> > wrote:
> >
> > Another consideration for potential future packaging/distribution
> >> solutions
> >> is how the artifacts line up as files in a flat directory. For
> >> that
> >> it
> >> may
> >> be good to 

One more streaming engine in OSS

[Seetharam Venkatesh]

https://blog.twitter.com/2016/open-sourcing-twitter-heron

More the merrier for Beam? :-)

Venkatesh

Re: [DISCUSS] Developing new components -- branches, maturity, and committers

[Seetharam Venkatesh]

+1, this is a step in the right direction.

On Wed, May 18, 2016 at 10:02 PM Frances Perry 
wrote:

> Hi Beamers --
>
> I’m thrilled by the recent energy and activity on writing new Beam runners!
> But that also means it’s probably time for us to figure out how, as a
> community, we want to support this process. ;-)
>
> Back near the beginning, we had a thread [1] discussing that feature
> branches are the preferred way of doing development of features or
> components that may take a while to reach maturity. I think new components
> like runners and SDKs meet the bar to be started from a feature branch.
> (Other features, like an IO connector or library of PTransforms, might also
> qualify depending on complexity.)
>
> We should also lay out what it takes to be considered mature enough to be
> merged into master, since once that happens the component gets released to
> users and failing tests become blocking issues. Here are some initial
> thoughts to kick off the discussion...
>
> In order to be merged into master, new components / major features should:
>
>-
>
>have at least 2 contributors interested in maintaining it, and 1
>committer interested in supporting it
>-
>
>provide both end-user and developer-facing documentation
>-
>
>have at least a basic level of unit test coverage
>-
>
>run all existing applicable integration tests with other Beam components
>and create additional tests as appropriate
>
>
> In addition...
>
> A runner should:
>
>-
>
>be able to handle a subset of the model that address a significant set
>of use cases (aka. ‘traditional batch’ or ‘processing time streaming’)
>-
>
>update the capability matrix with the current status
>
>
> An SDK* should:
>
>-
>
>provide the ability to construct graphs with all the basic building
>blocks of the model (ParDo, GroupByKey, Window, Trigger, etc)
>-
>
>begin fleshing out the common composite transforms (Count, Join, etc)
>and IO connectors (Text, Kafka, etc)
>-
>
>have at least one runner that can execute the complete model (may be a
>direct runner)
>-
>
>provide integration tests for executing against current and future
>runners
>
>
> * A note on DSLs:  I think it’s important to separate out an SDK from a
> DSL, because in my mind the former is by definition equivalent to the Beam
> model, while the latter may select portions of the model or change the
> user-visible abstractions in order to provide a domain-specific experience.
> We may want to encourage some DSLs to live separately from Beam because
> they may look completely non-Beam-like to their end users. But we can
> probably punt this decision until we have concrete examples to discuss.
>
> Another fun part of this growth is that we’ll likely grow new committers.
> And given the breadth of Beam, I think it would be useful to annotate our
> committers [2] page with which components folks are the most knowledgeable
> about.
>
> Looking forward to your thoughts.
>
> [1]
>
> http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201602.mbox/%3CCAAzyFAymVNpjQgZdz2BoMknnE3H9rYRbdnUemamt9Pavw8ugsw%40mail.gmail.com%3E
>
> [2] http://beam.incubator.apache.org/team/
>

Re: Podling Report Reminder - May 2016

[Seetharam Venkatesh]

JB, I think you need to give folks a chance to participate and learn the
apache way. :-)

On Sun, Apr 24, 2016 at 11:57 PM Jean-Baptiste Onofré 
wrote:

>
>
> Hi
> I already prepared a report draft as I requested to report this month. I
> will send on the dev mailing list later today.
> RegardsJB
>
>  Original message 
> From: James Malone 
> Date: 25/04/2016  05:11  (GMT+00:00)
> To: dev@beam.incubator.apache.org
> Subject: Re: Podling Report Reminder - May 2016
>
> Hi everyone!
>
> I'd like to volunteer myself to take this report on and help out with these
> type of reports. To that end, I want to ask our project champion, JB, for
> his assistance to do so. That will help make sure we can deliver exactly
> what is needed now and in the future.
>
> JB -
>
> Since you've been involved with many Apache projects in the past I think
> you might have a lot of insight into this process. I'd also like to
> volunteer myself to help here as well. Might I ask if I can steal some of
> your time to understand the process and craft this report?
>
> Thanks!
>
> James
>
> On Sun, Apr 24, 2016 at 6:59 AM,  wrote:
>
> > Dear podling,
> >
> > This email was sent by an automated system on behalf of the Apache
> > Incubator PMC. It is an initial reminder to give you plenty of time to
> > prepare your quarterly board report.
> >
> > The board meeting is scheduled for Wed, 18 May 2016, 10:30 am PDT.
> > The report for your podling will form a part of the Incubator PMC
> > report. The Incubator PMC requires your report to be submitted 2 weeks
> > before the board meeting, to allow sufficient time for review and
> > submission (Wed, May 4th).
> >
> > Please submit your report with sufficient time to allow the Incubator
> > PMC, and subsequently board members to review and digest. Again, the
> > very latest you should submit your report is 2 weeks prior to the board
> > meeting.
> >
> > Thanks,
> >
> > The Apache Incubator PMC
> >
> > Submitting your Report
> >
> > --
> >
> > Your report should contain the following:
> >
> > *   Your project name
> > *   A brief description of your project, which assumes no knowledge of
> > the project or necessarily of its field
> > *   A list of the three most important issues to address in the move
> > towards graduation.
> > *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> > aware of
> > *   How has the community developed since the last report
> > *   How has the project developed since the last report.
> >
> > This should be appended to the Incubator Wiki page at:
> >
> > http://wiki.apache.org/incubator/May2016
> >
> > Note: This is manually populated. You may need to wait a little before
> > this page is created from a template.
> >
> > Mentors
> > ---
> >
> > Mentors should review reports for their project(s) and sign them off on
> > the Incubator wiki page. Signing off reports shows that you are
> > following the project - projects that are not signed may raise alarms
> > for the Incubator PMC.
> >
> > Incubator PMC
> >
>