Re: "retest this please" no longer working on the beam site repo

JB, did you ever hear back from INFRA? This came up again on another PR: https://github.com/apache/beam/pull/5800#issuecomment-400866581 On Thu, Jun 21, 2018 at 10:39 AM Lukasz Cwik wrote: > I have found that it is significantly delayed, takes 10's of mins before > Jenkins recognizes the PR

Re: [DISCUSS] Automation for Java code formatting

So, there are a few modes of checkstyle failure that can be induced by this: - lines get too long because of wrap & indent logic - a lot of our HTML is actually already broken and doesn't have tags where it should; spotless adds a blank line which makes checkstyle notice the errors I think

Re: [DISCUSS] Automation for Java code formatting

+1! Given this is the recommended default for Go projects (with it's own gofmt tool) I'm for similar tooling for this in other languages. I suspect we can add Gradle command to run gofmt over the go code too for a consistent hook to run for beam code. It would save folks from needing to set up

Re: Filtered Pre-commit triggering is BACK!

Two of my PRs with python changes did not trigger any pre-commits. Could it be related to this change? https://github.com/apache/beam/pull/5768 https://github.com/apache/beam/pull/5800 Ahmet On Tue, Jun 26, 2018 at 2:30 PM, Andrew Pilloud wrote: > Awesome! This will save so much time

Re: [DISCUSS] Automation for Java code formatting

+1 I already used it in SQL module and it's great. It makes sense to apply on all modules. Thanks ! Regards JB Le 27 juin 2018 à 12:15, à 12:15, Kenneth Knowles a écrit: >Hi all, > >I like readable code, but I don't like formatting it myself. And I >_really_ >don't like discussing in code

Re: [DISCUSS] Automation for Java code formatting

OK, I opened https://github.com/apache/beam/pull/5797. This thread is still less than a day, so there's no commitment yet. There's no big hurry - I can always discard the autoformat and rerun it. I'm sure there will always be conflicts so I will just have to rerun it and merge at some quiet period

Re: [Design Proposal] Improving Beam code review

Thanks for writing this up! I especially like the idea of automatically assigning code reviewers, e.g. via https://help.github.com/articles/about-codeowners/ On Wed, Jun 27, 2018 at 11:10 AM Scott Wegner wrote: > > Thanks for putting together this proposal Huygaa. Overall looks good to me; I >

Re: Beam Dependency Ownership

It's mentioned under "Dependency declarations may identify owners that are responsible for upgrading respective dependencies". Feel free to update if you think more details should be added to it. I think it'll be easier if we transfer data in spreadsheet to comments close to dependency

Re: Beam Dependency Ownership

Thanks Scott, I will supplement the missing packages to the spreadsheet. And, we expect this being kept up to date along with the Beam project growth. Shall we mention this in the Dependency Guide page , @Chamikara Jayalath ? On Wed, Jun 27, 2018

Re: Using user developped source in streamline python

It seems like a nice opportunity also to contribute them to the project in case you are able to work on them. Don't hesitate to contact us or ask for help if needed. On Wed, Jun 27, 2018 at 4:45 PM Ahmet Altay wrote: > Hi Sébastien, > > Currently there is no work in progress for including the

Re: Beam Dependency Ownership

Thanks Yifan. I added myself to some of the dependencies I'm involved with upgrading. In case anyone miss, there's a second tab for Python SDK :). On Wed, Jun 27, 2018 at 11:17 AM Scott Wegner wrote: > Thanks for kicking off this process Yifan-- I'll add my name to some > dependencies I'm

Re: Unbounded source translation for portable pipelines

It would be great to have the ValidatesRunner suite of tests start executing against Flink/ULR as it will make sure things don't break and are reproducible. On Wed, Jun 27, 2018 at 12:34 PM Eugene Kirpichov wrote: > Hi! > > Those instructions are not current and I think should be discarded as

Re: Unbounded source translation for portable pipelines

Hi! Those instructions are not current and I think should be discarded as they referred to a particular effort that is over - +Ankur Goenka is, I believe, working on the remaining finishing touches for running from a clean clone of Beam master and documenting how to do that; could you help

Re: ErrorProne and -Werror enabled for all Java projects

This is awesome, thanks to everybody involved! It's so good to have ./gradlew compileJava compileTestJava not produce heaps of warnings like it used to. On Wed, Jun 27, 2018 at 9:52 AM Andrew Pilloud wrote: > Looking at the diff I think you can replace "Default Setting" with "Only > Setting".

Re: [DISCUSS] Automation for Java code formatting

+1 I'll throw in my support for auto-formatting, especially if the entire project is auto-formatted in advance. On Wed, Jun 27, 2018 at 10:53 AM Huygaa Batsaikhan wrote: > +1. Global auto-formatting is cool! > > On Wed, Jun 27, 2018 at 10:17 AM Kenneth Knowles wrote: > >> I just mean that

Re: Jenkins Groovy files naming convention

The Jenkins source files are naming according to the Jenkins jobs they produce, which are named with Snake_Case. However this doesn't seem like a requirement, and I'm all in favor of keeping things and idiomatic as possible. So +1 to CamelCase.groovy FYI, the official Groovy style guide is here:

Re: [DISCUSS] Automation for Java code formatting

I just mean that because of how the tool works. But I guess if there were discretion then two different people could end up with autoformatting that disagrees, so again you get lines in the PR diff that aren't real changes. Kenn On Wed, Jun 27, 2018 at 10:16 AM Raghu Angadi wrote: > On Wed,

Re: [DISCUSS] Remove findbugs from sdks/java

Last time I checked, errorprone didn't enforce nullness typing. Does it now? IMO that's the only P0 in our static analysis toolchain. Kenn On Wed, Jun 27, 2018 at 9:52 AM Scott Wegner wrote: > FYI, now that the ErrorProne migration is complete [1] it's a good time to > discuss whether FindBugs

Re: [DISCUSS] Automation for Java code formatting

On Wed, Jun 27, 2018 at 10:13 AM Kenneth Knowles wrote: > Nope! No discretion allowed :-) > +1. Fair enough! > > On Wed, Jun 27, 2018 at 9:57 AM Raghu Angadi wrote: > >> +1. >> >> Wondering if it can be configured to reformat only what we care most >> about (2 space indentation etc),

Re: [DISCUSS] Automation for Java code formatting

Nope! No discretion allowed :-) On Wed, Jun 27, 2018 at 9:57 AM Raghu Angadi wrote: > +1. > > Wondering if it can be configured to reformat only what we care most about > (2 space indentation etc), allowing some discretion on the edges. An > example of inconsistent formatting that ends up in my

Re: [DISCUSS] Automation for Java code formatting

+1. Wondering if it can be configured to reformat only what we care most about (2 space indentation etc), allowing some discretion on the edges. An example of inconsistent formatting that ends up in my code: --- anObject.someLongMethodName(arg_number_1,

Re: ErrorProne and -Werror enabled for all Java projects

Looking at the diff I think you can replace "Default Setting" with "Only Setting". This is Awesome! Thanks guys! Andrew On Wed, Jun 27, 2018 at 9:50 AM Kenneth Knowles wrote: > Awesome! Can we remove the ability to disable it? :-) :-) :-) or anyhow > make it more obscure, not like an expected

Re: [DISCUSS] Remove findbugs from sdks/java

FYI, now that the ErrorProne migration is complete [1] it's a good time to discuss whether FindBugs should be removed or not. I haven't compared the set of checks we are doing with FindBugs vs ErrorProne. If ErrorProne can replace the checks from FindBugs then I'm in favor of getting rid of it.

Re: ErrorProne and -Werror enabled for all Java projects

Awesome! Can we remove the ability to disable it? :-) :-) :-) or anyhow make it more obscure, not like an expected top-level config choice. Kenn On Wed, Jun 27, 2018 at 9:45 AM Tim wrote: > Thanks also to you Scott > > Tim > > On 27 Jun 2018, at 18:39, Scott Wegner wrote: > > Six weeks ago

Re: ErrorProne and -Werror enabled for all Java projects

Thanks also to you Scott Tim > On 27 Jun 2018, at 18:39, Scott Wegner wrote: > > Six weeks ago [1] we began an effort to improve the quality of the Java > codebase via ErrorProne static analysis, and promoting compiler warnings to > errors. As of today, all of our Java projects have been

Re: Jenkins emails

The failure emails are so frequent that they've just become noise to me. I'm +1 for not mailing individuals. I would also be in favor of sending them to commits@ instead of dev@. Andrew On Wed, Jun 27, 2018 at 8:38 AM Kenneth Knowles wrote: > I think it is just a matter of consensus. I have

Re: [DISCUSS] Automation for Java code formatting

It wasn't clear to me that the intent was to autoformat all the code from the proposal initially. If thats the case, then the delta is quite small typically. Also, it would be easier if we recommended to users to run run "./gradlew spotlessApply" which will run spotless on all modules. On Wed,

ErrorProne and -Werror enabled for all Java projects

Six weeks ago [1] we began an effort to improve the quality of the Java codebase via ErrorProne static analysis, and promoting compiler warnings to errors. As of today, all of our Java projects have been migrated and this is now the default setting for Beam [2]. This was a community effort. The

Re: [DISCUSS] Automation for Java code formatting

On Wed, Jun 27, 2018 at 9:31 AM Kenneth Knowles wrote: > Luke: the proposal here solves exactly what you are talking about. > > The problem you describe happens when the PR author uses autoformat but > the baseline is not already autoformatted. What I am proposing is to make > sure the baseline

Re: [DISCUSS] Automation for Java code formatting

Luke: the proposal here solves exactly what you are talking about. The problem you describe happens when the PR author uses autoformat but the baseline is not already autoformatted. What I am proposing is to make sure the baseline is already autoformatted, so PRs never have extraneous formatting

Re: [DISCUSS] Automation for Java code formatting

I agree that PRs with formatting changes are impossible to review, but I think that is exactly what this is trying to avoid. If this is run nightly, some contributors will still autoformat their PRs and have a bunch of unrelated changes in the diff. If the code is always well formatted (and that

Re: Jenkins Groovy files naming convention

SGTM. The underscores are more-or-less substituting for directories. Maybe we can just put them in directories? On Wed, Jun 27, 2018 at 9:18 AM Lukasz Cwik wrote: > I don't really have a strong preference. > > On Wed, Jun 27, 2018 at 9:13 AM Łukasz Gajowy > wrote: > >> Hi all, >> >> I think we

Re: Jenkins Groovy files naming convention

+1 CamelCase. Feels "normal" to me in Groovy. On Wed, Jun 27, 2018 at 9:18 AM Lukasz Cwik wrote: > I don't really have a strong preference. > > On Wed, Jun 27, 2018 at 9:13 AM Łukasz Gajowy > wrote: > >> Hi all, >> >> I think we should change the naming convention that we have in >> jenkins

Re: [DISCUSS] Automation for Java code formatting

Luke: Anything that helps contributors and reviewers work better together - +1! :D On Wed, Jun 27, 2018 at 9:04 AM Lukasz Cwik wrote: > If spotless is run against a PR that is already well formatted its a > non-issue as the formatting changes are usually related to the change but I > have

Re: Jenkins Groovy files naming convention

I don't really have a strong preference. On Wed, Jun 27, 2018 at 9:13 AM Łukasz Gajowy wrote: > Hi all, > > I think we should change the naming convention that we have in > jenkins .groovy files. AFAIK, groovy is CamelCase, and we use snake_case > names there. I suppose this is because we

Jenkins Groovy files naming convention

Hi all, I think we should change the naming convention that we have in jenkins .groovy files. AFAIK, groovy is CamelCase, and we use snake_case names there. I suppose this is because we wanted to reflect jenkins job names (do we need this?) IMO, the convention should be CamelCase for all .groovy

Re: [DISCUSS] Automation for Java code formatting

If spotless is run against a PR that is already well formatted its a non-issue as the formatting changes are usually related to the change but I have reviewed a few PRs that have 100s of lines of formatting change which really obfuscates the work. Instead of asking contributors to run spotless,

Beam Dependency Ownership

Hi all, We now have the automated detections for Beam dependency updates and sending a weekly report to dev mailing list. In order to address the updates in time, we want to find owners for all dependencies of Beam, and finally, Jira bugs will be automatically created and assigned to the owners

Re: Jenkins emails

I think it is just a matter of consensus. I have opened https://github.com/apache/beam/pull/5784 to make it off, and then we can go through the individual job definitions to remove the parameter entirely. Two more reasons to remove this email config: - as job count scales up, email is just not

Re: Jenkins emails

Thanks for the explanation, Kenn, I appreciate. Indeed my commits were on the website, so probably that's it. I presume not, but can I do something to facilitate the process? Best, Stefano ⁣Sent from Blue ​ On Jun 27, 2018, 17:23, at 17:23, Kenneth Knowles wrote: >First let me say what I

Re: Jenkins emails

First let me say what I think the cause is: - we have most jobs set to "email people who broke the build" - this scrapes what Jenkins thinks all the changes since last time are, and for every registered address, sends email - when we create a new job with these settings, *all* changes are

Jenkins emails

Hi all, I've contributed a couple of lines of code a couple of years back. Ever since a week, I started receiving email notification from Jenkins. How do I unsubscribe from those? I don't have an account on builds.apache.org. Best, Stefano

Jenkins build is back to normal : beam_SeedJob_Standalone #1219

See

Re: [DISCUSS] Automation for Java code formatting

Good points, Dan. Checkstyle will still run, but just focused on the things that go beyond format. Kenn On Wed, Jun 27, 2018 at 8:03 AM Etienne Chauchot wrote: > +1 ! > It's my custom to avoid reformatting to spare meaningless diff burden to > the reviewer. Now it will be over, thanks. > >

Re: [DISCUSS] Automation for Java code formatting

+1 !It's my custom to avoid reformatting to spare meaningless diff burden to the reviewer. Now it will be over, thanks. Etienne Le mardi 26 juin 2018 à 21:15 -0700, Kenneth Knowles a écrit : > Hi all, > I like readable code, but I don't like formatting it myself. And I _really_ > don't like

Re: Going on leave for a bit

Congrats Kenn ! Spend some good time with your family. Etienne Le lundi 25 juin 2018 à 22:42 -0700, Kenneth Knowles a écrit : > Hi friends, > I think I did not mention on dev@ at the time, but my child #2 arrived March > 14 (Pi day!) and I took some weeks off. > Starting ~July 4 I will be taking

Jenkins build is back to normal : beam_SeedJob #2093

See

Re: Using user developped source in streamline python

Hi Sébastien, Currently there is no work in progress for including the write transforms for the locations you listed. You could develop your own version if interested. Please see WriteToBigquery transform [1] for reference. Ahmet [1]

Re: [DISCUSS] Automation for Java code formatting

I’m +1 to the idea of using something to handle the auto-formatting of code. HOWEVER, I want to make it clear that this will likely NOT remove checkstyle from the builds as there are things that checkstyle checks that aren’t part of this. Variable names, type names, javadoc things, etc….

Build failed in Jenkins: beam_SeedJob #2092

See -- GitHub pull request #4976 of commit c1d933b616b2067743a9245f11515a7c51b1be69, no merge conflicts. Setting status of c1d933b616b2067743a9245f11515a7c51b1be69 to PENDING with url

Re: [DISCUSS] Automation for Java code formatting

++1 - This solves the big mismatch of autoformatting using eclipse's google java style and the google-java-format plugin that makes a common source of unneeded diffs part of the reviews. - The spotless plugin makes this trivial to do and uniform. We need to update the instructions on formatting

Jenkins build is back to normal : beam_SeedJob #2091

See

Re: Unbounded source translation for portable pipelines

Hi Eugene, The basic streaming translation is already in place from the prototype, though I have not verified it on the master branch yet. Are the user instructions for the portable Flink runner at https://s.apache.org/beam-portability-team-doc current? (I don't have a dependency on SDF since

Re: [DISCUSS] Automation for Java code formatting

+1 to automating this with Gradle and never have problems with formatting/forgetting about it again! :) śr., 27 cze 2018 o 07:40 Tim Robertson napisał(a): > ++1 > > > On Wed, Jun 27, 2018 at 7:36 AM, Ahmet Altay wrote: > >> +1 >> >> This is great idea. Does anyone know a similar tool for

Build failed in Jenkins: beam_SeedJob_Standalone #1218

See Changes: [xiliu] [BEAM-4640] Samza runner postcommit ValidatesRunner job [xiliu] Remove unused previous names -- Started by timer [EnvInject] - Loading node

Re: Using user developped source in streamline python

Hi, Thanks for your answer. Ok looking forward and ready to test alpha on this. Because we have actually some use cases to send data to CloudSQL or Spanner or BigTable or Firestore. As far as I read the documentation, there is no native support for them and so we have already implemented a

Re: Going on leave for a bit

Congrats and enjoy! :) Best regards, Łukasz Gajowy wt., 26 cze 2018 o 20:33 Mikhail Gryzykhin napisał(a): > Enjoy time off. > > --Mikhail > > Have feedback ? > > > On Tue, Jun 26, 2018 at 11:24 AM Xinyu Liu wrote: > >> Congrats! Enjoy the time without sleep. >> >>

Build failed in Jenkins: beam_SeedJob #2090

See Changes: [xiliu] [BEAM-4640] Samza runner postcommit ValidatesRunner job [xiliu] Remove unused previous names [amaliujia] replace byte[] with String as Map key. [scott] Update Findbugs annotations dependency

Re: Auto closing stale PRs label

Glad to see it is working. The requests currently marked stale can be found with https://github.com/apache/beam/labels/stale On Tue, Jun 26, 2018 at 9:34 PM Rafael Fernandez wrote: > Neat! Thanks for showing me where the options are. > > On Tue, Jun 26, 2018 at 7:24 PM Kenneth Knowles wrote: >