FYI I've been trying to merge pr/7545 to fix things. However a number of flaky Beam tests are making it slow to get a green run.
On Fri, Jan 18, 2019 at 12:56 PM Kenneth Knowles <k...@google.com> wrote: > What I mean is that every Gradle project by virtue of having the Java (or > other) plugin has explicit lists of sources. > > FWIW I traced it to here: > https://github.com/diffplug/spotless/blob/master/plugin-gradle/src/main/java/com/diffplug/gradle/spotless/SpotlessTask.java#L194 > which eventually gets its input files from the Java plugin here: > https://github.com/diffplug/spotless/blob/a67fcf3a9df061774a3a646e44ffbae55ead64e5/plugin-gradle/src/main/java/com/diffplug/gradle/spotless/JavaExtension.java#L184 > > These are further filtered by the include/exclude. Caveat: I'm not really > an expert in Gradle plugin authoring or exeuction. > > Kenn > > On Fri, Jan 18, 2019 at 12:04 PM Reuven Lax <re...@google.com> wrote: > >> No includes _or_ excludes are specified in our spotless config, which >> implies that spotless should be scraping everything. >> >> On Fri, Jan 18, 2019 at 12:00 PM Kenneth Knowles <k...@google.com> wrote: >> >>> We have many other generated sources that would not pass spotless. It >>> sounds like the Antlr-generated sources are ending up in a source set that >>> spotless runs over. I would assume the Gradle plugin uses those, not a >>> filesystem scrape, to find the files it should process. Worth checking. >>> >>> Kenn >>> >>> On Fri, Jan 18, 2019 at 11:51 AM Alan Myrvold <amyrv...@google.com> >>> wrote: >>> >>>> The Jenkins test does a fresh clone of the repo, without generating >>>> code before the spotless test. >>>> >>>> On Fri, Jan 18, 2019 at 11:41 AM Kenneth Knowles <k...@apache.org> >>>> wrote: >>>> >>>>> Those are the paths that cause the Jenkins job to be run. It doesn't >>>>> affect the Gradle task. >>>>> >>>>> Kenn >>>>> >>>>> On Fri, Jan 18, 2019 at 11:34 AM Reuven Lax <re...@google.com> wrote: >>>>> >>>>>> FYI, Jenkins works because it explicitly specifies which paths to run >>>>>> spotless on, as below. As a result, Jenkins (correctly) does not run >>>>>> spotless on generated src. >>>>>> >>>>>> PrecommitJobBuilder builder = new PrecommitJobBuilder( >>>>>> scope: this, >>>>>> nameBase: 'Spotless', >>>>>> gradleTask: 'spotlessCheck', >>>>>> triggerPathPatterns: [ >>>>>> '^buildSrc/.*$', >>>>>> '^sdks/java/.*$', >>>>>> '^runners/.*$', >>>>>> '^examples/java/.*$', >>>>>> ] >>>>>> ) >>>>>> >>>>>> >>>>>> On Fri, Jan 18, 2019 at 10:08 AM Reuven Lax <re...@google.com> wrote: >>>>>> >>>>>>> Thanks, working on a PR now to exclude generated code. I wonder if >>>>>>> this is why spotless has always been so slow. >>>>>>> >>>>>>> On Fri, Jan 18, 2019 at 8:28 AM Ismaël Mejía <ieme...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> What command are you running to build? >>>>>>>> This issue was reported also by other users in the slack channel. >>>>>>>> Agree the fix should be trivial >>>>>>>> >>>>>>>> On Fri, Jan 18, 2019 at 5:13 PM Reuven Lax <re...@google.com> >>>>>>>> wrote: >>>>>>>> > >>>>>>>> > Does this only happen on fresh clones? I created a fresh branch >>>>>>>> synced to origin/master, and I can't reproduce this still. >>>>>>>> > >>>>>>>> > If spotless is running against generated code, that seems like a >>>>>>>> bug in our spotless setup. Should be trivial to fix by creating a >>>>>>>> target >>>>>>>> block in our spotless config. >>>>>>>> > >>>>>>>> > Reuven >>>>>>>> > >>>>>>>> > On Fri, Jan 18, 2019 at 8:07 AM Ismaël Mejía <ieme...@gmail.com> >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >> Just make a fresh clone and run `./gradlew check -p >>>>>>>> sdks/java/core` it >>>>>>>> >> should break. If you add 'spotlessApply' the build passes but >>>>>>>> this >>>>>>>> >> should not be the default, no?, the default is 'spotlessCheck' >>>>>>>> >> >>>>>>>> >> On Fri, Jan 18, 2019 at 5:02 PM Reuven Lax <re...@google.com> >>>>>>>> wrote: >>>>>>>> >> > >>>>>>>> >> > I don't get those errors when I run spotlessApply, and I don't >>>>>>>> see those errors happening on Jenkins. Are you doing anything special >>>>>>>> to >>>>>>>> run spotless? In general, I don't think spotless was running on >>>>>>>> generated >>>>>>>> code before. >>>>>>>> >> > >>>>>>>> >> > On Fri, Jan 18, 2019 at 6:06 AM Ismaël Mejía < >>>>>>>> ieme...@gmail.com> wrote: >>>>>>>> >> >> >>>>>>>> >> >> When running the build on master we got an error message. >>>>>>>> >> >> Looks related to the recent inclusion/generation of stuff >>>>>>>> with ANTLR. >>>>>>>> >> >> Can Reuven or someone else involved in this specific changes >>>>>>>> please >>>>>>>> >> >> take a look? >>>>>>>> >> >> >>>>>>>> >> >> > Task :beam-sdks-java-core:spotlessJava FAILED >>>>>>>> >> >> >>>>>>>> >> >> FAILURE: Build failed with an exception. >>>>>>>> >> >> >>>>>>>> >> >> * What went wrong: >>>>>>>> >> >> Execution failed for task ':beam-sdks-java-core:spotlessJava'. >>>>>>>> >> >> > The following files had format violations: >>>>>>>> >> >> >>>>>>>> >>>>>>>> sdks/java/core/build/generated-src/antlr/main/java/org/apache/beam/sdk/schemas/parser/generated/FieldSpecifierNotationParser.java >>>>>>>> >> >> @@ -1,5 +1,3 @@ >>>>>>>> >> >> >>>>>>>> >>>>>>>> -//·Generated·from·java/org/apache/beam/sdk/schemas/parser/generated/FieldSpecifierNotation.g4·by·ANTLR·4.7 >>>>>>>> >> >> - >>>>>>>> >> >> /* >>>>>>>> >> >> >>>>>>>> ·*·Licensed·to·the·Apache·Software·Foundation·(ASF)·under·one >>>>>>>> >> >> >>>>>>>> ·*·or·more·contributor·license·agreements.··See·the·NOTICE·file >>>>>>>> >> >> @@ -19,609 +17,737 @@ >>>>>>>> >> >> ·*/ >>>>>>>> >> >> >>>>>>>> package·org.apache.beam.sdk.schemas.parser.generated; >>>>>>>> >> >> >>>>>>>> >> >> +import·java.util.List; >>>>>>>> >> >> +import·org.antlr.v4.runtime.*; >>>>>>>> >> >> import·org.antlr.v4.runtime.atn.*; >>>>>>>> >> >> import·org.antlr.v4.runtime.dfa.DFA; >>>>>>>> >> >> -import·org.antlr.v4.runtime.*; >>>>>>>> >> >> import·org.antlr.v4.runtime.misc.*; >>>>>>>> >> >> import·org.antlr.v4.runtime.tree.*; >>>>>>>> >> >> -import·java.util.List; >>>>>>>> >> >> -import·java.util.Iterator; >>>>>>>> >> >> -import·java.util.ArrayList; >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> @SuppressWarnings({"all",·"warnings",·"unchecked",·"unused",·"cast"}) >>>>>>>> >> >> >>>>>>>> public·class·FieldSpecifierNotationParser·extends·Parser·{ >>>>>>>> >> >> >>>>>>>> >>>>>>>> -\tstatic·{·RuntimeMetaData.checkVersion("4.7",·RuntimeMetaData.VERSION);·} >>>>>>>> >> >> +··static·{ >>>>>>>> >> >> >>>>>>>> +····RuntimeMetaData.checkVersion("4.7",·RuntimeMetaData.VERSION); >>>>>>>> >> >> +··} >>>>>>>> >> >> >>>>>>>> >> >> -\tprotected·static·final·DFA[]·_decisionToDFA; >>>>>>>> >> >> >>>>>>>> -\tprotected·static·final·PredictionContextCache·_sharedContextCache·= >>>>>>>> >> >> -\t\tnew·PredictionContextCache(); >>>>>>>> >> >> -\tpublic·static·final·int >>>>>>>> >> >> >>>>>>>> >>>>>>>> -\t\tT__0=1,·T__1=2,·T__2=3,·T__3=4,·T__4=5,·IDENTIFIER=6,·WILDCARD=7,·WS=8; >>>>>>>> >> >> -\tpublic·static·final·int >>>>>>>> >> >> >>>>>>>> >>>>>>>> -\t\tRULE_fieldSpecifier·=·0,·RULE_dotExpression·=·1,·RULE_dotExpressionComponent·=·2,· >>>>>>>> >> >> >>>>>>>> >>>>>>>> -\t\tRULE_qualifiedComponent·=·3,·RULE_qualifierList·=·4,·RULE_arrayQualifier·=·5,· >>>>>>>> >> >> -\t\tRULE_mapQualifier·=·6; >>>>>>>> >> >> -\tpublic·static·final·String[]·ruleNames·=·{ >>>>>>>> >> >> >>>>>>>> >>>>>>>> -\t\t"fieldSpecifier",·"dotExpression",·"dotExpressionComponent",·"qualifiedComponent",· >>>>>>>> >> >> >>>>>>>> -\t\t"qualifierList",·"arrayQualifier",·"mapQualifier" >>>>>>>> >> >> -\t}; >>>>>>>> >> >> +··protected·static·final·DFA[]·_decisionToDFA; >>>>>>>> >> >> >>>>>>>> >>>>>>>> +··protected·static·final·PredictionContextCache·_sharedContextCache·=·new·PredictionContextCache(); >>>>>>>> >> >> +··public·static·final·int·T__0·=·1, >>>>>>>> >> >> +······T__1·=·2, >>>>>>>> >> >> +······T__2·=·3, >>>>>>>> >> >> +······T__3·=·4, >>>>>>>> >> >> +······T__4·=·5, >>>>>>>> >> >> +······IDENTIFIER·=·6, >>>>>>>> >> >> ... (1268 more lines that didn't fit) >>>>>>>> >> >> Violations also present in: >>>>>>>> >> >> >>>>>>>> >>>>>>>> sdks/java/core/build/generated-src/antlr/main/java/org/apache/beam/sdk/schemas/parser/generated/FieldSpecifierNotationVisitor.java >>>>>>>> >> >> >>>>>>>> >>>>>>>> sdks/java/core/build/generated-src/antlr/main/java/org/apache/beam/sdk/schemas/parser/generated/FieldSpecifierNotationBaseListener.java >>>>>>>> >> >> >>>>>>>> >>>>>>>> sdks/java/core/build/generated-src/antlr/main/java/org/apache/beam/sdk/schemas/parser/generated/FieldSpecifierNotationBaseVisitor.java >>>>>>>> >> >> >>>>>>>> >>>>>>>> sdks/java/core/build/generated-src/antlr/main/java/org/apache/beam/sdk/schemas/parser/generated/FieldSpecifierNotationLexer.java >>>>>>>> >> >> >>>>>>>> >>>>>>>> sdks/java/core/build/generated-src/antlr/main/java/org/apache/beam/sdk/schemas/parser/generated/FieldSpecifierNotationListener.java >>>>>>>> >> >> Run 'gradlew spotlessApply' to fix these violations. >>>>>>>> >>>>>>>
