+1 to fix precommit slowness! It is really depressing to run precommit on my 128 core box and watch how slow it is :)
I'll try to add (read: open an issue!) a chart to our nightly benchmarks tracking precommit time over time ... Mike McCandless http://blog.mikemccandless.com On Fri, Jan 8, 2021 at 4:43 PM Dawid Weiss <[email protected]> wrote: > I don't remember, to be honest. I think javadocs, html linters - they > all contribute. We could remove some of them (via -x) on github if > they're too heavy. > > On Fri, Jan 8, 2021 at 9:08 PM Michael Sokolov <[email protected]> wrote: > > > > Whoops! Well I guess we'll see. I wonder what other checks we are > running now that's were not part of precommit, aside from spotless? > > > > On Fri, Jan 8, 2021, 2:28 PM Dawid Weiss <[email protected]> wrote: > >> > >> The switch to full check (minus tests) makes the "precommit" really > >> heavy... it's around 14 minutes on my not-so-fancy machine. I'm > >> curious if it's going to pass on github or timeout somehow. > >> > >> On Fri, Jan 8, 2021 at 8:26 PM Houston Putman <[email protected]> > wrote: > >> > > >> > Yeah, that should work Dawid. I'll create a PR that tests it out. > >> > > >> > - Houston > >> > > >> > On Fri, Jan 8, 2021 at 2:02 PM Dawid Weiss <[email protected]> > wrote: > >> >> > >> >> Can those jobs just run a sequence of two commands - > >> >> > >> >> ./gradlew localSettings > >> >> ./gradlew check -x test > >> >> > >> >> This would solve the problem as proper JVM settings (tuned for the > >> >> machine/ image it's running on) would be written and used on > >> >> subsequent run. > >> >> > >> >> Dawid > >> >> > >> >> On Fri, Jan 8, 2021 at 7:44 PM Uwe Schindler <[email protected]> > wrote: > >> >> > > >> >> > The problembcomes from the fact that Gradle sets a Xmx for itself > in the settings file. Jenkins installs a settings file before. > >> >> > > >> >> > As every GitHub run is a clean checkout in new working dir, the > settings never persist. > >> >> > > >> >> > IMHO, we should change the command line and pass JVM options to > set heap size as it is written to the settings file. > >> >> > > >> >> > Uwe > >> >> > > >> >> > Am January 8, 2021 6:13:01 PM UTC schrieb David Smiley < > [email protected]>: > >> >> >> > >> >> >> Perhaps the OOMs are because > .github/workflows/gradle-precommit.yml yesterday switched from doing > "gradlew check -x test" to "gradlew precomit" ? CC Michael Sokolov > >> >> >> > >> >> >> ~ David Smiley > >> >> >> Apache Lucene/Solr Search Developer > >> >> >> http://www.linkedin.com/in/davidwsmiley > >> >> >> > >> >> >> > >> >> >> On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter < > [email protected]> wrote: > >> >> >>> > >> >> >>> Same for my PR too ... OOMs about 14 minutes in ... > >> >> >>> > >> >> >>> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman < > [email protected]> wrote: > >> >> >>>> > >> >> >>>> Weirdly enough, Github PR precommit actions have started to > OOM. Not sure if it's a github thing or something that changed on our end... > >> >> >>>> > >> >> >>>> On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein < > [email protected]> wrote: > >> >> >>>>> > >> >> >>>>> It turned out to be this while I merged branches: > >> >> >>>>> > >> >> >>>>> warning: inexact rename detection was skipped due to too many > files. > >> >> >>>>> > >> >> >>>>> warning: you may want to set your merge.renamelimit variable > to at least 1639 and retry the command. > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> Joel Bernstein > >> >> >>>>> http://joelsolr.blogspot.com/ > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein < > [email protected]> wrote: > >> >> >>>>>> > >> >> >>>>>> Thanks Eric, I'll do a fresh clone, something must be out of > wack with my local repo. > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> Joel Bernstein > >> >> >>>>>> http://joelsolr.blogspot.com/ > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh < > [email protected]> wrote: > >> >> >>>>>>> > >> >> >>>>>>> It ran for me just fine. I *think* you may not be up to > date, as dataimporthandler/ is no longer in master! > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein < > [email protected]> wrote: > >> >> >>>>>>> > >> >> >>>>>>> I'm getting failing gradle precommits in master: > >> >> >>>>>>> > >> >> >>>>>>> > Task :solr:contrib:validateSourcePatterns FAILED > >> >> >>>>>>> tabs instead spaces: > /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml > >> >> >>>>>>> tabs instead spaces: > /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml > >> >> >>>>>>> tabs instead spaces: > /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml > >> >> >>>>>>> tabs instead spaces: > /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml > >> >> >>>>>>> tabs instead spaces: > /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml > >> >> >>>>>>> tabs instead spaces: > /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml > >> >> >>>>>>> tabs instead spaces: > /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml > >> >> >>>>>>> tabs instead spaces: > /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml > >> >> >>>>>>> > >> >> >>>>>>> FAILURE: Build failed with an exception. > >> >> >>>>>>> > >> >> >>>>>>> * Where: > >> >> >>>>>>> Script > '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle' > line: 324 > >> >> >>>>>>> > >> >> >>>>>>> * What went wrong: > >> >> >>>>>>> Execution failed for task > ':solr:contrib:validateSourcePatterns'. > >> >> >>>>>>> > Found 8 violations in source files (tabs instead spaces). > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> Are others seeing this as well? I'm not seeing Jenkins > emails about this. > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> Joel Bernstein > >> >> >>>>>>> http://joelsolr.blogspot.com/ > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> _______________________ > >> >> >>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | > 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy > >> >> >>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > >> >> >>>>>>> This e-mail and all contents, including attachments, is > considered to be Company Confidential unless explicitly stated otherwise, > regardless of whether attachments are marked as such. > >> >> >>>>>>> > >> >> > > >> >> > -- > >> >> > Uwe Schindler > >> >> > Achterdiek 19, 28357 Bremen > >> >> > https://www.thetaphi.de > >> >> > >> >> --------------------------------------------------------------------- > >> >> To unsubscribe, e-mail: [email protected] > >> >> For additional commands, e-mail: [email protected] > >> >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
