OK I opened https://github.com/mikemccand/luceneutil/issues/96
Patches welcome! Mike McCandless http://blog.mikemccandless.com On Mon, Jan 11, 2021 at 9:01 AM Michael McCandless < [email protected]> wrote: > +1 to fix precommit slowness! It is really depressing to run precommit on > my 128 core box and watch how slow it is :) > > I'll try to add (read: open an issue!) a chart to our nightly benchmarks > tracking precommit time over time ... > > Mike McCandless > > http://blog.mikemccandless.com > > > On Fri, Jan 8, 2021 at 4:43 PM Dawid Weiss <[email protected]> wrote: > >> I don't remember, to be honest. I think javadocs, html linters - they >> all contribute. We could remove some of them (via -x) on github if >> they're too heavy. >> >> On Fri, Jan 8, 2021 at 9:08 PM Michael Sokolov <[email protected]> >> wrote: >> > >> > Whoops! Well I guess we'll see. I wonder what other checks we are >> running now that's were not part of precommit, aside from spotless? >> > >> > On Fri, Jan 8, 2021, 2:28 PM Dawid Weiss <[email protected]> wrote: >> >> >> >> The switch to full check (minus tests) makes the "precommit" really >> >> heavy... it's around 14 minutes on my not-so-fancy machine. I'm >> >> curious if it's going to pass on github or timeout somehow. >> >> >> >> On Fri, Jan 8, 2021 at 8:26 PM Houston Putman <[email protected]> >> wrote: >> >> > >> >> > Yeah, that should work Dawid. I'll create a PR that tests it out. >> >> > >> >> > - Houston >> >> > >> >> > On Fri, Jan 8, 2021 at 2:02 PM Dawid Weiss <[email protected]> >> wrote: >> >> >> >> >> >> Can those jobs just run a sequence of two commands - >> >> >> >> >> >> ./gradlew localSettings >> >> >> ./gradlew check -x test >> >> >> >> >> >> This would solve the problem as proper JVM settings (tuned for the >> >> >> machine/ image it's running on) would be written and used on >> >> >> subsequent run. >> >> >> >> >> >> Dawid >> >> >> >> >> >> On Fri, Jan 8, 2021 at 7:44 PM Uwe Schindler <[email protected]> >> wrote: >> >> >> > >> >> >> > The problembcomes from the fact that Gradle sets a Xmx for itself >> in the settings file. Jenkins installs a settings file before. >> >> >> > >> >> >> > As every GitHub run is a clean checkout in new working dir, the >> settings never persist. >> >> >> > >> >> >> > IMHO, we should change the command line and pass JVM options to >> set heap size as it is written to the settings file. >> >> >> > >> >> >> > Uwe >> >> >> > >> >> >> > Am January 8, 2021 6:13:01 PM UTC schrieb David Smiley < >> [email protected]>: >> >> >> >> >> >> >> >> Perhaps the OOMs are because >> .github/workflows/gradle-precommit.yml yesterday switched from doing >> "gradlew check -x test" to "gradlew precomit" ? CC Michael Sokolov >> >> >> >> >> >> >> >> ~ David Smiley >> >> >> >> Apache Lucene/Solr Search Developer >> >> >> >> http://www.linkedin.com/in/davidwsmiley >> >> >> >> >> >> >> >> >> >> >> >> On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter < >> [email protected]> wrote: >> >> >> >>> >> >> >> >>> Same for my PR too ... OOMs about 14 minutes in ... >> >> >> >>> >> >> >> >>> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman < >> [email protected]> wrote: >> >> >> >>>> >> >> >> >>>> Weirdly enough, Github PR precommit actions have started to >> OOM. Not sure if it's a github thing or something that changed on our end... >> >> >> >>>> >> >> >> >>>> On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein < >> [email protected]> wrote: >> >> >> >>>>> >> >> >> >>>>> It turned out to be this while I merged branches: >> >> >> >>>>> >> >> >> >>>>> warning: inexact rename detection was skipped due to too many >> files. >> >> >> >>>>> >> >> >> >>>>> warning: you may want to set your merge.renamelimit variable >> to at least 1639 and retry the command. >> >> >> >>>>> >> >> >> >>>>> >> >> >> >>>>> Joel Bernstein >> >> >> >>>>> http://joelsolr.blogspot.com/ >> >> >> >>>>> >> >> >> >>>>> >> >> >> >>>>> On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein < >> [email protected]> wrote: >> >> >> >>>>>> >> >> >> >>>>>> Thanks Eric, I'll do a fresh clone, something must be out of >> wack with my local repo. >> >> >> >>>>>> >> >> >> >>>>>> >> >> >> >>>>>> Joel Bernstein >> >> >> >>>>>> http://joelsolr.blogspot.com/ >> >> >> >>>>>> >> >> >> >>>>>> >> >> >> >>>>>> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh < >> [email protected]> wrote: >> >> >> >>>>>>> >> >> >> >>>>>>> It ran for me just fine. I *think* you may not be up to >> date, as dataimporthandler/ is no longer in master! >> >> >> >>>>>>> >> >> >> >>>>>>> >> >> >> >>>>>>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein < >> [email protected]> wrote: >> >> >> >>>>>>> >> >> >> >>>>>>> I'm getting failing gradle precommits in master: >> >> >> >>>>>>> >> >> >> >>>>>>> > Task :solr:contrib:validateSourcePatterns FAILED >> >> >> >>>>>>> tabs instead spaces: >> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml >> >> >> >>>>>>> tabs instead spaces: >> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml >> >> >> >>>>>>> tabs instead spaces: >> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml >> >> >> >>>>>>> tabs instead spaces: >> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml >> >> >> >>>>>>> tabs instead spaces: >> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml >> >> >> >>>>>>> tabs instead spaces: >> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml >> >> >> >>>>>>> tabs instead spaces: >> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml >> >> >> >>>>>>> tabs instead spaces: >> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml >> >> >> >>>>>>> >> >> >> >>>>>>> FAILURE: Build failed with an exception. >> >> >> >>>>>>> >> >> >> >>>>>>> * Where: >> >> >> >>>>>>> Script >> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle' >> line: 324 >> >> >> >>>>>>> >> >> >> >>>>>>> * What went wrong: >> >> >> >>>>>>> Execution failed for task >> ':solr:contrib:validateSourcePatterns'. >> >> >> >>>>>>> > Found 8 violations in source files (tabs instead spaces). >> >> >> >>>>>>> >> >> >> >>>>>>> >> >> >> >>>>>>> Are others seeing this as well? I'm not seeing Jenkins >> emails about this. >> >> >> >>>>>>> >> >> >> >>>>>>> >> >> >> >>>>>>> Joel Bernstein >> >> >> >>>>>>> http://joelsolr.blogspot.com/ >> >> >> >>>>>>> >> >> >> >>>>>>> >> >> >> >>>>>>> _______________________ >> >> >> >>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | >> 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy >> >> >> >>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >> >> >> >>>>>>> This e-mail and all contents, including attachments, is >> considered to be Company Confidential unless explicitly stated otherwise, >> regardless of whether attachments are marked as such. >> >> >> >>>>>>> >> >> >> > >> >> >> > -- >> >> >> > Uwe Schindler >> >> >> > Achterdiek 19, 28357 Bremen >> >> >> > https://www.thetaphi.de >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> >> To unsubscribe, e-mail: [email protected] >> >> >> For additional commands, e-mail: [email protected] >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: [email protected] >> >> For additional commands, e-mail: [email protected] >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >>
