Thanks for digging into this! I can confirm that my lucene-*-9.0.0-SNAPSHOT.jar files have META-INF/services/ with all the expected codecs. I did remove the lucene subfolder from the maven repo, re-ran `gradlew mavenToLocalRepo`, manually verified that the expected META-INF/services/ listed all codecs, and re-ran the Solr build (with `--refresh-dependencies`, and after having added the `mavenLocal()` repo, changed `versions.props` to point at "9.0.0-SNAPSHO", and run `gradlew --write-locks` to update the versions.lock file). I'm still seeing the same exception ...
Michael On Wed, Jun 2, 2021 at 8:01 AM Uwe Schindler <[email protected]> wrote: > > FYI, > > I did a recompile locally and checked the JAR files in my local Maven > repository ("./gradlew mavenToLocalRepo"): All contain META-INF/services with > all Codecs. > > I have no idea what goes wrong on your side. Next I will try to setup > mavenLocal() and a SNAPSHOT dependency, too. > > ----- > Uwe Schindler > Achterdiek 19, D-28357 Bremen > https://www.thetaphi.de > eMail: [email protected] > > > -----Original Message----- > > From: Uwe Schindler <[email protected]> > > Sent: Wednesday, June 2, 2021 12:40 PM > > To: [email protected] > > Subject: RE: Solr/Lucene joint development workflow? > > > > I'd suggest to remove the whole Lucene subfolder from your local Maven repo. > > E.g., rm -rf ~/.m2/repository/org/apache/lucene > > > > Then compile and deploy Lucene again. > > > > Uwe > > > > ----- > > Uwe Schindler > > Achterdiek 19, D-28357 Bremen > > https://www.thetaphi.de > > eMail: [email protected] > > > > > -----Original Message----- > > > From: Uwe Schindler <[email protected]> > > > Sent: Wednesday, June 2, 2021 12:38 PM > > > To: '[email protected]' <[email protected]> > > > Subject: RE: Solr/Lucene joint development workflow? > > > > > > Hi, > > > > > > this error helps. It looks like the lucene-core.jar file does not have > > > the META- > > > INF/services. I wonder how Solr works! > > > > > > It would be good to figure out what's going on and why the JAR files in > > > your > > > local repository do not have the required META-INF/services subfolder. > > > > > > Uwe > > > > > > ----- > > > Uwe Schindler > > > Achterdiek 19, D-28357 Bremen > > > https://www.thetaphi.de > > > eMail: [email protected] > > > > > > > -----Original Message----- > > > > From: Michael Gibney <[email protected]> > > > > Sent: Wednesday, June 2, 2021 6:05 AM > > > > To: [email protected] > > > > Subject: Re: Solr/Lucene joint development workflow? > > > > > > > > Uwe, > > > > Thanks for this advice! I followed this as closely as I could > > > > (including the gradle `--refresh-dependencies` flag), and was again > > > > able to compile (and compileTest) against mavenLocal. But on actually > > > > running tests, afaict all tests fail with the same stacktrace (copied > > > > below). iirc this was the same error I was getting previously with the > > > > SNAPSHOT build (before the introduction of the prerelease dependency). > > > > I'd be grateful for any advice on addressing this remaining error (and > > > > having encountered the error, I continue to be curious whether anyone > > > > has this workflow or something similar working in practice). > > > > > > > > David, your suggestion to add "mavenLocalI()" to Gradle's repo list > > > > seems reasonable -- perhaps a more viable option now that the main > > > > branch is pegged to specific builds (as opposed to a SNAPSHOT)? I'm > > > > happy to help contribute to documentation of this process (once I have > > > > it working!). > > > > > > > > One other challenge I encountered was determining which commit the > > > > current prerelease build is based on. The desired workflow would be to > > > > update the Lucene feature branch by merging upstream changes as far as > > > > the commit of the prerelease build (for use as a dependency of the > > > > Solr-side feature branch). > > > > > > > > The exception I'm getting on running tests is: > > > > > > > > java.lang.ExceptionInInitializerError > > > > at org.apache.lucene.codecs.Codec.getDefault(Codec.java:141) > > > > at > > > > > > > > > org.apache.lucene.util.TestRuleSetupAndRestoreClassEnv.before(TestRuleSetup > > > > AndRestoreClassEnv.java:133) > > > > at > > > > > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfter > > > > Rule.java:42) > > > > at > > > > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stateme > > > > ntAdapter.java:36) > > > > at > > > > > > > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClass > > > > Name.java:38) > > > > at > > > > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethod > > > > sRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > > > > at > > > > > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethod > > > > sRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > > > > at > > > > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stateme > > > > ntAdapter.java:36) > > > > at > > > > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stateme > > > > ntAdapter.java:36) > > > > at > > > > > > > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertio > > > > nsRequired.java:53) > > > > at > > > > > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfter > > > > Rule.java:43) > > > > at > > > > > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja > > > > va:44) > > > > at > > > > > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn > > > > oreAfterMaxFailures.java:60) > > > > at > > > > > > > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTest > > > > Suites.java:47) > > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > > > > at > > > > > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stateme > > > > ntAdapter.java:36) > > > > at > > > > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run( > > > > ThreadLeakControl.java:370) > > > > at > > > > > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutin > > > > gTask$0(ThreadLeakControl.java:826) > > > > at java.base/java.lang.Thread.run(Thread.java:834) > > > > > > > > Caused by: > > > > java.lang.IllegalArgumentException: An SPI class of type > > > > org.apache.lucene.codecs.Codec with name 'Lucene90' does not exist. > > > > You need to add the corresponding JAR file supporting this SPI to your > > > > classpath. The current classpath supports the following names: [] > > > > at > > > > org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113) > > > > at > > > > org.apache.lucene.codecs.Codec$Holder.<clinit>(Codec.java:58) > > > > ... 19 more > > > > > > > > > > > > On Fri, May 28, 2021 at 4:03 PM David Smiley <[email protected]> > > wrote: > > > > > > > > > > To whomever does this first, it would be great if the steps could be > > written > > > > down carefully in more detail so that it can be shared for its eventual > > > inclusion > > > > in /dev-docs > > > > > > > > > > Maybe "mavenLocalI()" should simply be added to Gradle's repo list by > > > > default? This will speed up some first-time builds in some environments > > who > > > > already have lots of local Maven dependencies. > > > > > > > > > > ~ David Smiley > > > > > Apache Lucene/Solr Search Developer > > > > > http://www.linkedin.com/in/davidwsmiley > > > > > > > > > > > > > > > On Thu, May 27, 2021 at 12:09 PM Uwe Schindler <[email protected]> > > > > wrote: > > > > >> > > > > >> Hi Michael, > > > > >> > > > > >> since yesterday, the Lucene dependency is no longer a snapshot one, > > > > >> so > > > > parts of your mail are no longer valid. But the worksflow is very > > > > similar. > > > > >> > > > > >> If the Solr team wants to use a newer preview build of Lucene, there > > > > >> is a > > > > workflow using ASF Jenkins that can build a new "prerelease" release of > > > > the > > > > Lucene main branch and deploy it on some ASF data dump server as maven > > > > repository. > > > > >> > > > > >> The Gradle build of Solr is referring to the recent repository and > > > > >> picks a > > > > specific version (the build number of ASF jenkins). > > > > >> > > > > >> For joint development, I'd suggest to work like this: > > > > >> - Change the dependency in the gradle versions.props file to SNAPSHOT > > > > (only local, don't submit this as PR) > > > > >> - Add mavenLocal() as suggested in your mail (only local, don't > > > > >> submit > > this > > > as > > > > PR) > > > > >> - develop changes in lucene and install them in local repo > > > > >> - use them from solr through the snapshot dependency. BUT: make sure > > > the > > > > snapshot is updated, this can be enforced by passing a gradlew command > > line > > > > parameter to redownload all dependencies. > > > > >> - once you are done with joint development create pull requests in > > > > >> both > > > > repositories > > > > >> - Lucene's change should be merged first. Once this is done and all > > > > >> tests > > > > pass, ask a committer to trigger the Jenkins job to build a new > > > > prerelease > > > > >> - Update the Solr pull request and enter the new repository > > > > >> coordinates > > > > created by Jenkins (needs to be done in 2 files, see below). > > > > >> > > > > >> For more details how to update Lucene dependencies in Solr, run: > > > > >> $ gradlew :helpDeps > > > > >> > > > > >> Uwe > > > > >> > > > > >> ----- > > > > >> Uwe Schindler > > > > >> Achterdiek 19, D-28357 Bremen > > > > >> https://www.thetaphi.de > > > > >> eMail: [email protected] > > > > >> > > > > >> > -----Original Message----- > > > > >> > From: Michael Gibney <[email protected]> > > > > >> > Sent: Wednesday, May 26, 2021 8:02 PM > > > > >> > To: [email protected] > > > > >> > Subject: Solr/Lucene joint development workflow? > > > > >> > > > > > >> > I'm working on some features that involve changes to both Lucene > > > > >> > and > > > > >> > Solr. Post-TLP-split, I'm wondering whether anyone has recommended > > > > >> > techniques to handle this kind of situation. > > > > >> > > > > > >> > Ideally one would work on Lucene changes, get them merged, and then > > > > >> > proceed with Solr development; but realistically even if this were > > > > >> > as > > > > >> > straightforward in practice as it sounds in principal, there are > > > > >> > cases > > > > >> > where one would still want to develop in parallel. > > > > >> > > > > > >> > I haven't been able to find any documented recommendation on this > > > > >> > subject. It's possible to have a locally built Lucene snapshot (via > > > > >> > `gradlew mavenToLocalRepo`); but I was only able to actually > > > > >> > _build_ > > > > >> > Solr against the local Lucene artifact by adding `mavenLocal()` to > > > > >> > the > > > > >> > `allprojects/repositories` block in `gradle/defaults.gradle` -- > > > > >> > and I > > > > >> > have yet to figure a way get the local Lucene artifact on the test > > > > >> > classpath (so I'm as yet unable to run Solr tests that depend on > > > > >> > unmerged upstream changes to Lucene). > > > > >> > > > > > >> > It's also possible that the partially-functional approach described > > > > >> > above will have to change now that Solr main depends on a specific > > > > >> > Lucene snapshot version. > > > > >> > > > > > >> > Is anybody doing something like this? Or perhaps I'm asking the > > > > >> > wrong > > > > >> > question? I can think of solutions that involve setting up my own > > > > >> > maven repository, to which I publish my own pinned versions of > > Lucene, > > > > >> > and refer to such pinned versions/repo as part of a given Solr > > > > >> > "patch". But that feels both half-baked _and_ bloated, so I don't > > > > >> > want > > > > >> > to go down that road unless I feel convinced there's no better > > > > >> > alternative. > > > > >> > > > > > >> > Michael > > > > >> > > > > > >> > --------------------------------------------------------------------- > > > > >> > To unsubscribe, e-mail: [email protected] > > > > >> > For additional commands, e-mail: [email protected] > > > > >> > > > > >> > > > > >> --------------------------------------------------------------------- > > > > >> To unsubscribe, e-mail: [email protected] > > > > >> For additional commands, e-mail: [email protected] > > > > >> > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
