Thanks, I had probably missed some of what Uwe was saying in regards to the limitation of what would be possible even if the suggested changes were made. Given your points and the fact that it would take a while to have any of the changes filter into a release version, I decided to develop a patch-lucene.sh script to validate if the changes would allow me to access the required parts of Lucene. While it may not provide the full functionality provided by SPI including analysis chains and codecs, I worked out it will allow the use of the basics.
The patch script below adds Automatic module names to the four Lucene libraries I needed (not strictly required, but I was interested to check if it would work). For now, any duplicate packages I identified have been moved into the core library (as identified in suggested changes). Given, I'm not using the SPI functionality, I simply deleted the standard analysers from the services list (required or the module system complains they are missing). Thankfully Gradle doesn't validate jars in the cache so the patch script only needs to be run once. Once the patch script was applied, I was able to use Ngram analysers and the query parser without issue. This will provide the essentials of what we needed for an embedded solution (location indexing, short text indexing with ngrams tokenizer and lower case filter, searching by location with nearest, bounding box and radius search, and partial text searches using the query parser). Of course, I've only scratched the surface of the APIs and right now it isn't clear if/when the SPI functionality might cause further issues or be required. So, I've validated that the suggested changes would allow basic functionality to work under the Java module system. If there's anything else I can do to help progress the suggested changes please let me know. Regards, David. --------- patch-lucene.sh -- works on osx ------ mkdir -p patch-lucene/core mkdir -p patch-lucene/analyzers mkdir -p patch-lucene/sandbox mkdir -p patch-lucene/queryparser cd patch-lucene # copy the jars from the gradle cache. cp ~/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-core/8.5.0/3f9ea85fff4fc3f7c83869dddb9b0ef7818c0cae/lucene-core-8.5.0.jar . cp ~/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-analyzers-common/8.5.0/7156f2e545fd6e7faaee4781d15eb60cf5f07646/lucene-analyzers-common-8.5.0.jar . cp ~/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-sandbox/8.5.0/2b275921f2fd92b15b4f1a2a565467c3fa221ef9/lucene-sandbox-8.5.0.jar . cp ~/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-queryparser/8.5.0/13c38f39b1a7d10c4749ba789fa95da5868d4885/lucene-queryparser-8.5.0.jar . # Add automatic module name to core. cd core jar -xf ../lucene-core-8.5.0.jar sed -i '' -e '/Created-By:/a\'$'\n''Automatic-Module-Name: org.apache.lucene.core'$'\n' META-INF/MANIFEST.MF # Add automiatc module name, move standard analysis classes into core. Remove any classes in standard from service lists. cd ../analyzers jar -xf ../lucene-analyzers-common-8.5.0.jar sed -i '' -e '/Created-By:/a\'$'\n''Automatic-Module-Name: org.apache.lucene.analyzers.common'$'\n' META-INF/MANIFEST.MF sed -i '' -e '/standard/d' META-INF/services/* mv org/apache/lucene/analysis/standard/* ../core/org/apache/lucene/analysis/standard rmdir org/apache/lucene/analysis/standard jar -cfm ../lucene-analyzers-common-8.5.0-fix.jar META-INF/MANIFEST.MF . # Add automatic module name, move search and document packages into core. move Java 9 specific search into core. cd ../sandbox jar -xf ../lucene-sandbox-8.5.0.jar sed -i '' -e '/Created-By:/a\'$'\n''Automatic-Module-Name: org.apache.lucene.sandbox'$'\n' META-INF/MANIFEST.MF mv org/apache/lucene/search/* ../core/org/apache/lucene/search rmdir org/apache/lucene/search mv org/apache/lucene/document/* ../core/org/apache/lucene/document rmdir org/apache/lucene/document mv META-INF/versions/9/org/apache/lucene/search/* ../core/META-INF/versions/9/org/apache/lucene/search rm -rf META-INF/versions jar -cfm ../lucene-sandbox-8.5.0-fix.jar META-INF/MANIFEST.MF . # Package up core with changes. cd ../core jar -cfm ../lucene-core-8.5.0-fix.jar META-INF/MANIFEST.MF . # Add automatic module name. cd ../queryparser jar -xf ../lucene-queryparser-8.5.0.jar sed -i '' -e '/Created-By:/a\'$'\n''Automatic-Module-Name: org.apache.lucene.queryparser'$'\n' META-INF/MANIFEST.MF jar -cfm ../lucene-queryparser-8.5.0-fix.jar META-INF/MANIFEST.MF . # Copy the fixed versions back into gradle cache. cd .. cp lucene-core-8.5.0-fix.jar ~/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-core/8.5.0/3f9ea85fff4fc3f7c83869dddb9b0ef7818c0cae/lucene-core-8.5.0.jar cp lucene-analyzers-common-8.5.0-fix.jar ~/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-analyzers-common/8.5.0/7156f2e545fd6e7faaee4781d15eb60cf5f07646/lucene-analyzers-common-8.5.0.jar cp lucene-sandbox-8.5.0-fix.jar ~/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-sandbox/8.5.0/2b275921f2fd92b15b4f1a2a565467c3fa221ef9/lucene-sandbox-8.5.0.jar cp lucene-queryparser-8.5.0-fix.jar ~/.gradle/caches/modules-2/files-2.1/org.apache.lucene/lucene-queryparser/8.5.0/13c38f39b1a7d10c4749ba789fa95da5868d4885/lucene-queryparser-8.5.0.jar ------------ On Sat, Apr 11, 2020 at 11:21 AM Chris Hostetter <[email protected]> wrote: > > : If the changes I proposed are still viewed as having too many downstream > : impacts, my fallback position will be to patch the jars. This involves > : using the gradle import system to get the jars from Maven, then using a > : patch script to manually unzip the jars, move the offending classes into > : other jars which share the same package name and rezip. So far, I've been > > I'm no expert here, but i trust that Uwe is, and i feel like your followup > questions/suggests have still avoided his primary point about *why* > Lucene/Solr hasn't attempted jigsaw modulariation... > > : >>> There is currently some preparatory things to move forward with > modules, > : >>> so although you might be able to actually compile Lucene with module > system > : >>> (by limiting to a subset of JAR files), it currently won’t work > : >>> cross-module due to the way how it handles ServiceLoader interfaces > : >>> (codecs, postings formats, analyzers, see > : >>> https://issues.apache.org/jira/browse/LUCENE-9281). The only way to > : >>> make it work at runtime is to put all of Lucene into one module. > > ...so, IIUC, even if you patch the *current* jars, any Lucene code you use > that depends on SPI (like analysis chains, codecs, etc...) isn't going to > work unless follow Uwe's primary suggestion for folks who care about > modules... > > : >>> Th general recommendation is to combine all required Lucene libraries > : >>> into a separate JAR file during the maven / gradle build (e.g. using > the > : >>> Maven Shade plugin). Keep in mind that Lucene is also not suitable > for use > > > -Hoss > http://www.lucidworks.com/ > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected]
