[jira] [Commented] (SOLR-17297) Classloading issue with plugin and modules
[
https://issues.apache.org/jira/browse/SOLR-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848000#comment-17848000
]
David Smiley commented on SOLR-17297:
-
I believe it's only one classloader across modules & shardLib. On the other
hand, "packages" are isolated.
If one day modules get classloader isolation, then I think we would have to
invoke the Lucene SPI stuff for each separate classloader.
> Classloading issue with plugin and modules
> --
>
> Key: SOLR-17297
> URL: https://issues.apache.org/jira/browse/SOLR-17297
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: 9.3
>Reporter: Patson Luk
>Priority: Critical
>
> h2. Summary
> Using plugin jar and enabling any modules could trigger
> {{java.lang.ClassNotFoundException}}
> h2. Description
> 1. An implementation of {{org.apache.lucene.codecs.PostingsFormat}} with the
> jar within the /lib
> 2. Enable modules in solr.xml for example name="modules">opentelemetry
> 3. Now on startup. As a part of {{NodeConfig#setupSharedLib}}, it would load
> all the SPIs, it locates our jar and loads the class with a
> {{FactoryURLClassLoader}} with the classpaths point at the jar of the lib,
> which is correct
> 4. After {{NodeConfig#setupSharedLib}}, {{NodeConfig#initModules}} is
> invoked, which eventually calls {{SolrResourceLoader#addURLsToClassLoader}}
> that closes the previous class loader, which is the one used in 3.
> 5. Now a core is loaded with that codec, it runs the code which
> references other classes within our plugin jar, but unfortunately it would
> use the Classloader that loads our class in step 3., and such loader is
> marked as "closed" hence no longer load the correct resource/class. This
> triggers ClassNotFoundException.
> I have tried several things, the only thing that seems to work so far is
> commenting out {{IOUtils.closeWhileHandlingException(oldLoader);}} in
> {{SolrResourceLoader#addURLsToClassLoader}}, which is likely not the right
> workaround as the {{closeWhileHandlingException}} should be there for a
> reason ;)
> Switching {{setupSharedLib}} and {{initModules}} might work too (haven't
> tested), but I don't want to try any weird changes since I don't really know
> the ordering significance.
> Would appreciate some helps from the Solr experts! :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17297) Classloading issue with plugin and modules
[
https://issues.apache.org/jira/browse/SOLR-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847407#comment-17847407
]
Patson Luk commented on SOLR-17297:
---
Thank you for the swift response [~dsmiley]!
>Switching setupSharedLib and initModules might work too
Yes. Tried and it did work as well
>Moving the plugin jar to WEB-INF/lib
Yes. This works too!
Putting it in WEB-INF/lib could be an acceptable solution. However, based on
the [lib
directories|https://solr.apache.org/guide/solr/latest/configuration-guide/libs.html]
doc, putting plugin jars in /lib/ seems legit too (that's why we
put it there at first place, and it mostly works, until we started using
modules).
In particular, with the current code any Lucene SPIs defined in plugin could
expose to the same issue due to the
[loading|https://github.com/apache/solr/blob/ad0a7975c286790c78c08aa9d2d854167fabe5b7/solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java#L240-L256]?
I'm not too sure about the proper fix here? whether we want to reconsider the
load ordering: setupSharedLib vs initModules, based on the
SolrResourceLoader.addToClassLoader contract, setupSharedLib SHOULD appear
after initModules since it does load resource. However, I simply don't know
enough to tell whether it could trigger other issues (modules dependency on
shared lib?)
On the other end, we could simply document that any SPI should be placed in
WEB-INF/lib due to classloading ordering?
> Classloading issue with plugin and modules
> --
>
> Key: SOLR-17297
> URL: https://issues.apache.org/jira/browse/SOLR-17297
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: 9.3
>Reporter: Patson Luk
>Priority: Critical
>
> h2. Summary
> Using plugin jar and enabling any modules could trigger
> {{java.lang.ClassNotFoundException}}
> h2. Description
> 1. An implementation of {{org.apache.lucene.codecs.PostingsFormat}} with the
> jar within the /lib
> 2. Enable modules in solr.xml for example name="modules">opentelemetry
> 3. Now on startup. As a part of {{NodeConfig#setupSharedLib}}, it would load
> all the SPIs, it locates our jar and loads the class with a
> {{FactoryURLClassLoader}} with the classpaths point at the jar of the lib,
> which is correct
> 4. After {{NodeConfig#setupSharedLib}}, {{NodeConfig#initModules}} is
> invoked, which eventually calls {{SolrResourceLoader#addURLsToClassLoader}}
> that closes the previous class loader, which is the one used in 3.
> 5. Now a core is loaded with that codec, it runs the code which
> references other classes within our plugin jar, but unfortunately it would
> use the Classloader that loads our class in step 3., and such loader is
> marked as "closed" hence no longer load the correct resource/class. This
> triggers ClassNotFoundException.
> I have tried several things, the only thing that seems to work so far is
> commenting out {{IOUtils.closeWhileHandlingException(oldLoader);}} in
> {{SolrResourceLoader#addURLsToClassLoader}}, which is likely not the right
> workaround as the {{closeWhileHandlingException}} should be there for a
> reason ;)
> Switching {{setupSharedLib}} and {{initModules}} might work too (haven't
> tested), but I don't want to try any weird changes since I don't really know
> the ordering significance.
> Would appreciate some helps from the Solr experts! :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-17297) Classloading issue with plugin and modules
[
https://issues.apache.org/jira/browse/SOLR-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847147#comment-17847147
]
David Smiley commented on SOLR-17297:
-
Interesting. SolrResourceLoader.addToClassLoader has a clear contract in its
javadocs that it MUST only be called prior to using to get any resources. Yet
I suspect the underlying class loader here has already loaded stuff, based on
your description.
bq. Switching setupSharedLib and initModules might work too (haven't tested)
My thoughts exactly. Try that :-)
FWIW where I work, our Lucene PostingsFormat/Codec level stuff goes into
WEB-INF/lib because we ran into issues with it in sharedLib. I'm glad to see
you've looked closely at the matter.
> Classloading issue with plugin and modules
> --
>
> Key: SOLR-17297
> URL: https://issues.apache.org/jira/browse/SOLR-17297
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: 9.3
>Reporter: Patson Luk
>Priority: Critical
>
> h2. Summary
> Using plugin jar and enabling any modules could trigger
> {{java.lang.ClassNotFoundException}}
> h2. Description
> 1. An implementation of {{org.apache.lucene.codecs.PostingsFormat}} with the
> jar within the /lib
> 2. Enable modules in solr.xml for example name="modules">opentelemetry
> 3. Now on startup. As a part of {{NodeConfig#setupSharedLib}}, it would load
> all the SPIs, it locates our jar and loads the class with a
> {{FactoryURLClassLoader}} with the classpaths point at the jar of the lib,
> which is correct
> 4. After {{NodeConfig#setupSharedLib}}, {{NodeConfig#initModules}} is
> invoked, which eventually calls {{SolrResourceLoader#addURLsToClassLoader}}
> that closes the previous class loader, which is the one used in 3.
> 5. Now a core is loaded with that codec, it runs the code which
> references other classes within our plugin jar, but unfortunately it would
> use the Classloader that loads our class in step 3., and such loader is
> marked as "closed" hence no longer load the correct resource/class. This
> triggers ClassNotFoundException.
> I have tried several things, the only thing that seems to work so far is
> commenting out {{IOUtils.closeWhileHandlingException(oldLoader);}} in
> {{SolrResourceLoader#addURLsToClassLoader}}, which is likely not the right
> workaround as the {{closeWhileHandlingException}} should be there for a
> reason ;)
> Switching {{setupSharedLib}} and {{initModules}} might work too (haven't
> tested), but I don't want to try any weird changes since I don't really know
> the ordering significance.
> Would appreciate some helps from the Solr experts! :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
