[
https://issues.apache.org/jira/browse/SOLR-6887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257613#comment-14257613
]
Martijn Koster edited comment on SOLR-6887 at 12/23/14 10:27 PM:
-----------------------------------------------------------------
Looking through some system call traces, I think this is a ordering issue: The
"exists()" results in a stat with the full path including the
"gettingstarted_shard1_replica1" and the "../../..", and that fails because the
gettingstarted_shard1_replica1 directory has not yet been created, so the walk
down-and-up fails. If you canonicalise, that directory is removed from the
string, an its non-existence on-disk no longer matters.
was (Author: makuk66):
Looking through some system call traces, I think this is a race-condition: The
"exists()" results in a stat with the full path including the
"gettingstarted_shard1_replica1" and the "../../..", and that fails because the
gettingstarted_shard1_replica1 directory has not yet been created, so the walk
down-and-up fails. If you canonicalise, that directory is removed from the
string, an its non-existence on-disk no longer matters.
> SolrResourceLoader does not canonicalise the path
> -------------------------------------------------
>
> Key: SOLR-6887
> URL: https://issues.apache.org/jira/browse/SOLR-6887
> Project: Solr
> Issue Type: Bug
> Affects Versions: 4.10.2
> Reporter: Martijn Koster
> Priority: Minor
>
> I get
> {quote}
> Can't find (or read) directory to add to classloader
> {quote}
> errors for valid config files.
> To reproduce:
> Step 1: run up a Solr with a zookeeper (default collection, 1 node, 1 shard):
> {noformat}
> tar xvf ~/Downloads/solr-4.10.2.tgz
> cd solr-4.10.2/
> ./bin/solr -e cloud
> Welcome to the SolrCloud example!
> This interactive session will help you launch a SolrCloud cluster on your
> local workstation.
> To begin, how many Solr nodes would you like to run in your local cluster?
> (specify 1-4 nodes) [2] 1
> Ok, let's start up 1 Solr nodes for your example SolrCloud cluster.
> Please enter the port for node1 [8983]
> 8983
> Cloning /Users/mak/solr-4.10.2/example into /Users/mak/solr-4.10.2/node1
> Starting up SolrCloud node1 on port 8983 using command:
> solr start -cloud -d node1 -p 8983
> Waiting to see Solr listening on port 8983 [/]
> Started Solr server on port 8983 (pid=14245). Happy searching!
> Now let's create a new collection for indexing documents in your 1-node
> cluster.
> Please provide a name for your new collection: [gettingstarted]
> gettingstarted
> How many shards would you like to split gettingstarted into? [2] 1
> 1
> How many replicas per shard would you like to create? [2] 1
> 1
> Please choose a configuration for the gettingstarted collection, available
> options are: default or schemaless [default]
> default
> Deploying default Solr configuration files to embedded ZooKeeper using
> command:
> /Users/mak/solr-4.10.2/example/scripts/cloud-scripts/zkcli.sh -zkhost
> localhost:9983 -cmd upconfig -confdir
> /Users/mak/solr-4.10.2/example/solr/collection1/conf -confname default
> Successfully deployed the
> /Users/mak/solr-4.10.2/example/solr/collection1/conf configuration directory
> to ZooKeeper as default
> Creating new collection gettingstarted with 1 shards and replication factor 1
> using Collections API command:
> http://localhost:8983/solr/admin/collections?action=CREATE&name=gettingstarted&replicationFactor=1&numShards=1&collection.configName=default&maxShardsPerNode=1&wt=json&indent=2
> For more information about the Collections API, please see:
> https://cwiki.apache.org/confluence/display/solr/Collections+API
> {
> "responseHeader":{
> "status":0,
> "QTime":2139},
> "success":{
> "":{
> "responseHeader":{
> "status":0,
> "QTime":1906},
> "core":"gettingstarted_shard1_replica1"}}}
> {noformat}
> Verify the server is running on http://localhost:8983/solr/#/
> Step 2: duplicate the zookeeper config:
> {noformat}
> mkdir zkshell
> cd zkshell/
> virtualenv venv
> source venv/bin/activate
> pip install zk_shell
> zk-shell localhost:9983
> Welcome to zk-shell (0.99.05)
> (CONNECTING) />
> (CONNECTED) />
> (CONNECTED) /> cd configs
> (CONNECTED) /configs> cp myconf myconf2 true
> (CONNECTED) /configs> cd myconf
> (CONNECTED) /configs/myconf> get solrconfig.xml
> (CONNECTED) /configs> quit
> {noformat}
> admire the config file, and note the {{<lib
> dir="../../../contrib/extraction/lib" regex=".*\.jar" />}}. That
> configuration comes from [somewhere like
> this|https://github.com/apache/lucene-solr/blob/lucene_solr_4_10_2/solr/example/solr/collection1/conf/solrconfig.xml#L75].
> Step 3: create a collection with the new config:
> {noformat}
> curl
> 'http://localhost:8983/solr/admin/collections?action=CREATE&name=collection2&collection.configName=myconf2&numShards=1'
> {noformat}
> Step 4: check the logs:
> {noformat}
> grep org.apache.solr.core.SolrResourceLoader ./node1/logs/solr.log
> ...
> INFO - 2014-12-23 18:32:55.165; org.apache.solr.core.SolrResourceLoader; new
> SolrResourceLoader for directory:
> '/Users/mak/solr-4.10.2/node1/solr/collection2_shard1_replica1/'
> WARN - 2014-12-23 18:32:55.218; org.apache.solr.core.SolrResourceLoader;
> Can't find (or read) directory to add to classloader:
> ../../../contrib/extraction/lib (resolved as:
> /Users/mak/solr-4.10.2/node1/solr/collection2_shard1_replica1/../../../contrib/extraction/lib).
> WARN - 2014-12-23 18:32:55.218; org.apache.solr.core.SolrResourceLoader;
> Can't find (or read) directory to add to classloader: ../../../dist/
> (resolved as:
> /Users/mak/solr-4.10.2/node1/solr/collection2_shard1_replica1/../../../dist).
> {noformat}
> Note the error for
> {{/Users/mak/solr-4.10.2/node1/solr/collection2_shard1_replica1/../../../dist}}.
> But that path does exist:
> {noformat}
> ls
> /Users/mak/solr-4.10.2/node1/solr/collection2_shard1_replica1/../../../dist |
> grep '\.jar$' | wc -l
> 32
> {noformat}
> but the `../../..` causes trouble here.
> The error message comes from
> https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java#L192
>
> If I modify the code to add a File.getCanonicalFile()
> ([gist|https://gist.github.com/makuk66/20b04e4a8e4ff682714f], which also has
> added some expanded error checking, which I've not verified), it works
> without error:
> To patch:
> {noformat}
> mak@crab 529 lucene-solr [detached] $ git co lucene_solr_4_10_2
> HEAD is now at be4a1a6... Lucene/Solr release 4.10.2
> mak@crab 545 lucene-solr [detached] $ curl -# -o p
> https://gist.githubusercontent.com/makuk66/20b04e4a8e4ff682714f/raw/c0981917ffad96e53939902e4a7938f2ee264d89/gistfile1.diff
>
> ########################################################################
> 100.0%
> mak@crab 546 lucene-solr [detached] $ patch -p1 < p
> patching file solr/core/src/java/org/apache/solr/core/SolrResourceLoader.java
> mak@crab 548 lucene-solr [detached] $ ant compile && (cd solr; ant dist-war)
> ...
> mak@crab 552 lucene-solr [detached] $ killall java
> mak@crab 554 lucene-solr [detached] $ rm -fr ~/solr-4.10.2/
> mak@crab 537 ~ $ tar xf ~/Downloads/solr-4.10.2.tgz
> mak@crab 541 ~ $ cp ~/github/lucene-solr/solr/dist/solr-4.10.2-SNAPSHOT.war
> solr-4.10.2/dist/solr-4.10.2.war
> mak@crab 544 ~ $ cp ~/github/lucene-solr/solr/dist/solr-4.10.2-SNAPSHOT.war
> solr-4.10.2//example/webapps/solr.war
> {noformat}
> Then repro as above, but with "collection3" instead of "collection2", and now
> we get:
> {noformat}
> INFO - 2014-12-23 19:12:34.406; org.apache.solr.core.SolrResourceLoader; new
> SolrResourceLoader for directory:
> '/Users/mak/solr-4.10.2/node1/solr/collection3_shard1_replica1/'
> INFO - 2014-12-23 19:12:34.453; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/Users/mak/solr-4.10.2/contrib/extraction/lib/apache-mime4j-core-0.7.2.jar'
> to classloader
> {noformat}
> So I conclude that the canonicalisation is desirable here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
