Re: Elevation in dataDir in Solr Cloud
Thank you! I just filed the bug in Jira: https://issues.apache.org/jira/browse/SOLR-15170 About the workaround you mentioned, we ran a quick test on one server and it apparently worked, but we did not check it properly in a cluster (we decided that it is better not to go with this in production anyway). Just out of curiosity, even if we could load the elevate.xml file from the data folder in Cloud (or changing it directly in zk), does this make sense in terms of performance? We have frequent commits and a big elevate.xml file. We are considering your suggestion of using the elevation directly in the queries (I have seen work to improve this by removing the requirement of having an elevate.xml file at all). It seems to be straightforward to apply in some cases, but not so much when you need normalization. -- Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. If you are not the named addressee you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
Re: Elevation in dataDir in Solr Cloud
: Of course, here is the full stack trace (collection 'techproducts' with : just one core to make it easier): Ah yeah ... see -- this looks like a mistake introduced at some point... : Caused by: org.apache.solr.core.SolrResourceNotFoundException: Can't : find resource 'elevate.xml' in classpath or : '/configs/techproductsConfExp', cwd=/usr/share/solr-7.7.2/server : at org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:130) : at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:362) : at org.apache.solr.core.Config.(Config.java:120) : at org.apache.solr.core.Config.(Config.java:90) : at org.apache.solr.handler.component.QueryElevationComponent.loadElevationProvider(QueryElevationComponent.java:366) ...this bit of code is *expecting* to be able to init a Config object from the SolrResourceLoader, even thought this bit of code... : at org.apache.solr.handler.component.QueryElevationComponent.getElevationProvider(QueryElevationComponent.java:321) : at org.apache.solr.handler.component.QueryElevationComponent.loadElevationConfiguration(QueryElevationComponent.java:259) ...has already established that there is no "Config" file available from the resource loader, and we should be initializing an ElevationProvider that can raed from the data dir. 9and this code seems to be unchanged on branch_8x) Can you please file a jira pointing out that this doesn't work along with the full stack trace, and then add a comment copy/pasting my comments here that the code makes no sense? I'm not sure if/when someone who understands the code well enough will be able to help fix this (and write a test for it) ... was the experiment / work around i suggested viable? ... : > I don't know if it will work, but one thing you might want to experiment : > with is putting your elevate.xml back the configset in zk, and updating it : > on the fly in zk -- then see if it gets reloaded by each core the next : > time the index changes (NOTE that there will almost certainly need to be : > an index change for it to re-load, since I don't see any indication that : > it's watching for changes in zk) : > : > FWIW: the way most people seem to be using QEC these days is to have an : > empty elevate.xml file, and then have their application use some other : > key/val store, or more complex matching logic, to decide which documents : > to elevate, and then use the "elevateIds" param to pass that info to solr. -Hoss http://www.lucidworks.com/
Re: Elevation in dataDir in Solr Cloud
Of course, here is the full stack trace (collection 'techproducts' with just one core to make it easier): org.apache.solr.common.SolrException: Unable to reload core [techproducts2_shard1_replica_n1] at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1472) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$2(CoreAdminOperation.java:131) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent at org.apache.solr.core.SolrCore.(SolrCore.java:1048) at org.apache.solr.core.SolrCore.reload(SolrCore.java:666) at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1448) ... 44 more Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent at org.apache.solr.handler.component.QueryElevationComponent.handleInitializationException(QueryElevationComponent.java:277) at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:165)
Re: Elevation in dataDir in Solr Cloud
: I need to have the elevate.xml file updated frequently and I was wondering : if it is possible to put this file in the dataDir folder when using Solr : Cloud. I know that this is possible in the standalone mode, and I haven't : seen in the documentation [1] that it can not be done in Cloud. : : I am using Solr 7.7.2 and ZooKeeper. After creating the techproducts : collection for the tests, I remove the elevate.xml file from the : configuration and I put it in the dataDir folder of the cores. When I : update the collection with that configuration, I get the following error: : "Can't find resource 'elevate.xml' in classpath or : '/configs/techproductsConfExp'". Is this expected or I am doing something : wrong? Hmmm... can you share the full stack trace of that error? (I suspect at some point someone made a sloppy assumption in the QEC code that no one would ever try to keep elevate.xml in the data dir in cloud mode.) I don't know if it will work, but one thing you might want to experiment with is putting your elevate.xml back the configset in zk, and updating it on the fly in zk -- then see if it gets reloaded by each core the next time the index changes (NOTE that there will almost certainly need to be an index change for it to re-load, since I don't see any indication that it's watching for changes in zk) FWIW: the way most people seem to be using QEC these days is to have an empty elevate.xml file, and then have their application use some other key/val store, or more complex matching logic, to decide which documents to elevate, and then use the "elevateIds" param to pass that info to solr. -Hoss http://www.lucidworks.com/
Elevation in dataDir in Solr Cloud
Hi, I need to have the elevate.xml file updated frequently and I was wondering if it is possible to put this file in the dataDir folder when using Solr Cloud. I know that this is possible in the standalone mode, and I haven't seen in the documentation [1] that it can not be done in Cloud. I am using Solr 7.7.2 and ZooKeeper. After creating the techproducts collection for the tests, I remove the elevate.xml file from the configuration and I put it in the dataDir folder of the cores. When I update the collection with that configuration, I get the following error: "Can't find resource 'elevate.xml' in classpath or '/configs/techproductsConfExp'". Is this expected or I am doing something wrong? Thanks a lot for your help. [1] https://lucene.apache.org/solr/guide/7_7/the-query-elevation-component.html -- Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. If you are not the named addressee you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
