You still haven't answered _why_ you think splitting even a 20G index is desirable. We regularly see 200G+ indexes per replica in the field, so what's the point? Have you measured different setups to see if it's a good idea? A 200G index needs some beefy hardware admittedly....
If you have adequate response times with a 20G index and need to increase the QPS rate, just add more replicas. Having more than one shard inevitably adds overhead which may (or may not) be made up for by parallelizing some of the work. It's nearly always better to use only one shard if it meets your response time requirements. Best, Erick On Wed, Jun 20, 2018 at 10:39 AM, Sushant Vengurlekar <svengurle...@curvolabs.com> wrote: > The index size is small because this is my local development copy. The > production index is more than 20GB. So I am working on getting the index > split and replicated on different nodes. Our current instance on prod is > single instance solr 6 which we are working on moving towards solrcloud 7 > > On Wed, Jun 20, 2018 at 10:30 AM Erick Erickson <erickerick...@gmail.com> > wrote: > >> Use the indexupgrader tool or optimize your index before using splitshard. >> >> Since this is a small index (< 5G), optimizing will not create an >> overly-large segment, so that pitfall is avoided. >> >> You haven't yet explained why you think splitting the index would be >> beneficial. Splitting an index this small is unlikely to improve query >> performance appreciably. This feels a lot like an "XY" problem, you're >> asking how to do X thinking it will solve Y but not telling us what Y >> is. >> >> Best, >> Erick >> >> On Wed, Jun 20, 2018 at 9:40 AM, Sushant Vengurlekar >> <svengurle...@curvolabs.com> wrote: >> > How can I resolve this error? >> > >> > On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch < >> arafa...@gmail.com> >> > wrote: >> > >> >> This seems more related to an old index upgraded to latest Solr rather >> than >> >> the split itself. >> >> >> >> Regards, >> >> Alex >> >> >> >> On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, < >> >> svengurle...@curvolabs.com> wrote: >> >> >> >> > Thanks for the reply Alessandro! Appreciate it. >> >> > >> >> > Below is the full request and the error received >> >> > >> >> > curl ' >> >> > >> >> > http://localhost:8081/solr/admin/collections?action= >> >> SPLITSHARD&collection=dev-transactions&shard=shard1 >> >> > ' >> >> > >> >> > { >> >> > >> >> > "responseHeader":{ >> >> > >> >> > "status":500, >> >> > >> >> > "QTime":7920}, >> >> > >> >> > "success":{ >> >> > >> >> > "solr-1:8081_solr":{ >> >> > >> >> > "responseHeader":{ >> >> > >> >> > "status":0, >> >> > >> >> > "QTime":1190}, >> >> > >> >> > "core":"dev-transactions_shard1_0_replica_n3"}, >> >> > >> >> > "solr-1:8081_solr":{ >> >> > >> >> > "responseHeader":{ >> >> > >> >> > "status":0, >> >> > >> >> > "QTime":1047}, >> >> > >> >> > "core":"dev-transactions_shard1_1_replica_n4"}, >> >> > >> >> > "solr-1:8081_solr":{ >> >> > >> >> > "responseHeader":{ >> >> > >> >> > "status":0, >> >> > >> >> > "QTime":6}}, >> >> > >> >> > "solr-1:8081_solr":{ >> >> > >> >> > "responseHeader":{ >> >> > >> >> > "status":0, >> >> > >> >> > "QTime":1009}}}, >> >> > >> >> > "failure":{ >> >> > >> >> > >> >> > "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$ >> >> RemoteSolrException:Error >> >> > from server at http://solr-1:8081/solr: >> >> > java.lang.IllegalArgumentException: >> >> > Cannot merge a segment that has been created with major version 6 into >> >> this >> >> > index which has been created by major version 7"}, >> >> > >> >> > "Operation splitshard caused >> >> > >> >> > exception:":"org.apache.solr.common.SolrException:org. >> >> apache.solr.common.SolrException: >> >> > SPLITSHARD failed to invoke SPLIT core admin command", >> >> > >> >> > "exception":{ >> >> > >> >> > "msg":"SPLITSHARD failed to invoke SPLIT core admin command", >> >> > >> >> > "rspCode":500}, >> >> > >> >> > "error":{ >> >> > >> >> > "metadata":[ >> >> > >> >> > "error-class","org.apache.solr.common.SolrException", >> >> > >> >> > "root-error-class","org.apache.solr.common.SolrException"], >> >> > >> >> > "msg":"SPLITSHARD failed to invoke SPLIT core admin command", >> >> > >> >> > "trace":"org.apache.solr.common.SolrException: SPLITSHARD failed >> to >> >> > invoke SPLIT core admin command\n\tat >> >> > >> >> > org.apache.solr.client.solrj.SolrResponse.getException( >> >> SolrResponse.java:53)\n\tat >> >> > >> >> > org.apache.solr.handler.admin.CollectionsHandler.invokeAction( >> >> CollectionsHandler.java:258)\n\tat >> >> > >> >> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody( >> >> CollectionsHandler.java:230)\n\tat >> >> > >> >> > org.apache.solr.handler.RequestHandlerBase.handleRequest( >> >> RequestHandlerBase.java:195)\n\tat >> >> > >> >> > org.apache.solr.servlet.HttpSolrCall.handleAdmin( >> >> HttpSolrCall.java:736)\n\tat >> >> > >> >> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest( >> >> HttpSolrCall.java:717)\n\tat >> >> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat >> >> > >> >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter( >> >> SolrDispatchFilter.java:384)\n\tat >> >> > >> >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter( >> >> SolrDispatchFilter.java:330)\n\tat >> >> > >> >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain. >> >> doFilter(ServletHandler.java:1629)\n\tat >> >> > >> >> > org.eclipse.jetty.servlet.ServletHandler.doHandle( >> >> ServletHandler.java:533)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ScopedHandler.handle( >> >> ScopedHandler.java:143)\n\tat >> >> > >> >> > org.eclipse.jetty.security.SecurityHandler.handle( >> >> SecurityHandler.java:548)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle( >> >> HandlerWrapper.java:132)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ScopedHandler. >> >> nextHandle(ScopedHandler.java:190)\n\tat >> >> > >> >> > org.eclipse.jetty.server.session.SessionHandler. >> >> doHandle(SessionHandler.java:1595)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ScopedHandler. >> >> nextHandle(ScopedHandler.java:188)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ContextHandler. >> >> doHandle(ContextHandler.java:1253)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ScopedHandler. >> >> nextScope(ScopedHandler.java:168)\n\tat >> >> > >> >> > org.eclipse.jetty.servlet.ServletHandler.doScope( >> >> ServletHandler.java:473)\n\tat >> >> > >> >> > org.eclipse.jetty.server.session.SessionHandler. >> >> doScope(SessionHandler.java:1564)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ScopedHandler. >> >> nextScope(ScopedHandler.java:166)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ContextHandler. >> >> doScope(ContextHandler.java:1155)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ScopedHandler.handle( >> >> ScopedHandler.java:141)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle( >> >> ContextHandlerCollection.java:219)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.HandlerCollection. >> >> handle(HandlerCollection.java:126)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle( >> >> HandlerWrapper.java:132)\n\tat >> >> > >> >> > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle( >> >> RewriteHandler.java:335)\n\tat >> >> > >> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle( >> >> HandlerWrapper.java:132)\n\tat >> >> > org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat >> >> > >> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat >> >> > >> >> > org.eclipse.jetty.server.HttpConnection.onFillable( >> >> HttpConnection.java:256)\n\tat >> >> > org.eclipse.jetty.io >> >> > >> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\ >> >> n\tat >> >> > org.eclipse.jetty.io >> .FillInterest.fillable(FillInterest.java:102)\n\tat >> >> > org.eclipse.jetty.io >> .ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\ >> >> tat >> >> > >> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill. >> >> doProduce(EatWhatYouKill.java:247)\n\tat >> >> > >> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill. >> >> produce(EatWhatYouKill.java:140)\n\tat >> >> > >> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run( >> >> EatWhatYouKill.java:131)\n\tat >> >> > >> >> > >> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run( >> >> ReservedThreadExecutor.java:382)\n\tat >> >> > >> >> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( >> >> QueuedThreadPool.java:708)\n\tat >> >> > >> >> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run( >> >> QueuedThreadPool.java:626)\n\tat >> >> > java.lang.Thread.run(Thread.java:748)\n", >> >> > >> >> > "code":500}} >> >> > >> >> > >> >> > On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti < >> >> > a.benede...@sease.io> >> >> > wrote: >> >> > >> >> > > Hi, >> >> > > in the first place, why do you want to split 2 Gb indexes ? >> >> > > Nowadays is a fairly small index. >> >> > > >> >> > > Secondly what you reported is incomplete. >> >> > > I would expect a Caused By section in the stacktrace. >> >> > > >> >> > > This are generic recommendations, always spend time in analysing the >> >> > > problem >> >> > > you had scrupulously. >> >> > > - SolrCloud problems often involve more than one node. Be sure to >> check >> >> > the >> >> > > logs of all the nodes possibly involved. >> >> > > - Report the full stack trace to the community >> >> > > - Report your full request which provoked the exception >> >> > > >> >> > > Help is much easier this way :) >> >> > > >> >> > > Regards >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > ----- >> >> > > --------------- >> >> > > Alessandro Benedetti >> >> > > Search Consultant, R&D Software Engineer, Director >> >> > > Sease Ltd. - www.sease.io >> >> > > -- >> >> > > Sent from: >> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >> >> > > >> >> > >> >> >>
