Hi Hendrik and Shalin, Really appreciate your valuable inputs on this.
I looked up to the two issues that were being referred to (SOLR-8804 and SOLR-10720) and thats exactly what I'm running into. Glad they have been fixed in later versions. Thanks much , Atita On Tue, Mar 13, 2018 at 10:38 AM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > I think you are running into race conditions in the API which have been > fixed. See SOLR-8804 and SOLR-10720. The first is available in 5.5.1 but > the latter fix will be released in the upcoming 7.3 release. The best > workaround for your version is to just retry a few times until the API > succeeds. > > On Sun, Mar 11, 2018 at 11:57 PM, Atita Arora <atitaar...@gmail.com> > wrote: > > > Hi , > > > > I am working on an application which involves working on a highly > > distributed Solr cloud environment. The application supports > multi-tenancy > > and we have around 250-300 collections on Solr where each client has > their > > own collection with a new shard being created as > clientid-<timestamp_long> > > where the timestamp is whenever the new data comes in for the client > > (typically every 4-8 hrs) , the reason for this convention is to make > sure > > when the Indexes are being built (on demand) the timestamp matches > closely > > to the time when the last indexing was run (the earlier shard is > > de-provisioned as soon as the new one is created). Whenever the indexing > is > > triggered it first makes a DB entry and then creates a catalog with > > timestamp in solr. > > The Solr cloud has 10 Nodes distributed geographically among 10 > > datacenters. > > The replication factor is 2. The Solr version is 5.3.2. > > Coming to my problem - I had to write a utility to ensure that the DB > > insert timestamp matches closely to the Solr index timestamp wherein I > can > > ensure that if the difference between DB timestamp and Solr Index > tinestamp > > is <= 2 hrs , we have fresh index. The new index contains revised prices > of > > products or offers etc which are critical to be updated as in when they > > come. Hence this utility is to track that the required updates have been > > successfully made. > > I used *CLUSTERSTATUS* api for this task. It is serving the purpose well > so > > far , but pretty recently our solr cloud started complaining of strange > > things because of which the *CLUSTERSTATUS* api keeps returning as error. > > > > The error claims to be of missing config & sometime missing collections > > like. > > > > org.apache.solr.common.SolrException: Could not find collection : > > > 1785-1520548816454 > > > > org.apache.solr.common.SolrException: Could not find collection : > > 1785-1520548816454 > > at > > org.apache.solr.common.cloud.ClusterState.getCollection( > > ClusterState.java:165) > > at > > org.apache.solr.handler.admin.ClusterStatus.getClusterStatus( > > ClusterStatus.java:110) > > at > > org.apache.solr.handler.admin.CollectionsHandler$ > > CollectionOperation$19.call(CollectionsHandler.java:614) > > at > > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody( > > CollectionsHandler.java:166) > > at > > org.apache.solr.handler.RequestHandlerBase.handleRequest( > > RequestHandlerBase.java:143) > > at > > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest( > > HttpSolrCall.java:678) > > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444) > > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter( > > SolrDispatchFilter.java:215) > > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter( > > SolrDispatchFilter.java:179) > > at > > org.eclipse.jetty.servlet.ServletHandler$CachedChain. > > doFilter(ServletHandler.java:1652) > > at > > org.eclipse.jetty.servlet.ServletHandler.doHandle( > ServletHandler.java:585) > > > > The other times it would complain of missing the config for same or > > different client id- timestamp like : > > > > 1532-1518669619526_shard1_replica3: > > org.apache.solr.common.cloud.ZooKeeperException:org.apache. > > solr.common.cloud.ZooKeeperException: > > Specified config does not exist in ZooKeeper:1532-1518669619526I > > > > I would really appreciate if : > > > > > > 1. Someone can possibly guide me as to whats going on Solr Cloud > > 2. If CLUSTERSTATUS is the right pick to build such utility. Do we > > have any other option? > > > > > > Thanks for any pointers and suggestions. > > > > Appreciate your attention looking this through. > > > > Atita > > > > > > -- > Regards, > Shalin Shekhar Mangar. >