I just checked the shards again (with &distrib=false) and it seems that
I was mistaken, the document does *not* reside in _different_ shards -
everything good in this respect.
However, I still have the issue that deleteById those not work whereas
deleteByQuery works. Specifically, the following line does *not* work:
UpdateResponse response = solrClient.deleteById(collection, <id>);
And the following line works:
UpdateResponse response = solrClient.deleteByQuery(collection, "id:" +
<id>);
I do not touch/change any other code when switching between these two
modes and in both scenarios I use CloudSolrClient.
Am 31.05.2016 um 05:32 schrieb Erick Erickson:
> bq: I checked in the Solr Admin and noticed that the same document
> resided in both shards on the same node....
>
> If this means two _different_ shards (as opposed to two replicas in
> the _same_ shard) showed the
> document, then that's the proverbial "smoking gun", somehow your setup
> isn't what you think
> it is, perhaps you are somehow using implicit routing and routing the
> doc with the same ID to
> two different shards?
>
> try querying each of your replicas with &distrib=false to see if the
> doc is somehow on two different
> shards. If so, I suspect that's the root of your problems and figuring
> out _how_ that happened
> is the next step I'd recommend.
>
> As to why the raw URL deletes should work and CloudSolrClient doesn't,
> CloudSolrClient
> tries to send updates only to the shard that they should end up on. So
> if your routing is
> odd or you somehow have the same doc on two shards, the "wrong" shard wouldn't
> see the delete. There's some speculation here BTW, I didn't trace
> through the code...
>
> But this functionality is tested in the unit tests
> (CloudSolrClientTest.java), so I suspect it's
> something odd in your setup....
>
> Best,
> Erick
>
> On Mon, May 30, 2016 at 12:33 PM, Moritz Becker <moritz.bec...@gmx.at> wrote:
>> Hi,
>>
>> I have the following issue:
>> I initially started with a Solr 5.3.1 + Zookeeper 3.4.6 cloud setup with 2
>> solr nodes and with one collection consisting of 2 shards and 2 replicas.
>>
>> I am accessing the cluster using the CloudSolrClient. When I tried to delete
>> a document, no error occurred but after deletion and subsequent commit, the
>> document was still available via index queries.
>> I checked in the Solr Admin and noticed that the same document resided in
>> both shards on the same node which I thought was odd.
>> Also after deleting the collection and recreating it, the issue remained.
>>
>> Then I tried upgrading to latest Solr 6.0.1 with the same setup. Again, I
>> recreated the collection but I still could not delete the documents. Here is
>> a log snippet of the deletion attempt of a single document:
>>
>> --------------------------------------------
>>
>> 126023 INFO (qtp12209492-16) [c:cc5363_dm_documentversion s:shard1
>> r:core_node4 x:cc5363_dm_documentversion_shard1_replica1]
>> o.a.s.u.p.LogUpdateProcessorFactory
>> [cc5363_dm_documentversion_shard1_replica1] webapp=/solr path=/update
>> params={update.distrib=FROMLEADER&distrib.from=http://localhost:8983/solr/cc5363_dm_documentversion_shard1_replica2/&wt=javabin&version=2}{delete=[100002535
>> (-1535773473331216384)]} 0 16
>> 126024 INFO (commitScheduler-15-thread-1) [c:cc5363_dm_documentversion
>> s:shard1 r:core_node4 x:cc5363_dm_documentversion_shard1_replica1]
>> o.a.s.u.DirectUpdateHandler2 start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>> 126036 INFO (commitScheduler-15-thread-1) [c:cc5363_dm_documentversion
>> s:shard1 r:core_node4 x:cc5363_dm_documentversion_shard1_replica1]
>> o.a.s.c.SolrCore SolrIndexSearcher has not changed - not re-opening:
>> org.apache.solr.search.SolrIndexSearcher
>> 126038 INFO (commitScheduler-15-thread-1) [c:cc5363_dm_documentversion
>> s:shard1 r:core_node4 x:cc5363_dm_documentversion_shard1_replica1]
>> o.a.s.u.DirectUpdateHandler2 end_commit_flush
>> 126049 INFO (qtp12209492-20) [c:cc5363_dm_documentversion s:shard2
>> r:core_node1 x:cc5363_dm_documentversion_shard2_replica1]
>> o.a.s.u.DirectUpdateHandler2 start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
>> 126050 INFO (qtp12209492-20) [c:cc5363_dm_documentversion s:shard2
>> r:core_node1 x:cc5363_dm_documentversion_shard2_replica1]
>> o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
>> 126051 INFO (qtp12209492-19) [c:cc5363_dm_documentversion s:shard1
>> r:core_node4 x:cc5363_dm_documentversion_shard1_replica1]
>> o.a.s.u.DirectUpdateHandler2 start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
>> 126054 INFO (qtp12209492-20) [c:cc5363_dm_documentversion s:shard2
>> r:core_node1 x:cc5363_dm_documentversion_shard2_replica1] o.a.s.c.SolrCore
>> SolrIndexSearcher has not changed - not re-opening:
>> org.apache.solr.search.SolrIndexSearcher
>> 126056 INFO (qtp12209492-20) [c:cc5363_dm_documentversion s:shard2
>> r:core_node1 x:cc5363_dm_documentversion_shard2_replica1]
>> o.a.s.u.DirectUpdateHandler2 end_commit_flush
>> 126055 INFO (qtp12209492-19) [c:cc5363_dm_documentversion s:shard1
>> r:core_node4 x:cc5363_dm_documentversion_shard1_replica1]
>> o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
>> 126057 INFO (qtp12209492-20) [c:cc5363_dm_documentversion s:shard2
>> r:core_node1 x:cc5363_dm_documentversion_shard2_replica1]
>> o.a.s.u.p.LogUpdateProcessorFactory
>> [cc5363_dm_documentversion_shard2_replica1] webapp=/solr path=/update
>> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://localhost:8983/solr/cc5363_dm_documentversion_shard2_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
>> 0 10
>> 126059 INFO (qtp12209492-19) [c:cc5363_dm_documentversion s:shard1
>> r:core_node4 x:cc5363_dm_documentversion_shard1_replica1] o.a.s.c.SolrCore
>> SolrIndexSearcher has not changed - not re-opening:
>> org.apache.solr.search.SolrIndexSearcher
>> 126063 INFO (qtp12209492-19) [c:cc5363_dm_documentversion s:shard1
>> r:core_node4 x:cc5363_dm_documentversion_shard1_replica1]
>> o.a.s.u.DirectUpdateHandler2 end_commit_flush
>> 126064 INFO (qtp12209492-19) [c:cc5363_dm_documentversion s:shard1
>> r:core_node4 x:cc5363_dm_documentversion_shard1_replica1]
>> o.a.s.u.p.LogUpdateProcessorFactory
>> [cc5363_dm_documentversion_shard1_replica1] webapp=/solr path=/update
>> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://localhost:8983/solr/cc5363_dm_documentversion_shard2_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
>> 0 13
>> --------------------------------------------
>>
>> I used the CloudSolrClient.deleteById(collection, id); to delete the
>> document.
>>
>> According to the logs, Solr thinks that nothing has changed and does not
>> recreate the searcher so I tried to restart the instances but the document
>> was still there.
>> Finally, I was able to manually delete the document via the following
>> request:
>>
>> POST
>> http://localhost:7574/solr/cc5363_dm_documentversion_shard2_replica1/update?commit=true
>> <delete><query>id:100002535</query></delete>
>>
>> So I tried CloudSolrClient.deleteByQuery(collection, "id:" + id); and it
>> worked as well.
>> This could be a bug in the CloudSolrClient.deleteById method or in the
>> server side handler.
>>
>> Since my schema defines a uniqueKey, deleteById should work, right? Here is
>> the relevant schema snippet:
>>
>> <field name="id" type="int" indexed="true" stored="true" required="true"
>> multiValued="false"/>
>> <fieldType name="int" class="solr.TrieIntField" precisionStep="0"
>> positionIncrementGap="0"/>
>> <uniqueKey>id</uniqueKey>
>>
>> I also tried string type for id but it did not make any difference.
>>
>>
>> <field name="id" type="string" indexed="true" stored="true" required="true"
>> multiValued="false"/>
>> <fieldType name="string" class="solr.StrField" sortMissingLast="true" />
>>
>> Thanks,
>>
>> Moritz
