[
https://issues.apache.org/jira/browse/SOLR-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17349952#comment-17349952
]
xuanyu huang edited comment on SOLR-15417 at 5/23/21, 8:31 AM:
---------------------------------------------------------------
Hi Takashi,
I removed that line
syncDocFieldsWithSOLR(doc);
still same result, see the log,
{code:java}
2021-05-23 18:20:19,377 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=81cf5d59-a9a6-4f43-88ee-1f46202fdeaf to updateRequest
2021-05-23 18:20:19,378 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=278ebc89-cb34-40f0-b656-a642707f06a1 to updateRequest
2021-05-23 18:20:19,378 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=7b2eb702-15a5-4e70-89fa-f888cbdbc1e0_invalid to updateRequest
2021-05-23 18:20:19,378 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=0446987c-596c-42b6-a4e9-b0f53a24b8e5 to updateRequest
2021-05-23 18:20:19,378 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=6ea4e405-468e-428d-8e82-8f2a03038f52 to updateRequest
2021-05-23 18:20:28,120 DEBUG: [SolrIndexDAOImpl] - xhk205 process = {NOTE=the
request is processed in a background stream}2021-05-23 18:20:28,400 ERROR:
[ConcurrentUpdateSolrClient] -
errororg.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://localhost:8983/solr/biocache: Conflict
request:
http://localhost:8983/solr/biocache/update?commit=true&softCommit=false&waitSearcher=false&failOnVersionConflicts=false&wt=javabin&version=2Remote
error message: Document not found for update.
id=7b2eb702-15a5-4e70-89fa-f888cbdbc1e0_invalid at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:394)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:191)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor{code}
This is the query result:
{code:java}
{"responseHeader": {"zkConnected": true,"status": 0,"QTime": 2,"params": {"q":
"*:*","fq": "hasUserAssertions:true"}},"response": {"numFound": 2,"start":
0,"docs":
[
{"id": "81cf5d59-a9a6-4f43-88ee-1f46202fdeaf","scientificName": "Circus
assimilis","vernacularName": "Spotted Harrier","taxonConceptID":
"urn:lsid:biodiversity.org.au:afd.taxon:75801261-975f-436f-b1c7-d395a06dc067","dataResourceUid":
"dr0","month": 2,"year": 1994,"lft": 178256,"rgt": 178256,"decimalLatitude":
-29.3,"decimalLongitude": 115.6,"recordedBy": ["A (Tony) J McAleer"],"kingdom":
"Animalia","phylum": "Chordata","class": "Aves","order":
"Falconiformes","family": "Accipitridae","genus": "Circus","userAssertions":
"50005","hasUserAssertions": true,"lastAssertionDate":
"2021-05-23T08:19:38Z","assertionUserId": ["56528"]},
{"id": "278ebc89-cb34-40f0-b656-a642707f06a1","scientificName": "Circus
assimilis","vernacularName": "Spotted Harrier","taxonConceptID":
"urn:lsid:biodiversity.org.au:afd.taxon:75801261-975f-436f-b1c7-d395a06dc067","dataResourceUid":
"dr0","month": 2,"year": 1976,"lft": 178256,"rgt": 178256,"decimalLatitude":
-37.9985,"decimalLongitude": 144.8346,"recordedBy": ["RAOU VIC historical
observer code 363"],"kingdom": "Animalia","phylum": "Chordata","class":
"Aves","order": "Falconiformes","family": "Accipitridae","genus":
"Circus","userAssertions": "50005","hasUserAssertions":
true,"lastAssertionDate": "2021-05-23T08:19:59Z","assertionUserId":
["56528"]}]}{code}
Check the ids of the records and the lastAssertionsDate, "lastAssertionDate":
"2021-05-23T08:19:59Z"
I added an assertion to each record and then do the update, so the
lastAssertionDate will be a little ahead of update time but very close. I
cleared solr and database every time before the test. AssertionDate so close to
update time in log also proves the update is done in this test (not something
left uncleaned in database/solr)
was (Author: hxuanyu):
Hi Takashi,
I removed that line
syncDocFieldsWithSOLR(doc);
still same result, see the log,
{code:java}
2021-05-23 18:20:19,377 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=81cf5d59-a9a6-4f43-88ee-1f46202fdeaf to updateRequest
2021-05-23 18:20:19,378 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=278ebc89-cb34-40f0-b656-a642707f06a1 to updateRequest
2021-05-23 18:20:19,378 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=7b2eb702-15a5-4e70-89fa-f888cbdbc1e0_invalid to updateRequest
2021-05-23 18:20:19,378 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=0446987c-596c-42b6-a4e9-b0f53a24b8e5 to updateRequest
2021-05-23 18:20:19,378 DEBUG: [SolrIndexDAOImpl] - Added solr doc for record:
id=6ea4e405-468e-428d-8e82-8f2a03038f52 to updateRequest
2021-05-23 18:20:28,120 DEBUG: [SolrIndexDAOImpl] - xhk205 process = {NOTE=the
request is processed in a background stream}2021-05-23 18:20:28,400 ERROR:
[ConcurrentUpdateSolrClient] -
errororg.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://localhost:8983/solr/biocache: Conflict
request:
http://localhost:8983/solr/biocache/update?commit=true&softCommit=false&waitSearcher=false&failOnVersionConflicts=false&wt=javabin&version=2Remote
error message: Document not found for update.
id=7b2eb702-15a5-4e70-89fa-f888cbdbc1e0_invalid at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:394)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:191)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor{code}
This is the query result:
{code:java}
{"responseHeader": {"zkConnected": true,"status": 0,"QTime": 2,"params": {"q":
"*:*","fq": "hasUserAssertions:true"}},"response": {"numFound": 2,"start":
0,"docs":
[
{"id": "81cf5d59-a9a6-4f43-88ee-1f46202fdeaf","scientificName": "Circus
assimilis","vernacularName": "Spotted Harrier","taxonConceptID":
"urn:lsid:biodiversity.org.au:afd.taxon:75801261-975f-436f-b1c7-d395a06dc067","dataResourceUid":
"dr0","month": 2,"year": 1994,"lft": 178256,"rgt": 178256,"decimalLatitude":
-29.3,"decimalLongitude": 115.6,"recordedBy": ["A (Tony) J McAleer"],"kingdom":
"Animalia","phylum": "Chordata","class": "Aves","order":
"Falconiformes","family": "Accipitridae","genus": "Circus","userAssertions":
"50005","hasUserAssertions": true,"lastAssertionDate":
"2021-05-23T08:19:38Z","assertionUserId": ["56528"]},
{"id": "278ebc89-cb34-40f0-b656-a642707f06a1","scientificName": "Circus
assimilis","vernacularName": "Spotted Harrier","taxonConceptID":
"urn:lsid:biodiversity.org.au:afd.taxon:75801261-975f-436f-b1c7-d395a06dc067","dataResourceUid":
"dr0","month": 2,"year": 1976,"lft": 178256,"rgt": 178256,"decimalLatitude":
-37.9985,"decimalLongitude": 144.8346,"recordedBy": ["RAOU VIC historical
observer code 363"],"kingdom": "Animalia","phylum": "Chordata","class":
"Aves","order": "Falconiformes","family": "Accipitridae","genus":
"Circus","userAssertions": "50005","hasUserAssertions":
true,"lastAssertionDate": "2021-05-23T08:19:59Z","assertionUserId":
["56528"]}]}{code}
Check the ids of the records and the lastAssertionsDate, "lastAssertionDate":
"2021-05-23T08:19:59Z"
I added an assertion to each record and then do the update, so each update
includes the latest assertion date. I cleared solr and database every time
before the test. AssertionDate so close to update time in log also proves the
update is done in this test (not something left uncleaned in database/solr)
> exception in updateRequest caused all subsequent update fail
> ------------------------------------------------------------
>
> Key: SOLR-15417
> URL: https://issues.apache.org/jira/browse/SOLR-15417
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: UpdateRequestProcessors
> Affects Versions: 8.5.1
> Reporter: xuanyu huang
> Priority: Minor
>
> Hi there,
> I'm using solrj 8.8.2 for a 8.5.1 solr server. I have a list of records and
> in a for loop I construct an updateRequest to update each record.
> Code looks like this
> {code:java}
> for (Map<String, Object> map : maps) {
> if (map.containsKey("record_uuid")) {
> UpdateRequest updateRequest = new UpdateRequest();
> updateRequest.setAction( UpdateRequest.ACTION.COMMIT, false, false);
> SolrInputDocument doc = new SolrInputDocument();
> if (idx == 3) {
> doc.addField("id", map.get("record_uuid") + "_invalid");
> } else {
> doc.addField("id", map.get("record_uuid"));
> }
> idx++;
> doc.addField("hasUserAssertions", new HashMap<String, Object>() {{
> put("set", true); }});
> // this makes sure update only succeeds when record with specified id
> exists
> doc.addField("_version_", 1);
> logger.debug("Added solr doc for record: " + doc.get("id"));
> updateRequest.add(doc);
> try {
> updateRequest.setParam("failOnVersionConflicts", "false");
> UpdateResponse process = updateRequest.process(solrClient);
> System.out.println("xhk205 process = " + process.toString());
> } catch (Exception e) {
> logger.error("Failed to update solr doc, error message: " +
> e.getMessage(), e);
> }
> }{code}
> There are 5 requests in total and I intentionally set the id in 3rd request
> to be an invalid id so that updateRequet for 3rd record should fail. (This is
> to mimic the situation where the record to be updated no longer exists in
> solr, so I only want those updates with a valid id to succeed, those updates
> with an invalid id should fail/rejected instead of creating a new reocrd in
> solr, so I used __version__=1).
>
> Also I used the syntax to do partial update.
> The variable doc looks like this
> {code:java}
> {
> "id":"2d4b625d-8809-461f-b19b-d0c963e038ed",
> "hasUserAssertions":{"set":true}
> }
> {code}
>
> {color:#de350b}Since each update is put into its own request, I suppose only
> the 3rd request will fail because there's no record with that id and I've set
> __version__{color} {color:#de350b}to 1. But the reality is, only the first 2
> records were updated and other 3 not.{color}
> {color:#de350b}When I queried in solr admin console after the update, with
> [http://localhost:8983/solr/biocache/select?fq=hasUserAssertions:true&q=*:*]
> there were only 2 records returned instead of 4.{color}
>
> Below is the log of IntelliJ IDEA:
>
> {code:java}
> - Added solr doc for record: id=429cfa88-2e18-46b0-ab9f-f4efd9e36c3c
> xhk205 process = {NOTE=the request is processed in a background stream}
> - Added solr doc for record: id=5a80561b-a68d-46a3-a59b-03d267f35d0e
> xhk205 process = {NOTE=the request is processed in a background stream}
> - Added solr doc for record: id=ff2dcbee-9c05-491f-91a8-9f1fec348546_invalid
> xhk205 process = {NOTE=the request is processed in a background stream}
> - Added solr doc for record: id=baf7af1f-1525-403a-95bf-e28e432f1b12
> xhk205 process = {NOTE=the request is processed in a background stream}
> - Added solr doc for record: id=4ea76605-c262-409b-845e-213f11ea4e34
> xhk205 process = {NOTE=the request is processed in a background stream}{code}
> {code:java}
> 2021-05-19 14:12:16,827 ERROR: [ConcurrentUpdateSolrClient] - error
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
> from server at http://localhost:8983/solr/biocache: Conflict request:
> http://localhost:8983/solr/biocache/update?commit=true&softCommit=false&waitSearcher=false&failOnVersionConflicts=false&wt=javabin&version=2
> Remote error message: Document not found for update.
> id=ff2dcbee-9c05-491f-91a8-9f1fec348546_invalid at
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:394)
> at
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:191)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0{code}
>
>
> {color:#de350b}The 3rd update obviously caused an exception. But why 4th and
> 5th updates didn't succeed? Is it possible that this exception caused solr
> client or server in some non-useable state so all subsequent updates
> failed?{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
