[ https://issues.apache.org/jira/browse/SOLR-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062793#comment-14062793 ]
Nathan Neulinger commented on SOLR-6251: ---------------------------------------- 16.24 = POD SRV 16.204 = SOLR 1 16.207 = SOLR 2 16.24 ⇒ 16.204 CAP 1 11344 14:29:49.299883 POST /solr/d-_v22/update/json?commit=true HTTP/1.1 host: d01-solr.srv.hivepoint.com Accept-Encoding: gzip,deflate Content-Type: application/json; charset=UTF-8 request_id: null 8677c2fb-8b92-4220-bb73-1e4c610d95be 2057 User-Agent: HivePoint (Factory JSON client:null:2056) X-Forwarded-For: 10.220.16.229 X-Forwarded-Port: 80 X-Forwarded-Proto: http Content-Length: 1555 Connection: keep-alive { "add": { "commitWithin" : 5000, "doc" : {"hive":"vdates","at":"2014-07-10T21:28:41Z","timestamp":1405027721000,"type":"MESSAGE","channel":["dev"],"from":"pr...@sevogle.com","to":["a...@sevogle.com","vi...@sevogle.com","d...@sevogle.com","s...@hive.sevogle.com"],"subject":"Re: Deployments - B and then C","body":"eve.....SNIP.......stem. ","id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1","message_id":"2014-07-10-77a6614c-66e4-4ddb-8566-dff4bfb743d1"} } } 16.204 ⇒ 16.207 CAP 1 POST /solr/d-_v22_shard1_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F10.220.16.204%3A8983%2Fsolr%2Fd-_v22_shard1_replica1%2F&wt=javabin&version=2 HTTP/1.1 User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrServer] 1.0 Content-Type: application/javabin Transfer-Encoding: chunked Host: 10.220.16.207:8983 Connection: Keep-Alive 64c ...¶ms...update.distrib(TOLEADER.,distrib.from?.http://10.220.16.204:8983/solr/d-_v22_shard1_replica1/.&delByQ..'docsMap.....?....$hive&vdates."at42014-07-10T21:28:41Z.)timestampx.......$type'MESSAGE.'channel.#dev.$from1pr...@sevogle.com."to.0adam@sevogle.com1vi...@sevogle.com/dev@sevogle.com4...@hive.sevogle.com.'subject>Re: Deployments - B and then C.$body?#eve.....SNIP.......tem. ."id?.4b2c4d09-31e2-4fe2-b767-3868efbdcda1.*message_id?.2014-07-10-77a6614c-66e4-4ddb-8566-dff4bfb743d1 .."ow.."cwX... 0 16.207 ⇒ 16.204 CAP 1 11368 14:29:49.495301 HTTP/1.1 200 OK Content-Type: application/octet-stream Content-Length: 40 ....responseHeader..&status......%QTimeK 16.24 ⇒ 16.204 CAP 1 11371 14:29:49.496308 INDEX COMPLETE HTTP/1.1 200 OK Content-Type: text/plain;charset=UTF-8 Transfer-Encoding: chunked 2C {"responseHeader":{"status":0,"QTime":195}} 0 16.24 ⇒ 16.207 CAP 2 9218 14:29:57.065156 9232 14:29”57.099274 Search (two different search results to two servers?) that show the timestamp is set. POST /solr/d-_v22/select?indent=on&wt=json HTTP/1.1 host: d01-solr.srv.hivepoint.com Accept-Encoding: gzip,deflate Content-Type: application/x-www-form-urlencoded; charset=UTF-8 request_id: null 957d1ca5-7200-4058-9c70-16a17fc64c19 2069 User-Agent: HivePoint (Factory JSON client:null:2068) X-Forwarded-For: 10.220.16.229 X-Forwarded-Port: 80 X-Forwarded-Proto: http Content-Length: 244 Connection: keep-alive q=%2B%28*%29&fq=%2Bhive%3Avdates+AND+%2Bchannel%3A%28adam+bethany+dev+notifications+preet+share%29+AND+at%3A%5B2014-07-10T21%3A27%3A56Z+TO+*%5D&start=0&rows=300&sort=at+desc%2C+id+desc&fl=id,hive,timestamp,type,message_id,file_instance_id,scoreHTTP/1.1 200 OK Content-Type: text/plain;charset=UTF-8 Transfer-Encoding: chunked 2BB { "responseHeader":{ "status":0, "QTime":3, "params":{ "fl":"id,hive,timestamp,type,message_id,file_instance_id,score", "sort":"at desc, id desc", "indent":"on", "start":"0", "q":"+(*)", "wt":"json", "fq":"+hive:vdates AND +channel:(adam bethany dev notifications preet share) AND at:[2014-07-10T21:27:56Z TO *]", "rows":"300"}}, "response":{"numFound":1,"start":0,"maxScore":1.0,"docs":[ { "hive":"vdates", "timestamp":1405027721000, "type":"MESSAGE", "id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1", "message_id":"2014-07-10-77a6614c-66e4-4ddb-8566-dff4bfb743d1", "score":1.0}] }} 0 16.24 ⇒ 16.207 CAP 2 9415 14:30:00.310995 Update Channel POST /solr/d-_v22/update?commit=true HTTP/1.1 host: d01-solr.srv.hivepoint.com Accept-Encoding: gzip,deflate Content-Type: application/json; charset=UTF-8 request_id: null 92fa6c11-78d8-44cc-a143-9ff3e4c132f4 2115 User-Agent: HivePoint (Factory JSON client:null:2114) X-Forwarded-For: 10.220.16.229 X-Forwarded-Port: 80 X-Forwarded-Proto: http Content-Length: 102 Connection: keep-alive [{"id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1","channel": {"add": "preet"},"channel": {"add": "adam"}}]HTTP/1.1 400 Bad Request Content-Type: text/plain;charset=UTF-8 Transfer-Encoding: chunked 96 {"responseHeader":{"status":400,"QTime":1},"error":{"msg":"[doc=4b2c4d09-31e2-4fe2-b767-3868efbdcda1] missing required field: timestamp","code":400}} 0 CAP 2 9602 14:30:08.082758 Subsequent search, after update POST /solr/d-_v22/select?indent=on&wt=json HTTP/1.1 host: d01-solr.srv.hivepoint.com Accept-Encoding: gzip,deflate Content-Type: application/x-www-form-urlencoded; charset=UTF-8 request_id: null 196bee69-e79c-455e-b0cb-6ad6ecdab4e0 1813 User-Agent: HivePoint (Factory JSON client:null:1811) X-Forwarded-For: 10.220.16.230 X-Forwarded-Port: 80 X-Forwarded-Proto: http Content-Length: 261 Connection: keep-alive q=%2B%28*+-%28%28foo%29%29%29&fq=%2Bhive%3Avdates+AND+%2Bchannel%3A%28adam+bethany+dev+notifications+preet+share%29+AND+at%3A%5B2014-07-10T21%3A28%3A07Z+TO+*%5D&start=0&rows=300&sort=at+desc%2C+id+desc&fl=id,hive,timestamp,type,message_id,file_instance_id,scoreHTTP/1.1 200 OK Content-Type: text/plain;charset=UTF-8 Transfer-Encoding: chunked 2C4 { "responseHeader":{ "status":0, "QTime":3, "params":{ "fl":"id,hive,timestamp,type,message_id,file_instance_id,score", "sort":"at desc, id desc", "indent":"on", "start":"0", "q":"+(* -((foo)))", "wt":"json", "fq":"+hive:vdates AND +channel:(adam bethany dev notifications preet share) AND at:[2014-07-10T21:28:07Z TO *]", "rows":"300"}}, "response":{"numFound":1,"start":0,"maxScore":1.0,"docs":[ { "hive":"vdates", "timestamp":1405027721000, "type":"MESSAGE", "id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1", "message_id":"2014-07-10-77a6614c-66e4-4ddb-8566-dff4bfb743d1", "score":1.0}] }} 0 > incorrect 'missing required field' during update - document definitely has it > ----------------------------------------------------------------------------- > > Key: SOLR-6251 > URL: https://issues.apache.org/jira/browse/SOLR-6251 > Project: Solr > Issue Type: Bug > Components: SolrCloud > Affects Versions: 4.8 > Environment: 4.8.0. Two nodes, SolrCloud, external ZK ensemble. All > on EC2. The two hosts are round-robin'd behind an ELB. > Reporter: Nathan Neulinger > Labels: replication > > Document added on solr1. We can see the distribute take place from solr1 to > solr2 and returning a success. Subsequent searches returning document, > clearly showing the field as being there. Later on, an update is done to add > to an element of the document - and the update fails. The update was sent to > solr2 instance. > Schema marks the 'timestamp' field as required, so the initial insert should > not work if the field isn't present. > Symptom is intermittent - we're seeing this randomly, with no warning or > triggering that we can see, but in all cases, it's getting the error in > response to an update when the instance tries to distribute the change to the > other node. > Searches that were run AFTER the update also show the field as being present > in the document. > Will add full trace of operations in the comments shortly. pcap captures of > ALL traffic for the two nodes on 8983 is available if requested. -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org