Perhaps - although missing_content_stream seems to imply that it had at least partly read 4 requests which later failed. Also, wouldn't there be something in the output log which would give us a clue as to what happened?
Is there any post-hiccup spelunking I can reasonably do? Or should I try to reproduce the problem with more diagnostics on? Karl -----Original Message----- From: ext Bernd Fondermann [mailto:[email protected]] Sent: Monday, June 07, 2010 10:54 AM To: [email protected] Subject: Re: Solr spewage and dropped documents, while indexing Looks like a server-side problem to me. Maybe the server ran out of sockets or other resources and just replied with a 400 error? Bernd [email protected] wrote: > Hi folks, > > This morning I was experimenting with using multiple threads while indexing > some 20,000,000 records worth of content. In fact, my test spun up some 50 > threads, and happily chugged away for a couple of hours before I saw the > following output from my test code: > > Http protocol error: HTTP/1.1 400 missing_content_stream, while trying to > index record 6469124 > Http protocol error: HTTP/1.1 400 missing_content_stream, while trying to > index record 6469551 > Http protocol error: HTTP/1.1 400 missing_content_stream, while trying to > index record 6470592 > Http protocol error: HTTP/1.1 400 missing_content_stream, while trying to > index record 6472454 > java.net.SocketException: Connection reset > at java.net.SocketInputStream.read(SocketInputStream.java:168) > at HttpPoster.getResponse(HttpPoster.java:280) > at HttpPoster.indexPost(HttpPoster.java:191) > at ParseAndLoad$PostThread.run(ParseAndLoad.java:638) > <<<<<< > > Looking at the solr-side output, I see nothing interesting at all: > > Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/update/extract > params={literal.nokia_longitude=9.78518981933594&literal.nokia_phone=%2B497971910474&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_district=Münster&literal.nokia_placerating=0&literal.id=6472724&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=1&literal.nokia_ppid=276u0wyw-c8cb7f4d6cd84a639a4e7d3570bf8814&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9985514322917&literal.nokia_postalcode=74405&literal.nokia_street=WeinhaldenstraÃe&literal.nokia_title=Dorfgemeinschaft+Münster+e.V.&literal.nokia_category=261} > status=0 QTime=1 > Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/update/extract > params={literal.nokia_longitude=9.76717020670573&literal.nokia_phone=%2B497971950725&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_placerating=0&literal.id=6472737&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=13&literal.nokia_ppid=276u0wyw-d3bed6449fcb41b0adc50ae08e041f8d&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9974405924479&literal.nokia_fax=%2B497971950712&literal.nokia_postalcode=74405&literal.nokia_street=KochstraÃe&literal.nokia_title=BayWa+AG+Bau-+%26+Gartenmarkt&literal.nokia_category=194} > status=0 QTime=0 > Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/update/extract > params={literal.nokia_longitude=9.77591044108073&literal.nokia_phone=%2B49797124009&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_district=Unterrot&literal.nokia_placerating=0&literal.id=6472739&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=28&literal.nokia_ppid=276u0wyw-d534d7a9235a4edf878d5e32a34bad8b&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9791788736979&literal.nokia_fax=%2B49797123431&literal.nokia_postalcode=74405&literal.nokia_street=HauptstraÃe&literal.nokia_title=Gastel+R.&literal.nokia_category=5} > status=0 QTime=1 > Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/update/extract > params={literal.nokia_longitude=9.76935&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_placerating=5&literal.id=6472698&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=15&literal.nokia_ppid=276u0wyw-9544100e68d74162aff54783b9376134&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9981&literal.nokia_postalcode=74405&literal.nokia_street=KanzleistraÃe&literal.nokia_tag=Steuerberater&literal.nokia_tag=Business+%26+Service&literal.nokia_title=Consultis+GmbH&literal.nokia_category=215} > status=0 QTime=92 > Jun 7, 2010 9:57:48 AM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/update/extract > params={literal.nokia_longitude=9.77173970540364&literal.nokia_phone=%2B4979713238&literal.nokia_type=0&literal.nokia_boost=1&literal.nokia_placerating=0&literal.id=6472699&literal.nokia_visitcount=0&literal.nokia_country=DEU&literal.nokia_housenumber=37&literal.nokia_ppid=276u0wyw-9600016fd0d248c9b442111838350f64&literal.nokia_language=de&literal.nokia_city=Gaildorf&literal.nokia_latitude=48.9987182617188&literal.nokia_fax=%2B497971911639&literal.nokia_postalcode=74405&literal.nokia_street=KarlstraÃe&literal.nokia_title=Videothek,+5th+avenue+Peltekis+Apostolos&literal.nokia_category=5} > status=0 QTime=93 > <<<<<< > > It is unlikely (but, of course, not out of the question) that this hiccup is > due to some reentrancy problem in my test code. It is much more likely to be > some kind of a Solr multi-threaded race condition - especially since it looks > like a number of requests all failed at precisely the same time. This is a > Solr 1.5 build from mid-late March, FWIW. Does anyone know of an > extractingUpdateRequestHandler re-entrancy bug of this kind? > > Thanks, > Karl > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
