bq: If there is a problem writing the segment, a permission error, Highly doubtful that this'll occur. When an IndexWriter is opened, the first thing that's (usually) done is write to the lock file to keep other Solr's from writing. That should fail right off the bat, far before any docs are actually indexed, perhaps with a lock obtain timeout error message.
And, for that matter, when Solr first starts up it creates the ./data, ./data/index and (perhaps) the ./data/tlog directories and any permissions errors should be hit then. I suppose there's some "interesting" stuff possible if someone out there changing directory permissions while Solr is running, in which case you should find then and then slap them silly ;) IOW I've certainly seen Solr _fail_ to start when it can't access the right directories, but not fail part way through. Best, Erick On Tue, Sep 29, 2015 at 1:55 AM, Alessandro Benedetti <benedetti.ale...@gmail.com> wrote: > Hi Matteo, at this point I would suggest you this reading by Erick: > > https://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ > > If i am not wrong when the document is indexed ( simplifying) : > 1) The document is added to the current segment in memory > 2) When a soft commit happens, we get the visibility ( no flush happens to > the disk, but the document is searchable) > 3) When the hard commit happens, we get the durability, we truncate the > segment in memory and we flush it to the disk, so if a problem happens > here, you should see an error Solr side, but this does not imply that the > document indexing is failed, actually only the last flush has failed. > > Related point 3, I am not sure what are the Solr reaction to this fail. > I should investigate. > > Cheers > > > > 2015-09-29 8:53 GMT+01:00 Matteo Grolla <matteo.gro...@gmail.com>: > >> Hi Erik, >> it's a curiosity question. When I add a document it's buffered by Solr >> and can (apparently is) be parsed to verify it matches the schema. But it's >> not written to a segment file until a commit is issued. If there is a >> problem writing the segment, a permission error, isn't this a case where I >> would report everything OK when in fact documents are not there? >> >> thanks >> >> 2015-09-29 2:12 GMT+02:00 Erick Erickson <erickerick...@gmail.com>: >> >> > You shouldn't be losing errors with HttpSolrServer. Are you >> > seeing evidence that you are or is this mostly a curiosity question? >> > >> > Do not it's better to batch up docs, your throughput will increase >> > a LOT. That said, when you do batch (e.g. send 500 docs per update >> > or whatever) and you get an error back, you're not quite sure what >> > doc failed. So what people do is retry a failed batch one document >> > at a time when the batch has errors and rely on Solr overwriting >> > any docs in the batch that were indexed the first time. >> > >> > Best, >> > Erick >> > >> > On Mon, Sep 28, 2015 at 2:27 PM, Matteo Grolla <matteo.gro...@gmail.com> >> > wrote: >> > > Hi, >> > > if I need fine grained error reporting I use Http Solr server and >> > send >> > > 1 doc per request using the add method. >> > > I report errors on exceptions of the add method, >> > > I'm using autocommit so I'm not seing errors related to commit. >> > > Am I loosing some errors? Is there a better way? >> > > >> > > Thanks >> > >> > > > > -- > -------------------------- > > Benedetti Alessandro > Visiting card - http://about.me/alessandro_benedetti > Blog - http://alexbenedetti.blogspot.co.uk > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England
