On Tue, Aug 14, 2012 at 5:37 PM, Jonatan Fournier <jonatan.fourn...@gmail.com> wrote: > On Tue, Aug 14, 2012 at 10:25 AM, Erick Erickson > <erickerick...@gmail.com> wrote: >> This is quite odd, it really sounds like you're not >> actually committing. So, some questions. >> >> 1> What happens if you search before you shut >> down your tomcat? Do you see docs then? If so, >> somehow you're doing soft commits and never >> doing a hard commit.
Yeah I just realized the behavior is the same as softCommit, is it the default for commitWithin? Cheers, /jonathan >> >> 2> What happens if, as the last statement in your SolrJ >> program you do a commit()? > > When using commitWithin, if I introduce server.commit() within the > data load process the data gets commited ( I didn't reproduce with my > 89G of data...), if I shutdown my EmbeddedServer and restart it and > send a commit, like on Tomcat, all data gets wiped out too. So I guess > that there's state loss somewhere. > > Cheers, > > /jonathan > >> >> 3> While you're indexing, what do you see in your index >> directory? You should see multiple segments being >> created, and possibly merged so the number of >> files should go up and down. If you only have a single >> set of files, you're somehow not doing a commit. >> >> 4> Is there something really silly going on like your >> restart scripts delete the index directory? Or you're >> using a VM that restores a blank image? >> >> 5> When you do restart, are there any files at all >> in your index directory? >> >> I really suspect you've got some configuration problem >> here.... >> >> Best >> Erick >> >> >> >> On Mon, Aug 13, 2012 at 9:11 AM, Jonatan Fournier >> <jonatan.fourn...@gmail.com> wrote: >>> Hi, >>> >>> I'm using Solr 4.0.0-ALPHA and the EmbeddedSolrServer. >>> >>> Within my SolrJ application, the documents are added to the server >>> using the commitWithin parameter (in my case 60s). After 1 day my 125 >>> millions document are all added to the server and I can see 89G of >>> index data files. I stop my SolrJ application and reload my Solr >>> instance in Tomcat. >>> >>> From the Solr admin panel related to my Core (collection1) I see this info: >>> >>> >>> Last Modified: >>> Num Docs:0 >>> Max Doc:0 >>> Version:1 >>> Segment Count:0 >>> Optimized: (green check) >>> Current: (green check) >>> Master: >>> Version: 0 >>> Gen: 1 >>> Size: 88.14 GB >>> >>> >>> From the general Core Admin panel I see: >>> >>> lastModified: >>> version:1 >>> numDocs:0 >>> maxDoc:0 >>> optimized: (red circle) >>> current: (green check) >>> hasDeletions: (red circle) >>> >>> If I query my index for *:* I get 0 result. If I trigger optimize it >>> wipes ALL my data inside the index and reset to empty. I've played >>> around my EmbeddedServer initially using autoCommit/softCommit and it >>> was working fine. Now that I've switched to commitWithin the document >>> add query, it always do that! I'm never able to reload my index within >>> Tomcat/Solr. >>> >>> Any idea? >>> >>> Cheers, >>> >>> /jonathan
