Re: Index not loading
On Tue, Aug 14, 2012 at 5:37 PM, Jonatan Fournier jonatan.fourn...@gmail.com wrote: On Tue, Aug 14, 2012 at 10:25 AM, Erick Erickson erickerick...@gmail.com wrote: This is quite odd, it really sounds like you're not actually committing. So, some questions. 1 What happens if you search before you shut down your tomcat? Do you see docs then? If so, somehow you're doing soft commits and never doing a hard commit. Yeah I just realized the behavior is the same as softCommit, is it the default for commitWithin? Cheers, /jonathan 2 What happens if, as the last statement in your SolrJ program you do a commit()? When using commitWithin, if I introduce server.commit() within the data load process the data gets commited ( I didn't reproduce with my 89G of data...), if I shutdown my EmbeddedServer and restart it and send a commit, like on Tomcat, all data gets wiped out too. So I guess that there's state loss somewhere. Cheers, /jonathan 3 While you're indexing, what do you see in your index directory? You should see multiple segments being created, and possibly merged so the number of files should go up and down. If you only have a single set of files, you're somehow not doing a commit. 4 Is there something really silly going on like your restart scripts delete the index directory? Or you're using a VM that restores a blank image? 5 When you do restart, are there any files at all in your index directory? I really suspect you've got some configuration problem here Best Erick On Mon, Aug 13, 2012 at 9:11 AM, Jonatan Fournier jonatan.fourn...@gmail.com wrote: Hi, I'm using Solr 4.0.0-ALPHA and the EmbeddedSolrServer. Within my SolrJ application, the documents are added to the server using the commitWithin parameter (in my case 60s). After 1 day my 125 millions document are all added to the server and I can see 89G of index data files. I stop my SolrJ application and reload my Solr instance in Tomcat. From the Solr admin panel related to my Core (collection1) I see this info: Last Modified: Num Docs:0 Max Doc:0 Version:1 Segment Count:0 Optimized: (green check) Current: (green check) Master: Version: 0 Gen: 1 Size: 88.14 GB From the general Core Admin panel I see: lastModified: version:1 numDocs:0 maxDoc:0 optimized: (red circle) current: (green check) hasDeletions: (red circle) If I query my index for *:* I get 0 result. If I trigger optimize it wipes ALL my data inside the index and reset to empty. I've played around my EmbeddedServer initially using autoCommit/softCommit and it was working fine. Now that I've switched to commitWithin the document add query, it always do that! I'm never able to reload my index within Tomcat/Solr. Any idea? Cheers, /jonathan
Re: Index not loading
This is quite odd, it really sounds like you're not actually committing. So, some questions. 1 What happens if you search before you shut down your tomcat? Do you see docs then? If so, somehow you're doing soft commits and never doing a hard commit. 2 What happens if, as the last statement in your SolrJ program you do a commit()? 3 While you're indexing, what do you see in your index directory? You should see multiple segments being created, and possibly merged so the number of files should go up and down. If you only have a single set of files, you're somehow not doing a commit. 4 Is there something really silly going on like your restart scripts delete the index directory? Or you're using a VM that restores a blank image? 5 When you do restart, are there any files at all in your index directory? I really suspect you've got some configuration problem here Best Erick On Mon, Aug 13, 2012 at 9:11 AM, Jonatan Fournier jonatan.fourn...@gmail.com wrote: Hi, I'm using Solr 4.0.0-ALPHA and the EmbeddedSolrServer. Within my SolrJ application, the documents are added to the server using the commitWithin parameter (in my case 60s). After 1 day my 125 millions document are all added to the server and I can see 89G of index data files. I stop my SolrJ application and reload my Solr instance in Tomcat. From the Solr admin panel related to my Core (collection1) I see this info: Last Modified: Num Docs:0 Max Doc:0 Version:1 Segment Count:0 Optimized: (green check) Current: (green check) Master: Version: 0 Gen: 1 Size: 88.14 GB From the general Core Admin panel I see: lastModified: version:1 numDocs:0 maxDoc:0 optimized: (red circle) current: (green check) hasDeletions: (red circle) If I query my index for *:* I get 0 result. If I trigger optimize it wipes ALL my data inside the index and reset to empty. I've played around my EmbeddedServer initially using autoCommit/softCommit and it was working fine. Now that I've switched to commitWithin the document add query, it always do that! I'm never able to reload my index within Tomcat/Solr. Any idea? Cheers, /jonathan
Re: Index not loading
Hi Erick, On Tue, Aug 14, 2012 at 10:25 AM, Erick Erickson erickerick...@gmail.com wrote: This is quite odd, it really sounds like you're not actually committing. So, some questions. 1 What happens if you search before you shut down your tomcat? Do you see docs then? If so, somehow you're doing soft commits and never doing a hard commit. No I'm not seeing any documents if I do search for anything. Like mentioned above, Num and Max docs are 0. Like I mentioned below, my index files are not deleted when I start/restart tomcat, but when within tomcat I send a commit/optimize command. On thing I noticed that was different in the log output from the embedded server was that when I use the solrconfig.xml autoCommit, after the delay I see some stdout message about commiting to the index. But when relying on the commitWithin, I never see the solr server output freeze for a moment while commiting, I only see all my add document stdout message. Should the behavior be the same? Or the commit messages pass by so fast I don't see them? It must be trying to do some kind of commit/merge, because when I was monitoring the memory I could see periodic memory increase (when I assumed it was merging) then memory decreased until the next delay... 2 What happens if, as the last statement in your SolrJ program you do a commit()? Let me try that and come back to you, for now here's the commands I was using in the 3 test scenarios: SolrInputDocument doc = new SolrInputDocument(); solrDoc.addField(id, someId); ... server.add(doc); // In the case I have either autoCommit maxTime6/maxTime enabled in the solrconfig.xml or autoSoftCommit // Both scenarios works, in those 2 cases when I shutdown my embeddedserver and restart tomcat I have all my data indexed/commited or server.add(doc, 6) // In the case I don't have autoCommit enabled, try to rely on commitWithin param. 3 While you're indexing, what do you see in your index directory? You should see multiple segments being created, and possibly merged so the number of files should go up and down. If you only have a single set of files, you're somehow not doing a commit. No I do see a bunch of files being created/merged, at the end I had a bout 89G in many many files. Another thing I was playing around when trying to use the commitWithin is to change the useCompoundFiletrue/useCompoundFile and mergeFactor10/mergeFactor to reduce the number of files created. Could it impact things? 4 Is there something really silly going on like your restart scripts delete the index directory? Or you're using a VM that restores a blank image? No VM, no scripts, no replication. 5 When you do restart, are there any files at all in your index directory? When I restart tomcat I do see all the same 89G files that was created using the embedded server, they only vanish when I force a commit or optimize, then it's like if my data directory didn't exist and the 2 initial segment files are being created and all the rest deleted. I really suspect you've got some configuration problem here Maybe, but other than playing with the compound file thingy I don't have any fancy config changes. Cheers, /jonathan Best Erick On Mon, Aug 13, 2012 at 9:11 AM, Jonatan Fournier jonatan.fourn...@gmail.com wrote: Hi, I'm using Solr 4.0.0-ALPHA and the EmbeddedSolrServer. Within my SolrJ application, the documents are added to the server using the commitWithin parameter (in my case 60s). After 1 day my 125 millions document are all added to the server and I can see 89G of index data files. I stop my SolrJ application and reload my Solr instance in Tomcat. From the Solr admin panel related to my Core (collection1) I see this info: Last Modified: Num Docs:0 Max Doc:0 Version:1 Segment Count:0 Optimized: (green check) Current: (green check) Master: Version: 0 Gen: 1 Size: 88.14 GB From the general Core Admin panel I see: lastModified: version:1 numDocs:0 maxDoc:0 optimized: (red circle) current: (green check) hasDeletions: (red circle) If I query my index for *:* I get 0 result. If I trigger optimize it wipes ALL my data inside the index and reset to empty. I've played around my EmbeddedServer initially using autoCommit/softCommit and it was working fine. Now that I've switched to commitWithin the document add query, it always do that! I'm never able to reload my index within Tomcat/Solr. Any idea? Cheers, /jonathan
Re: Index not loading
If you send a dummy document using a curl command, without the commit option, does it auto-commit and become visible in 1 minute? -- Jack Krupansky -Original Message- From: Jonatan Fournier Sent: Tuesday, August 14, 2012 11:03 AM To: solr-user@lucene.apache.org ; erickerick...@gmail.com Subject: Re: Index not loading Hi Erick, On Tue, Aug 14, 2012 at 10:25 AM, Erick Erickson erickerick...@gmail.com wrote: This is quite odd, it really sounds like you're not actually committing. So, some questions. 1 What happens if you search before you shut down your tomcat? Do you see docs then? If so, somehow you're doing soft commits and never doing a hard commit. No I'm not seeing any documents if I do search for anything. Like mentioned above, Num and Max docs are 0. Like I mentioned below, my index files are not deleted when I start/restart tomcat, but when within tomcat I send a commit/optimize command. On thing I noticed that was different in the log output from the embedded server was that when I use the solrconfig.xml autoCommit, after the delay I see some stdout message about commiting to the index. But when relying on the commitWithin, I never see the solr server output freeze for a moment while commiting, I only see all my add document stdout message. Should the behavior be the same? Or the commit messages pass by so fast I don't see them? It must be trying to do some kind of commit/merge, because when I was monitoring the memory I could see periodic memory increase (when I assumed it was merging) then memory decreased until the next delay... 2 What happens if, as the last statement in your SolrJ program you do a commit()? Let me try that and come back to you, for now here's the commands I was using in the 3 test scenarios: SolrInputDocument doc = new SolrInputDocument(); solrDoc.addField(id, someId); ... server.add(doc); // In the case I have either autoCommit maxTime6/maxTime enabled in the solrconfig.xml or autoSoftCommit // Both scenarios works, in those 2 cases when I shutdown my embeddedserver and restart tomcat I have all my data indexed/commited or server.add(doc, 6) // In the case I don't have autoCommit enabled, try to rely on commitWithin param. 3 While you're indexing, what do you see in your index directory? You should see multiple segments being created, and possibly merged so the number of files should go up and down. If you only have a single set of files, you're somehow not doing a commit. No I do see a bunch of files being created/merged, at the end I had a bout 89G in many many files. Another thing I was playing around when trying to use the commitWithin is to change the useCompoundFiletrue/useCompoundFile and mergeFactor10/mergeFactor to reduce the number of files created. Could it impact things? 4 Is there something really silly going on like your restart scripts delete the index directory? Or you're using a VM that restores a blank image? No VM, no scripts, no replication. 5 When you do restart, are there any files at all in your index directory? When I restart tomcat I do see all the same 89G files that was created using the embedded server, they only vanish when I force a commit or optimize, then it's like if my data directory didn't exist and the 2 initial segment files are being created and all the rest deleted. I really suspect you've got some configuration problem here Maybe, but other than playing with the compound file thingy I don't have any fancy config changes. Cheers, /jonathan Best Erick On Mon, Aug 13, 2012 at 9:11 AM, Jonatan Fournier jonatan.fourn...@gmail.com wrote: Hi, I'm using Solr 4.0.0-ALPHA and the EmbeddedSolrServer. Within my SolrJ application, the documents are added to the server using the commitWithin parameter (in my case 60s). After 1 day my 125 millions document are all added to the server and I can see 89G of index data files. I stop my SolrJ application and reload my Solr instance in Tomcat. From the Solr admin panel related to my Core (collection1) I see this info: Last Modified: Num Docs:0 Max Doc:0 Version:1 Segment Count:0 Optimized: (green check) Current: (green check) Master: Version: 0 Gen: 1 Size: 88.14 GB From the general Core Admin panel I see: lastModified: version:1 numDocs:0 maxDoc:0 optimized: (red circle) current: (green check) hasDeletions: (red circle) If I query my index for *:* I get 0 result. If I trigger optimize it wipes ALL my data inside the index and reset to empty. I've played around my EmbeddedServer initially using autoCommit/softCommit and it was working fine. Now that I've switched to commitWithin the document add query, it always do that! I'm never able to reload my index within Tomcat/Solr. Any idea? Cheers, /jonathan
Re: Index not loading
On Tue, Aug 14, 2012 at 11:14 AM, Jack Krupansky
j...@basetechnology.com wrote:
If you send a dummy document using a curl command, without the commit
option, does it auto-commit and become visible in 1 minute?
Sending a JSON document using curl:
{
add: {
commitWithin: 6,
overwrite: false,
doc: {
id : 1,
type : foo
}
}
}
This worked fine. But If use the EmbeddedServer.add(doc, commitWithin)
it doesn't show up in the search result.
From this article:
http://www.cominvent.com/2011/09/09/discover-commitwithin-in-solr/
I see there's is multiple ways to specify this commitWithin options:
https://issues.apache.org/jira/browse/SOLR-2742 introduced it to the
.add() methods for SolrServer, could it be broken only there?
I will go try this syntax:
UpdateRequest req = new UpdateRequest();
req.add(mySolrInputDocument);
req.setCommitWithin(1);
req.process(server);
Cheers,
/jonathan
-- Jack Krupansky
-Original Message- From: Jonatan Fournier
Sent: Tuesday, August 14, 2012 11:03 AM
To: solr-user@lucene.apache.org ; erickerick...@gmail.com
Subject: Re: Index not loading
Hi Erick,
On Tue, Aug 14, 2012 at 10:25 AM, Erick Erickson
erickerick...@gmail.com wrote:
This is quite odd, it really sounds like you're not
actually committing. So, some questions.
1 What happens if you search before you shut
down your tomcat? Do you see docs then? If so,
somehow you're doing soft commits and never
doing a hard commit.
No I'm not seeing any documents if I do search for anything. Like
mentioned above, Num and Max docs are 0.
Like I mentioned below, my index files are not deleted when I
start/restart tomcat, but when within tomcat I send a commit/optimize
command.
On thing I noticed that was different in the log output from the
embedded server was that when I use the solrconfig.xml autoCommit,
after the delay I see some stdout message about commiting to the
index. But when relying on the commitWithin, I never see the solr
server output freeze for a moment while commiting, I only see all my
add document stdout message. Should the behavior be the same? Or the
commit messages pass by so fast I don't see them?
It must be trying to do some kind of commit/merge, because when I was
monitoring the memory I could see periodic memory increase (when I
assumed it was merging) then memory decreased until the next delay...
2 What happens if, as the last statement in your SolrJ
program you do a commit()?
Let me try that and come back to you, for now here's the commands I
was using in the 3 test scenarios:
SolrInputDocument doc = new SolrInputDocument();
solrDoc.addField(id, someId);
...
server.add(doc); // In the case I have either autoCommit
maxTime6/maxTime enabled in the solrconfig.xml or
autoSoftCommit
// Both scenarios works, in those 2 cases when I shutdown my
embeddedserver and restart tomcat I have all my data indexed/commited
or
server.add(doc, 6) // In the case I don't have autoCommit enabled,
try to rely on commitWithin param.
3 While you're indexing, what do you see in your index
directory? You should see multiple segments being
created, and possibly merged so the number of
files should go up and down. If you only have a single
set of files, you're somehow not doing a commit.
No I do see a bunch of files being created/merged, at the end I had a
bout 89G in many many files.
Another thing I was playing around when trying to use the commitWithin
is to change the useCompoundFiletrue/useCompoundFile and
mergeFactor10/mergeFactor to reduce the number of files created.
Could it impact things?
4 Is there something really silly going on like your
restart scripts delete the index directory? Or you're
using a VM that restores a blank image?
No VM, no scripts, no replication.
5 When you do restart, are there any files at all
in your index directory?
When I restart tomcat I do see all the same 89G files that was created
using the embedded server, they only vanish when I force a commit or
optimize, then it's like if my data directory didn't exist and the 2
initial segment files are being created and all the rest deleted.
I really suspect you've got some configuration problem
here
Maybe, but other than playing with the compound file thingy I don't
have any fancy config changes.
Cheers,
/jonathan
Best
Erick
On Mon, Aug 13, 2012 at 9:11 AM, Jonatan Fournier
jonatan.fourn...@gmail.com wrote:
Hi,
I'm using Solr 4.0.0-ALPHA and the EmbeddedSolrServer.
Within my SolrJ application, the documents are added to the server
using the commitWithin parameter (in my case 60s). After 1 day my 125
millions document are all added to the server and I can see 89G of
index data files. I stop my SolrJ application and reload my Solr
instance in Tomcat.
From the Solr admin panel related to my Core (collection1) I see this
info:
Last Modified:
Num Docs:0
Max
Re: Index not loading
,
_p_nrm.cfs, _8_Lucene40_0.tip, _j_nrm.cfs, _q_Lucene40_0.prx, _g.si,
_l.fnm, _p.fnm, _k.fdt, _k.fdx, _h_nrm.cfe, _s.fnm, _a.fdt,
_9_Lucene40_0.prx, _a.fdx, _l_Lucene40_0.frq, _g.fnm, _6_nrm.cfs,
_p_Lucene40_0.tim, _h_nrm.cfs, _p_Lucene40_0.tip, _0.si, _5.fnm,
_9_Lucene40_0.tim, _j_Lucene40_0.prx, _6_nrm.cfe, _0_nrm.cfs, _s.fdx,
_j.fnm, _0_nrm.cfe, _5.fdx, _0.fdx, _8.fdx, _i.fnm, _0.fdt,
segments_4, _8.fdt]
commit{dir=/mnt/data/solr/couids/data/index,segFN=segments_5,generation=5,filenames=[_5_nrm.cfe,
_v.fdx, _s.fdt, _l.si, _w_Lucene40_0.prx, _k.fnm, _0_Lucene40_0.prx,
_r_nrm.cfs, _m.si, _8.si, _8_Lucene40_0.frq, _a_Lucene40_0.frq,
_v.fnm, _w.fnm, _r_nrm.cfe, _0_Lucene40_0.tim, _w.fdt, _s.si, _w.fdx,
_t_Lucene40_0.tim, _9.fdt, _t_Lucene40_0.tip, _9.fdx,
_u_Lucene40_0.frq, _9_Lucene40_0.frq, _0_Lucene40_0.tip, _5_nrm.cfs,
_l.fdx, _l_Lucene40_0.prx, _l.fdt, _6.fdt, _t.fdt, _a.fnm, _j.fdx,
_k_Lucene40_0.tim, _w.si, _m_Lucene40_0.frq, _k_Lucene40_0.tip,
_r_Lucene40_0.frq, _j.fdt, _6.fdx, _a_Lucene40_0.tim, _u.fdx,
_t_Lucene40_0.prx, _a_Lucene40_0.tip, _v_Lucene40_0.frq,
_m_Lucene40_0.tip, _m_Lucene40_0.tim, _k_Lucene40_0.frq, _r.fdt,
_r.fnm, _u.fnm, _5_Lucene40_0.tim, _0_Lucene40_0.frq,
_5_Lucene40_0.tip, _r.fdx, _r_Lucene40_0.tim, _r_Lucene40_0.tip,
_m_Lucene40_0.prx, _j.si, _v.si, _9.fnm, _p.si, _j_Lucene40_0.tip,
_v_Lucene40_0.prx, _p_Lucene40_0.prx, _j_Lucene40_0.tim,
_v_Lucene40_0.tip, _s_Lucene40_0.prx, _m.fdt, _v_Lucene40_0.tim,
_m.fdx, _6.si, _6.fnm, _5_Lucene40_0.prx, _8_nrm.cfe,
_8_Lucene40_0.tim, _p.fdx, _5.fdt, _l_nrm.cfe, _6_Lucene40_0.tim,
_p.fdt, _6_Lucene40_0.tip, _u_Lucene40_0.tip, _t_Lucene40_0.frq,
_s_Lucene40_0.frq, _u_Lucene40_0.tim, _l_Lucene40_0.tim, _l_nrm.cfs,
_l_Lucene40_0.tip, _9_nrm.cfs, _k_Lucene40_0.prx, _9_Lucene40_0.tip,
_9.si, _j_Lucene40_0.frq, _m.fnm, _k.si, _s_nrm.cfe, _m_nrm.cfe,
_p_Lucene40_0.frq, _5_Lucene40_0.frq, _a_nrm.cfe, _k_nrm.cfe, _0.fnm,
_j_nrm.cfe, _a_Lucene40_0.prx, _9_nrm.cfe, _8_Lucene40_0.prx,
_s_Lucene40_0.tip, _s_Lucene40_0.tim, _a.si, _a_nrm.cfs,
_r_Lucene40_0.prx, _s_nrm.cfs, _6_Lucene40_0.frq, _p_nrm.cfe,
_8_nrm.cfs, _5.si, _k_nrm.cfs, _8.fnm, _m_nrm.cfs, _u.si, _u.fdt,
_6_Lucene40_0.prx, _r.si, _p_nrm.cfs, _8_Lucene40_0.tip, _j_nrm.cfs,
_l.fnm, _t.fnm, _p.fnm, _k.fdt, _w_Lucene40_0.tip, _k.fdx, _s.fnm,
_a.fdt, _w_Lucene40_0.tim, _t_nrm.cfs, _9_Lucene40_0.prx, _v_nrm.cfs,
_a.fdx, _l_Lucene40_0.frq, _t.si, _6_nrm.cfs, _u_nrm.cfs,
_p_Lucene40_0.tim, _p_Lucene40_0.tip, _w_nrm.cfe, _0.si,
_w_Lucene40_0.frq, _u_Lucene40_0.prx, _5.fnm, _9_Lucene40_0.tim,
_j_Lucene40_0.prx, _v.fdt, _u_nrm.cfe, _6_nrm.cfe, _w_nrm.cfs,
_0_nrm.cfs, _s.fdx, _j.fnm, _0_nrm.cfe, _t.fdx, _5.fdx, _v_nrm.cfe,
_t_nrm.cfe, _0.fdx, _8.fdx, segments_5, _0.fdt, _8.fdt]
Aug 14, 2012 1:02:59 PM org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: newest commit = 5
Aug 14, 2012 1:02:59 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
I don't think my config is wrong, since using the dummy commitWithin
JSON update is working, that my autoCommit is always working... What
else could be wrong other than the SolrServer in SolrJ?
Cheers,
/jonathan
On Tue, Aug 14, 2012 at 12:30 PM, Jonatan Fournier
jonatan.fourn...@gmail.com wrote:
On Tue, Aug 14, 2012 at 11:14 AM, Jack Krupansky
j...@basetechnology.com wrote:
If you send a dummy document using a curl command, without the commit
option, does it auto-commit and become visible in 1 minute?
Sending a JSON document using curl:
{
add: {
commitWithin: 6,
overwrite: false,
doc: {
id : 1,
type : foo
}
}
}
This worked fine. But If use the EmbeddedServer.add(doc, commitWithin)
it doesn't show up in the search result.
From this article:
http://www.cominvent.com/2011/09/09/discover-commitwithin-in-solr/
I see there's is multiple ways to specify this commitWithin options:
https://issues.apache.org/jira/browse/SOLR-2742 introduced it to the
.add() methods for SolrServer, could it be broken only there?
I will go try this syntax:
UpdateRequest req = new UpdateRequest();
req.add(mySolrInputDocument);
req.setCommitWithin(1);
req.process(server);
Cheers,
/jonathan
-- Jack Krupansky
-Original Message- From: Jonatan Fournier
Sent: Tuesday, August 14, 2012 11:03 AM
To: solr-user@lucene.apache.org ; erickerick...@gmail.com
Subject: Re: Index not loading
Hi Erick,
On Tue, Aug 14, 2012 at 10:25 AM, Erick Erickson
erickerick...@gmail.com wrote:
This is quite odd, it really sounds like you're not
actually committing. So, some questions.
1 What happens if you search before you shut
down your tomcat? Do you see docs then? If so,
somehow you're doing soft commits and never
doing a hard commit.
No I'm not seeing any documents if I do search for anything. Like
mentioned above, Num and Max docs are 0.
Like I mentioned below, my index files are not deleted when I
start/restart tomcat, but when within tomcat
Re: Index not loading
On Tue, Aug 14, 2012 at 10:25 AM, Erick Erickson erickerick...@gmail.com wrote: This is quite odd, it really sounds like you're not actually committing. So, some questions. 1 What happens if you search before you shut down your tomcat? Do you see docs then? If so, somehow you're doing soft commits and never doing a hard commit. 2 What happens if, as the last statement in your SolrJ program you do a commit()? When using commitWithin, if I introduce server.commit() within the data load process the data gets commited ( I didn't reproduce with my 89G of data...), if I shutdown my EmbeddedServer and restart it and send a commit, like on Tomcat, all data gets wiped out too. So I guess that there's state loss somewhere. Cheers, /jonathan 3 While you're indexing, what do you see in your index directory? You should see multiple segments being created, and possibly merged so the number of files should go up and down. If you only have a single set of files, you're somehow not doing a commit. 4 Is there something really silly going on like your restart scripts delete the index directory? Or you're using a VM that restores a blank image? 5 When you do restart, are there any files at all in your index directory? I really suspect you've got some configuration problem here Best Erick On Mon, Aug 13, 2012 at 9:11 AM, Jonatan Fournier jonatan.fourn...@gmail.com wrote: Hi, I'm using Solr 4.0.0-ALPHA and the EmbeddedSolrServer. Within my SolrJ application, the documents are added to the server using the commitWithin parameter (in my case 60s). After 1 day my 125 millions document are all added to the server and I can see 89G of index data files. I stop my SolrJ application and reload my Solr instance in Tomcat. From the Solr admin panel related to my Core (collection1) I see this info: Last Modified: Num Docs:0 Max Doc:0 Version:1 Segment Count:0 Optimized: (green check) Current: (green check) Master: Version: 0 Gen: 1 Size: 88.14 GB From the general Core Admin panel I see: lastModified: version:1 numDocs:0 maxDoc:0 optimized: (red circle) current: (green check) hasDeletions: (red circle) If I query my index for *:* I get 0 result. If I trigger optimize it wipes ALL my data inside the index and reset to empty. I've played around my EmbeddedServer initially using autoCommit/softCommit and it was working fine. Now that I've switched to commitWithin the document add query, it always do that! I'm never able to reload my index within Tomcat/Solr. Any idea? Cheers, /jonathan
Index not loading
Hi, I'm using Solr 4.0.0-ALPHA and the EmbeddedSolrServer. Within my SolrJ application, the documents are added to the server using the commitWithin parameter (in my case 60s). After 1 day my 125 millions document are all added to the server and I can see 89G of index data files. I stop my SolrJ application and reload my Solr instance in Tomcat. From the Solr admin panel related to my Core (collection1) I see this info: Last Modified: Num Docs:0 Max Doc:0 Version:1 Segment Count:0 Optimized: (green check) Current: (green check) Master: Version: 0 Gen: 1 Size: 88.14 GB From the general Core Admin panel I see: lastModified: version:1 numDocs:0 maxDoc:0 optimized: (red circle) current: (green check) hasDeletions: (red circle) If I query my index for *:* I get 0 result. If I trigger optimize it wipes ALL my data inside the index and reset to empty. I've played around my EmbeddedServer initially using autoCommit/softCommit and it was working fine. Now that I've switched to commitWithin the document add query, it always do that! I'm never able to reload my index within Tomcat/Solr. Any idea? Cheers, /jonathan
