Re: SolR 5.3.1 deletes index files

2016-01-18 Thread Moll, Dr. Andreas
> I still think you should look at ensuring your merge policy is turned off
>in solrconfig.xml (if I understand your scenario, you have 1 instance which
>is read-only for searching, and another writing to the same index
>location), and did your turn infostream on as Erick suggested?

Thank you for the hint. I have tried



but it did not solve my problem. And yes I have enabled infostream and loglevel 
debug.

Best regards

Andreas Moll

Vertraulichkeitshinweis
Diese Information und jeder uebermittelte Anhang beinhaltet vertrauliche 
Informationen und ist nur fuer die Personen oder das Unternehmen bestimmt, an 
welche sie tatsaechlich gerichtet ist. Sollten Sie nicht der 
Bestimmungsempfaenger sein, weisen wir Sie darauf hin, dass die Verbreitung, 
das (auch teilweise) Kopieren sowie der Gebrauch der empfangenen E-Mail und der 
darin enthaltenen Informationen gesetzlich verboten sein kann und 
gegebenenfalls Schadensersatzpflichten ausloesen kann. Sollten Sie diese 
Nachricht aufgrund eines Uebermittlungsfehlers erhalten haben, bitten wir Sie 
den Sender unverzueglich hiervon in Kenntnis zu setzen.
Sicherheitswarnung: Bitte beachten Sie, dass das Internet kein sicheres 
Kommunikationsmedium ist. Obwohl wir im Rahmen unseres Qualitaetsmanagements 
und der gebotenen Sorgfalt Schritte eingeleitet haben, um einen 
Computervirenbefall weitestgehend zu verhindern, koennen wir wegen der Natur 
des Internets das Risiko eines Computervirenbefalls dieser E-Mail nicht 
ausschliessen.


Re: SolR 5.3.1 deletes index files

2016-01-15 Thread Daniel Collins
I know Solr used to have issues with indexes on NFS, there was a
segments.gen file specifically for issues around that, though that was
removed in 5.0. But you say this happens on local disks too, so that would
rule NFS out of it.

I still think you should look at ensuring your merge policy is turned off
in solrconfig.xml (if I understand your scenario, you have 1 instance which
is read-only for searching, and another writing to the same index
location), and did your turn infostream on as Erick suggested?

2016-01-15 16:01 GMT+00:00 Moll, Dr. Andreas :

> Hi,
>
> If you look at the files at the ls-Output in my last post you will see
> that SolR has deleted the
> segments_f -file. Thus the index can no longer be loaded.
>
> I also had other cases in which the data directory of SolR was empty after
> the SolR shutdown.
>
> And yes, it ist bad.
>
> Best regards
>
> Andreas Moll
>
> >Can I just clarify something.  The title of this thread implies Solr is
> >losing data when it shuts down which would be really bad(!)
> >The core isn't
> >deleting any data, it is performing a merge, so the data exists, just in
> >fewer larger segments instead of all the smaller segments you had before.
>
>
> Vertraulichkeitshinweis
> Diese Information und jeder uebermittelte Anhang beinhaltet vertrauliche
> Informationen und ist nur fuer die Personen oder das Unternehmen bestimmt,
> an welche sie tatsaechlich gerichtet ist. Sollten Sie nicht der
> Bestimmungsempfaenger sein, weisen wir Sie darauf hin, dass die
> Verbreitung, das (auch teilweise) Kopieren sowie der Gebrauch der
> empfangenen E-Mail und der darin enthaltenen Informationen gesetzlich
> verboten sein kann und gegebenenfalls Schadensersatzpflichten ausloesen
> kann. Sollten Sie diese Nachricht aufgrund eines Uebermittlungsfehlers
> erhalten haben, bitten wir Sie den Sender unverzueglich hiervon in Kenntnis
> zu setzen.
> Sicherheitswarnung: Bitte beachten Sie, dass das Internet kein sicheres
> Kommunikationsmedium ist. Obwohl wir im Rahmen unseres
> Qualitaetsmanagements und der gebotenen Sorgfalt Schritte eingeleitet
> haben, um einen Computervirenbefall weitestgehend zu verhindern, koennen
> wir wegen der Natur des Internets das Risiko eines Computervirenbefalls
> dieser E-Mail nicht ausschliessen.
>


Re: SolR 5.3.1 deletes index files

2016-01-15 Thread Moll, Dr. Andreas
Hi,

If you look at the files at the ls-Output in my last post you will see that 
SolR has deleted the
segments_f -file. Thus the index can no longer be loaded.

I also had other cases in which the data directory of SolR was empty after the 
SolR shutdown.

And yes, it ist bad.

Best regards

Andreas Moll

>Can I just clarify something.  The title of this thread implies Solr is
>losing data when it shuts down which would be really bad(!)
>The core isn't
>deleting any data, it is performing a merge, so the data exists, just in
>fewer larger segments instead of all the smaller segments you had before.


Vertraulichkeitshinweis
Diese Information und jeder uebermittelte Anhang beinhaltet vertrauliche 
Informationen und ist nur fuer die Personen oder das Unternehmen bestimmt, an 
welche sie tatsaechlich gerichtet ist. Sollten Sie nicht der 
Bestimmungsempfaenger sein, weisen wir Sie darauf hin, dass die Verbreitung, 
das (auch teilweise) Kopieren sowie der Gebrauch der empfangenen E-Mail und der 
darin enthaltenen Informationen gesetzlich verboten sein kann und 
gegebenenfalls Schadensersatzpflichten ausloesen kann. Sollten Sie diese 
Nachricht aufgrund eines Uebermittlungsfehlers erhalten haben, bitten wir Sie 
den Sender unverzueglich hiervon in Kenntnis zu setzen.
Sicherheitswarnung: Bitte beachten Sie, dass das Internet kein sicheres 
Kommunikationsmedium ist. Obwohl wir im Rahmen unseres Qualitaetsmanagements 
und der gebotenen Sorgfalt Schritte eingeleitet haben, um einen 
Computervirenbefall weitestgehend zu verhindern, koennen wir wegen der Natur 
des Internets das Risiko eines Computervirenbefalls dieser E-Mail nicht 
ausschliessen.


Re: SolR 5.3.1 deletes index files

2016-01-15 Thread Daniel Collins
Can I just clarify something.  The title of this thread implies Solr is
losing data when it shuts down which would be really bad(!)  The core isn't
deleting any data, it is performing a merge, so the data exists, just in
fewer larger segments instead of all the smaller segments you had before.

So the issue you have is why does the core do a merge on shutdown?  Now
that is a valid question, but in terms of risk, its a much lesser problem.
There is no data loss here, right??

I think what you need to do is investigate your MergePolicy configuration.
Sounds like you want NoMergePolicy, which I assume from the name means it
never merges. :)

On 15 January 2016 at 10:58, Moll, Dr. Andreas  wrote:

> Hi,
>
> we still have the problem that SolR deletes index files on closing the
> application if the index was changed in the meantime from the production
> application (which has an embedded SolR-Server).
> The problem also occurs if we use a local file system instead of a NFS.
> I have changed the loglevel to DEBUG and got some interesting lines,
> especially:
>
> 1140211 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class
> org.apache.lucene.index.MergeTrigger from
> WebAppClassLoader=174573182@a67c67e
>
> Why does SolR try a merge action on closing the core, if this SolR instance
> did not write any changes on the index?
>
> I have also tried to add the following lines to the solrconfig, but it
> didn't change anything:
>
> 
>   1000
>   1000
>   1000DAY
> 
>
> Below you find the log and an ls-Output before and after shuting down SolR.
> Maybe someone can help us along?
>
> Thanks and best regards
>
> Andreas Moll
>
> 1140203 INFO  (Thread-0) [   x:recht] o.a.s.c.SolrCore [recht]  CLOSING
> SolrCore org.apache.solr.core.SolrCore@7cb0d12c
> 1140206 INFO  (Thread-0) [   x:recht] o.a.s.u.UpdateHandler closing
> DirectUpdateHandler2{commits=0,autocommit
> maxTime=15000ms,autocommits=0,soft
> autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
> 1140207 INFO  (Thread-0) [   x:recht] o.a.s.u.SolrCoreState Closing
> SolrCoreState
> 1140207 INFO  (Thread-0) [   x:recht] o.a.s.u.DefaultSolrCoreState
> SolrCoreState ref count has reached 0 - closing IndexWriter
> 1140207 INFO  (Thread-0) [   x:recht] o.a.s.u.DefaultSolrCoreState closing
> IndexWriter with IndexWriterCloser
> 1140208 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class org.apache.solr.request.LocalSolrQueryRequest from
> WebAppClassLoader=174573182@a67c67e
> 1140209 DEBUG (Thread-0) [   x:recht] o.a.s.u.SolrIndexWriter Closing
> Writer DirectUpdateHandler2
> 1140211 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class org.apache.lucene.index.MergeTrigger from
> WebAppClassLoader=174573182@a67c67e
> 1140213 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class org.apache.lucene.store.FileSwitchDirectory from
> WebAppClassLoader=174573182@a67c67e
> 1140236 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class java.nio.file.FileSystem
> 1140236 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class java.nio.file.FileSystem from null
> 1140238 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class java.nio.file.FileStore
> 1140238 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class java.nio.file.FileStore from null
> 1140259 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class java.nio.file.attribute.FileTime
> 1140259 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded
> class java.nio.file.attribute.FileTime from null
> 1140518 DEBUG (Thread-0) [   x:recht] o.a.s.c.CachingDirectoryFactory
> Releasing directory:
> /mnt/solr/jpprodt1/abzug/prodman_solrhome/solr_recht/core_recht/data/index
> 1 false
> 1140520 INFO  (Thread-0) [   x:recht] o.a.s.c.SolrCore [recht] Closing
> main searcher on request.
> 1140521 DEBUG (Thread-0) [   x:recht] o.a.s.s.SolrIndexSearcher Closing
> Searcher@664499[recht] main
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}
>
> filterCache{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}
>
> queryResultCache{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}
>
> documentCache{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}
>

Re: SolR 5.3.1 deletes index files

2016-01-15 Thread Moll, Dr. Andreas
Hi,

we still have the problem that SolR deletes index files on closing the
application if the index was changed in the meantime from the production
application (which has an embedded SolR-Server).
The problem also occurs if we use a local file system instead of a NFS.
I have changed the loglevel to DEBUG and got some interesting lines, especially:

1140211 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class
org.apache.lucene.index.MergeTrigger from WebAppClassLoader=174573182@a67c67e

Why does SolR try a merge action on closing the core, if this SolR instance
did not write any changes on the index?

I have also tried to add the following lines to the solrconfig, but it
didn't change anything:


  1000
  1000
  1000DAY


Below you find the log and an ls-Output before and after shuting down SolR.
Maybe someone can help us along?

Thanks and best regards

Andreas Moll

1140203 INFO  (Thread-0) [   x:recht] o.a.s.c.SolrCore [recht]  CLOSING 
SolrCore org.apache.solr.core.SolrCore@7cb0d12c
1140206 INFO  (Thread-0) [   x:recht] o.a.s.u.UpdateHandler closing 
DirectUpdateHandler2{commits=0,autocommit maxTime=15000ms,autocommits=0,soft 
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
1140207 INFO  (Thread-0) [   x:recht] o.a.s.u.SolrCoreState Closing 
SolrCoreState
1140207 INFO  (Thread-0) [   x:recht] o.a.s.u.DefaultSolrCoreState 
SolrCoreState ref count has reached 0 - closing IndexWriter
1140207 INFO  (Thread-0) [   x:recht] o.a.s.u.DefaultSolrCoreState closing 
IndexWriter with IndexWriterCloser
1140208 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
org.apache.solr.request.LocalSolrQueryRequest from 
WebAppClassLoader=174573182@a67c67e
1140209 DEBUG (Thread-0) [   x:recht] o.a.s.u.SolrIndexWriter Closing Writer 
DirectUpdateHandler2
1140211 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
org.apache.lucene.index.MergeTrigger from WebAppClassLoader=174573182@a67c67e
1140213 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
org.apache.lucene.store.FileSwitchDirectory from 
WebAppClassLoader=174573182@a67c67e
1140236 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
java.nio.file.FileSystem
1140236 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
java.nio.file.FileSystem from null
1140238 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
java.nio.file.FileStore
1140238 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
java.nio.file.FileStore from null
1140259 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
java.nio.file.attribute.FileTime
1140259 DEBUG (Thread-0) [   x:recht] o.e.j.w.WebAppClassLoader loaded class 
java.nio.file.attribute.FileTime from null
1140518 DEBUG (Thread-0) [   x:recht] o.a.s.c.CachingDirectoryFactory Releasing 
directory: 
/mnt/solr/jpprodt1/abzug/prodman_solrhome/solr_recht/core_recht/data/index 1 
false
1140520 INFO  (Thread-0) [   x:recht] o.a.s.c.SolrCore [recht] Closing main 
searcher on request.
1140521 DEBUG (Thread-0) [   x:recht] o.a.s.s.SolrIndexSearcher Closing 
Searcher@664499[recht] main

fieldValueCache{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}

filterCache{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}

queryResultCache{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}

documentCache{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}

perSegFilter{lookups=0,hits=0,hitratio=0.0,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.0,cumulative_inserts=0,cumulative_evictions=0}
1140534 DEBUG (Thread-0) [   x:recht] o.a.s.c.CachingDirectoryFactory Releasing 
directory: 
/mnt/solr/jpprodt1/abzug/prodman_solrhome/solr_recht/core_recht/data/index 0 
false
1140535 INFO  (Thread-0) [   x:recht] o.a.s.c.CachingDirectoryFactory Closing 
NRTCachingDirectoryFactory - 2 directories currently being tracked
1140536 DEBUG (Thread-0) [   x:recht] o.a.s.c.CachingDirectoryFactory Closing 
NRTCachingDirectoryFactory - currently tracking: 
CachedDir<>
1140536 DEBUG (Thread-0) [   x:recht] o.a.s.c.CachingDirectoryFactory Closing 
NRTCachingDirectoryFactory - currently tracking: 
CachedDir

Re: SolR 5.3.1 deletes index files

2015-12-18 Thread Erick Erickson
Andreas:

Let me see if I understand correctly:
You have two Solr instances pointing at the _same_ NFS-mounted
directory. The lock type of "single" implies this. And you're totally
and absolutely sure that only _one_ Solr instance writes to that
directory _ever_, right? It's not even the case that the two Solrs
take turns writing to the index

It's a little hard to debug this one remotely, but consider adding this

true

to your solrconfigs, see:
https://cwiki.apache.org/confluence/display/solr/IndexConfig+in+SolrConfig
That'll produce a _lot_ of information in the logs, but does log
detailed information and if this is some weird path in Solr it should
show up in one of the logs.


Best,
Erick

2015-12-18 0:08 GMT-08:00 Moll, Dr. Andreas :
> Hi Shawn,
>
>>To figure out what's going on, we will need information about your
>>server, exactly how you installed Solr, how it is started, how it is
>>stopped, etc.
>
> thanks for your interest in our problem.
>
> I currently start SolR on Linux via
>
> solr-5.3.1/bin/solr start -f
>
> because we are still in a developing phase.
>
> The data/index directory looks like
>
> -rw-r--r-- 1 solr users 23100968011 Dez 15 09:57 _3by.fdt
> -rw-r--r-- 1 solr users 5796206 Dez 15 09:54 _3by.fdx
> -rw-r--r-- 1 solr users   56477 Dez 15 09:54 _3by.fnm
> -rw-r--r-- 1 solr users 24574620575 Dez 15 10:00 _3by_Lucene50_0.doc
> -rw-r--r-- 1 solr users36234501 Dez 15 09:46 _3by_Lucene50_0.dvd
> -rw-r--r-- 1 solr users 203 Dez 15 09:46 _3by_Lucene50_0.dvm
> -rw-r--r-- 1 solr users 59974695274 Dez 15 10:07 _3by_Lucene50_0.pos
> -rw-r--r-- 1 solr users  3089766837 Dez 15 10:07 _3by_Lucene50_0.tim
> -rw-r--r-- 1 solr users42080038 Dez 15 10:07 _3by_Lucene50_0.tip
> -rw-r--r-- 1 solr users 545 Dez 15 09:54 _3by.si
> -rw-r--r-- 1 solr users 59566692561 Dez 15 09:54 _3by.tvd
> -rw-r--r-- 1 solr users 9279030 Dez 15 09:46 _3by.tvx
> -rw-r--r-- 1 solr users 167 Dez 15 09:54 segments_e
>
> We produce the index via a separate instance of SolR.
> Die Filesystem used by the two SolR instances is a NFS share.
>
> The Index-lock-file is configured as
>
> single
>
> When the production SolR writes changes in the index directory while the SolR 
> search server is running,
> the search server deletes all index files on shutdown which is initiated via 
> CTRL-C in the starting shell window.
> SolR also deletes the index-files if I perform a core reload via the 
> web-interface.
>
> I thought it might be an error in the production SolR, but the files are also 
> deleted if I shutdown the production
> SolR and wait a minute before I reload the index or shutdown the SolR search 
> instance.
> If I shut down the SolR production instance and the index files are not used 
> in the search instance, they are not deleted.
>
>
> Thanks and best regards
>
> Andreas Moll
>
> Vertraulichkeitshinweis
> Diese Information und jeder uebermittelte Anhang beinhaltet vertrauliche 
> Informationen und ist nur fuer die Personen oder das Unternehmen bestimmt, an 
> welche sie tatsaechlich gerichtet ist. Sollten Sie nicht der 
> Bestimmungsempfaenger sein, weisen wir Sie darauf hin, dass die Verbreitung, 
> das (auch teilweise) Kopieren sowie der Gebrauch der empfangenen E-Mail und 
> der darin enthaltenen Informationen gesetzlich verboten sein kann und 
> gegebenenfalls Schadensersatzpflichten ausloesen kann. Sollten Sie diese 
> Nachricht aufgrund eines Uebermittlungsfehlers erhalten haben, bitten wir Sie 
> den Sender unverzueglich hiervon in Kenntnis zu setzen.
> Sicherheitswarnung: Bitte beachten Sie, dass das Internet kein sicheres 
> Kommunikationsmedium ist. Obwohl wir im Rahmen unseres Qualitaetsmanagements 
> und der gebotenen Sorgfalt Schritte eingeleitet haben, um einen 
> Computervirenbefall weitestgehend zu verhindern, koennen wir wegen der Natur 
> des Internets das Risiko eines Computervirenbefalls dieser E-Mail nicht 
> ausschliessen.


RE: SolR 5.3.1 deletes index files

2015-12-18 Thread Moll, Dr. Andreas
Hi Shawn,

>To figure out what's going on, we will need information about your
>server, exactly how you installed Solr, how it is started, how it is
>stopped, etc.

thanks for your interest in our problem.

I currently start SolR on Linux via

solr-5.3.1/bin/solr start -f

because we are still in a developing phase.

The data/index directory looks like

-rw-r--r-- 1 solr users 23100968011 Dez 15 09:57 _3by.fdt
-rw-r--r-- 1 solr users 5796206 Dez 15 09:54 _3by.fdx
-rw-r--r-- 1 solr users   56477 Dez 15 09:54 _3by.fnm
-rw-r--r-- 1 solr users 24574620575 Dez 15 10:00 _3by_Lucene50_0.doc
-rw-r--r-- 1 solr users36234501 Dez 15 09:46 _3by_Lucene50_0.dvd
-rw-r--r-- 1 solr users 203 Dez 15 09:46 _3by_Lucene50_0.dvm
-rw-r--r-- 1 solr users 59974695274 Dez 15 10:07 _3by_Lucene50_0.pos
-rw-r--r-- 1 solr users  3089766837 Dez 15 10:07 _3by_Lucene50_0.tim
-rw-r--r-- 1 solr users42080038 Dez 15 10:07 _3by_Lucene50_0.tip
-rw-r--r-- 1 solr users 545 Dez 15 09:54 _3by.si
-rw-r--r-- 1 solr users 59566692561 Dez 15 09:54 _3by.tvd
-rw-r--r-- 1 solr users 9279030 Dez 15 09:46 _3by.tvx
-rw-r--r-- 1 solr users 167 Dez 15 09:54 segments_e

We produce the index via a separate instance of SolR.
Die Filesystem used by the two SolR instances is a NFS share.

The Index-lock-file is configured as

single

When the production SolR writes changes in the index directory while the SolR 
search server is running,
the search server deletes all index files on shutdown which is initiated via 
CTRL-C in the starting shell window.
SolR also deletes the index-files if I perform a core reload via the 
web-interface.

I thought it might be an error in the production SolR, but the files are also 
deleted if I shutdown the production
SolR and wait a minute before I reload the index or shutdown the SolR search 
instance.
If I shut down the SolR production instance and the index files are not used in 
the search instance, they are not deleted.


Thanks and best regards

Andreas Moll

Vertraulichkeitshinweis
Diese Information und jeder uebermittelte Anhang beinhaltet vertrauliche 
Informationen und ist nur fuer die Personen oder das Unternehmen bestimmt, an 
welche sie tatsaechlich gerichtet ist. Sollten Sie nicht der 
Bestimmungsempfaenger sein, weisen wir Sie darauf hin, dass die Verbreitung, 
das (auch teilweise) Kopieren sowie der Gebrauch der empfangenen E-Mail und der 
darin enthaltenen Informationen gesetzlich verboten sein kann und 
gegebenenfalls Schadensersatzpflichten ausloesen kann. Sollten Sie diese 
Nachricht aufgrund eines Uebermittlungsfehlers erhalten haben, bitten wir Sie 
den Sender unverzueglich hiervon in Kenntnis zu setzen.
Sicherheitswarnung: Bitte beachten Sie, dass das Internet kein sicheres 
Kommunikationsmedium ist. Obwohl wir im Rahmen unseres Qualitaetsmanagements 
und der gebotenen Sorgfalt Schritte eingeleitet haben, um einen 
Computervirenbefall weitestgehend zu verhindern, koennen wir wegen der Natur 
des Internets das Risiko eines Computervirenbefalls dieser E-Mail nicht 
ausschliessen.


Re: SolR 5.3.1 deletes index files

2015-12-17 Thread Shawn Heisey
On 12/17/2015 8:00 AM, Moll, Dr. Andreas wrote:
> we are using SolR for some years now and are currently switching from SolR 
> 3.6 to 5.3.1.
> SolR 5.3.1 deletes all index files when it shuts down and there were external 
> changes on the index-files
> (in our case from a second SolR-server which produces the index).

I have *never* seen Solr delete the index files without outside
influence.  Either there's a misconfiguration, or something in your
environment is doing the delete.

If the DirectoryFactory were changed to use RAMDirectoryFactory, then
all the data would be in memory, and that would be purged on shutdown,
because it doesn't exist anywhere else.

Assuming you're using a standard directory implementation that puts
files on the disk, there is only one feature that I'm aware of that can
automatically delete information -- it's possible to index documents
with an expiration date, so it's automatically deleted once the
expiration date is reached.  I would not expect this to delete
everything on shutdown, though.

To figure out what's going on, we will need information about your
server, exactly how you installed Solr, how it is started, how it is
stopped, etc.

Thanks,
Shawn