Re: Solr 3.1 Upgrade - Reindex necessary ?
1. Do we have to reindex all content again to use Solr 3.1 ? 2. If we don't reindex all content are there any potential issues ? (I read somewhere that first commit would change the 1.4.1 format to 3.1. have the analyzer's behavior changed which warrants reindexing ?) 3. Apart from deploying the new solr 3.1 war; Is it just enough to set luceneMatchVersionLUCENE_31/luceneMatchVersion to get all the goodies and bug fixes of the LUCENE/SOLR 3.1 ? HI Solr-3.1 version usage the the latest version of Lucene jars so if you are planning to Upgrade then it is necessary to Re index all the content with Solr3.1 version. Not re-indexing will possibly cause of index corruption because newer version of lucene will create indexes in Newer version which is backward compatible for read only. setting luceneMatchVersionLUCENE_31/luceneMatchVersion is not enough because it will not get the lucene 3.1 jar automatically. - Thanx: Grijesh www.gettinhahead.co.in -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-3-1-Upgrade-Reindex-necessary-tp2919679p2922645.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 3.1 Upgrade - Reindex necessary ?
Thanks Grijesh for responding. I meant that I will use the Lucene 3.1 jars for indexing also from now on. My current index already has a million docs indexed with solr 1.4.1 version, I read somewhere that once server is upgraded to 3.1, it is said that the first commit will change the indexes to 3.1 format automatically. Is this true or do I have to literally reindex the million docs again ? Thanks, Ravi Kiran Bhaskar On Tuesday, May 10, 2011, Grijesh pintu.grij...@gmail.com wrote: 1. Do we have to reindex all content again to use Solr 3.1 ? 2. If we don't reindex all content are there any potential issues ? (I read somewhere that first commit would change the 1.4.1 format to 3.1. have the analyzer's behavior changed which warrants reindexing ?) 3. Apart from deploying the new solr 3.1 war; Is it just enough to set luceneMatchVersionLUCENE_31/luceneMatchVersion to get all the goodies and bug fixes of the LUCENE/SOLR 3.1 ? HI Solr-3.1 version usage the the latest version of Lucene jars so if you are planning to Upgrade then it is necessary to Re index all the content with Solr3.1 version. Not re-indexing will possibly cause of index corruption because newer version of lucene will create indexes in Newer version which is backward compatible for read only. setting luceneMatchVersionLUCENE_31/luceneMatchVersion is not enough because it will not get the lucene 3.1 jar automatically. - Thanx: Grijesh www.gettinhahead.co.in -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-3-1-Upgrade-Reindex-necessary-tp2919679p2922645.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr 3.1 Upgrade - Reindex necessary ?
: Thanks Grijesh for responding. I meant that I will use the Lucene 3.1 : jars for indexing also from now on. My current index already has a : million docs indexed with solr 1.4.1 version, I read somewhere that : once server is upgraded to 3.1, it is said that the first commit will : change the indexes to 3.1 format automatically. Is this true or do I : have to literally reindex the million docs again ? index versioning happens on a segment basis, so once you start using Solr 3.1, as new docs are added and segments are merged those segments will be updated to the new file format -- the way to ensure that all segments are updated is to optimize your index. : 1. Do we have to reindex all content again to use Solr 3.1 ? you should not need to, know. : 3. Apart from deploying the new solr 3.1 war; Is it just enough to set : luceneMatchVersionLUCENE_31/luceneMatchVersion to get all the : goodies and bug fixes of the LUCENE/SOLR 3.1 ? It's not mandatory to change the luceneMatchVersion/ to upgrade -- if you do want to change the luceneMatchVersion/ then you should reindex, as that change causes analyzers/query parsers to behave differently (in ways thta might be incompatible with how they behave previously. this change is unrelated to the index fileformat -- optimizing your index to force the 3.1 fileformat has no impact on how what esoteric/broken behavior a tokenizer might have had in the past that changed once the luceneMatchVersion/ setting is updated. The purpose of luceneMatchVersion/ is to say i want the behavior of X.Y, even when it's been decided that that behavior was bad, because it's what matches the terms i've already indexed -Hoss
Re: Solr 3.1 Upgrade - Reindex necessary ?
Hoss, Thank you very much for clearly delineating the difference. Just to be clear - My intent to move to 3.1 was driven by my desire to improve my replication performance - Deducing from your explanation, I believe the replication/indexing related changes/bug fixes like the following will be available to me even without specifying luceneMatchVersionLUCENE_31/luceneMatchVersion am I right ?? faster exact PhraseQuery; merging favors segments with deletions; primary key lookup is faster; IndexWriter.addIndexes(Directory[]) uses file copy instead of merging; various Directory performance improvements; compound file is dynamically turned off for large segments; fully deleted segments are dropped on commit; faster snowball analyzers (in contrib); ConcurrentMergeScheduler is more careful about setting priority of merge threads. Ravi Kiran Bhaskar On Tue, May 10, 2011 at 2:49 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : Thanks Grijesh for responding. I meant that I will use the Lucene 3.1 : jars for indexing also from now on. My current index already has a : million docs indexed with solr 1.4.1 version, I read somewhere that : once server is upgraded to 3.1, it is said that the first commit will : change the indexes to 3.1 format automatically. Is this true or do I : have to literally reindex the million docs again ? index versioning happens on a segment basis, so once you start using Solr 3.1, as new docs are added and segments are merged those segments will be updated to the new file format -- the way to ensure that all segments are updated is to optimize your index. : 1. Do we have to reindex all content again to use Solr 3.1 ? you should not need to, know. : 3. Apart from deploying the new solr 3.1 war; Is it just enough to set : luceneMatchVersionLUCENE_31/luceneMatchVersion to get all the : goodies and bug fixes of the LUCENE/SOLR 3.1 ? It's not mandatory to change the luceneMatchVersion/ to upgrade -- if you do want to change the luceneMatchVersion/ then you should reindex, as that change causes analyzers/query parsers to behave differently (in ways thta might be incompatible with how they behave previously. this change is unrelated to the index fileformat -- optimizing your index to force the 3.1 fileformat has no impact on how what esoteric/broken behavior a tokenizer might have had in the past that changed once the luceneMatchVersion/ setting is updated. The purpose of luceneMatchVersion/ is to say i want the behavior of X.Y, even when it's been decided that that behavior was bad, because it's what matches the terms i've already indexed -Hoss
Solr 3.1 Upgrade - Reindex necessary ?
Hello All, I am planning to upgrade from Solr 1.4.1 to Solr 3.1. I saw some deprecation warnings in the log as shown below [#|2011-05-09T12:37:18.762-0400|WARNING|sun-appserver9.1|org.apache.solr.analysis.BaseTokenStreamFactory|_ThreadID=53;_ThreadName=httpSSLWorkerThread-9001-13 ;_RequestID=de32fd3f-e968-4228-a071-9bb175bfb549;|StopFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0|#] [#|2011-05-09T12:37:18.765-0400|WARNING|sun-appserver9.1|org.apache.solr.analysis.BaseTokenStreamFactory|_ThreadID=53;_ThreadName=httpSSLWorkerThread-9001-13 ;_RequestID=de32fd3f-e968-4228-a071-9bb175bfb549;|WordDelimiterFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and re index to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0|#] [#|2011-05-09T12:37:18.767-0400|WARNING|sun-appserver9.1|org.apache.solr.analysis.BaseTokenStreamFactory|_ThreadID=53;_ThreadName=httpSSLWorkerThread-9001-13 ;_RequestID=de32fd3f-e968-4228-a071-9bb175bfb549;|EnglishPorterFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and re index to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0|#] so I would love the experts advise on the following questions 1. Do we have to reindex all content again to use Solr 3.1 ? 2. If we don't reindex all content are there any potential issues ? (I read somewhere that first commit would change the 1.4.1 format to 3.1. have the analyzer's behavior changed which warrants reindexing ?) 3. Apart from deploying the new solr 3.1 war; Is it just enough to set luceneMatchVersionLUCENE_31/luceneMatchVersion to get all the goodies and bug fixes of the LUCENE/SOLR 3.1 ? Thank You, Ravi Kiran Bhaskar
