Thanks Erick.
Couple of Questions :
Our transaction logs are huge as we have disabled auto commit. The biggest
one is 6.1 GB.
*567M* autosuggest/data/tlog
*22M* avmediaCore/data/tlog
*388M* booksCore/data/tlog
*4.9G * books/data/tlog
*6.1G * mp3-downloads/data/tlog ( 150 % of index Size)
1*.5G* next-5/data/tlog
690M queries/data/tlog ( 25 % of Index Size )
207M queryProduct/data/tlog (100 % of Index Size)
Btw, I am surprised by the size of transaction log, because that is a
significant amount of index size itself
2.6G autosuggest/data/index
992M avmediaCore/data/index
12G booksCore/data/index
4.2G mp3-downloads-new/data/index
45G next-5/data/index
2.9G queries/data/index
*220M* queryProduct/data/index
We use DIH and have turned off the Auto commit because we have to sometimes
build index from Scratch (clean=true) and we not want to
Our master server sees a lot of restarts, sometimes 2-3 times a day. It
polls other Data Sources for updates which are quite a few. Master
maintains a version of last committed version and can handle uncommitted
changes.
Given the frequent restarts, We can't really afford a huge start up at this
point.
In the worst case, does Solr allow for disabling transactional log ?
Here is our Index Config
<indexConfig>
<!-- Values here affect all index writers and act as a default unless
overridden. -->
<useCompoundFile>false</useCompoundFile>
<mergeFactor>10</mergeFactor>
<ramBufferSizeMB>32</ramBufferSizeMB>
<maxMergeDocs>2147483647</maxMergeDocs>
<maxFieldLength>50000</maxFieldLength>
<writeLockTimeout>1000</writeLockTimeout>
<commitLockTimeout>10000</commitLockTimeout>
<lockType>single</lockType>
<!-- options specific to the main on-disk lucene index -->
<useCompoundFile>false</useCompoundFile>
<ramBufferSizeMB>32</ramBufferSizeMB>
<mergeFactor>5</mergeFactor>
<!-- Deprecated -->
<!--<maxBufferedDocs>1000</maxBufferedDocs>-->
<maxMergeDocs>2147483647</maxMergeDocs>
<maxFieldLength>50000</maxFieldLength>
<unlockOnStartup>false</unlockOnStartup>
<deletionPolicy class="solr.SolrDeletionPolicy">
<!-- The number of commit points to be kept -->
<str name="maxCommitsToKeep">5</str>
<!-- The number of optimized commit points to be kept -->
<str name="maxOptimizedCommitsToKeep">0</str>
<str name="maxCommitAge">2HOUR</str>
</deletionPolicy>
</indexConfig>
Thanks & Regards
Umesh Prasad
On Wed, Apr 17, 2013 at 4:57 PM, Erick Erickson <erickerick...@gmail.com>wrote:
> How big are you transaction logs? They can be replayed on startup.
> They are truncated and a new one started when you do a hard commit
> (openSearcher true or false doesn't matter).
>
> So a quick test of this theory would be to just stop your indexing
> process, issue a hard commit on all your cores and _then_ try to
> restart. If it comes up immediately, you've identified your problem.
>
> Best
> Erick
>
> On Tue, Apr 16, 2013 at 8:33 AM, Umesh Prasad <umesh.i...@gmail.com>
> wrote:
> > Hi,
> > We are migrating to Solr 4.2 from Solr 3.6 and Solr 4.2 is throwing
> > Exception on Restart. What is More, it take a hell lot of Time ( More
> than
> > one hour to get Up and Running)
> >
> >
> > THE exception After Restart ...
> > =========================
> > "Apr 16, 2013 4:47:31 PM org.apache.solr.update.UpdateLog$RecentUpdates
> > update
> > WARNING: Unexpected log entry or corrupt log. Entry=11
> > java.lang.ClassCastException: java.lang.Long cannot be cast to
> > java.util.List
> > at
> > org.apache.solr.update.UpdateLog$RecentUpdates.update(UpdateLog.java:929)
> > at
> >
> org.apache.solr.update.UpdateLog$RecentUpdates.access$000(UpdateLog.java:863)
> > at
> > org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1014)
> > at org.apache.solr.update.UpdateLog.init(UpdateLog.java:253)
> > at
> > org.apache.solr.update.UpdateHandler.initLog(UpdateHandler.java:82)
> > at
> > org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137)
> > at
> > org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:123)
> > at
> >
> org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:95)
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> > at
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> > at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> > at
> org.apache.solr.core.SolrCore.createInstance(SolrCore.java:525)
> > at
> > org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:596)
> > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:806)
> > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:618)
> > at
> >
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:1021)
> > at
> > org.apache.solr.core.CoreContainer.create(CoreContainer.java:1051)
> > at
> org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:634)
> > at
> org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629)
> > at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> > at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > at java.lang.Thread.run(Thread.java:662)
> > Apr 16, 2013 4:47:31 PM org.apache.solr.update.UpdateLog$RecentUpdates
> > update
> > WARNING: Unexpected log entry or corrupt log. Entry=8120?785879438123
> > java.lang.ClassCastException: java.lang.String cannot be cast to
> > java.util.List
> > at
> > org.apache.solr.update.UpdateLog$RecentUpdates.update(UpdateLog.java:929)
> > at
> >
> org.apache.solr.update.UpdateLog$RecentUpdates.access$000(UpdateLog.java:863)
> > at
> > org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1014)
> > at org.apache.solr.update.UpdateLog.init(UpdateLog.java:253)
> > at
> > org.apache.solr.update.UpdateHandler.initLog(UpdateHandler.java:82)
> > at
> > org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:137)
> > at
> > org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:123)
> > at
> >
> org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:95)
> >
> > =========================
> >
> > And Once Restarted, I start getting replication errors
> >
> >
> > Apr 16, 2013 5:20:30 PM org.apache.solr.handler.SnapPuller
> fetchLatestIndex
> > SEVERE: Master at: http://localhost:25280/solr/accessories is not
> > available. Index fetch failed. Exception:
> > org.apache.solr.client.solrj.SolrServerException: IOException occured
> when
> > talking to server at: http://localhost:25280/solr/accessories
> > Apr 16, 2013 5:20:30 PM org.apache.solr.handler.SnapPuller
> fetchLatestIndex
> > SEVERE: Master at: http://localhost:25280/solr/newQueries is not
> available.
> > Index fetch failed. Exception:
> > org.apache.solr.client.solrj.SolrServerException: IOException occured
> when
> > talking to server at: http://localhost:25280/solr/newQueries
> > Apr 16, 2013 5:21:00 PM org.apache.solr.handler.SnapPuller
> fetchLatestIndex
> > SEVERE: Master at: http://localhost:25280/solr/phcare is not available.
> > Index fetch failed. Exception:
> > org.apache.solr.client.solrj.SolrServerException: Timeout occured while
> > waiting response from server at: http://localhost:25280/solr/phcare
> > Apr 16, 2013 5:21:00 PM org.apache.solr.handler.SnapPuller
> fetchLatestIndex
> > SEVERE: Master at: http://localhost:25280/solr/audioplayersCore is not
> > available. Index fetch failed. Exception:
> > org.apache.solr.client.solrj.SolrServerException: Timeout occured while
> > waiting response from server at:
> > http://localhost:25280/solr/audioplayersCore
> > Apr 16, 2013 5:21:24 PM org.apache.solr.handler.SnapPuller
> fetchLatestIndex
> > SEVERE: Master at: http://localhost:25280/solr/cameras is not available.
> > Index fetch failed. Exception:
> > org.apache.solr.client.solrj.SolrServerException: Timeout occured while
> > waiting response from server at: http://localhost:25280/solr/cameras
> >
> >
> > Before Restart : Server was running Incremental Indexing ( Triggered by
> a
> > Cron). The cron triggers every 5 mins for each of about 40 Cores. This
> was
> > the same with Solr 3.5 also. But we never faced any issues.
> >
> >
> >
> > --
> > ---
> > Thanks & Regards
> > Umesh Prasad
>
--
---
Thanks & Regards
Umesh Prasad
