I would say definitely investigate the performance of the query, but also since you're using CachedSqlEntityProcessor, you might want to back off on the transaction isolation to READ_COMMITTED, which I think is the lowest one that Oracle supports:
http://wiki.apache.org/solr/DataImportHandler#Configuring_JdbcDataSource Michael Della Bitta Applications Developer o: +1 646 532 3062 | c: +1 917 477 7906 appinions inc. “The Science of Influence Marketing” 18 East 41st Street New York, NY 10017 t: @appinions <https://twitter.com/Appinions> | g+: plus.google.com/appinions w: appinions.com <http://www.appinions.com/> On Fri, Jun 28, 2013 at 2:52 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Hi, > > I'd go talk to the DBA. How long does this query take if you run it > directly against Oracle? How long if you run it locally vs. from a > remove server (like Solr is in relation to your Oracle server(s)). > What happens if you increase batchSize? > > Otis > -- > Solr & ElasticSearch Support -- http://sematext.com/ > Performance Monitoring -- http://sematext.com/spm > > > > On Thu, Jun 27, 2013 at 6:41 PM, srinalluri <nallurisr...@yahoo.com> > wrote: > > Hello, > > > > I am using Solr 4.3.2 and Oracle DB. The sub entity is using > > CachedSqlEntityProcessor. The dataSource is having batchSize="500". The > > full-import is failed with 'ORA-01555: snapshot too old: rollback segment > > number with name "" too small ORA-22924: snapshot too old' Exception > after > > 5 hours. > > > > We already increased the undo space 4 times at the database end. Number > of > > records in the jan_story table is 800,000 only. Tomcat is with 4GB JVM > > memory. > > > > Following is the entity (there are other sub-entities, I didn't mention > them > > here. As the import failed with article_details entity. article_details > is > > the first sub-entity) > > > > <entity name="par8-article-testingprod" dataSource="par8_prod" pk="VCMID" > > preImportDeleteQuery="content_type:article AND > > repository:par8qatestingprod" > > query="select ID as VCMID from jan_story"> > > <entity name="article_details" dataSource="par8_prod" > > transformer="TemplateTransformer,ClobTransformer,RegexTransformer" > > query="select bb.recordid, aa.ID as DID,aa.STORY_TITLE, > > aa.STORY_HEADLINE, aa.SOURCE, aa.DECK, regexp_replace(aa.body, > > '\<p\>\[(pullquote|summary)\]\</p\>|\[video [0-9]+?\]|\[youtube > > .+?\]', '') as BODY, aa.PUBLISHED_DATE, aa.MODIFIED_DATE, aa.DATELINE, > > aa.REPORTER_NAME, aa.TICKER_CODES,aa.ADVERTORIAL_CONTENT from jan_story > > aa,mapp bb where aa.id=bb.keystring1" cacheKey="DID" > > cacheLookup="par8-article-testingprod.VCMID" > > processor="CachedSqlEntityProcessor" > > > <field column="content_type" template="article" /> > > <field column="RECORDID" name="native_id" /> > > <field column="repository" template="par8qatestingprod" /> > > <field column="STORY_TITLE" name="title" /> > > <field column="DECK" name="description" clob="true" /> > > <field column="PUBLISHED_DATE" name="date" /> > > <field column="MODIFIED_DATE" name="last_modified_date" /> > > <field column="BODY" name="body" clob="true" /> > > <field column="SOURCE" name="source" /> > > <field column="DATELINE" name="dateline" /> > > <field column="STORY_HEADLINE" name="export_headline" /> > > </entity> > > </entity> > > > > > > The full-import without CachedSqlEntityProcessor is taking 7 days. That > is > > why I am doing all this. > > > > > > > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/full-import-failed-after-5-hours-with-Exception-ORA-01555-snapshot-too-old-rollback-segment-number-wd-tp4073822.html > > Sent from the Solr - User mailing list archive at Nabble.com. >
