I guess the indexing of a doc is taking too long (may be because of the de-dup patch) and the resultset gets closed automaticallly (timed out) --Noble
On Mon, Jan 5, 2009 at 5:14 PM, Marc Sturlese <marc.sturl...@gmail.com> wrote: > > Donig this fix I get the same error :( > > I am going to try to set up the last nigthly build... let's see if I have > better luck. > > The thing is it stop indexing at the doc num 150.000 aprox... and give me > that mysql exception error... Without DeDuplication patch I can index 2 > milion docs without problems... > > I am pretty lost with this... :( > > > Shalin Shekhar Mangar wrote: >> >> Yes I meant the 05/01/2008 build. The fix is a one line change >> >> Add the following as the last line of DataConfig.Entity.clearCache() >> dataSrc = null; >> >> >> >> On Mon, Jan 5, 2009 at 4:22 PM, Marc Sturlese >> <marc.sturl...@gmail.com>wrote: >> >>> >>> Shalin you mean I should test the 05/01/2008 nighlty? maybe with this one >>> works? If the fix you did is not really big can u tell me where in the >>> source is and what is it for? (I have been debuging and tracing a lot the >>> dataimporthandler source and I I would like to know what the imporovement >>> is >>> about if it is not a problem...) >>> >>> Thanks! >>> >>> >>> Shalin Shekhar Mangar wrote: >>> > >>> > Marc, I've just committed a fix which may have caused the bug. Can you >>> use >>> > svn trunk (or the next nightly build) and confirm? >>> > >>> > On Mon, Jan 5, 2009 at 3:10 PM, Noble Paul നോബിള് नोब्ळ् < >>> > noble.p...@gmail.com> wrote: >>> > >>> >> looks like a bug w/ DIH with the recent fixes. >>> >> --Noble >>> >> >>> >> On Mon, Jan 5, 2009 at 2:36 PM, Marc Sturlese >>> <marc.sturl...@gmail.com> >>> >> wrote: >>> >> > >>> >> > Hey there, >>> >> > I was using the Deduplication patch with Solr 1.3 release and >>> >> everything >>> >> was >>> >> > working perfectly. Now I upgraded to a nigthly build (20th december) >>> to >>> >> be >>> >> > able to use new facet algorithm and other stuff and DeDuplication is >>> >> not >>> >> > working any more. I have followed exactly the same steps to apply >>> the >>> >> patch >>> >> > to the source code. I am geting this error: >>> >> > >>> >> > WARNING: Error reading data >>> >> > com.mysql.jdbc.CommunicationsException: Communications link failure >>> due >>> >> to >>> >> > underlying exception: >>> >> > >>> >> > ** BEGIN NESTED EXCEPTION ** >>> >> > >>> >> > java.io.EOFException >>> >> > >>> >> > STACKTRACE: >>> >> > >>> >> > java.io.EOFException >>> >> > at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1905) >>> >> > at >>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2404) >>> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) >>> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) >>> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) >>> >> > at >>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) >>> >> > at >>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) >>> >> > at com.mysql.jdbc.ResultSet.next(ResultSet.java:6144) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:294) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) >>> >> > >>> >> > >>> >> > ** END NESTED EXCEPTION ** >>> >> > Last packet sent to the server was 202481 ms ago. >>> >> > at >>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2563) >>> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) >>> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) >>> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) >>> >> > at >>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) >>> >> > at >>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) >>> >> > at com.mysql.jdbc.ResultSet.next(ResultSet.java:6144) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:294) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) >>> >> > Jan 5, 2009 10:06:16 AM >>> >> org.apache.solr.handler.dataimport.JdbcDataSource >>> >> > logError >>> >> > WARNING: Exception while closing result set >>> >> > com.mysql.jdbc.CommunicationsException: Communications link failure >>> due >>> >> to >>> >> > underlying exception: >>> >> > >>> >> > ** BEGIN NESTED EXCEPTION ** >>> >> > >>> >> > java.io.EOFException >>> >> > >>> >> > STACKTRACE: >>> >> > >>> >> > java.io.EOFException >>> >> > at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1905) >>> >> > at >>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2351) >>> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) >>> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) >>> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) >>> >> > at >>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) >>> >> > at >>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) >>> >> > at >>> com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:150) >>> >> > at com.mysql.jdbc.ResultSet.realClose(ResultSet.java:6488) >>> >> > at com.mysql.jdbc.ResultSet.close(ResultSet.java:736) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.close(JdbcDataSource.java:312) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:304) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) >>> >> > >>> >> > >>> >> > ** END NESTED EXCEPTION ** >>> >> > >>> >> > Last packet sent to the server was 202482 ms ago. >>> >> > at >>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2563) >>> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) >>> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) >>> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) >>> >> > at >>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) >>> >> > at >>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) >>> >> > at >>> com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:150) >>> >> > at com.mysql.jdbc.ResultSet.realClose(ResultSet.java:6488) >>> >> > at com.mysql.jdbc.ResultSet.close(ResultSet.java:736) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.close(JdbcDataSource.java:312) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:304) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) >>> >> > at >>> >> > >>> >> >>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) >>> >> > >>> >> > Mark do you know if I should to some extra hack to run DeDuplication >>> >> patch >>> >> > in the new solr version (nightly)? Has something changed in the >>> source >>> >> that >>> >> > make it not work or it's simply that i am doing something wrong? >>> >> > >>> >> > Thanks in advanced >>> >> > -- >>> >> > View this message in context: >>> >> >>> http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21287327.html >>> >> > Sent from the Solr - User mailing list archive at Nabble.com. >>> >> > >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> --Noble Paul >>> >> >>> > >>> > >>> > >>> > -- >>> > Regards, >>> > Shalin Shekhar Mangar. >>> > >>> > >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21288939.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>> >> >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> > > -- > View this message in context: > http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21289643.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- --Noble Paul
