Donig this fix I get the same error :( I am going to try to set up the last nigthly build... let's see if I have better luck.
The thing is it stop indexing at the doc num 150.000 aprox... and give me that mysql exception error... Without DeDuplication patch I can index 2 milion docs without problems... I am pretty lost with this... :( Shalin Shekhar Mangar wrote: > > Yes I meant the 05/01/2008 build. The fix is a one line change > > Add the following as the last line of DataConfig.Entity.clearCache() > dataSrc = null; > > > > On Mon, Jan 5, 2009 at 4:22 PM, Marc Sturlese > <marc.sturl...@gmail.com>wrote: > >> >> Shalin you mean I should test the 05/01/2008 nighlty? maybe with this one >> works? If the fix you did is not really big can u tell me where in the >> source is and what is it for? (I have been debuging and tracing a lot the >> dataimporthandler source and I I would like to know what the imporovement >> is >> about if it is not a problem...) >> >> Thanks! >> >> >> Shalin Shekhar Mangar wrote: >> > >> > Marc, I've just committed a fix which may have caused the bug. Can you >> use >> > svn trunk (or the next nightly build) and confirm? >> > >> > On Mon, Jan 5, 2009 at 3:10 PM, Noble Paul നോബിള് नोब्ळ् < >> > noble.p...@gmail.com> wrote: >> > >> >> looks like a bug w/ DIH with the recent fixes. >> >> --Noble >> >> >> >> On Mon, Jan 5, 2009 at 2:36 PM, Marc Sturlese >> <marc.sturl...@gmail.com> >> >> wrote: >> >> > >> >> > Hey there, >> >> > I was using the Deduplication patch with Solr 1.3 release and >> >> everything >> >> was >> >> > working perfectly. Now I upgraded to a nigthly build (20th december) >> to >> >> be >> >> > able to use new facet algorithm and other stuff and DeDuplication is >> >> not >> >> > working any more. I have followed exactly the same steps to apply >> the >> >> patch >> >> > to the source code. I am geting this error: >> >> > >> >> > WARNING: Error reading data >> >> > com.mysql.jdbc.CommunicationsException: Communications link failure >> due >> >> to >> >> > underlying exception: >> >> > >> >> > ** BEGIN NESTED EXCEPTION ** >> >> > >> >> > java.io.EOFException >> >> > >> >> > STACKTRACE: >> >> > >> >> > java.io.EOFException >> >> > at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1905) >> >> > at >> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2404) >> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) >> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) >> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) >> >> > at >> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) >> >> > at >> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) >> >> > at com.mysql.jdbc.ResultSet.next(ResultSet.java:6144) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:294) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) >> >> > >> >> > >> >> > ** END NESTED EXCEPTION ** >> >> > Last packet sent to the server was 202481 ms ago. >> >> > at >> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2563) >> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) >> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) >> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) >> >> > at >> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) >> >> > at >> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) >> >> > at com.mysql.jdbc.ResultSet.next(ResultSet.java:6144) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:294) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) >> >> > Jan 5, 2009 10:06:16 AM >> >> org.apache.solr.handler.dataimport.JdbcDataSource >> >> > logError >> >> > WARNING: Exception while closing result set >> >> > com.mysql.jdbc.CommunicationsException: Communications link failure >> due >> >> to >> >> > underlying exception: >> >> > >> >> > ** BEGIN NESTED EXCEPTION ** >> >> > >> >> > java.io.EOFException >> >> > >> >> > STACKTRACE: >> >> > >> >> > java.io.EOFException >> >> > at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1905) >> >> > at >> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2351) >> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) >> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) >> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) >> >> > at >> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) >> >> > at >> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) >> >> > at >> com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:150) >> >> > at com.mysql.jdbc.ResultSet.realClose(ResultSet.java:6488) >> >> > at com.mysql.jdbc.ResultSet.close(ResultSet.java:736) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.close(JdbcDataSource.java:312) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:304) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) >> >> > >> >> > >> >> > ** END NESTED EXCEPTION ** >> >> > >> >> > Last packet sent to the server was 202482 ms ago. >> >> > at >> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2563) >> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) >> >> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) >> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) >> >> > at >> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) >> >> > at >> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) >> >> > at >> com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:150) >> >> > at com.mysql.jdbc.ResultSet.realClose(ResultSet.java:6488) >> >> > at com.mysql.jdbc.ResultSet.close(ResultSet.java:736) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.close(JdbcDataSource.java:312) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:304) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) >> >> > at >> >> > >> >> >> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) >> >> > >> >> > Mark do you know if I should to some extra hack to run DeDuplication >> >> patch >> >> > in the new solr version (nightly)? Has something changed in the >> source >> >> that >> >> > make it not work or it's simply that i am doing something wrong? >> >> > >> >> > Thanks in advanced >> >> > -- >> >> > View this message in context: >> >> >> http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21287327.html >> >> > Sent from the Solr - User mailing list archive at Nabble.com. >> >> > >> >> > >> >> >> >> >> >> >> >> -- >> >> --Noble Paul >> >> >> > >> > >> > >> > -- >> > Regards, >> > Shalin Shekhar Mangar. >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21288939.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21289643.html Sent from the Solr - User mailing list archive at Nabble.com.
