OK, setting the wait_timeout back to its previous value and adding readOnly didn't help, I got the stack overflow again. I re-upped the mysql timeout value again.
-jsd- On Tue, May 15, 2012 at 2:42 PM, Jon Drukman <jdruk...@gmail.com> wrote: > I fixed it for now by upping the wait_timeout on the mysql server. > Apparently Solr doesn't like having its connection yanked out from under > it and/or isn't smart enough to reconnect if the server goes away. I'll > set it back the way it was and try your readOnly option. > > Is there an option with DataImportHandler to have it transmit one or more > arbitrary SQL statements after connecting? If there was, I could just send > "SET wait_timeout=86400;" after connecting. That would probably prevent > this issue. > > -jsd- > > On Tue, May 15, 2012 at 2:35 PM, Dyer, James <james.d...@ingrambook.com>wrote: > >> Shot in the dark here, but try adding readOnly="true" to your dataSource >> tag. >> >> <dataSource readOnly="true" type="JdbcDataSource" ... /> >> >> This sets autocommit to true and sets the Holdability to >> ResultSet.CLOSE_CURSORS_AT_COMMIT. DIH does not explicitly close >> resultsets and maybe if your JDBC driver also manages this poorly you could >> end up with strange conditions like the one you're getting? It could be a >> case where your data has grown just over the limit your setup can handle >> under such an unfortunate circumstance. >> >> Let me know if this solves it. If so, we probably should open a bug >> report and get this fixed in DIH. >> >> James Dyer >> E-Commerce Systems >> Ingram Content Group >> (615) 213-4311 >> >> >> -----Original Message----- >> From: Jon Drukman [mailto:jdruk...@gmail.com] >> Sent: Tuesday, May 15, 2012 4:12 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Exception in DataImportHandler (stack overflow) >> >> i don't think so, my config is straightforward: >> >> <dataConfig> >> <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" >> url="jdbc:mysql://xxxxx/xxxxxx" >> user="xxxxx" password="xxxxx" batchSize="-1" /> >> <document> >> <entity name="content" >> query="select content_id, description, title, add_date from >> content_solr where active = '1'"> >> <entity name="tag" >> query="select tag_id from tags_assoc where content_id = >> '${content.content_id}'" /> >> <entity name="likes" >> query="select count(1) as likes from votes where content_id = >> '${content.content_id}'" /> >> <entity name="views" >> query="select sum(views) as views from media_views mv join >> content_media cm USING (media_id) WHERE cm.content_id = >> '${content.content_id}'" /> >> </entity> >> </document> >> </dataConfig> >> >> i'm triggering the import with: >> >> http://localhost:8983/solr/dataimport?command=full-import&clean=true&commit=true >> >> >> >> On Tue, May 15, 2012 at 2:07 PM, Michael Della Bitta < >> michael.della.bi...@appinions.com> wrote: >> >> > Hi, Jon: >> > >> > Well, you don't see that every day! >> > >> > Is it possible that you have something weird going on in your DDL >> > and/or queries, like a tree schema that now suddenly has a cyclical >> > reference? >> > >> > Michael >> > >> > On Tue, May 15, 2012 at 4:33 PM, Jon Drukman <jdruk...@gmail.com> >> wrote: >> > > I have a machine which does a full update using DataImportHandler >> every >> > > hour. It worked up until a little while ago. I did not change the >> > > dataconfig.xml or version of Solr. >> > > >> > > Here is the beginning of the error in the log (the real thing runs for >> > > thousands of lines) >> > > >> > > 2012-05-15 12:44:30.724166500 SEVERE: Full Import >> > > failed:org.apache.solr.handler.dataimport.DataImportHandlerException: >> > > java.lang.StackOverflowError >> > > 2012-05-15 12:44:30.724168500 at >> > > >> > >> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:669) >> > > 2012-05-15 12:44:30.724169500 at >> > > >> > >> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:268) >> > > 2012-05-15 12:44:30.724171500 at >> > > >> > >> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:187) >> > > 2012-05-15 12:44:30.724219500 at >> > > >> > >> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359) >> > > 2012-05-15 12:44:30.724221500 at >> > > >> > >> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427) >> > > 2012-05-15 12:44:30.724223500 at >> > > >> > >> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408) >> > > 2012-05-15 12:44:30.724224500 Caused by: java.lang.StackOverflowError >> > > 2012-05-15 12:44:30.724225500 at >> > > java.lang.String.checkBounds(String.java:404) >> > > 2012-05-15 12:44:30.724234500 at >> java.lang.String.<init>(String.java:450) >> > > 2012-05-15 12:44:30.724235500 at >> java.lang.String.<init>(String.java:523) >> > > 2012-05-15 12:44:30.724236500 at >> > > java.net.SocketOutputStream.socketWrite0(Native Method) >> > > 2012-05-15 12:44:30.724238500 at >> > > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109) >> > > 2012-05-15 12:44:30.724239500 at >> > > java.net.SocketOutputStream.write(SocketOutputStream.java:153) >> > > 2012-05-15 12:44:30.724253500 at >> > > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) >> > > 2012-05-15 12:44:30.724254500 at >> > > java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) >> > > 2012-05-15 12:44:30.724256500 at >> > > com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3345) >> > > 2012-05-15 12:44:30.724257500 at >> > > com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1983) >> > > 2012-05-15 12:44:30.724259500 at >> > > com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163) >> > > 2012-05-15 12:44:30.724267500 at >> > > com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2618) >> > > 2012-05-15 12:44:30.724268500 at >> > > >> > >> com.mysql.jdbc.StatementImpl.executeSimpleNonQuery(StatementImpl.java:1644) >> > > 2012-05-15 12:44:30.724270500 at >> > > com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:198) >> > > 2012-05-15 12:44:30.724271500 at >> > > com.mysql.jdbc.ResultSetImpl.realClose(ResultSetImpl.java:7617) >> > > 2012-05-15 12:44:30.724273500 at >> > > com.mysql.jdbc.ResultSetImpl.close(ResultSetImpl.java:907) >> > > 2012-05-15 12:44:30.724280500 at >> > > com.mysql.jdbc.StatementImpl.realClose(StatementImpl.java:2478) >> > > 2012-05-15 12:44:30.724282500 at >> > > >> > >> com.mysql.jdbc.ConnectionImpl.closeAllOpenStatements(ConnectionImpl.java:1584) >> > > 2012-05-15 12:44:30.724283500 at >> > > com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4364) >> > > 2012-05-15 12:44:30.724285500 at >> > > com.mysql.jdbc.ConnectionImpl.cleanup(ConnectionImpl.java:1360) >> > > 2012-05-15 12:44:30.724286500 at >> > > com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2652) >> > > 2012-05-15 12:44:30.724321500 at >> > > >> > >> com.mysql.jdbc.StatementImpl.executeSimpleNonQuery(StatementImpl.java:1644) >> > > 2012-05-15 12:44:30.724322500 at >> > > com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:198) >> > > 2012-05-15 12:44:30.724324500 at >> > > com.mysql.jdbc.ResultSetImpl.realClose(ResultSetImpl.java:7617) >> > > 2012-05-15 12:44:30.724325500 at >> > > com.mysql.jdbc.ResultSetImpl.close(ResultSetImpl.java:907) >> > > 2012-05-15 12:44:30.724327500 at >> > > com.mysql.jdbc.StatementImpl.realClose(StatementImpl.java:2478) >> > > 2012-05-15 12:44:30.724334500 at >> > > >> > >> com.mysql.jdbc.ConnectionImpl.closeAllOpenStatements(ConnectionImpl.java:1584) >> > > 2012-05-15 12:44:30.724335500 at >> > > com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4364) >> > > 2012-05-15 12:44:30.724336500 at >> > > com.mysql.jdbc.ConnectionImpl.cleanup(ConnectionImpl.java:1360) >> > > 2012-05-15 12:44:30.724338500 at >> > > com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2652) >> > > 2012-05-15 12:44:30.724339500 at >> > > >> > >> com.mysql.jdbc.StatementImpl.executeSimpleNonQuery(StatementImpl.java:1644) >> > > 2012-05-15 12:44:30.724345500 at >> > > com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:198) >> > > 2012-05-15 12:44:30.724347500 at >> > > com.mysql.jdbc.ResultSetImpl.realClose(ResultSetImpl.java:7617) >> > > 2012-05-15 12:44:30.724348500 at >> > > com.mysql.jdbc.ResultSetImpl.close(ResultSetImpl.java:907) >> > > 2012-05-15 12:44:30.724350500 at >> > > com.mysql.jdbc.StatementImpl.realClose(StatementImpl.java:2478) >> > > 2012-05-15 12:44:30.724351500 at >> > > >> > >> com.mysql.jdbc.ConnectionImpl.closeAllOpenStatements(ConnectionImpl.java:1584) >> > > 2012-05-15 12:44:30.724363500 at >> > > com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4364) >> > > 2012-05-15 12:44:30.724364500 at >> > > com.mysql.jdbc.ConnectionImpl.cleanup(ConnectionImpl.java:1360) >> > > 2012-05-15 12:44:30.724366500 at >> > > com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2652) >> > > 2012-05-15 12:44:30.724367500 at >> > > >> > >> com.mysql.jdbc.StatementImpl.executeSimpleNonQuery(StatementImpl.java:1644) >> > > 2012-05-15 12:44:30.724369500 at >> > > com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:198) >> > >> > >
