Hi I am currently using the DIH to connect to and import data from a MS SQL
Server, and in general doing full, delta or deletes seems to work perfectly.
The issue is that I spotted some errors being logged in the tomcat logs for
SOLR which are :
19-Sep-2011 07:45:25 org.apache.solr.common.SolrException log
SEVERE: Exception in entity :
product:org.apache.solr.handler.dataimport.DataImportHandlerException:
com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 125)
was deadlocked on lock resources with another process and has been chosen as
the deadlock victim. Rerun the transaction.
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:64)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:339)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$600(JdbcDataSource.java:228)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:262)
at
org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:77)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:75)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:238)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:591)
at
org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:302)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:178)
at
org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:390)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:429)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction
(Process ID 125) was deadlocked on lock resources with another process and has
been chosen as the deadlock victim. Rerun the transaction.
at
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:213)
at
com.microsoft.sqlserver.jdbc.SQLServerResultSet$FetchBuffer.nextRow(SQLServerResultSet.java:4713)
at
com.microsoft.sqlserver.jdbc.SQLServerResultSet.fetchBufferNext(SQLServerResultSet.java:1671)
at
com.microsoft.sqlserver.jdbc.SQLServerResultSet.next(SQLServerResultSet.java:944)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:331)
... 11 more
19-Sep-2011 07:45:25 org.apache.solr.handler.dataimport.DocBuilder doDelta
INFO: Delta Import completed successfully
19-Sep-2011 07:45:25 org.apache.solr.handler.dataimport.DocBuilder finish
Now this SQL error I can deal with and I will probably switch to snapshot
Isolation as these are constantly updated tables, but my issue is not the sql
error but the fact that the delta import still reported that it had imported
successfully and still wrote out the last updated time to the
dataimport.properties file, so the next time it ran it missed a bunch of
documents that should have been indexed.
If it had failed and just rolled back the changes and not updated the
dataimport.properties file it would (assuming no more deadlocks) caught all of
the missed documents on the next delta import.
My connection to MS SQL is using the "responseBuffering=adaptive" setting to
reduce memory overhead, So I guess what I am asking is there any way I can
cause the DIH to roll back the import if an error occurs and to not update the
"dataimport.properties" file.
Any help or suggestions would be appreciated
Thanks
Barry H
DISCLAIMER: This email and its attachments may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of Misco UK Ltd. Any unauthorised use or dissemination of this
communication is strictly prohibited. If you have received this communication
in error, please immediately notify the sender by return e-mail message and
delete all copies of the original communication. Thank you for your
cooperation. Misco UK Ltd, registered in Scotland Number 114143. Registered
Office: Caledonian Exchange, 19a Canning Street, Edinburgh EH3 8EG. Telephone
+44 (0)1933 686000. This e-mail message has been scanned by CA Gateway Security.
