Hi I am currently using the DIH to connect to and import data from a MS SQL 
Server, and in general doing full, delta or deletes seems to work perfectly.

The issue is that I spotted some errors being logged in the tomcat logs for 
SOLR which are :

19-Sep-2011 07:45:25 org.apache.solr.common.SolrException log
SEVERE: Exception in entity : 
product:org.apache.solr.handler.dataimport.DataImportHandlerException: 
com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 125) 
was deadlocked on lock resources with another process and has been chosen as 
the deadlock victim. Rerun the transaction.
        at 
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:64)
        at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:339)
        at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$600(JdbcDataSource.java:228)
        at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:262)
        at 
org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:77)
        at 
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:75)
        at 
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:238)
        at 
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:591)
        at 
org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:302)
        at 
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:178)
        at 
org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:390)
        at 
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:429)
        at 
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction 
(Process ID 125) was deadlocked on lock resources with another process and has 
been chosen as the deadlock victim. Rerun the transaction.
        at 
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:213)
        at 
com.microsoft.sqlserver.jdbc.SQLServerResultSet$FetchBuffer.nextRow(SQLServerResultSet.java:4713)
        at 
com.microsoft.sqlserver.jdbc.SQLServerResultSet.fetchBufferNext(SQLServerResultSet.java:1671)
        at 
com.microsoft.sqlserver.jdbc.SQLServerResultSet.next(SQLServerResultSet.java:944)
        at 
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:331)
        ... 11 more

19-Sep-2011 07:45:25 org.apache.solr.handler.dataimport.DocBuilder doDelta
INFO: Delta Import completed successfully
19-Sep-2011 07:45:25 org.apache.solr.handler.dataimport.DocBuilder finish


Now this SQL error I can deal with and I will probably switch to snapshot 
Isolation as these are constantly updated tables, but my issue is not the sql 
error but the fact that the delta import still reported that it had imported 
successfully and still wrote out the last updated time to the 
dataimport.properties file, so the next time it ran it missed a bunch of 
documents that should have been indexed.

If it had failed and just rolled back the changes and not updated the 
dataimport.properties file it would (assuming no more deadlocks) caught all of 
the missed documents on the next delta import.

My connection to MS SQL is using the "responseBuffering=adaptive" setting to 
reduce memory overhead, So I guess what I am asking is there any way I can 
cause the DIH to roll back the import if an error occurs and to not update the 
"dataimport.properties" file.

Any help or suggestions would be appreciated

Thanks

Barry H
 
DISCLAIMER: This email and its attachments may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Misco UK Ltd. Any unauthorised use or dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please immediately notify the sender by return e-mail message and 
delete all copies of the original communication. Thank you for your 
cooperation. Misco UK Ltd, registered in Scotland Number 114143. Registered 
Office: Caledonian Exchange, 19a Canning Street, Edinburgh EH3 8EG. Telephone 
+44 (0)1933 686000. This e-mail message has been scanned by CA Gateway Security.

Reply via email to