[
https://issues.apache.org/jira/browse/SOLR-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gunnlaugur Thor Briem updated SOLR-1191:
----------------------------------------
Attachment: SOLR-1191.patch
Patch to resolve this. It resolves deltaQuery columns against pk when they
differ by prefix (and report error more helpfully when no column matches, or
more than one column matches).
No unit test, sorry (but there's not much deltaQuery coverage anyway). All
existing unit tests pass, and this is working fine for me in production.
> NullPointerException in delta import
> ------------------------------------
>
> Key: SOLR-1191
> URL: https://issues.apache.org/jira/browse/SOLR-1191
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 1.3, 1.4
> Environment: OS: Windows & Linux.
> Java: 1.6
> DB: MySQL & SQL Server
> Reporter: Ali Syed
> Assignee: Noble Paul
> Fix For: 1.4
>
> Attachments: SOLR-1191.patch
>
>
> Seeing few of these NullPointerException during delta imports. Once this
> happens delta import stops working and keeps giving the same error.
> java.lang.NullPointerException
> at
> org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:622)
> at
> org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:240)
> at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:159)
> at
> org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:337)
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:376)
> at
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:355)
> Running delta import for a particular entity fixes the problem and delta
> import start working again.
> Here is the log just before & after the exception
> 05/27 11:59:29 86987686 INFO btpool0-538 org.apache.solr.core.SolrCore -
> [localhost] webapp=/solr path=/dataimport
> params={command=delta-import&optimize=false} status=0 QTime=0
> 05/27 11:59:29 86987687 INFO Thread-4162
> org.apache.solr.handler.dataimport.SolrWriter - Read dataimport.properties
> 05/27 11:59:29 86987687 INFO Thread-4162
> org.apache.solr.handler.dataimport.DataImporter - Starting Delta Import
> 05/27 11:59:29 86987687 INFO Thread-4162
> org.apache.solr.handler.dataimport.SolrWriter - Read dataimport.properties
> 05/27 11:59:29 86987687 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Starting delta collection.
> 05/27 11:59:29 86987690 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Running ModifiedRowKey() for
> Entity: content
> 05/27 11:59:29 86987690 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed ModifiedRowKey for
> Entity: content rows obtained : 0
> 05/27 11:59:29 86987690 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed DeletedRowKey for
> Entity: content rows obtained : 0
> 05/27 11:59:29 86987692 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed parentDeltaQuery
> for Entity: content
> 05/27 11:59:29 86987692 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Running ModifiedRowKey() for
> Entity: job
> 05/27 11:59:29 86987692 INFO Thread-4162
> org.apache.solr.handler.dataimport.JdbcDataSource - Creating a connection
> for entity job with URL: jdbc:sqlserver://localhost;databaseName=TestDB
> 05/27 11:59:29 86987704 INFO Thread-4162
> org.apache.solr.handler.dataimport.JdbcDataSource - Time taken for
> getConnection(): 12
> 05/27 11:59:29 86987707 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed ModifiedRowKey for
> Entity: job rows obtained : 0
> 05/27 11:59:29 86987707 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed DeletedRowKey for
> Entity: job rows obtained : 0
> 05/27 11:59:29 86987707 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed parentDeltaQuery
> for Entity: job
> 05/27 11:59:29 86987707 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Delta Import completed
> successfully
> 05/27 11:59:29 86987707 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Starting delta collection.
> 05/27 11:59:29 86987709 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Running ModifiedRowKey() for
> Entity: user
> 05/27 11:59:29 86987709 INFO Thread-4162
> org.apache.solr.handler.dataimport.JdbcDataSource - Creating a connection
> for entity user with URL: jdbc:sqlserver://localhost;databaseName=TestDB
> 05/27 11:59:29 86987716 INFO Thread-4162
> org.apache.solr.handler.dataimport.JdbcDataSource - Time taken for
> getConnection(): 7
> 05/27 11:59:29 86987873 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed ModifiedRowKey for
> Entity: user rows obtained : 46
> 05/27 11:59:29 86987873 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed DeletedRowKey for
> Entity: user rows obtained : 0
> 05/27 11:59:29 86987873 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed parentDeltaQuery
> for Entity: user
> 05/27 11:59:29 86987882 INFO Thread-4162 org.apache.solr.core.SolrCore -
> SolrDeletionPolicy.onInit: commits:num=1
>
> commit{dir=/usr/share/solr/work/localhost/data/index,segFN=segments_m0,version=1242658668035,generation=792,filenames=[_wv.fdt,
> _vf.prx, _z2.frq, _z5.frq, _z2.tis, _wv.fdx, _z6.nrm, _z3.tis, segments_
> m0, _uh_1.del, _uj.fnm, _yd.fnm, _yz.fdt, _wa.fnm, _ui.prx, _yz.tis, _z0.nrm,
> _z7.fdt, _vf_6.del, _cy.fnm, _z5.nrm, _yz.fdx, _cy.tii, _z6.fdx, _uh.prx,
> _z6.fdt, _z7.fdx, _ui.nrm, _uh.fnm, _yz.tii, _uu.fdt, _x
> g.fnm, _z3.fdt, _uu.fdx, _z3.fdx, _z3_1.del, _wa.tis, _uj.tis, _ui.frq,
> _wa.tii, _z0.frq, _wv.nrm, _cy.tis, _xr.prx, _z3.tii, _z4.prx, _z7.prx,
> _xr.fdt, _z4.fdt, _xr.fdx, _z4.fdx, _uh.nrm, _y2.frq, _y2_1.del,
> _uj.tii, _z2.tii, _z2.nrm, _yo.prx, _z5.fnm, _y2.prx, _uu.nrm, _z0_1.del,
> _cy.prx, _yd.frq, _yo.fnm, _cy.nrm, _z0.fdx, _xg.tis, _z0.fdt, _z2_1.del,
> _xg.tii, _cy.frq, _xg.frq, _yd.prx, _z7.tis, _z0.tis, _z7.t
> ii, _yd.nrm, _vf.fnm, _z6.frq, _z2.fdx, _ui.fnm, _z5.tis, _z6_1.del, _z5.tii,
> _z1.fdt, _z0.tii, _z5_1.del, _uu.tis, _uu.tii, _z1.prx, _z1.fdx, _z6.tii,
> _y2.fnm, _xr.frq, _z4.frq, _z2.fdt, _z6.tis, _vf.frq, _y
> o.frq, _uj.fdx, _wv.fnm, _cy.fdt, _z6.prx, _xr.tis, _z4.tis, _yz.fnm,
> _uu.fnm, _z2.prx, _yo.tii, _z0.prx, _yd_1.del, _z1.frq, _yo.tis, _z7.nrm,
> _z5.prx, _yd.fdx, _yz.nrm, _cy.fdx, _yd.fdt, _wa_6.del, _z3.prx,
> _uj_h.del, _ui_e.del, _z7.fnm, _z3.fnm, _uu.prx, _uh.tii, _z6.fnm, _xg.prx,
> _xg.fdx, _xg.nrm, _wv.prx, _xg.fdt, _uu_b.del, _xr.nrm, _z4.nrm, _uj.prx,
> _xr.fnm, _z4.fnm, _uj.nrm, _yz_2.del, _uh.fdx, _vf.fdt, _
> uh.fdt, _yo.nrm, _xr.tii, _z4.tii, _vf.fdx, _uh.tis, _uj.fdt, _uj.frq,
> _yo.fdx, _wa.fdt, _ui.fdt, _z1.tii, _xg_1.del, _wa.fdx, _ui.fdx, _yz.frq,
> _ui.tii, _z5.fdt, _z5.fdx, _wv.frq, _wa.nrm, _yd.tii, _yd.tis,
> _yo_2.del, _y2.tii, _uu.frq, _cy_jd.del, _uh.frq, _y2.tis, _z1_1.del,
> _wa.prx, _z1.fnm, _ui.tis, _z2.fnm, _wa.frq, _wv.tii, _z0.fnm, _y2.fdx,
> _y2.fdt, _z7.frq, _wv.tis, _z3.frq, _y2.nrm, _yz.prx, _xr_2.del, _
> z1.nrm, _vf.tis, _vf.tii, _vf.nrm, _z3.nrm, _z1.tis, _yo.fdt]
> 05/27 11:59:29 86987882 INFO Thread-4162 org.apache.solr.core.SolrCore -
> last commit = 1242658668035
> 05/27 11:59:29 86987944 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Delta Import completed
> successfully
> 05/27 11:59:29 86987944 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Starting delta collection.
> 05/27 11:59:29 86987946 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Running ModifiedRowKey() for
> Entity: candidateContent
> 05/27 11:59:29 86987946 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed ModifiedRowKey for
> Entity: candidateContent rows obtained : 0
> 05/27 11:59:29 86987946 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed DeletedRowKey for
> Entity: candidateContent rows obtained : 0
> 05/27 11:59:29 86987948 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed parentDeltaQuery
> for Entity: candidateContent
> 05/27 11:59:29 86987950 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Running ModifiedRowKey() for
> Entity: candidateMessage
> 05/27 11:59:29 86987950 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed ModifiedRowKey for
> Entity: candidateMessage rows obtained : 0
> 05/27 11:59:29 86987950 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed DeletedRowKey for
> Entity: candidateMessage rows obtained : 0
> 05/27 11:59:29 86987950 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed parentDeltaQuery
> for Entity: candidateMessage
> 05/27 11:59:29 86987952 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Running ModifiedRowKey() for
> Entity: candidateCustom
> 05/27 11:59:29 86987953 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed ModifiedRowKey for
> Entity: candidateCustom rows obtained : 0
> 05/27 11:59:29 86987953 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed DeletedRowKey for
> Entity: candidateCustom rows obtained : 0
> 05/27 11:59:29 86987953 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Completed parentDeltaQuery
> for Entity: candidateCustom
> 05/27 11:59:29 86987953 INFO Thread-4162
> org.apache.solr.handler.dataimport.DocBuilder - Running ModifiedRowKey() for
> Entity: candidate
> 05/27 11:59:29 86987953 INFO Thread-4162
> org.apache.solr.handler.dataimport.JdbcDataSource - Creating a connection
> for entity candidate with URL: jdbc:sqlserver://localhost;databaseName=TestDB
> 05/27 11:59:29 86987960 INFO Thread-4162
> org.apache.solr.handler.dataimport.JdbcDataSource - Time taken for
> getConnection(): 7
> 05/27 11:59:29 86988066 ERROR Thread-4162
> org.apache.solr.handler.dataimport.DataImporter - Delta Import Failed
> java.lang.NullPointerException
> at
> org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:622)
> at
> org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:240)
> at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:159)
> at
> org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:337)
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:376)
> at
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:355)
> 05/27 11:59:29 86988066 INFO Thread-4162
> org.apache.solr.update.UpdateHandler - start rollback
> 05/27 11:59:29 86988071 INFO Thread-4162
> org.apache.solr.update.UpdateHandler - end_rollback
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
