[
https://issues.apache.org/jira/browse/SOLR-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179174#comment-13179174
]
Hoss Man commented on SOLR-2804:
--------------------------------
I've only skimmed the patch, but it seems like overkill for the bug as i
understand it -- but maybe i'm just missing something...
*Has anyone actually seen this bug reproduce in any situation where they
_weren't_ using DIH multithreaded mode?*
bq. It blocks all users who index in multiple threads whether via SolrJ or DIH
That should not be the case at all for SOlrJ users and people using DIH in
single threaded mode -- every UpdateProcessorFactory (including the
LogUpdateprocessorFactory) creates new UpdateProcessor instances for every
update request -- so no concurrency protection should be required for multiple
threads processing concurrent requests individually. A simple NamedList to
collect the results to be logged _for each request_ should be completely
sufficient.
Based on the comments so far, it sounds like the bug is multiple threads in DIH
attempting to update the same NamedList concurrently -- which is most certainly
not safe for DIH to do (NamedList has never been, nor been documented to
suggest it might be, thread safe)
Changing LogUpdateprocessorFactory to add concurrency boundaries to something
that is already suppose to only be single thread access seems like it will only
slow down all the use cases that are already doing the right thing and working
fine -- let's fix the concurrency issue in the place where there is concurrency
code: figure out when/why/where DIH is letting multiple threads manipulate the
toLog NamedList and fix that.
> Logging error causes entire DIH process to fail
> -----------------------------------------------
>
> Key: SOLR-2804
> URL: https://issues.apache.org/jira/browse/SOLR-2804
> Project: Solr
> Issue Type: Bug
> Components: contrib - DataImportHandler
> Affects Versions: 4.0
> Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03-384-10M3425)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02-384, mixed mode)
> Model Name: MacBook Pro
> Model Identifier: MacBookPro8,2
> Processor Name: Intel Core i7
> Processor Speed: 2.2 GHz
> Number of Processors: 1
> Total Number of Cores: 4
> L2 Cache (per Core): 256 KB
> L3 Cache: 6 MB
> Memory: 4 GB
> System Software Overview:
> System Version: Mac OS X 10.6.8 (10K549)
> Kernel Version: Darwin 10.8.0
> Reporter: Pulkit Singhal
> Labels: dih
> Attachments: SOLR-2804-3x.patch, SOLR-2804.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> SEVERE: Full Import failed:java.lang.ClassCastException:
> java.util.ArrayList cannot be cast to java.lang.String
> at org.apache.solr.common.util.NamedList.getName(NamedList.java:127)
> at org.apache.solr.common.util.NamedList.toString(NamedList.java:263)
> at java.lang.String.valueOf(String.java:2826)
> at java.lang.StringBuilder.append(StringBuilder.java:115)
> at
> org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:188)
> at
> org.apache.solr.handler.dataimport.SolrWriter.close(SolrWriter.java:57)
> at
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:265)
> at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:372)
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:440)
> at
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:421)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
