Re: Solr Cloud error with shard update
On 3/13/2014 12:54 PM, cpk wrote: We're seeing the same behavior with Solr 4.6.0 and 4.7. DataInputHandler loads documents, but the updates to the replica fail because of the limited support for the BigDecimal type in SolrCloud. We've successfully worked around the issue by setting convertType=true in the DIH config. This tells DIH to convert the BigDecimal to the supported Solr type (float, double, etc) defined in your schema.xml for the field before submitting to Solr. In my opinion, this is more of a issue with DIH, than SolrCloud. DIH shouldn't try to submit the BigDecimal type, if its not well supported by Solr. SolrCloud should try to support BigDecimal, but that suggestion has been pending for a while. The real problem here is not DIH, but the JDBC driver. The convertType parameter that you have set is for your JDBC driver. For most people, these details don't matter, because they are writing Java code themselves and can adjust according to the peculiarities of a specific JDBC driver. DIH is a *generic* solution that can only deal with standard types. BigDecimal is not a standard java data type. Although it is included in the standard JVM, it is part of the *math* package, it is not built into Java. http://docs.oracle.com/javase/7/docs/api/java/math/BigDecimal.html You might wonder why DIH doesn't convert the data. The answer is that without the programmer explicitly providing code to detect each nonstandard type, it won't know *HOW* to convert it. Solr and Lucene can't be expected to support every data type, especially if the data type is not even available until you import a class. Thanks, Shawn
Re: Solr Cloud error with shard update
In case anyone else runs across this issue, I think we've found a work-around. We're seeing the same behavior with Solr 4.6.0 and 4.7. DataInputHandler loads documents, but the updates to the replica fail because of the limited support for the BigDecimal type in SolrCloud. We've successfully worked around the issue by setting convertType=true in the DIH config. This tells DIH to convert the BigDecimal to the supported Solr type (float, double, etc) defined in your schema.xml for the field before submitting to Solr. In my opinion, this is more of a issue with DIH, than SolrCloud. DIH shouldn't try to submit the BigDecimal type, if its not well supported by Solr. SolrCloud should try to support BigDecimal, but that suggestion has been pending for a while. Hope this helps. Chris -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Cloud-error-with-shard-update-tp4106260p4123486.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Cloud error with shard update
I have created Jira issue here: https://issues.apache.org/jira/browse/SOLR-5551 -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Cloud-error-with-shard-update-tp4106260p4106448.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Cloud error with shard update
On 12/11/2013 5:01 PM, dboychuck wrote: > I am running a data import and it is throwing all kinds of errors. I am > upgrading to 4.6 from 4.5.1 with the exact schema and solrconfig and dih > configs. > > Here is the error I am getting: > > org.apache.solr.common.SolrException: ERROR: [doc=k-690kohler!670614] Error > adding field 'weight'='java.math.BigDecimal:9.28' msg=For input string: > "java.math.BigDecimal:9.28" > at > org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167) > at > org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77) > at > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:215) > at > org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) > at > org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) > at > org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:569) > at > org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:705) > at > org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435) > at org.apache.solr.update.PeerSync.handleUpdates(PeerSync.java:482) > at org.apache.solr.update.PeerSync.handleResponse(PeerSync.java:330) > at org.apache.solr.update.PeerSync.sync(PeerSync.java:260) > at > org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:387) > at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:244) > Caused by: java.lang.NumberFormatException: For input string: > "java.math.BigDecimal:9.28" > at > sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1241) > at java.lang.Float.parseFloat(Float.java:452) > at org.apache.solr.schema.TrieField.createField(TrieField.java:593) > at org.apache.solr.schema.TrieField.createFields(TrieField.java:650) > at > org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47) > at > org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118) > ... 12 more I have been discussing this with David on IRC. Everything works perfectly for him on 4.5.1 with identical configs. On 4.6.0, the update succeeds on the server where DIH is running, but fails when the update is forwarded to SolrCloud replicas. The source database is MSSQL, with a decimal DB column and a TrieFloatField in Solr. I am not sure yet what the problem is. I know that there's been a lot of code tightening going on in order to fix SolrCloud indexing deadlocks. Thanks, Shawn
Re: Solr Cloud error with shard update
Also I have tried setting my the schema defintion to float for the offending fields in the schema.xml as well as tried to cast my columns to strings in the query. Both still give the same result. My java version is: java version "1.7.0_10" Java(TM) SE Runtime Environment (build 1.7.0_10-b18) Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode) -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Cloud-error-with-shard-update-tp4106260p4106271.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Cloud error with shard update
r.processAdd(DistributedUpdateProcessor.java:435) at org.apache.solr.update.PeerSync.handleUpdates(PeerSync.java:482) at org.apache.solr.update.PeerSync.handleResponse(PeerSync.java:330) at org.apache.solr.update.PeerSync.sync(PeerSync.java:260) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:387) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:244) Caused by: java.lang.NumberFormatException: For input string: "java.math.BigDecimal:9.28" at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1241) at java.lang.Float.parseFloat(Float.java:452) at org.apache.solr.schema.TrieField.createField(TrieField.java:593) at org.apache.solr.schema.TrieField.createFields(TrieField.java:650) at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47) at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118) ... 13 more -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Cloud-error-with-shard-update-tp4106260p4106270.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Cloud error with shard update
current.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Cloud-error-with-shard-update-tp4106260p4106269.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Cloud error with shard update
I am running a data import and it is throwing all kinds of errors. I am upgrading to 4.6 from 4.5.1 with the exact schema and solrconfig and dih configs. Here is the error I am getting: org.apache.solr.common.SolrException: ERROR: [doc=k-690kohler!670614] Error adding field 'weight'='java.math.BigDecimal:9.28' msg=For input string: "java.math.BigDecimal:9.28" at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:167) at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:215) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:569) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:705) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435) at org.apache.solr.update.PeerSync.handleUpdates(PeerSync.java:482) at org.apache.solr.update.PeerSync.handleResponse(PeerSync.java:330) at org.apache.solr.update.PeerSync.sync(PeerSync.java:260) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:387) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:244) Caused by: java.lang.NumberFormatException: For input string: "java.math.BigDecimal:9.28" at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1241) at java.lang.Float.parseFloat(Float.java:452) at org.apache.solr.schema.TrieField.createField(TrieField.java:593) at org.apache.solr.schema.TrieField.createFields(TrieField.java:650) at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47) at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118) ... 12 more It looks like there is a problem with the readJavaFormat() function on a tried field? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Cloud-error-with-shard-update-tp4106260.html Sent from the Solr - User mailing list archive at Nabble.com.
