[
https://issues.apache.org/jira/browse/GORA-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178651#comment-14178651
]
Lewis John McGibbney commented on GORA-388:
-------------------------------------------
Hi [~drazzib],
bq. we have to implement this transformation in each driver (which seems error
prone)
this sounds like an excellent opportunity to actually check this one out.
Baring in mind that not all libraries are currently using filters in the same
way right now! gora-cassandra for example is behind w.r.t filtering operations.
I think we could substantiate some tests on a datastore-wide basis e.g. from
within DataStoreTestBase --> DataStoreTestUtil
wdyt? I would be willing to have a crack at this.
> MongoStore: Fix handling of Utf8 in filters
> -------------------------------------------
>
> Key: GORA-388
> URL: https://issues.apache.org/jira/browse/GORA-388
> Project: Apache Gora
> Issue Type: Bug
> Components: gora-mongodb
> Affects Versions: 0.5
> Reporter: Damien Raude-Morvan
>
> Gora 0.5 MongoDB driver fail to properly serialize Utf8 instance passed to
> its local filters implementation.
> Here is sample code
> {noformat}
> filter.getOperands().add(new Utf8(batchId));
> {noformat}
> that will trigger the following exception at runtime
> {noformat}
> java.lang.IllegalArgumentException: can't serialize class
> org.apache.avro.util.Utf8
> at org.bson.BasicBSONEncoder._putObjectField(BasicBSONEncoder.java:284)
> at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:185)
> at org.bson.BasicBSONEncoder.putObject(BasicBSONEncoder.java:131)
> at com.mongodb.DefaultDBEncoder.writeObject(DefaultDBEncoder.java:33)
> at com.mongodb.OutMessage.putObject(OutMessage.java:289)
> at com.mongodb.OutMessage.writeQuery(OutMessage.java:211)
> at com.mongodb.OutMessage.query(OutMessage.java:86)
> at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:81)
> at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:66)
> at com.mongodb.DBCursor._check(DBCursor.java:458)
> at com.mongodb.DBCursor._hasNext(DBCursor.java:546)
> at com.mongodb.DBCursor.hasNext(DBCursor.java:571)
> at
> org.apache.gora.mongodb.query.MongoDBResult.nextInner(MongoDBResult.java:69)
> at org.apache.gora.query.impl.ResultBase.next(ResultBase.java:114)
> at
> org.apache.gora.mapreduce.GoraRecordReader.nextKeyValue(GoraRecordReader.java:119)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:531)
> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {noformat}
> This pattern is extensively used by Apache Nutch :
> *
> [FetcherJob|https://github.com/apache/nutch/blob/2.x/src/java/org/apache/nutch/fetcher/FetcherJob.java#L218]
> *
> [ParserJob|https://github.com/apache/nutch/blob/2.x/src/java/org/apache/nutch/parse/ParserJob.java#L274]
> *
> [DbUpdaterJob|https://github.com/apache/nutch/blob/2.x/src/java/org/apache/nutch/crawl/DbUpdaterJob.java#L128]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
