[
https://issues.apache.org/jira/browse/NIFI-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641165#comment-16641165
]
ASF GitHub Bot commented on NIFI-5664:
--
Github user snagacarl commented on the issue:
https://github.com/apache/nifi/pull/3049
Great suggestions. I can incorporate and re-test this week. Thanks for
your feedback!
> LookupRecord cannot handle JSON array properties from MongoDBLookupService
> --
>
> Key: NIFI-5664
> URL: https://issues.apache.org/jira/browse/NIFI-5664
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0, 1.7.1
>Reporter: Carl Gieringer
>Priority: Major
>
> We are using a LookupRecord processor to add an array property to a Record.
> The LookupRecord's `Lookup Service` is a MongoDBLookupService. The idea is
> for the LookupRecord to query Mongo documents by the Record's `_id`, to
> extract the array property at `f` on that document, and to insert that array
> into the Record at the RecordPath `f`. This flow results in the conversion
> error copied below, which completely prevents the FlowFile from proceeding
> (routes to Failure/Unmatched.) It appears that the `MongoDBLookupService` (or
> perhaps its `MongoDBControllerService`) stores Mongo document arrays as
> `ArrayList`s, which `DataTypeUtils#toArray` does not support. A simple fix
> appears to be to add a few lines to `DataTypeUtils#toArray` to support
> `ArrayList`s. I will provide a PR for this issue.
> This issue appears in 1.7.1, and is extant in 1.8.0-SNAPSHOT (2018-10-02
> b4c8e0179). I am not aware of any workaround to enrich a Record with an array
> from Mongo.
> LookupRecord Configuration
> ||Property||Value||
> |Result RecordPath|/f|
> |Record Result Contents|Insert Entire Record|
> |_id|/_id|
> MongoDBLookupService Configuration
> ||Property||Value||
> |Lookup Value Field|f|
> Conversion error (with irrelevant Mongo document and schema properties
> elided):
> {code:java}
> 2018-10-04 15:00:31,348 ERROR [Timer-Driven Process Thread-1]
> o.a.n.processors.standard.LookupRecord
> LookupRecord[id=339bf72c-91c3-36f0-ac14-bc8b734e2582] Failed to write
> MapRecord[{... f=[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}],
> i=[Ljava.lang.Object;@35dee8fc, ...] with schema {
> "namespace": "nifi",
> "name": "MongoPosting",
> "type": "record",
> "fields": [
> ...
> { "name": "f", "type": [
> "null",
> { "type": "array", "items": {
> "type": "record", "name": "PostingFeatureDetails", "fields": [
> { "name": "fid", "type": "string" },
> { "name": "l" , "type": ["null", "double"] }
> ]}
> }
> ]},
> ...
> ]
> } as a JSON Object due to
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class
> java.util.ArrayList to Object Array for field f:
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class
> java.util.ArrayList to Object Array for field f
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class
> java.util.ArrayList to Object Array for field f
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.toArray(DataTypeUtils.java:347)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:153)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:115)
> at
> org.apache.nifi.json.WriteJsonResult.writeValue(WriteJsonResult.java:284)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:187)
> at
> org.apa
