Mike Thomsen created NIFI-7159:
----------------------------------
Summary: Mongo processors appear to not support Decimal 128 data
types
Key: NIFI-7159
URL: https://issues.apache.org/jira/browse/NIFI-7159
Project: Apache NiFi
Issue Type: Bug
Reporter: Mike Thomsen
Assignee: Mike Thomsen
This is the verbatim response from the user posting on nifi-dev:
*_Here’s a really stripped back example._*
*_I have an input as below_*
*_{_*
*_"value": "123456.00"_*
*_}_*
*_And I want to write the value into MongoDB as a numeric value, but
critically not as a double (these will be currency values)_*
*_In order to try to enforce the type used to write to Mongo we created a
simple Avro schema, for the example above this would be as follows;_*
*_{_*
*_"type": "record",_*
*_"name": "TransactionEvent",_*
*_"namespace": "com.example.demo",_*
*_"fields": [_*
*_{_*
*_"name": "value",_*
*_"type": [_*
*_"null",_*
*_{_*
*_"type": "bytes",_*
*_"logicalType": "decimal",_*
*_"precision": 10,_*
*_"scale": 2_*
*_}_*
*_]_*
*_}_*
*_]_*
*_}_*
*_Hoping that this would map to a Decimal128 in Mongo, however we consistently
see double as the type in Mongo regardless of any variations of Avro schema we
have tried._*
*_On having a quick look into the code I’ve identified 2 possible problem
areas._*
*_1. The conversion of the Avro schema into the internal representation
which seems to treat Avro logical decimal types as double (ref
org.apache.nifi.avro.AvroTypeUtil – line 343)_*
*_2. The Mongo processor which uses this type information to decide what
Mongo types to persist data as._*
*_For a quick win, which would hopefully have a smaller impact, I was hoping
that I could fork the Mongo processor and keep the changes local to that but
since the information about the Avro logical type is lost before the schema
information gets to MongoDB i’m not sure that will be possible now._*
*_When we reached this point and the changes we were looking at seemed like
they could be a little more complex than hoped we wanted to reach out to see
if_*
*_1. We’re doing something wrong_*
*_2. Anybody else had encountered a similar situation_*
*_3. If we did look to introduce changes either to the Mongo processor or
more widely for support of BigDecimal would this be of wider use?_*
It would appear to be a distinctly different type than Double:
https://mongodb.github.io/mongo-java-driver/3.5/javadoc/?org/bson/types/Decimal128.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
