Thanks Mike,
Here’s a really stripped back example.
I have an input as below
{
"value": "123456.00"
}
And I want to write the value into MongoDB as a numeric value, but critically
not as a double (these will be currency values)
In order to try to enforce the type used to write to Mongo we created a simple
Avro schema, for the example above this would be as follows;
{
"type": "record",
"name": "TransactionEvent",
"namespace": "com.example.demo",
"fields": [
{
"name": "value",
"type": [
"null",
{
"type": "bytes",
"logicalType": "decimal",
"precision": 10,
"scale": 2
}
]
}
]
}
Hoping that this would map to a Decimal128 in Mongo, however we consistently
see double as the type in Mongo regardless of any variations of Avro schema we
have tried.
On having a quick look into the code I’ve identified 2 possible problem areas.
1. The conversion of the Avro schema into the internal representation which
seems to treat Avro logical decimal types as double (ref
org.apache.nifi.avro.AvroTypeUtil – line 343)
2. The Mongo processor which uses this type information to decide what Mongo
types to persist data as.
For a quick win, which would hopefully have a smaller impact, I was hoping that
I could fork the Mongo processor and keep the changes local to that but since
the information about the Avro logical type is lost before the schema
information gets to MongoDB i’m not sure that will be possible now.
When we reached this point and the changes we were looking at seemed like they
could be a little more complex than hoped we wanted to reach out to see if
1. We’re doing something wrong
2. Anybody else had encountered a similar situation
3. If we did look to introduce changes either to the Mongo processor or more
widely for support of BigDecimal would this be of wider use?
Many thanks
Chris
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
________________________________
From: Mike Thomsen <mikerthom...@gmail.com>
Sent: Saturday, February 15, 2020 7:48:29 PM
To: dev@nifi.apache.org <dev@nifi.apache.org>
Subject: Re: BigDecimal support
Chris,
Drop us some details about the delta between what you're expecting and what
you see happening so we can investigate and see what we can do to help.
Thanks,
Mike
On Sat, Feb 15, 2020 at 10:09 AM Christopher Manniex <cmann...@hotmail.com>
wrote:
> Hi All,
>
> Hi All, not sure this is the right forum, I've been evaluating Nifi for
> use in one of our solutions but am hitting issues preserving type
> information for BigDecimal through to Mongo... am I missing something
> obvious or is this a possibility for an improvement (happy to contribute if
> i can be pointed in the right direction)
>
> Regards
>
> Chris
>
> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
