That could be a legitimate issue. I'll write up a Jira ticket.
On Sun, Feb 16, 2020 at 5:42 AM Christopher Manniex <cmann...@hotmail.com>
wrote:
> Thanks Mike,
>
> Here’s a really stripped back example.
>
> I have an input as below
>
> {
> "value": "123456.00"
> }
>
> And I want to write the value into MongoDB as a numeric value, but
> critically not as a double (these will be currency values)
>
> In order to try to enforce the type used to write to Mongo we created a
> simple Avro schema, for the example above this would be as follows;
>
> {
> "type": "record",
> "name": "TransactionEvent",
> "namespace": "com.example.demo",
> "fields": [
> {
> "name": "value",
> "type": [
> "null",
> {
> "type": "bytes",
> "logicalType": "decimal",
> "precision": 10,
> "scale": 2
> }
> ]
> }
> ]
> }
>
> Hoping that this would map to a Decimal128 in Mongo, however we
> consistently see double as the type in Mongo regardless of any variations
> of Avro schema we have tried.
>
> On having a quick look into the code I’ve identified 2 possible problem
> areas.
>
>
> 1. The conversion of the Avro schema into the internal representation
> which seems to treat Avro logical decimal types as double (ref
> org.apache.nifi.avro.AvroTypeUtil – line 343)
> 2. The Mongo processor which uses this type information to decide what
> Mongo types to persist data as.
>
> For a quick win, which would hopefully have a smaller impact, I was hoping
> that I could fork the Mongo processor and keep the changes local to that
> but since the information about the Avro logical type is lost before the
> schema information gets to MongoDB i’m not sure that will be possible now.
>
> When we reached this point and the changes we were looking at seemed like
> they could be a little more complex than hoped we wanted to reach out to
> see if
>
>
> 1. We’re doing something wrong
> 2. Anybody else had encountered a similar situation
> 3. If we did look to introduce changes either to the Mongo processor or
> more widely for support of BigDecimal would this be of wider use?
>
> Many thanks
>
> Chris
>
>
> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
> ________________________________
> From: Mike Thomsen <mikerthom...@gmail.com>
> Sent: Saturday, February 15, 2020 7:48:29 PM
> To: dev@nifi.apache.org <dev@nifi.apache.org>
> Subject: Re: BigDecimal support
>
> Chris,
>
> Drop us some details about the delta between what you're expecting and what
> you see happening so we can investigate and see what we can do to help.
>
> Thanks,
>
> Mike
>
> On Sat, Feb 15, 2020 at 10:09 AM Christopher Manniex <cmann...@hotmail.com
> >
> wrote:
>
> > Hi All,
> >
> > Hi All, not sure this is the right forum, I've been evaluating Nifi for
> > use in one of our solutions but am hitting issues preserving type
> > information for BigDecimal through to Mongo... am I missing something
> > obvious or is this a possibility for an improvement (happy to contribute
> if
> > i can be pointed in the right direction)
> >
> > Regards
> >
> > Chris
> >
> > Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for
> > Windows 10
> >
> >
>
