GorkemG created NIFI-9256:
-----------------------------
Summary: QueryDatabaseTable Processor Default Decimal Scale not
working
Key: NIFI-9256
URL: https://issues.apache.org/jira/browse/NIFI-9256
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.14.0
Environment: Linux HOSTNAME 4.18.0-305.19.1.el8_4.x86_64 #1 SMP Tue
Sep 7 07:07:31 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
openjdk version "1.8.0_302"
OpenJDK Runtime Environment (build 1.8.0_302-b08)
OpenJDK 64-Bit Server VM (build 25.302-b08, mixed mode)
Reporter: GorkemG
When using QueryDatabaseTable processor , and to infer schemea from table set
Use Avro Logical Types = True, whether you change Default Decimal Precision
value to other than default 10 or not, and set Default Decimal Scale to desired
number (other than default 0) , in the output avro schema scale is "always" set
to whatever Default Decimal Precision is.
E.g. if Default Decimal Precision = 38 -> Default Decimal Scale is always 38
(irrelevant to value)
On the other hand ExecuteSQL Processor is working as intended, changing scale
and precision affects avro schema (using same OracleSQL connection).
Note that QueryDatabaseTable is handy for incremental fetching, and if Avro
Logical Types is false (default) columns are always string type and when you
convert to for example Parquet, you miss the schema types and there is no handy
way to pass out that information.
Usually when reading Parquet files Precision=38 and Scale=18 is required (due
to other software conversion missing features).
Using QueryDatabaseTable
"fields":[{"name":"ID","type":["null",{"type":"bytes","logicalType":"decimal","precision":38,*"scale":38*}]}
Using ExecuteSQL
{"name":"ID","type":["null",\{"type":"bytes","logicalType":"decimal","precision":38,"scale":18}]}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
