[ https://issues.apache.org/jira/browse/NIFI-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15460710#comment-15460710 ]
Toivo Adams edited comment on NIFI-2624 at 9/3/16 8:49 AM: ----------------------------------------------------------- Note!! BigInteger handling is not yet changed. I am not sure how Avro supports DECIMAL types. Current solution looks clumsy. You can’t put BigDecimal value directly to Avro record, like rec.put(i - 1, bigDecimalValue); Avro will throw org.apache.avro.AvroRuntimeException: Unknown datum type java.math.BigDecimal: 38 Instead you need to convert value to bytes, like Schema decimalSchema = getDecimalSchema(schema, meta.getColumnName(i)); LogicalType logicalType = LogicalTypes.fromSchema(decimalSchema); ByteBuffer byteBuffer = decimalConversion.toBytes((BigDecimal) value, decimalSchema, logicalType); rec.put(i - 1, byteBuffer); And getting BigDecimal value from record is equally weird: record = dataFileReader.next(record); DecimalConversion decimalConversion = new DecimalConversion(); Schema schema = record.getSchema(); Schema decimalSchema = getDecimalSchema(schema, "Chairman"); LogicalType logicalType = LogicalTypes.fromSchema(decimalSchema); ByteBuffer buffer = (ByteBuffer) record.get("Chairman"); BigDecimal resultBD = decimalConversion.fromBytes(buffer, schema, logicalType); I am not sure how to handle BigInteger, because its seems DecimalConversion supports only BigDecimal. Thanks Toivo was (Author: toivo adams): Note!! BigInteger handling is not yet changed. I am not sure how Avro supports DECIMAL types. Current solution looks clumsy. You can’ put BigDecimal value directly to Avro record, like rec.put(i - 1, bigDecimalValue); Avro will throw org.apache.avro.AvroRuntimeException: Unknown datum type java.math.BigDecimal: 38 Instead you need to convert value to bytes, like Schema decimalSchema = getDecimalSchema(schema, meta.getColumnName(i)); LogicalType logicalType = LogicalTypes.fromSchema(decimalSchema); ByteBuffer byteBuffer = decimalConversion.toBytes((BigDecimal) value, decimalSchema, logicalType); rec.put(i - 1, byteBuffer); And getting BigDecimal value from record is equally weird: record = dataFileReader.next(record); DecimalConversion decimalConversion = new DecimalConversion(); Schema schema = record.getSchema(); Schema decimalSchema = getDecimalSchema(schema, "Chairman"); LogicalType logicalType = LogicalTypes.fromSchema(decimalSchema); ByteBuffer buffer = (ByteBuffer) record.get("Chairman"); BigDecimal resultBD = decimalConversion.fromBytes(buffer, schema, logicalType); I am not sure how to handle BigInteger, because its seems DecimalConversion supports only BigDecimal. Thanks Toivo > JDBC-to-Avro processors handle BigDecimals as Strings > ----------------------------------------------------- > > Key: NIFI-2624 > URL: https://issues.apache.org/jira/browse/NIFI-2624 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions > Reporter: Matt Burgess > Assignee: Toivo Adams > > The original SQL processors implemented BigDecimal values as Strings for > Avro, as the version of Avro it used (1.7.6) did not support DECIMAL type. > As of Avro 1.7.7 (AVRO-1402), this type is supported and so the SQL/HiveQL > processors should be updated to handle BigDecimals correctly if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)