[
https://issues.apache.org/jira/browse/AVRO-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224936#comment-17224936
]
Werner Daehn commented on AVRO-2952:
------------------------------------
Is the travis error:
[INFO] Running org.apache.avro.compiler.specific.TestSpecificCompiler
6477
[ERROR] Tests run: 38, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.783
s <<< FAILURE! - in org.apache.avro.compiler.specific.TestSpecificCompiler
6478
[ERROR]
org.apache.avro.compiler.specific.TestSpecificCompiler.testConversionInstanceWithDecimalLogicalTypeDisabled
Time elapsed: 0.377 s <<< FAILURE!
6479
org.junit.ComparisonFailure: Should use null for decimal if the flag is off
expected:<n[ull]> but was:<n[ew
org.apache.avro.Conversions.DecimalConversion()]>
caused by me or something else? I cannot check for myself because the
avro-compiler package does not build with the error that "Idl" cannot be found.
I can't find this class either.
Can anybody help me here?
> Logical Types and Conversions enhancements
> ------------------------------------------
>
> Key: AVRO-2952
> URL: https://issues.apache.org/jira/browse/AVRO-2952
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.10.1
> Reporter: Werner Daehn
> Priority: Critical
>
> *Summary*:
> * AvroDatatype Field.getDataType(): returns an object with common data type
> related methods.
> * Add more trivial LogicalTypes to allow better database integration, e.g.
> the LogicalType VARCHAR(10) is a STRING that carries the information that
> only ASCII chars are in the payload and up to 10 chars max.
> For example, to set an ENUM value for a field f the call
> {{testRecord.put(f.name(),
> f.getDataType().convertToRawType(myEnum.male.name()));}}
> does all the conversion. I considered adding that conversion to the put()
> method but did not for fear of side effects.
> *Reasoning*:
> I am working with Avro (Kafka) for two years now and have implemented
> improvements around Logical Types. These I merged into the Avro code with
> zero side effects - pure additions. No breaking changes for other Avro users
> but a great help for them.
> Imagine you connect two databases via Kafka using Avro as the message payload.
> # The first problem you will be facing is that RawTypes and LogicalTypes are
> handled differently. For LogicalTypes there are conversion functions that
> provide metadata (e.g. getConvertedType returns that a Java Instant is the
> best data type for a timestamp-millis plus conversion logic. For raw types
> there is no such thing. A Boolean can be provided as true, "TRUE", 1,...
> # Second problem will be the lack of getObject()/setObject() methods similar
> to JDBC. The result are endless switch-case lists to call the correct
> methods. In every single project for every user.
> # Number three is the usage of the Converters as such. The intended usage is
> to add converters to the GenericData and the reader/writer uses the best
> suited converter. What I have seen most people do however is to use the
> converters manually and assign the raw value directly. While adding
> converters is possible still, the conversion at GenericRecord.put() and
> GenericRecord.get() is easy now.
> # For a data exchange format like Avro, it is important to carry as much
> metadata as possible. For example, purely seen from Avro a STRING data type
> is just fine. 99% of the string data types in a database are VARCHAR(length)
> and NVARCHAR(length). While putting an ASCII String of length 10 into a
> STRING is no problem, on the consumer side the only matching data type is a
> NCLOB - the worst for a data base. The LogicalTypes provide such nice methods
> to carry such metadata, e.g. a LogicalType VARCHAR(10) backed by a String.
> These Logical Types do not have any conversion, they just exist for the
> metadata. You have such a thing already with the UUID LogicalType.
>
> *Changes*:
> * A new package logicaltypes created. It includes all new LogicalTypes and
> the AvroDataType implementations for the various raw data types.
> * The existing LogicalTypes are unchanged. The corresponding classes in the
> logicaltype package just extend them.
> * For that some LogicalType fields needed to be made public.
> * The LogicalTypes return the more detailed logicaltype.* classes.
> * A test class created.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
