Re: Consider Python < 2.7 obsolete

[Ivan Greene]

One more task for this subject, the python 3 implementation does not yet have 
support for Avro logical types as far as I’ve been able to tell. So a decent 
amount of code would need to be ported there.

—Ivan

> On Nov 22, 2019, at 7:40 AM, Ryan Skraba  wrote:
> 
> Thanks for the info; it sounds reasonable to me!  (A big +1 to getting
> rid of ant, of course).
> 
> On Thu, Nov 21, 2019 at 12:56 PM Michael A. Smith  
> wrote:
>> 
>> i would like to update and maintain the py colleges and deprecate and
>> eventually remove the py3 one.
>> 
>> 1. Despite being less modern, the py codebase has been kept somewhat more
>> pythonic. Capitalizing `schema.Parse` and the literal translation of the
>> java parsing normal form implementation are two oddities we could address.
>> There are several issues and pull requests inquiring why the two python
>> implementations aren't API compatible.
>> 2. Several modules in py3 were never completed. I called out txipc as
>> broken, but the tether stuff is missing entirely.
>> 
>> Things we need to do to make this possible:
>> 
>> 1. Make the py codebase compatible with py3.5. I've been working on this,
>> while still trying to maintain 2.7 compatibility for now.
>> 2. I want to port py3's setup approach, making it possible to package and
>> test py without ant. There are lots of benefits, but the only thing on
>> topic here is to be able to be able to use multiple python versions at the
>> same time. (We should look at tox soon.)
>> 
>> What do you think?
>> 
>> On Thu, Nov 21, 2019 at 04:23 Ryan Skraba  wrote:
>> 
>>> Tick-tock... just bumping this up as the year end approaches!  Any
>>> interest in making a statement or plan for python2 support for future
>>> releases of Avro?
>>> 
>>> There should be one more maintenance release of python 2.7 in 2020
>>> (after sunset) for the accumulated fixes.
>>> 
>>> I'm in the context of looking at the docker+build scripts: keeping or
>>> dropping the python2 runtime has little significant impact.
>>> 
>>> Ryan
>>> 
>>> 
>>> 
>>> On Tue, Jun 25, 2019 at 1:22 PM Michael A. Smith 
>>> wrote:
 
 Inline…
 
 On Tue, Jun 25, 2019 at 05:03 Ismaël Mejía  wrote:
 
> Probably is a good idea that we publish our policy around python
> support [1] as other projects have done [2].
> I think supporting python 2 makes sense at least for our latest
> release of this year so probably 1.9.x or eventually 1.10.x.
 
 
 i agree wholeheartedly, but only python 2.7.
 
 I am not at all familiar with our python3 codebase, are we feature
> equivalent? otherwise maybe worth to create JIRAs and work on those.
 
 
 Not perfectly, and there is work on that, but the biggest gap is that
 lang/py is much more extensively tested, but its tests use pyant, which I
 have not yet figured out how to port.
 
 [1] https://pythonclock.org/
> [2] https://python3statement.org/
> 
> On Tue, Jun 25, 2019 at 9:38 AM Driesprong, Fokko >>> 
> wrote:
>> 
>> I'm not sure how much effort we should put into Python2.7 in general,
> since
>> this version is EOL after this year.
>> 
>> Cheers, Fokko
>> 
>> Op ma 24 jun. 2019 om 03:20 schreef Michael A. Smith <
> mich...@smith-li.com>:
>> 
>>> There's some not-insignificant complexity in the lang/py codebase
>>> to
>>> support derelict versions of Python. There are polyfills for json,
> structs,
>>> a whole "StoppableHTTPServer" in avro.tool.
>>> 
>>> I created AVRO-2445 and will start removing this stuff now, but
>>> wanted
> to
>>> bounce the idea around the list in case there's some obscure
>>> reason to
> keep
>>> these things around.
>>> 
> 
>>> 

[jira] [Commented] (AVRO-2498) UUID generation is not working avro 1.9 version

[Ivan Greene (Jira)]

[ 
https://issues.apache.org/jira/browse/AVRO-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16947119#comment-16947119
 ] 

Ivan Greene commented on AVRO-2498:
---

[~ryanskraba] The problem with UUID logical type is setting a stringType in the 
generator configuration causes the logical type to be ignored. See AVRO-2548 
for info. So the specific compiler will generate java.util.UUID with this issue 
resolved. I have a PR out for that one waiting on review

> UUID generation is not working avro 1.9 version
> ---
>
> Key: AVRO-2498
> URL: https://issues.apache.org/jira/browse/AVRO-2498
> Project: Apache Avro
>  Issue Type: Bug
>Reporter: Raj G
>Priority: Major
>
> Hi, 
>  
> I've downloaded avro-tools 1.9 jar and trying to generate a java file from an 
> avro schema using below command
>  
> java -jar avro-tools-1.9.0.jar compile schema ./myschema.avsc  ./avro_java
>  
> and have below schema in my .avsc file to generate a UUID field
> {
>  "name": "myField",
>  "type": {
>  "type": "long",
>  "logicalType": "uuid"
>  }
> }
>  
> But this is generating a long field instead of UUID type. Please suggest.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Python 3 namespace validation incorrect

[Ivan Greene]

I think this makes sense as generally an identifier part may not start with a 
number (2 here). I’m unfamiliar with the influence that a schema’s namespace 
has on code generation in Python but this makes sense from the Java perspective 
at least.

—Ivan

> On Sep 26, 2019, at 1:20 PM, Neil McBride 
>  wrote:
> 
> Apologies Ivan, I used a bad example. A specific case is where the name =
> "ADFORM__DELIVERY" and the namespace = "io.2sixty":
> 
>>>> fullname='io.2sixty.ADFORM__DELIVERY'
>>>> _RE_FULL_NAME = re.compile(
> ... r'^'
> ... r'[.]?(?:[A-Za-z_][A-Za-z0-9_]*[.])*'  # optional namespace
> ... r'([A-Za-z_][A-Za-z0-9_]*)'# name
> ... r'$'
> ... )
>>>> print(_RE_FULL_NAME.match(fullname))
> None
> 
> On Thu, 26 Sep 2019 at 19:09, Ivan Greene  wrote:
> 
>> In a Python 3 repl, a full name like ‘org.apache.avro.ipc.Foo’ matches
>> this pattern. It looks like the pattern you suggest is not used to prevent
>> repeating full stops.
>> 
>> —Ivan
>> 
>>> On Sep 26, 2019, at 12:42 PM, Neil McBride
>>  wrote:
>>> 
>>> Hello,
>>> 
>>> We're unable to make use of the Python 3 Avro package as it does not
>> allow
>>> full stops within the namespace; they will fail with "Invalid schema name
>>> %r infered from name %r and namespace %r.". An example namespace is
>>> org.apache.avro.ipc.
>>> 
>>> The regex is in:
>>> 
>>> https://github.com/apache/avro/blob/master/lang/py3/avro/schema.py#L236
>>> 
>>> _RE_FULL_NAME = re.compile(
>>>   r'^'
>>>   r'[.]?(?:[A-Za-z_][A-Za-z0-9_]*[.])*'  # optional namespace
>>>   r'([A-Za-z_][A-Za-z0-9_]*)'# name
>>>   r'$'
>>> )
>>> 
>>> Is there a reason why the validation for _RE_FULL_NAME could not be
>>> r'[A-Za-z_][A-Za-z0-9_.]*
>>> 
>>> Thanks
>> 
>> 

Re: Python 3 namespace validation incorrect

[Ivan Greene]

In a Python 3 repl, a full name like ‘org.apache.avro.ipc.Foo’ matches this 
pattern. It looks like the pattern you suggest is not used to prevent repeating 
full stops.

—Ivan

> On Sep 26, 2019, at 12:42 PM, Neil McBride 
>  wrote:
> 
> Hello,
> 
> We're unable to make use of the Python 3 Avro package as it does not allow
> full stops within the namespace; they will fail with "Invalid schema name
> %r infered from name %r and namespace %r.". An example namespace is
> org.apache.avro.ipc.
> 
> The regex is in:
> 
> https://github.com/apache/avro/blob/master/lang/py3/avro/schema.py#L236
> 
> _RE_FULL_NAME = re.compile(
>r'^'
>r'[.]?(?:[A-Za-z_][A-Za-z0-9_]*[.])*'  # optional namespace
>r'([A-Za-z_][A-Za-z0-9_]*)'# name
>r'$'
> )
> 
> Is there a reason why the validation for _RE_FULL_NAME could not be
> r'[A-Za-z_][A-Za-z0-9_.]*
> 
> Thanks

[jira] [Comment Edited] (AVRO-2498) UUID generation is not working avro 1.9 version

[Ivan Greene (Jira)]

[ 
https://issues.apache.org/jira/browse/AVRO-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933650#comment-16933650
 ] 

Ivan Greene edited comment on AVRO-2498 at 9/19/19 6:17 PM:


The 'uuid' logical type applies to fields of avro type 'string' and not 'long'. 
Change your field to

{
"name": "myField",
"type":
{ "type": "text", "logicalType": "uuid" }
}



was (Author: igreene):
The 'uuid' logical type applies to fields of avro type 'string' and not 'long'. 
Change your field to
{{
{
"name": "myField",
"type":
{ "type": "text", "logicalType": "uuid" }
}
}}

> UUID generation is not working avro 1.9 version
> ---
>
> Key: AVRO-2498
> URL: https://issues.apache.org/jira/browse/AVRO-2498
> Project: Apache Avro
>  Issue Type: Bug
>Reporter: Raj G
>Priority: Major
>
> Hi, 
>  
> I've downloaded avro-tools 1.9 jar and trying to generate a java file from an 
> avro schema using below command
>  
> java -jar avro-tools-1.9.0.jar compile schema ./myschema.avsc  ./avro_java
>  
> and have below schema in my .avsc file to generate a UUID field
> {
>  "name": "myField",
>  "type": {
>  "type": "long",
>  "logicalType": "uuid"
>  }
> }
>  
> But this is generating a long field instead of UUID type. Please suggest.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (AVRO-2498) UUID generation is not working avro 1.9 version

[Ivan Greene (Jira)]

[ 
https://issues.apache.org/jira/browse/AVRO-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933650#comment-16933650
 ] 

Ivan Greene commented on AVRO-2498:
---

The 'uuid' logical type applies to fields of avro type 'string' and not 'long'. 
Change your field to
{{
{
"name": "myField",
"type":
{ "type": "text", "logicalType": "uuid" }
}
}}

> UUID generation is not working avro 1.9 version
> ---
>
> Key: AVRO-2498
> URL: https://issues.apache.org/jira/browse/AVRO-2498
> Project: Apache Avro
>  Issue Type: Bug
>Reporter: Raj G
>Priority: Major
>
> Hi, 
>  
> I've downloaded avro-tools 1.9 jar and trying to generate a java file from an 
> avro schema using below command
>  
> java -jar avro-tools-1.9.0.jar compile schema ./myschema.avsc  ./avro_java
>  
> and have below schema in my .avsc file to generate a UUID field
> {
>  "name": "myField",
>  "type": {
>  "type": "long",
>  "logicalType": "uuid"
>  }
> }
>  
> But this is generating a long field instead of UUID type. Please suggest.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: C# POCO serializer/deserializer

[Ivan Greene]

Patrick,

This sounds as though it would be the C# equivalent of Java's ReflectData, 
which has been part of the Avro API for several years now, so it’s likely there 
would be interest. Your best bet is to open a Jira describing the planned 
feature and open a pull request on Github to start the discussion. The 
contributing page on the wiki is a bit out of date and still recommends 
submitting patches on Jira but I believe that Github is now the official 
repository.

> On May 1, 2019, at 4:31 PM, Patrick Farry  wrote:
> 
> Hi All,
> 
> I have written code that implements Avro serialization for POCO classes - 
> i.e. classes that are not generated by Avro codegen and do not implement 
> ISpecificRecord. The idea was to make it work as much like JSON.net as 
> possible.
> 
> The serializer inherits from SpecificDefaultWriter and the deserializer from 
> SpecificDefaultReader.
> 
> Avro fields are mapped to C# properties either by matching the field name and 
> property name or by using an attribute to specify the field sequence number. 
> 
> Is this something that would be of interest to the Avro project? My company 
> has approved committing the code and I’d be available and happy to maintain 
> this code and work on other parts of the C# implementation.
> 
> Regards. 
> 
> 

[jira] [Created] (AVRO-2375) Allow repeating AvroMeta use

[Ivan Greene (JIRA)]

Ivan Greene created AVRO-2375:
-

 Summary: Allow repeating AvroMeta use
 Key: AVRO-2375
 URL: https://issues.apache.org/jira/browse/AVRO-2375
 Project: Apache Avro
  Issue Type: Improvement
  Components: java
Reporter: Ivan Greene
 Fix For: 1.9.0


AvroMeta should be a repeatable annotation, to allow the association of more 
than one key-value pairs of metadata with a type or field within a schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Feature freeze Apache Avro 1.9

[Ivan Greene]

Just to clarify, the UUID logical type was included in 1.8.2 (and perhaps a 
release prior to that?), it was just undocumented. So I think one option could 
be providing a conversion both to/from string and to/from 16-length fixed, 
similar to the way that the decimal type provides conversion to/from fixed as 
well as to/from bytes, which can be chosen by the user.

—Ivan

> On Apr 9, 2019, at 9:02 AM, Driesprong, Fokko  wrote:
> 
> Hi Ivan,
> 
> Thanks for the input. Personally, I'm not using the UUID at all. Any
> comments on it Ryan? Since it isn't introduced in any release, we could
> remove it if there is a consensus if it doesn't add any value to the Avro
> format.
> 
> Apart from that, I did some testing last week on some internal projects,
> and it went surprisingly well. I would like to proceed into cutting a first
> RC.
> 
> Cheers, Fokko
> 
> Op di 2 apr. 2019 om 03:49 schreef Ivan Greene :
> 
>> Somehow only now after I sent this last message did I realize the
>> conversion can already be controlled by adding the LogicalType to
>> ReflectData as in TestReflectLogicalTypes.java :) So never mind on that
>> second part.
>> 
>> However it's worth making a consideration about whether we are interested
>> in revising the UUID logicalType to be 16-byte fixed rather than a 36-byte
>> string before it is documented in this release.
>> 
>> -- Ivan
>> 
>> On Mon, Apr 1, 2019 at 8:30 PM Ivan Greene 
>> wrote:
>> 
>>> Another outstanding bit for the release in my mind is the state of the
>>> UUID logicalType. As Ryan commented here:
>>> 
>> https://issues.apache.org/jira/browse/AVRO-2021?focusedCommentId=16242524=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16242524
>>> there is currently an implemented logical type of 'uuid', however it uses
>>> the naive toString/fromString rather than a more compact 16-byte fixed. I
>>> think we should think about whether we really need this logical type as
>> it
>>> doesn't provide much benefit as it is. What do you think?
>>> 
>>> Related is UUID support for ReflectData, which is currently nil (we
>> cannot
>>> even annotate @Stringable; UUID has no String constructor). Basically the
>>> only option for using UUIDs in ReflectData is the user implementing their
>>> own CustomEncoding. A long unanswered query on another Jira is here:
>>> 
>> https://issues.apache.org/jira/browse/AVRO-1497?focusedCommentId=16283656=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16283656
>>> We could either:
>>> - Provide CustomEncoding implementation(s), such as UUIDAsStringEncoding
>>> and/or UUIDAsFixedEncoding
>>> - Add support for LogicalTypes in ReflectData definitions, through a new
>>> annotation a la @AvroLogicalType("timestamp-millis"), etc. where we can
>> use
>>> the uuid logicalType if we decide to keep that in
>>> 
>>> Taking a look at the history it seems as though LogicalTypes and
>>> CustomEncodings share some of the same intentions. It's not clear what
>> the
>>> better option is in this case. CustomEncoding predates LogicalType by a
>> few
>>> years, but CustomEncoding is more Java-centric.
>>> 
>>> Please let me know what the thoughts are. I should have some time to work
>>> on what we decide on this week. Would like to get these items in for 1.9.
>>> 
>>> -- Ivan
>>> 
>> 

[jira] [Commented] (AVRO-2369) Provide external way to construct Schema.Field with default value of 'null'

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811181#comment-16811181
 ] 

Ivan Greene commented on AVRO-2369:
---

Looks like it will work for our purposes, thanks for working on this. There 
should be a good path for us to be compatible with both 1.8.2 and 1.9.0 now.

> Provide external way to construct Schema.Field with default value of 'null'
> ---
>
> Key: AVRO-2369
> URL: https://issues.apache.org/jira/browse/AVRO-2369
> Project: Apache Avro
>  Issue Type: Task
>  Components: java
>    Reporter: Ivan Greene
>Assignee: Daniel Kulp
>Priority: Blocker
> Fix For: 1.9.0
>
>
> After making the Schema.Field constructor which takes the default value as a 
> JsonNode was made package private, there is no external way to construct a 
> field that has a default value of 'null'. Internally that constructor will 
> call {{JacksonUtils.toJsonNode(defaultValue)}}, which will return 'null' when 
> passed null, and the resulting Field will not have a default value (the json 
> node would need to be NullNode instead of simply null itself). This will 
> affect projects that need a way to dynamically build schemas and their fields.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-2369) Provide external way to construct Schema.Field with default value of 'null'

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811052#comment-16811052
 ] 

Ivan Greene commented on AVRO-2369:
---

[~dkulp] regarding this and the last issue I posted; do you think the 
difference between a field with default 'null' and no default at all needs to 
be significant? When dealing in generated SpecificRecords, there is essentially 
no difference, except in the schema text; i.e a field with type {{["null", 
"string"]}} that doesn't have an explicit default will build and serialize 
correctly without setting the field. Is there a strong reason we need to 
distinguish between implicit and explicit defaults of null, other than to 
provide a more meaningful exception message when a Union does not contain null?

> Provide external way to construct Schema.Field with default value of 'null'
> ---
>
> Key: AVRO-2369
> URL: https://issues.apache.org/jira/browse/AVRO-2369
> Project: Apache Avro
>  Issue Type: Task
>  Components: java
>Reporter: Ivan Greene
>Priority: Blocker
> Fix For: 1.9.0
>
>
> After making the Schema.Field constructor which takes the default value as a 
> JsonNode was made package private, there is no external way to construct a 
> field that has a default value of 'null'. Internally that constructor will 
> call {{JacksonUtils.toJsonNode(defaultValue)}}, which will return 'null' when 
> passed null, and the resulting Field will not have a default value (the json 
> node would need to be NullNode instead of simply null itself). This will 
> affect projects that need a way to dynamically build schemas and their fields.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (AVRO-2369) Provide external way to construct Schema.Field with default value of 'null'

[Ivan Greene (JIRA)]

Ivan Greene created AVRO-2369:
-

 Summary: Provide external way to construct Schema.Field with 
default value of 'null'
 Key: AVRO-2369
 URL: https://issues.apache.org/jira/browse/AVRO-2369
 Project: Apache Avro
  Issue Type: Task
  Components: java
Reporter: Ivan Greene
 Fix For: 1.9.0


After making the Schema.Field constructor which takes the default value as a 
JsonNode was made package private, there is no external way to construct a 
field that has a default value of 'null'. Internally that constructor will call 
{{JacksonUtils.toJsonNode(defaultValue)}}, which will return 'null' when passed 
null, and the resulting Field will not have a default value (the json node 
would need to be NullNode instead of simply null itself). This will affect 
projects that need a way to dynamically build schemas and their fields.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-2368) Provide way to distinguish whether a Schema.Field has a default value or its default is null

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810860#comment-16810860
 ] 

Ivan Greene commented on AVRO-2368:
---

Alright that makes sense. Thanks for getting that fix in so quickly.

> Provide way to distinguish whether a Schema.Field has a default value or its 
> default is null
> 
>
> Key: AVRO-2368
> URL: https://issues.apache.org/jira/browse/AVRO-2368
> Project: Apache Avro
>  Issue Type: Task
>  Components: java
>    Reporter: Ivan Greene
>Assignee: Daniel Kulp
>Priority: Blocker
> Fix For: 1.9.0
>
>
> After making {{Schema.Field#defaultValue}} package private, there is no way 
> to distinguish whether a Field has a default value of 'null' or has no 
> default at all. This is a breaking change, notably affecting Hortonworks 
> Schema Registry. See discussion here: 
> [https://github.com/hortonworks/registry/pull/547]
> I propose we should make that method public and still deprecated. Otherwise I 
> don't see a clear way to release a version of the Hortonworks registry that 
> will be compatible with both Avro 1.8.x and 1.9.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-2368) Provide way to distinguish whether a Schema.Field has a default value or its default is null

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810852#comment-16810852
 ] 

Ivan Greene commented on AVRO-2368:
---

[~dkulp] are you entirely opposed to exposing the original method again? 
WIthout that method I think our only option in Hortonworks is going to be 
reflection to create a build that can work with both avro 1.8.2 and 1.9.0

> Provide way to distinguish whether a Schema.Field has a default value or its 
> default is null
> 
>
> Key: AVRO-2368
> URL: https://issues.apache.org/jira/browse/AVRO-2368
> Project: Apache Avro
>  Issue Type: Task
>  Components: java
>    Reporter: Ivan Greene
>Assignee: Daniel Kulp
>Priority: Blocker
> Fix For: 1.9.0
>
>
> After making {{Schema.Field#defaultValue}} package private, there is no way 
> to distinguish whether a Field has a default value of 'null' or has no 
> default at all. This is a breaking change, notably affecting Hortonworks 
> Schema Registry. See discussion here: 
> [https://github.com/hortonworks/registry/pull/547]
> I propose we should make that method public and still deprecated. Otherwise I 
> don't see a clear way to release a version of the Hortonworks registry that 
> will be compatible with both Avro 1.8.x and 1.9.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (AVRO-2368) Provide way to distinguish whether a Schema.Field has a default value or its default is null

[Ivan Greene (JIRA)]

Ivan Greene created AVRO-2368:
-

 Summary: Provide way to distinguish whether a Schema.Field has a 
default value or its default is null
 Key: AVRO-2368
 URL: https://issues.apache.org/jira/browse/AVRO-2368
 Project: Apache Avro
  Issue Type: Task
  Components: java
Reporter: Ivan Greene
 Fix For: 1.9.0


After making {{Schema.Field#defaultValue}} package private, there is no way to 
distinguish whether a Field has a default value of 'null' or has no default at 
all. This is a breaking change, notably affecting Hortonworks Schema Registry. 
See discussion here: [https://github.com/hortonworks/registry/pull/547]

I propose we should make that method public and still deprecated. Otherwise I 
don't see a clear way to release a version of the Hortonworks registry that 
will be compatible with both Avro 1.8.x and 1.9.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Feature freeze Apache Avro 1.9

[Ivan Greene]

Somehow only now after I sent this last message did I realize the
conversion can already be controlled by adding the LogicalType to
ReflectData as in TestReflectLogicalTypes.java :) So never mind on that
second part.

However it's worth making a consideration about whether we are interested
in revising the UUID logicalType to be 16-byte fixed rather than a 36-byte
string before it is documented in this release.

-- Ivan

On Mon, Apr 1, 2019 at 8:30 PM Ivan Greene 
wrote:

> Another outstanding bit for the release in my mind is the state of the
> UUID logicalType. As Ryan commented here:
> https://issues.apache.org/jira/browse/AVRO-2021?focusedCommentId=16242524=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16242524
> there is currently an implemented logical type of 'uuid', however it uses
> the naive toString/fromString rather than a more compact 16-byte fixed. I
> think we should think about whether we really need this logical type as it
> doesn't provide much benefit as it is. What do you think?
>
> Related is UUID support for ReflectData, which is currently nil (we cannot
> even annotate @Stringable; UUID has no String constructor). Basically the
> only option for using UUIDs in ReflectData is the user implementing their
> own CustomEncoding. A long unanswered query on another Jira is here:
> https://issues.apache.org/jira/browse/AVRO-1497?focusedCommentId=16283656=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16283656
> We could either:
> - Provide CustomEncoding implementation(s), such as UUIDAsStringEncoding
> and/or UUIDAsFixedEncoding
> - Add support for LogicalTypes in ReflectData definitions, through a new
> annotation a la @AvroLogicalType("timestamp-millis"), etc. where we can use
> the uuid logicalType if we decide to keep that in
>
> Taking a look at the history it seems as though LogicalTypes and
> CustomEncodings share some of the same intentions. It's not clear what the
> better option is in this case. CustomEncoding predates LogicalType by a few
> years, but CustomEncoding is more Java-centric.
>
> Please let me know what the thoughts are. I should have some time to work
> on what we decide on this week. Would like to get these items in for 1.9.
>
> -- Ivan
>

Re: Feature freeze Apache Avro 1.9

[Ivan Greene]

Another outstanding bit for the release in my mind is the state of the UUID
logicalType. As Ryan commented here:
https://issues.apache.org/jira/browse/AVRO-2021?focusedCommentId=16242524=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16242524
there is currently an implemented logical type of 'uuid', however it uses
the naive toString/fromString rather than a more compact 16-byte fixed. I
think we should think about whether we really need this logical type as it
doesn't provide much benefit as it is. What do you think?

Related is UUID support for ReflectData, which is currently nil (we cannot
even annotate @Stringable; UUID has no String constructor). Basically the
only option for using UUIDs in ReflectData is the user implementing their
own CustomEncoding. A long unanswered query on another Jira is here:
https://issues.apache.org/jira/browse/AVRO-1497?focusedCommentId=16283656=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16283656
We could either:
- Provide CustomEncoding implementation(s), such as UUIDAsStringEncoding
and/or UUIDAsFixedEncoding
- Add support for LogicalTypes in ReflectData definitions, through a new
annotation a la @AvroLogicalType("timestamp-millis"), etc. where we can use
the uuid logicalType if we decide to keep that in

Taking a look at the history it seems as though LogicalTypes and
CustomEncodings share some of the same intentions. It's not clear what the
better option is in this case. CustomEncoding predates LogicalType by a few
years, but CustomEncoding is more Java-centric.

Please let me know what the thoughts are. I should have some time to work
on what we decide on this week. Would like to get these items in for 1.9.

-- Ivan

Re: Feature freeze Apache Avro 1.9

[Ivan Greene]

Hi Fokko, just now seeing your original email about features for 1.9. I have a 
small enhancement to ReflectData's protocol building ready for review here: 
https://github.com/apache/avro/pull/490 


I also opened a Jira earlier today about enhancements to the new ‘induce’ maven 
goal in the plugin: https://issues.apache.org/jira/browse/AVRO-2365 


I would love to address at least a few of the items listed in that Jira before 
1.9 is ultimately released, as the induce goal is an entirely new feature and 
we should make it as usable as possible out of the gate. I will try to get a PR 
opened up this weekend for those.

—Ivan

> On Mar 29, 2019, at 4:26 PM, Driesprong, Fokko  wrote:
> 
> Thanks all for the feedback.
> 
> Regarding the discussion Joda time, I've made a PR to completely remove the
> Joda dependency: https://github.com/apache/avro/pull/495
> 
> What do you guys think, could we push this for 1.9? This would obviously
> break backward compatibility, but then again, people should not use Joda
> time anymore.
> 
> Cheers, Fokko
> 
> Op vr 29 mrt. 2019 om 10:37 schreef Nandor Kollar
> :
> 
>> I totally support making JSR310 as the default instead of Joda
>> time. Should we consider removing the JSR310 from the class names, for
>> example rename Jsr310TimeConversions to TimeConversions and the existing
>> TimeConversions to JodaTimeConversions, and deprecate the latter?
>> 
>> On Thu, Mar 28, 2019 at 6:53 PM Daniel Kulp  wrote:
>> 
>>> I made few commits today that change the generated code from the Schema
>>> compiler a bit….  The changes make Avro 1.9 a bit more incompatible with
>>> 1.8, but since they are going to have to migrate anyway, I thought it
>> would
>>> be important to make the changes now rather then forcing everyone to do
>> so
>>> in 1.10.   The changes:
>>> 
>>> 1) Default for “time” classes change from Joda to JSR310.  There is a
>> flag
>>> to specify joda if they need/want it, but I think the “default” should be
>>> what we plan on supporting going forward, and I don’t think joda should
>> be
>>> it. At this point, joda should go away.  :)
>>> 
>>> 2) Private fields  - we were, by default, generating “@Deprecated public”
>>> fields.   The idea of generating deprecated code by default annoys me.
>>> Thus, I changed the default to “private”.  Again, there is a setting to
>>> make them public (or deprecated/public) if they want it, but for the
>>> default, I believe private is what we want.
>>> 
>>> While those will increase the effort to migrate from 1.8 to 1.9, I think
>>> it’s better to do that now than waiting any longer.
>>> 
>>> Thoughts?
>>> Dan
>>> 
>>> 
 On Mar 26, 2019, at 9:31 AM, Driesprong, Fokko 
>>> wrote:
 
 Hi all,
 
 I'd like to cut the branch for Apache Avro 1.9 release this Friday, and
 start moving to a release candidate so we can test. If there are any
 features that you would like to get in, please let me know.
 
 Cheers, Fokko
>>> 
>>> --
>>> Daniel Kulp
>>> dk...@apache.org  - http://dankulp.com/blog <
>>> http://dankulp.com/blog>
>>> Talend Community Coder - http://talend.com 
>>> 
>> 

[jira] [Created] (AVRO-2365) Enhancements to induce maven plugin goal

[Ivan Greene (JIRA)]

Ivan Greene created AVRO-2365:
-

 Summary: Enhancements to induce maven plugin goal
 Key: AVRO-2365
 URL: https://issues.apache.org/jira/browse/AVRO-2365
 Project: Apache Avro
  Issue Type: Improvement
  Components: java
Reporter: Ivan Greene
 Fix For: 1.9.0, 1.8.3


Before a release of 1.8.3 or 1.9.0 which will introduce the new 'induce' maven 
plugin goal (see AVRO-1749), there are a few issues with it which in my mind 
are critical to enable real-world use of the goal:
 * Allow excluding classes based on a pattern, and/or the ability to specify 
multiple sourceDirectories
 * Option to specify whether to use ReflectData.AllowNull or ReflectData
 * Throw MojoExecutionException with an informative message when appropriate, 
instead of just printing stacktraces. This is very important, we definitely 
need a build to fail when we hit a problem.
 * Use the current project defined encoding rather than hard coded UTF-8.
 * Use try-with-resources to ensure the PrintWriter gets closed in the event of 
exception

Nice to haves:
 * Specify custom ReflectData implementation (i.e user declared extensions) to 
be used
 * Specify separate output directories for schemata and protocols (these could 
both default to the single output directory if not specified individually)

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (AVRO-2357) (ReflectData) Support for generic types in protocol definitions

[Ivan Greene (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Greene updated AVRO-2357:
--
Description: 
For a Java interface extending another interface with type parameters, we may 
resolve actual type parameters and build the protocol definition based upon 
those.

For example, let's say we have a generic protocol defined by a Java interface:
{code:java}
public interface CrudProto {
  void persist(T record);

  T fetchById(I id);
}{code}
It would be natural to define a set of interfaces that extend this, such as:
{code:java}
public interface FooBarRecordProto extends CrudProto {}

public interface OtherRecordProto extends CrudProto {}
{code}
Calling ReflectData.get().getProtocol(FooBarRecordProto.class) should be able 
to resolve that this protocol deals in FooBarRecords and Strings, and build a 
protocol accordingly.

Currently, this call will produce an exception stating that a schema for 'T' 
cannot be resolved.

  was:
For a Java interface extending another interface with type parameters, we may 
resolve actual type parameters and build the protocol definition based upon 
those.

For example, let's say we have a generic protocol defined by a Java interface:
{code:java}
public interface CrudProto {
  void persist(T record);

  T fetchById(I id);
}{code}
It would be natural to define a set of interfaces that extend this, such as:
{code:java}
public interface FooBarRecordProto extends CrudProto {}

public interface OtherRecordProto extends CrudProto {}
{code}
Calling ReflectData.get().getProtocol(FooBarRecordProto.class) should be able 
to resolve that this protocol deals in FooBarRecords and Strings, and build a 
protocol accordingly.


> (ReflectData) Support for generic types in protocol definitions
> ---
>
> Key: AVRO-2357
> URL: https://issues.apache.org/jira/browse/AVRO-2357
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>    Reporter: Ivan Greene
>Priority: Minor
>
> For a Java interface extending another interface with type parameters, we may 
> resolve actual type parameters and build the protocol definition based upon 
> those.
> For example, let's say we have a generic protocol defined by a Java interface:
> {code:java}
> public interface CrudProto {
>   void persist(T record);
>   T fetchById(I id);
> }{code}
> It would be natural to define a set of interfaces that extend this, such as:
> {code:java}
> public interface FooBarRecordProto extends CrudProto {}
> public interface OtherRecordProto extends CrudProto {}
> {code}
> Calling ReflectData.get().getProtocol(FooBarRecordProto.class) should be able 
> to resolve that this protocol deals in FooBarRecords and Strings, and build a 
> protocol accordingly.
> Currently, this call will produce an exception stating that a schema for 'T' 
> cannot be resolved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (AVRO-2357) (ReflectData) Support for generic types in protocol definitions

[Ivan Greene (JIRA)]

Ivan Greene created AVRO-2357:
-

 Summary: (ReflectData) Support for generic types in protocol 
definitions
 Key: AVRO-2357
 URL: https://issues.apache.org/jira/browse/AVRO-2357
 Project: Apache Avro
  Issue Type: Improvement
  Components: java
Reporter: Ivan Greene


For a Java interface extending another interface with type parameters, we may 
resolve actual type parameters and build the protocol definition based upon 
those.

For example, let's say we have a generic protocol defined by a Java interface:
{code:java}
public interface CrudProto {
  void persist(T record);

  T fetchById(I id);
}{code}
It would be natural to define a set of interfaces that extend this, such as:
{code:java}
public interface FooBarRecordProto extends CrudProto {}

public interface OtherRecordProto extends CrudProto {}
{code}
Calling ReflectData.get().getProtocol(FooBarRecordProto.class) should be able 
to resolve that this protocol deals in FooBarRecords and Strings, and build a 
protocol accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-2327) Add support for java annotations containing elements

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787468#comment-16787468
 ] 

Ivan Greene commented on AVRO-2327:
---

[~rstull1250] please see my answer on your stackoverflow question; I believe 
this issue may be closed.

> Add support for java annotations containing elements
> 
>
> Key: AVRO-2327
> URL: https://issues.apache.org/jira/browse/AVRO-2327
> Project: Apache Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.8.2
>Reporter: Ryan Stull
>Priority: Minor
>  Labels: usability
>
> Currently it seems AVRO only supports java annotations that do not contain 
> [elements|https://docs.oracle.com/javase/tutorial/java/annotations/basics.html]
>  (which are basically the parameters to the annotation).  
> [Here's a stack overflow 
> issue|https://stackoverflow.com/questions/54675764/how-to-supply-argument-to-a-javaannotation-on-an-avro-idl-file]
>  which shows an example of this.
> It would be useful to have this feature because many java annotations require 
> elements, and not  supporting this means only a small portion of java 
> annotations can be supported.
> An example use case would be:  I want to store data in avro that I'm getting 
> from a 3rd party service as a JSON response, and I want my AVRO idl file to 
> have different field names from the response body. If I'm using GSON, or any 
> other JSON parser that uses annotations to configure alternative name 
> mapping, then there's no easy way to do this.
> To be more explicit, if I had a json response 
> {code:json}
> {
>"name": "Jeff"
> }
> {code}
> and an avro file
> {code:js}
> protocol Response {
>   record Response {
> string @javaAnnotation("com.google.gson.annotations.SerializedName") 
> userName;
>   }
> }
> {code}
> Then there's no way to tell the GSON annotation about the alternate name is 
> "name".  In java you would just use:
> {code:java}
> public class Response {
> @com.google.gson.annotations.SerializedName("name")
> String userName;
> }
> {code}
> Supporting this feature would allow the generated java classes to be easily 
> reused throughout the different layers of an app.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-1891) Generated Java code fails with union containing logical type

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787433#comment-16787433
 ] 

Ivan Greene commented on AVRO-1891:
---

Also curious; what are the remaining unresolved parts of this issue?

> Generated Java code fails with union containing logical type
> 
>
> Key: AVRO-1891
> URL: https://issues.apache.org/jira/browse/AVRO-1891
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java, logical types
>Affects Versions: 1.8.1
>Reporter: Ross Black
>Priority: Blocker
> Fix For: 1.8.3
>
> Attachments: AVRO-1891.patch, AVRO-1891.yshi.1.patch, 
> AVRO-1891.yshi.2.patch, AVRO-1891.yshi.3.patch, AVRO-1891.yshi.4.patch
>
>
> Example schema:
> {code}
> {
>   "type": "record",
>   "name": "RecordV1",
>   "namespace": "org.brasslock.event",
>   "fields": [
> { "name": "first", "type": ["null", {"type": "long", 
> "logicalType":"timestamp-millis"}]}
>   ]
> }
> {code}
> The avro compiler generates a field using the relevant joda class:
> {code}
> public org.joda.time.DateTime first
> {code}
> Running the following code to perform encoding:
> {code}
> final RecordV1 record = new 
> RecordV1(DateTime.parse("2016-07-29T10:15:30.00Z"));
> final DatumWriter datumWriter = new 
> SpecificDatumWriter<>(record.getSchema());
> final ByteArrayOutputStream stream = new ByteArrayOutputStream(8192);
> final BinaryEncoder encoder = 
> EncoderFactory.get().directBinaryEncoder(stream, null);
> datumWriter.write(record, encoder);
> encoder.flush();
> final byte[] bytes = stream.toByteArray();
> {code}
> fails with the exception stacktrace:
> {code}
>  org.apache.avro.AvroRuntimeException: Unknown datum type 
> org.joda.time.DateTime: 2016-07-29T10:15:30.000Z
> at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:741)
> at 
> org.apache.avro.specific.SpecificData.getSchemaName(SpecificData.java:293)
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:706)
> at 
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at 
> org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at 
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at 
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at 
> org.brasslock.avro.compiler.GeneratedRecordTest.shouldEncodeLogicalTypeInUnion(GeneratedRecordTest.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>

[jira] [Commented] (AVRO-1870) Avro compiler generate JAVA enum interfaces but writeEnum expects GenericEnumSymbol

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787430#comment-16787430
 ] 

Ivan Greene commented on AVRO-1870:
---

This issue may be closed; it is the same as AVRO-1810 which has been resolved

> Avro compiler generate JAVA enum interfaces but writeEnum expects 
> GenericEnumSymbol
> ---
>
> Key: AVRO-1870
> URL: https://issues.apache.org/jira/browse/AVRO-1870
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1
>Reporter: Ali Hashemi
>Priority: Major
>
> Avro compiler generate an enum JAVA class from an enum Avro type 
> and this class is used in the Java class of the records pointing to the enum 
> type. However, org.apache.avro.generic.GenericDatumWriter.writeEnum expects a 
> GenericEnumSymbol rather than Java Enum!
> This is problem because the class interfaces for Avro messages expect Java 
> Enum and not GenericEnumSymbol!
> Consider the following schema:
> TrafficLights.avsc:
> {"namespace": "com.comp.message",
>  "type": "record",
>  "name": "TrafficLights",
>  "fields": [
> {"name": "id",  "type": "string"},
> {"name": "signal",  "type": "Signals" }
>  ]
> }
> Signals.avsc:
> {
>"namespace": "com.comp.message",
>"type": "enum",
>"name":"Signals",
>"symbols" : ["GREEN", "YELLOW", "RED"]
> }
> Using maven-avro-plugin to import "Signals" as an external schema, Avro 
> compiler creates the following constructor for TrafficLights class:
> public TrafficLights(java.lang.CharSequence id, com.comp.message.Signals 
> signal){...}
> and an Enum class fot the Signals:
> package com.comp.message;
> @SuppressWarnings("all")
> @org.apache.avro.specific.AvroGenerated
> public enum Signals {
>   GREEN, YELLOW, RED  ;
>   public static final org.apache.avro.Schema SCHEMA$ = new 
> org.apache.avro.Schema.Parser().parse("{\"type\":\"enum\",\"name\":\"Signals\",\"namespace\":\"com.comp.message\",\"symbols\":[\"GREEN\",\"YELLOW\",\"RED\"]}");
>   public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }
> }
> However, when I try to serialize a TrafficLights message, 
> GenericDatumWriter.writeEnum throws an exeception:
> TrafficLights  mainTrafficLight = new TrafficLights  ("123",  Signals.GREEN);
> byte[] msg = new AvroMsgManager( mainTrafficLight .getSchema().toString() 
> ).serialize (mainTrafficLight))
> org.apache.avro.AvroTypeException: Not an enum: GREEN
> at 
> org.apache.avro.generic.GenericDatumWriter.writeEnum(GenericDatumWriter.java:164)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:106)
> at 
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at 
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at 
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at 
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-2136) All Outstreams are not getting closed in DataFileWriter

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16787427#comment-16787427
 ] 

Ivan Greene commented on AVRO-2136:
---

I agree this issue can be closed. It is not obvious but in DataFileWriter#init, 
'underlyingStream' is assigned to the OutputStream and 'out' is assigned to a 
BufferedFileOutputStream wrapping the OutputStream; when 'out' is closed, the 
original outputstream will also be closed.

> All Outstreams are not getting closed in DataFileWriter
> ---
>
> Key: AVRO-2136
> URL: https://issues.apache.org/jira/browse/AVRO-2136
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Reporter: Balu
>Priority: Critical
>
> underlyingStream is getting initialized with output stream provided while 
> writing the data, butis never getting closed. This is causing issue when a 
> file system accepts only one active connection to write the data.
>  
> When close method is called it is also close underlyingStream



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (AVRO-2212) GenericData.deepCopyRaw not working with java.lang.BigDecimal

[Ivan Greene (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782880#comment-16782880
 ] 

Ivan Greene commented on AVRO-2212:
---

I believe this has been resolved by [https://github.com/apache/avro/pull/329] . 
I am unable to reproduce on the current master.

> GenericData.deepCopyRaw not working with java.lang.BigDecimal
> -
>
> Key: AVRO-2212
> URL: https://issues.apache.org/jira/browse/AVRO-2212
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.2
>Reporter: Sean Policarpio
>Priority: Critical
> Attachments: SomethingWithDec.java
>
>
> With an AVDL of:
> {code:java}
>   @namespace("com.foo")
>   record SomethingWithDec {
> decimal(8,2) dec;
>   }
> {code}
> and the Scala/Java code of:
> {code}
> object Foo extends App {
>   import com.foo.SomethingWithDec
>   val x =  SomethingWithDec.newBuilder().setDec(new 
> java.math.BigDecimal("1234"))
>   val y = SomethingWithDec.newBuilder(x).build()
> }
> {code}
> I get the exception:
> {noformat}
> Exception in thread "main" java.lang.ClassCastException: java.math.BigDecimal 
> cannot be cast to java.nio.ByteBuffer
>   at 
> org.apache.avro.generic.GenericData.deepCopyRaw(GenericData.java:1089)
>   at org.apache.avro.generic.GenericData.deepCopy(GenericData.java:1065)
>   at com.foo.SomethingWithDec$Builder.(SomethingWithDec.java:167)
>   at com.foo.SomethingWithDec$Builder.(SomethingWithDec.java:150)
>   at com.foo.SomethingWithDec.newBuilder(SomethingWithDec.java:135)
> {noformat}
> I have attached the generated Java class for {{SomethingWithDec.}}
> This seems related to AVRO-1895, but I believe that was meant to have been 
> fixed in 1.8.2.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)