[jira] [Commented] (AVRO-2400) Avro 1.9.0 can't resolve schemas that can be resolved in 1.8.2

2019-05-29 Thread Raman Gupta (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16850568#comment-16850568
 ] 

Raman Gupta commented on AVRO-2400:
---

Having just encountered a situation in which I wanted to refactor how some 
schemas were organized by changing their namespaces, but otherwise leaving them 
exactly the same, and having quite a difficult time doing that with 1.9, I'd 
certainly appreciate a return to the 1.8 behavior of ignoring the namespace 
when checking schema compat. Alternately, allowing users to maintain 
compatibility by specifying a record "alias" that can be fully qualified to the 
old namespace + name would work also, and be more explicit, and allow record 
renames as well.

> Avro 1.9.0 can't resolve schemas that can be resolved in 1.8.2
> --
>
> Key: AVRO-2400
> URL: https://issues.apache.org/jira/browse/AVRO-2400
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Reporter: Jacob Tolar
>Priority: Blocker
> Fix For: 1.10.0, 1.9.1
>
>
> The failure occurs in ResolvingGrammarGenerator when reader and writer schema 
> have an array of records with different full names (e.g. different 
> namespace). 
> {code:java}
> Exception in thread "main" java.lang.ClassCastException: 
> org.apache.avro.Resolver$ReaderUnion cannot be cast to 
> org.apache.avro.Resolver$Container{code}
> Avro 1.8.2 allowed this behavior but it now fails in 1.9.0. Looking at the 
> jiras and code, I don't believe this was intentional ( AVRO-2275,  
> [https://github.com/apache/avro/commit/39d959e1c6a1f339f03dab18289e47f27c10be7f]
>   ).
>  
> It looks like there were some attempts to keep compatibility ( 
> [https://github.com/apache/avro/blob/branch-1.9/lang/java/avro/src/main/java/org/apache/avro/Resolver.java]
>  , e.g. see the commented out check for w.getFullName() in resolve()) but 
> this case was missed.
>  
> See this simple example to reproduce. 
> [https://gist.github.com/jacobtolar/c88d43ab4e8767227891e5cdc188ffad]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2383) Avro 1.9 applyAliases broken with self-referential records

2019-04-27 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2383:
--
Attachment: AVRO-2383.patch
Status: Patch Available  (was: Open)

Patch attached (same as pull in GitHub, use whichever is more convenient).

> Avro 1.9 applyAliases broken with self-referential records
> --
>
> Key: AVRO-2383
> URL: https://issues.apache.org/jira/browse/AVRO-2383
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Blocker
> Attachments: AVRO-2383.patch
>
>
> If a schema has a field with a self-referential record, the applyAliases 
> function produces a NullPointerException.
> The problem appears to be that the value for the record put into "seen" has 
> no fields, and then when the equality check is done, the `Schema.equals` 
> method assumes that fields is not null. Making the `equals` method more 
> defensive with null-checking "fields" solves the problem, as the `putAll` 
> method adds the fields back in, but I'm not sure if that is actually the best 
> fix.
> I will attach a patch with this fix.
> I'm marking this as a blocker for 1.9, as this behavior will cause a 
> NullPointerException in this situation any time data is decoded where the 
> writer and reader schema are different, and there are any aliases defined in 
> the reader (even if these aliases are unrelated to the self-referential 
> record).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AVRO-2383) Avro 1.9 applyAliases broken with self-referential records

2019-04-27 Thread Raman Gupta (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827823#comment-16827823
 ] 

Raman Gupta commented on AVRO-2383:
---

Created pull request: https://github.com/apache/avro/pull/510

> Avro 1.9 applyAliases broken with self-referential records
> --
>
> Key: AVRO-2383
> URL: https://issues.apache.org/jira/browse/AVRO-2383
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Blocker
>
> If a schema has a field with a self-referential record, the applyAliases 
> function produces a NullPointerException.
> The problem appears to be that the value for the record put into "seen" has 
> no fields, and then when the equality check is done, the `Schema.equals` 
> method assumes that fields is not null. Making the `equals` method more 
> defensive with null-checking "fields" solves the problem, as the `putAll` 
> method adds the fields back in, but I'm not sure if that is actually the best 
> fix.
> I will attach a patch with this fix.
> I'm marking this as a blocker for 1.9, as this behavior will cause a 
> NullPointerException in this situation any time data is decoded where the 
> writer and reader schema are different, and there are any aliases defined in 
> the reader (even if these aliases are unrelated to the self-referential 
> record).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (AVRO-2383) Avro 1.9 applyAliases broken with self-referential records

2019-04-27 Thread Raman Gupta (JIRA)
Raman Gupta created AVRO-2383:
-

 Summary: Avro 1.9 applyAliases broken with self-referential records
 Key: AVRO-2383
 URL: https://issues.apache.org/jira/browse/AVRO-2383
 Project: Apache Avro
  Issue Type: Bug
  Components: java
Affects Versions: 1.9.0
Reporter: Raman Gupta


If a schema has a field with a self-referential record, the applyAliases 
function produces a NullPointerException.

The problem appears to be that the value for the record put into "seen" has no 
fields, and then when the equality check is done, the `Schema.equals` method 
assumes that fields is not null. Making the `equals` method more defensive with 
null-checking "fields" solves the problem, as the `putAll` method adds the 
fields back in, but I'm not sure if that is actually the best fix.

I will attach a patch with this fix.

I'm marking this as a blocker for 1.9, as this behavior will cause a 
NullPointerException in this situation any time data is decoded where the 
writer and reader schema are different, and there are any aliases defined in 
the reader (even if these aliases are unrelated to the self-referential record).




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AVRO-2366) setValidateDefaults=false does not seem to work

2019-04-27 Thread Raman Gupta (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827608#comment-16827608
 ] 

Raman Gupta commented on AVRO-2366:
---

Thanks [~rumeshkrish]... you can see the problem by checking out commit 
3d2666f760b4834bcfdf2d2a96d6a277972d24c7 of  
https://github.com/rocketraman/gradle-avro-plugin 
(https://github.com/rocketraman/gradle-avro-plugin/tree/3d2666f760b4834bcfdf2d2a96d6a277972d24c7).

The failing test case, that worked on 1.8.0, is this one here -- just remove 
the @Ignored annotation:

https://github.com/rocketraman/gradle-avro-plugin/blob/3d2666f760b4834bcfdf2d2a96d6a277972d24c7/src/test/groovy/com/commercehub/gradle/plugin/avro/OptionsFunctionalSpec.groovy#L293

> setValidateDefaults=false does not seem to work
> ---
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure at compile-time if the default does not match 
> the schema. For example, a field definition of:
> {"name": "name", "type": "string", "default": null}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?
> The exception is:
> Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
> null not a {"type":"string","avro.java.string":"String"}
>   at org.apache.avro.Schema.validateDefault(Schema.java:1482)
>   at org.apache.avro.Schema.access$300(Schema.java:84)
>   at org.apache.avro.Schema$Field.(Schema.java:493)
>   at org.apache.avro.Schema$Field.(Schema.java:485)
>   at org.apache.avro.Schema$Field.(Schema.java:504)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2366) setValidateDefaults=false does not seem to work

2019-04-26 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2366:
--
Summary: setValidateDefaults=false does not seem to work  (was: 1.9 )

> setValidateDefaults=false does not seem to work
> ---
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure at compile-time if the default does not match 
> the schema. For example, a field definition of:
> {"name": "name", "type": "string", "default": null}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?
> The exception is:
> Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
> null not a {"type":"string","avro.java.string":"String"}
>   at org.apache.avro.Schema.validateDefault(Schema.java:1482)
>   at org.apache.avro.Schema.access$300(Schema.java:84)
>   at org.apache.avro.Schema$Field.(Schema.java:493)
>   at org.apache.avro.Schema$Field.(Schema.java:485)
>   at org.apache.avro.Schema$Field.(Schema.java:504)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2366) 1.9

2019-04-01 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2366:
--
Description: 
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure at compile-time if the default does not match the 
schema. For example, a field definition of:

{{
{"name": "name", "type": "string", "default": null}
}}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{
Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a {"type":"string","avro.java.string":"String"}
at org.apache.avro.Schema.validateDefault(Schema.java:1482)
at org.apache.avro.Schema.access$300(Schema.java:84)
at org.apache.avro.Schema$Field.(Schema.java:493)
at org.apache.avro.Schema$Field.(Schema.java:485)
at org.apache.avro.Schema$Field.(Schema.java:504)
at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)
at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)
at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)
at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)
}}

  was:
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure at compile-time if the default does not match the 
schema. For example, a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a \{"type":"string","avro.java.string":"String"}}}
{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}


> 1.9 
> 
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure at compile-time if the default does not match 
> the schema. For example, a field definition of:
> {{
> {"name": "name", "type": "string", "default": null}
> }}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?
> The exception is:
> {{
> Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
> null not a {"type":"string","avro.java.string":"String"}
>   at org.apache.avro.Schema.validateDefault(Schema.java:1482)
>   at org.apache.avro.Schema.access$300(Schema.java:84)
>   at org.apache.avro.Schema$Field.(Schema.java:493)
>   at org.apache.avro.Schema$Field.(Schema.java:485)
>   at org.apache.avro.Schema$Field.(Schema.java:504)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)
>   at 
> org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)
> }}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2366) 1.9

2019-04-01 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2366:
--
Description: 
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure at compile-time if the default does not match the 
schema. For example, a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a \{"type":"string","avro.java.string":"String"}}}
{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

  was:
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure at compile-time if the default does not match the 
schema. For example, a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a \{"type":"string","avro.java.string":"String"}}}
{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

 


> 1.9 
> 
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure at compile-time if the default does not match 
> the schema. For example, a field definition of:
> {"name": "name", "type": "string", "default": null}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?
> The exception is:
> {{Caused by: org.apache.avro.AvroTypeException: Invalid default for field 
> name: null not a \{"type":"string","avro.java.string":"String"}}}
> {{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
> {{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2366) 1.9

2019-04-01 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2366:
--
Description: 
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure at compile-time if the default does not match the 
schema. For example, a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a \{"type":"string","avro.java.string":"String"}}}
{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

 

  was:
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure at compile-time if the default does not match the 
schema. For example, a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{\{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
 \{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

 


> 1.9 
> 
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure at compile-time if the default does not match 
> the schema. For example, a field definition of:
> {"name": "name", "type": "string", "default": null}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?
> The exception is:
> {{Caused by: org.apache.avro.AvroTypeException: Invalid default for field 
> name: null not a \{"type":"string","avro.java.string":"String"}}}
> {{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
> {{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2366) 1.9

2019-04-01 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2366:
--
Description: 
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure at compile-time if the default does not match the 
schema. For example, a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{\{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
 \{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

 

  was:
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure if the default does not match the schema. For example, 
a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a {"type":"string","avro.java.string":"String"}}}
 \{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
 \{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

 


> 1.9 
> 
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure at compile-time if the default does not match 
> the schema. For example, a field definition of:
> {"name": "name", "type": "string", "default": null}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?
> The exception is:
> {\{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
>  \{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
>  \{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
>  \{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
>  \{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
>  \{{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
>  \{{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
>  \{{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
>  \{{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2366) 1.9

2019-04-01 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2366:
--
Description: 
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure if the default does not match the schema. For example, 
a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a {"type":"string","avro.java.string":"String"}}}
 \{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
 \{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
 \{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
 \{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

 

  was:
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure if the default does not match the schema. For example, 
a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a \{"type":"string","avro.java.string":"String"}}}
{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

 


> 1.9 
> 
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure if the default does not match the schema. For 
> example, a field definition of:
> {"name": "name", "type": "string", "default": null}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?
> The exception is:
> {{Caused by: org.apache.avro.AvroTypeException: Invalid default for field 
> name: null not a {"type":"string","avro.java.string":"String"}}}
>  \{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
>  \{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
>  \{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
>  \{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
>  \{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
>  \{{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
>  \{{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
>  \{{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
>  \{{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2366) 1.9

2019-04-01 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2366:
--
Description: 
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure if the default does not match the schema. For example, 
a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?

The exception is:

{{Caused by: org.apache.avro.AvroTypeException: Invalid default for field name: 
null not a \{"type":"string","avro.java.string":"String"}}}
{{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
{{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
{{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
{{ at 
org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}

 

  was:
Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure if the default does not match the schema. For example, 
a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?


> 1.9 
> 
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure if the default does not match the schema. For 
> example, a field definition of:
> {"name": "name", "type": "string", "default": null}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?
> The exception is:
> {{Caused by: org.apache.avro.AvroTypeException: Invalid default for field 
> name: null not a \{"type":"string","avro.java.string":"String"}}}
> {{ at org.apache.avro.Schema.validateDefault(Schema.java:1482)}}
> {{ at org.apache.avro.Schema.access$300(Schema.java:84)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:493)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:485)}}
> {{ at org.apache.avro.Schema$Field.(Schema.java:504)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:689)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.addStringType(SpecificCompiler.java:668)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compile(SpecificCompiler.java:598)}}
> {{ at 
> org.apache.avro.compiler.specific.SpecificCompiler.compileToDestination(SpecificCompiler.java:527)}}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2366) 1.9

2019-04-01 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2366:
--
Affects Version/s: 1.9.0

> 1.9 
> 
>
> Key: AVRO-2366
> URL: https://issues.apache.org/jira/browse/AVRO-2366
> Project: Apache Avro
>  Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Raman Gupta
>Priority: Major
>
> Followup to AVRO-2035.
> I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to 
> false, and still get a failure if the default does not match the schema. For 
> example, a field definition of:
> {"name": "name", "type": "string", "default": null}
> Should this definition succeed if `setValidateDefaults(false)` is called? If 
> not, should the `setValidateDefaults` method just be removed?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (AVRO-2366) 1.9

2019-04-01 Thread Raman Gupta (JIRA)
Raman Gupta created AVRO-2366:
-

 Summary: 1.9 
 Key: AVRO-2366
 URL: https://issues.apache.org/jira/browse/AVRO-2366
 Project: Apache Avro
  Issue Type: Bug
Reporter: Raman Gupta


Followup to AVRO-2035.

I am testing 1.9 SNAPSHOT. I set `Schema.Parser.setValidateDefaults` to false, 
and still get a failure if the default does not match the schema. For example, 
a field definition of:

{"name": "name", "type": "string", "default": null}

Should this definition succeed if `setValidateDefaults(false)` is called? If 
not, should the `setValidateDefaults` method just be removed?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AVRO-2360) Java 8 timestamp-millis / timestamp-micros type inconsistency

2019-03-28 Thread Raman Gupta (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804203#comment-16804203
 ] 

Raman Gupta commented on AVRO-2360:
---

Awesome, thanks for the quick resolution to both problems. Looking forward to 
the 1.9.0 release.

> Java 8 timestamp-millis / timestamp-micros type inconsistency
> -
>
> Key: AVRO-2360
> URL: https://issues.apache.org/jira/browse/AVRO-2360
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.9.0
> Environment: JDK 11.0.2 on Linux
>Reporter: Raman Gupta
>Assignee: Daniel Kulp
>Priority: Blocker
> Fix For: 1.9.0
>
>
> I am trying the new JSR 310 date/time types with a snapshot of 1.9.0.
> I see what seems to be an inconsistency. If I generate my code with a logical 
> type of "timestamp-millis", then the code is generated with `Instant`, as 
> expected. However, on JDK11 on Linux, if I do `Instant.now()` the Instant 
> value created contains microseconds. When setting this Instant on an instance 
> of the generated Avro SpecificRecord, I am unable to round-trip the data:
> Before serialization to bytes:
> System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665308Z
> After serializing to bytes, and then back into a specific record the 
> microseconds are now truncated:
> System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665Z
> I believe the compiler should generate a class that truncates the 
> microseconds at `setter` time for "timestamp-millis", so that the value 
> before serialization, and after deserialization, are the same. This can be 
> done with a call to the method `truncatedTo(ChronoUnit.MILLIS)`.
> Another, possibly related, oddity is that the "timestamp-micros" type 
> generates a class with `long` as the type of the field. Since Instant can 
> support both milli and micro-second precision, I don't see the reason for 
> this behavior.
> Followup on AVRO-2079.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2360) Java 8 timestamp-millis / timestamp-micros type inconsistency

2019-03-27 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2360:
--
Description: 
I am trying the new JSR 310 date/time types with a snapshot of 1.9.0.

I see what seems to be an inconsistency. If I generate my code with a logical 
type of "timestamp-millis", then the code is generated with `Instant`, as 
expected. However, on JDK11 on Linux, if I do `Instant.now()` the Instant value 
created contains microseconds. When setting this Instant on an instance of the 
generated Avro SpecificRecord, I am unable to round-trip the data:

Before serialization to bytes:

System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665308Z

After serializing to bytes, and then back into a specific record the 
microseconds are now truncated:

System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665Z

I believe the compiler should generate a class that truncates the microseconds 
at `setter` time for "timestamp-millis", so that the value before 
serialization, and after deserialization, are the same. This can be done with a 
call to the method `truncatedTo(ChronoUnit.MILLIS)`.

Another, possibly related, oddity is that the "timestamp-micros" type generates 
a class with `long` as the type of the field. Since Instant can support both 
milli and micro-second precision, I don't see the reason for this behavior.

Followup on AVRO-2079.

  was:
I am trying the new JSR 310 date/time types with a snapshot of 1.9.0.

I see what seems to be an inconsistency. If I generate my code with a logical 
type of "timestamp-millis", then the code is generated with `Instant`, as 
expected. However, on JDK11 on Linux, if I do `Instant.now()` the Instant value 
created contains microseconds. When setting this Instant on an instance of the 
generated Avro SpecificRecord, I am unable to round-trip the data:

Before serialization to bytes:

System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665308Z

After serializing to bytes, and then back into a specific record the 
microseconds are now truncated:

System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665Z

I believe the compiler should generate a class that truncates the microseconds 
at `setter` time for "timestamp-millis", so that the value before 
serialization, and after deserialization, are the same. This can be done with a 
call to the method `truncatedTo(ChronoUnit.MILLIS)`.

Another, possibly related, oddity is that the "timestamp-micros" type generates 
a class with `long` as the type of the field. Since Instant can support both 
milli and micro-second precision, I don't see the reason for this behavior.

 


> Java 8 timestamp-millis / timestamp-micros type inconsistency
> -
>
> Key: AVRO-2360
> URL: https://issues.apache.org/jira/browse/AVRO-2360
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.9.0
> Environment: JDK 11.0.2 on Linux
>Reporter: Raman Gupta
>Priority: Critical
>
> I am trying the new JSR 310 date/time types with a snapshot of 1.9.0.
> I see what seems to be an inconsistency. If I generate my code with a logical 
> type of "timestamp-millis", then the code is generated with `Instant`, as 
> expected. However, on JDK11 on Linux, if I do `Instant.now()` the Instant 
> value created contains microseconds. When setting this Instant on an instance 
> of the generated Avro SpecificRecord, I am unable to round-trip the data:
> Before serialization to bytes:
> System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665308Z
> After serializing to bytes, and then back into a specific record the 
> microseconds are now truncated:
> System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665Z
> I believe the compiler should generate a class that truncates the 
> microseconds at `setter` time for "timestamp-millis", so that the value 
> before serialization, and after deserialization, are the same. This can be 
> done with a call to the method `truncatedTo(ChronoUnit.MILLIS)`.
> Another, possibly related, oddity is that the "timestamp-micros" type 
> generates a class with `long` as the type of the field. Since Instant can 
> support both milli and micro-second precision, I don't see the reason for 
> this behavior.
> Followup on AVRO-2079.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AVRO-2360) Java 8 timestamp-millis / timestamp-micros type inconsistency

2019-03-27 Thread Raman Gupta (JIRA)


 [ 
https://issues.apache.org/jira/browse/AVRO-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raman Gupta updated AVRO-2360:
--
Environment: JDK 11.0.2 on Linux  (was: JDK 11.0.2 on Linux

Followup on AVRO-2079.)

> Java 8 timestamp-millis / timestamp-micros type inconsistency
> -
>
> Key: AVRO-2360
> URL: https://issues.apache.org/jira/browse/AVRO-2360
> Project: Apache Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.9.0
> Environment: JDK 11.0.2 on Linux
>Reporter: Raman Gupta
>Priority: Critical
>
> I am trying the new JSR 310 date/time types with a snapshot of 1.9.0.
> I see what seems to be an inconsistency. If I generate my code with a logical 
> type of "timestamp-millis", then the code is generated with `Instant`, as 
> expected. However, on JDK11 on Linux, if I do `Instant.now()` the Instant 
> value created contains microseconds. When setting this Instant on an instance 
> of the generated Avro SpecificRecord, I am unable to round-trip the data:
> Before serialization to bytes:
> System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665308Z
> After serializing to bytes, and then back into a specific record the 
> microseconds are now truncated:
> System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665Z
> I believe the compiler should generate a class that truncates the 
> microseconds at `setter` time for "timestamp-millis", so that the value 
> before serialization, and after deserialization, are the same. This can be 
> done with a call to the method `truncatedTo(ChronoUnit.MILLIS)`.
> Another, possibly related, oddity is that the "timestamp-micros" type 
> generates a class with `long` as the type of the field. Since Instant can 
> support both milli and micro-second precision, I don't see the reason for 
> this behavior.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (AVRO-2360) Java 8 timestamp-millis / timestamp-micros type inconsistency

2019-03-27 Thread Raman Gupta (JIRA)
Raman Gupta created AVRO-2360:
-

 Summary: Java 8 timestamp-millis / timestamp-micros type 
inconsistency
 Key: AVRO-2360
 URL: https://issues.apache.org/jira/browse/AVRO-2360
 Project: Apache Avro
  Issue Type: Bug
  Components: java
Affects Versions: 1.9.0
 Environment: JDK 11.0.2 on Linux

Followup on AVRO-2079.
Reporter: Raman Gupta


I am trying the new JSR 310 date/time types with a snapshot of 1.9.0.

I see what seems to be an inconsistency. If I generate my code with a logical 
type of "timestamp-millis", then the code is generated with `Instant`, as 
expected. However, on JDK11 on Linux, if I do `Instant.now()` the Instant value 
created contains microseconds. When setting this Instant on an instance of the 
generated Avro SpecificRecord, I am unable to round-trip the data:

Before serialization to bytes:

System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665308Z

After serializing to bytes, and then back into a specific record the 
microseconds are now truncated:

System.out.println(mySpecificRecord.tsMillis()) // 2019-03-27T23:16:09.665Z

I believe the compiler should generate a class that truncates the microseconds 
at `setter` time for "timestamp-millis", so that the value before 
serialization, and after deserialization, are the same. This can be done with a 
call to the method `truncatedTo(ChronoUnit.MILLIS)`.

Another, possibly related, oddity is that the "timestamp-micros" type generates 
a class with `long` as the type of the field. Since Instant can support both 
milli and micro-second precision, I don't see the reason for this behavior.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AVRO-2163) Release 1.8.3

2018-09-13 Thread Raman Gupta (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-2163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613638#comment-16613638
 ] 

Raman Gupta commented on AVRO-2163:
---

Not that it appears like 1.8.3 is gonna happen any time soon, BUT that being 
said, I'd like to submit AVRO-2079 for consideration. There is a pull request 
here: https://github.com/apache/avro/pull/309.

> Release 1.8.3
> -
>
> Key: AVRO-2163
> URL: https://issues.apache.org/jira/browse/AVRO-2163
> Project: Avro
>  Issue Type: Task
>  Components: community
>Affects Versions: 1.8.3
>Reporter: Sean Busbey
>Assignee: Suraj Acharya
>Priority: Major
> Fix For: 1.8.3
>
>
> Please link to any issues that should be considered blockers for the 1.8.3 
> release.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (AVRO-2210) Is The Avro Project Dead? Hopefully not.

2018-09-13 Thread Raman Gupta (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613625#comment-16613625
 ] 

Raman Gupta edited comment on AVRO-2210 at 9/13/18 3:15 PM:


It certainly appears dead-ish: https://i.imgur.com/sNkUglN.png


was (Author: rocketraman):
It certainly appears dead-ish:

!image-2018-09-13-11-14-44-791.png!

> Is The Avro Project Dead? Hopefully not.
> 
>
> Key: AVRO-2210
> URL: https://issues.apache.org/jira/browse/AVRO-2210
> Project: Avro
>  Issue Type: Wish
>Reporter: ldawson
>Priority: Major
>
> I'm just curious about it's future. It's an awesome technology that'd I'd 
> love to keep using and supporting. It's been over one year since the last 
> release, which is why I'm asking.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AVRO-2210) Is The Avro Project Dead? Hopefully not.

2018-09-13 Thread Raman Gupta (JIRA)


[ 
https://issues.apache.org/jira/browse/AVRO-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613625#comment-16613625
 ] 

Raman Gupta commented on AVRO-2210:
---

It certainly appears dead-ish:

!image-2018-09-13-11-14-44-791.png!

> Is The Avro Project Dead? Hopefully not.
> 
>
> Key: AVRO-2210
> URL: https://issues.apache.org/jira/browse/AVRO-2210
> Project: Avro
>  Issue Type: Wish
>Reporter: ldawson
>Priority: Major
>
> I'm just curious about it's future. It's an awesome technology that'd I'd 
> love to keep using and supporting. It's been over one year since the last 
> release, which is why I'm asking.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)