[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054327#comment-17054327
]
Andy Le commented on AVRO-2723:
---
[~sekikn] Thank you. I'll use the suggested name
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054334#comment-17054334
]
Andy Le commented on AVRO-2723:
---
[~rskraba] [~sekikn] Currently we have class `ReflectData.AllowNulls` declared
[at line
#175|https://github.com/apache/avro/blob/791ec6017b03e3cc79fbf546dd9d79d20d476b4e/lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectData.java#L75].
Regarding this issue, should I add implementation for
`ReflectData.AllowDefaults` (please see class DefaultReflector above) ?
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le updated AVRO-2723:
--
Attachment: Screen Shot 2020-03-08 at 16.13.29.png
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054340#comment-17054340
]
Andy Le commented on AVRO-2723:
---
[~rskraba] [~sekikn] I follow [this
guideline|https://cwiki.apache.org/confluence/display/AVRO/How+To+Contribute]
to contribute code. But when I issue commmand
{code:bash}
$ mvn javadoc:aggregate
{code}
I see various errors in other classes (not ReflectData). What should I do? Make
another PR to improve the documentation, right?
!Screen Shot 2020-03-08 at 16.13.29.png!
> Avro Java: Obtaining default values for POJO objects with ReflectData
> -
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2723) Avro Java: Obtaining default field values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Le updated AVRO-2723:
--
Summary: Avro Java: Obtaining default field values for POJO objects with
ReflectData (was: Avro Java: Obtaining default values for POJO objects with
ReflectData)
> Avro Java: Obtaining default field values for POJO objects with ReflectData
> ---
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: supporting a "unit" field for avro schema
To move this idea forward, and start exploring what an implementation of unit analysis in native Avro would really look like, I submitted a PR that implements unit analysis for python Avro schema: https://github.com/apache/avro/pull/841 I've also been playing around with a JSON-Schema that defines valid json for unit expressions and unit definitions: https://github.com/erikerlandson/unit-analysis-json-schema/blob/master/unit-analysis-schema.json The current state of the PR supports units on 'record' fields. Supporting units for 'array' also seems desirable. Important infra like unit testing and documentation would also need to be done, but I wanted to put something out there for the Avro community to examine and discuss. Cheers, Erik On Fri, Jun 28, 2019 at 2:58 PM Erik Erlandson wrote: > Hi Avro community, > > Recently I have been experimenting with avro schema that are extended with > a "unit" field. By "unit" I mean expressions like "second", or "megabyte" - > that is "units of measure". > > I delivered a short talk on my experiments at Berlin Buzzwords, which can > be viewed here: > https://www.youtube.com/watch?v=qrQmB2KFKE8 > I also wrote a short blog post that may be faster to ingest: > > http://erikerlandson.github.io/blog/2019/05/23/unit-types-for-avro-schema-integrating-avro-with-coulomb/ > > I received some audience interest in making this concept "first class" for > avro, and so I'm writing to see what the avro dev community thinks of the > idea. One issue is that this kind of unit checking is currently only > available for Scala (and specifically scala 2.13 +). > > The Scala project itself is here: > https://github.com/erikerlandson/coulomb > > Cheers, > Erik > >
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default field values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054576#comment-17054576
]
Kengo Seki commented on AVRO-2723:
--
bq. should I add implementation for `ReflectData.AllowDefaults` (please see
class DefaultReflector above) ?
That sounds good to me. But I'm not sure if {{AllowDefaults}} is a suitable
name, because users can also specify default values using the {{AvroDefault}}
annotations instead of that class. So {{UseInitialValueAsDefault}} or something
might be more precise?
bq. Make another PR to improve the documentation, right?
Yes please, that's very helpful! And before that, would you file a
corresponding JIRA issue and explain how to reproduce those errors first?
Because I didn't see them when I ran {{mvn javadoc:aggregate}} in a Docker
container, though many warnings were shown as follows. Maybe you're using Java
11?
{code}
~/avro$ ./build.sh docker
(snip)
sekikn@07b11f942aed:~/avro$ mvn javadoc:aggregate
(snip)
48 warnings
[WARNING] Javadoc Warnings
[WARNING]
/home/sekikn/avro/lang/java/avro/src/main/java/org/apache/avro/message/BinaryMessageEncoder.java:54:
warning - Tag @link: can't find encode(D) in
org.apache.avro.message.BinaryMessageEncoder
[WARNING]
/home/sekikn/avro/lang/java/avro/src/main/java/org/apache/avro/message/BinaryMessageEncoder.java:76:
warning - Tag @link: can't find encode(D) in
org.apache.avro.message.BinaryMessageEncoder
[WARNING]
/home/sekikn/avro/lang/java/avro/src/main/java/org/apache/avro/message/RawMessageEncoder.java:58:
warning - Tag @link: can't find encode(D) in
org.apache.avro.message.RawMessageEncoder
[WARNING]
/home/sekikn/avro/lang/java/avro/src/main/java/org/apache/avro/message/RawMessageEncoder.java:80:
warning - Tag @link: can't find encode(D) in
org.apache.avro.message.RawMessageEncoder
[WARNING]
/home/sekikn/avro/lang/java/avro/src/main/java/org/apache/avro/reflect/MapEntry.java:34:
warning - Tag @link: reference not found: Map.Entry
[WARNING]
/home/sekikn/avro/lang/java/compiler/src/main/java/org/apache/avro/compiler/schema/SchemaVisitor.java:32:
warning - @return tag has no arguments.
[WARNING]
/home/sekikn/avro/lang/java/compiler/src/main/java/org/apache/avro/compiler/schema/SchemaVisitor.java:40:
warning - @return tag has no arguments.
[WARNING]
/home/sekikn/avro/lang/java/compiler/src/main/java/org/apache/avro/compiler/schema/SchemaVisitor.java:48:
warning - @return tag has no arguments.
[WARNING]
/home/sekikn/avro/lang/java/compiler/src/main/java/org/apache/avro/compiler/specific/SpecificCompiler.java:813:
warning - @Deprecated is an unknown tag -- same as a known tag except for case.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:86:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:94:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:102:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:110:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:119:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:128:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:137:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:146:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:154:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:164:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:171:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:180:
warning - @parameter is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:180:
warning - @readonly is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/AbstractAvroMojo.java:180:
warning - @required is an unknown tag.
[WARNING]
/home/sekikn/avro/lang/java/maven-plugin/target/generated-sources/plu
[jira] [Commented] (AVRO-2723) Avro Java: Obtaining default field values for POJO objects with ReflectData
[
https://issues.apache.org/jira/browse/AVRO-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054694#comment-17054694
]
Andy Le commented on AVRO-2723:
---
[~sekikn]
> So UseInitialValueAsDefault or something might be more precise?
Fine for me
> didn't see them when I ran mvn javadoc:aggregate in a Docker container,
> though many warnings were shown as follows. Maybe you're using Java 11?...
My fault. I directly run the command without being inside the Docker
environment.
> Avro Java: Obtaining default field values for POJO objects with ReflectData
> ---
>
> Key: AVRO-2723
> URL: https://issues.apache.org/jira/browse/AVRO-2723
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
>Affects Versions: 1.9.1
>Reporter: Andy Le
>Priority: Critical
> Attachments: Screen Shot 2020-03-08 at 16.13.29.png
>
>
> Hi guys,
>
> I've got a simple app using Avro Reflection:
>
> {code:java}
> public class App {
> public static void main(String[] args) {
> testReflection();
> }
> static class User {
> public String first = "Andy";
> public String last = "Le";
> }
> static void testReflection(){
> // get the reflected schema for packets
> Schema schema = ReflectData.AllowNull.get().getSchema(User.class);
> System.out.println(schema.toString(true));
> }
> {code}
> The output on console will be:
> {noformat}
> {
> "type" : "record",
> "name" : "User",
> "namespace" : "App",
> "fields" : [ {
> "name" : "first",
> "type" : [ "null", "string" ],
> "default" : null
> }, {
> "name" : "last",
> "type" : [ "null", "string" ],
> "default" : null
> } ]
> }
> {noformat}
>
> As you can see, there's no default values for fields. Would you please tell
> me how to obtain such values?
> Thank you.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
