[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[ https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547243#comment-16547243 ] Ryan Blue commented on AVRO-1927: - Sounds like the problem here is actually that there is no validation in the avro-specific builders or in object creation. Avro is doing the right thing and not allowing you to serialize invalid data. I think it makes sense to add null checks to the generated builders. Feel free to fix this and open a PR! > If a default value is set, Avro allows null values in non-nullable fields. > -- > > Key: AVRO-1927 > URL: https://issues.apache.org/jira/browse/AVRO-1927 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.1 >Reporter: Andreas Maier >Priority: Major > > With an avro schema like > {code} > { > "name": "myfield", > "type": "string", > "default": "" > } > {code} > the following code should throw an exception > {code} > MyObject myObject = MyObject.newBuilder().setMyfield(null).build(); > {code} > But instead the value of myfield is set to null, which causes an exception > later when serializing myObject, because null is not a valid value for > myfield. > I believe in this case setMyfield(null) should throw an exception, > independent of the value of default. > See also > https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[ https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527320#comment-16527320 ] Mykola Polonskyi commented on AVRO-1927: hello (bump message) *same behavior from builder on 1.8.2 and same error onto serialization call* can any one share reference for spec that explain this behavior if its valid? or clarify statusĀ tnx > If a default value is set, Avro allows null values in non-nullable fields. > -- > > Key: AVRO-1927 > URL: https://issues.apache.org/jira/browse/AVRO-1927 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.1 >Reporter: Andreas Maier >Priority: Major > > With an avro schema like > {code} > { > "name": "myfield", > "type": "string", > "default": "" > } > {code} > the following code should throw an exception > {code} > MyObject myObject = MyObject.newBuilder().setMyfield(null).build(); > {code} > But instead the value of myfield is set to null, which causes an exception > later when serializing myObject, because null is not a valid value for > myfield. > I believe in this case setMyfield(null) should throw an exception, > independent of the value of default. > See also > https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[ https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568879#comment-15568879 ] Andreas Maier commented on AVRO-1927: - So I had a look at the generated Java code. And strangely enough, this code only throws an exception, if no default value is set: {code} protected void validate(Field field, Object value) { if(!isValidValue(field, value)) { if(field.defaultValue() == null) { // why this check? throw new AvroRuntimeException("Field " + field + " does not accept null values"); } } } {code} I don't understand why Avro checks, if {{if(field.defaultValue() == null)}} before throwing an exception. In my opinion it should always throw an exception if the field value is invalid. > If a default value is set, Avro allows null values in non-nullable fields. > -- > > Key: AVRO-1927 > URL: https://issues.apache.org/jira/browse/AVRO-1927 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.1 >Reporter: Andreas Maier > Labels: newbie > > With an avro schema like > {code} > { > "name": "myfield", > "type": "string", > "default": "" > } > {code} > the following code should throw an exception > {code} > MyObject myObject = MyObject.newBuilder().setMyfield(null).build(); > {code} > But instead the value of myfield is set to null, which causes an exception > later when serializing myObject, because null is not a valid value for > myfield. > I believe in this case setMyfield(null) should throw an exception, > independent of the value of default. > See also > https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[ https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568867#comment-15568867 ] Andreas Maier commented on AVRO-1927: - Sorry for the long delay. I was on vacation. I tried the code you suggested. You are right, it does throw a NullPointerException when I try to write the object: {code} java.lang.NullPointerException: null of string of de.am.MyObject at org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:132) at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:126) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60) at de.am.MyObjectTest.myObjectTest2(MyObjectTest.java:27) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) Caused by: java.lang.NullPointerException at org.apache.avro.specific.SpecificDatumWriter.writeString(SpecificDatumWriter.java:67) at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:115) at org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87) at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143) at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105) ... 30 more {code} > If a default value is set, Avro allows null values in non-nullable fields. > -- > > Key: AVRO-1927 > URL: https://issues.apache.org/jira/browse/AVRO-1927 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.1 >Reporter: Andreas Maier > Labels: newbie > > With an avro schema like > {code} > { > "name": "myfield", > "type": "string", > "default": "" > } > {code} > the following code should throw an exception > {code} > MyObject myObject = MyObject.newBuilder().setMyfield(null).build(); > {code} > But instead the value of myfield is set to null, which causes an exception > later when serializing myObject, because null is not a valid value for > myfield. > I believe in this case setMyfield(null) should throw an exception, > independent of the value of default. > See also > https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.
[ https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522012#comment-15522012 ] Yibing Shi commented on AVRO-1927: -- [~asmaier], currently the generated code doesn't check whether we are setting a null value for a non nullable column. In your case, null is a valid Java String value and thus it works. However, I believe AVRO would throw an exception when you try to write your object. Have you tried that? Does it fail or succeed? {code} ByteArrayOutputStream os = new ByteArrayOutputStream(1024); Encoder encoder = EncoderFactory.get().binaryEncoder(os, null); SpecificDatumWriter writer = new SpecificDatumWriter(MyObject.getClassSchema()); writer.write(myObject, encoder); {code} > If a default value is set, Avro allows null values in non-nullable fields. > -- > > Key: AVRO-1927 > URL: https://issues.apache.org/jira/browse/AVRO-1927 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.1 >Reporter: Andreas Maier > Labels: newbie > > With an avro schema like > {code} > { > "name": "myfield", > "type": "string", > "default": "" > } > {code} > the following code should throw an exception > {code} > MyObject myObject = MyObject.newBuilder().setMyfield(null).build(); > {code} > But instead the value of myfield is set to null, which causes an exception > later when serializing myObject, because null is not a valid value for > myfield. > I believe in this case setMyfield(null) should throw an exception, > independent of the value of default. > See also > https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default -- This message was sent by Atlassian JIRA (v6.3.4#6332)