[jira] [Commented] (AVRO-1340) use default to allow old readers to specify default enum value when encountering new enum symbols
[ https://issues.apache.org/jira/browse/AVRO-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457331#comment-16457331 ] ASF GitHub Bot commented on AVRO-1340: -- bellemare commented on issue #298: AVRO-1340: Added Enum Defaults and unit tests. URL: https://github.com/apache/avro/pull/298#issuecomment-385135011 One more quick statement - I noticed that there is next to no testing around the usage of the "@default" notation. I know that my changes work only because I cannot use the "@default" notation on my enums anymore. However, I do not have any tests to prove that. Should this also be part of this JIRA? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > use default to allow old readers to specify default enum value when > encountering new enum symbols > - > > Key: AVRO-1340 > URL: https://issues.apache.org/jira/browse/AVRO-1340 > Project: Avro > Issue Type: Improvement > Components: spec > Environment: N/A >Reporter: Jim Donofrio >Priority: Minor > > The schema resolution page says: > > if both are enums: > > if the writer's symbol is not present in the reader's enum, then an > error is signalled. > This makes it difficult to use enum's because you can never add a enum value > and keep old reader's compatible. Why not use the default option to refer to > one of enum values so that when a old reader encounters a enum ordinal it > does not recognize, it can default to the optional schema provided one. If > the old schema does not provide a default then the older reader can continue > to fail as it does today. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1340) use default to allow old readers to specify default enum value when encountering new enum symbols
[
https://issues.apache.org/jira/browse/AVRO-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457330#comment-16457330
]
ASF GitHub Bot commented on AVRO-1340:
--
bellemare commented on a change in pull request #298: AVRO-1340: Added Enum
Defaults and unit tests.
URL: https://github.com/apache/avro/pull/298#discussion_r184837513
##
File path: lang/java/avro/src/main/java/org/apache/avro/Schema.java
##
@@ -1354,9 +1359,14 @@ static Schema parse(JsonNode schema, Names names) {
} else
throw new SchemaParseException("Type not supported: "+type);
Iterator i = schema.getFieldNames();
+
+ Set reserved = SCHEMA_RESERVED;
+ if (type.equals("enum")) {
+reserved = ENUM_RESERVED;
+ }
Review comment:
The intention here was to not cause any unintended side effects.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> use default to allow old readers to specify default enum value when
> encountering new enum symbols
> -
>
> Key: AVRO-1340
> URL: https://issues.apache.org/jira/browse/AVRO-1340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Environment: N/A
>Reporter: Jim Donofrio
>Priority: Minor
>
> The schema resolution page says:
> > if both are enums:
> > if the writer's symbol is not present in the reader's enum, then an
> error is signalled.
> This makes it difficult to use enum's because you can never add a enum value
> and keep old reader's compatible. Why not use the default option to refer to
> one of enum values so that when a old reader encounters a enum ordinal it
> does not recognize, it can default to the optional schema provided one. If
> the old schema does not provide a default then the older reader can continue
> to fail as it does today.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-1340) use default to allow old readers to specify default enum value when encountering new enum symbols
[
https://issues.apache.org/jira/browse/AVRO-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457328#comment-16457328
]
ASF GitHub Bot commented on AVRO-1340:
--
bellemare commented on a change in pull request #298: AVRO-1340: Added Enum
Defaults and unit tests.
URL: https://github.com/apache/avro/pull/298#discussion_r184837417
##
File path: lang/java/avro/src/test/java/org/apache/avro/TestSchemaBuilder.java
##
@@ -561,6 +561,17 @@ public void testEnum() {
Assert.assertEquals(expected, schema);
}
+ @Test
+ public void testEnumWithDefault() {
+List symbols = Arrays.asList("a", "b");
+String enumDefault = "a";
+Schema expected = Schema.createEnum("myenum", null, null, symbols,
enumDefault);
+expected.addProp("p", "v");
+Schema schema = SchemaBuilder.enumeration("myenum")
+ .prop("p", "v").defaultSymbol(enumDefault).symbols("a", "b");
Review comment:
So this particular code decision means that the default must be set before
calling .symbols(). It will not work the other way around.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> use default to allow old readers to specify default enum value when
> encountering new enum symbols
> -
>
> Key: AVRO-1340
> URL: https://issues.apache.org/jira/browse/AVRO-1340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Environment: N/A
>Reporter: Jim Donofrio
>Priority: Minor
>
> The schema resolution page says:
> > if both are enums:
> > if the writer's symbol is not present in the reader's enum, then an
> error is signalled.
> This makes it difficult to use enum's because you can never add a enum value
> and keep old reader's compatible. Why not use the default option to refer to
> one of enum values so that when a old reader encounters a enum ordinal it
> does not recognize, it can default to the optional schema provided one. If
> the old schema does not provide a default then the older reader can continue
> to fail as it does today.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-1340) use default to allow old readers to specify default enum value when encountering new enum symbols
[ https://issues.apache.org/jira/browse/AVRO-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457326#comment-16457326 ] Adam Bellemare commented on AVRO-1340: -- Okay, I put up a new commit with the changes mentioned above. Please take a look and let me know about the changes made, especially around the definition of the ENUM_RESERVED component. > use default to allow old readers to specify default enum value when > encountering new enum symbols > - > > Key: AVRO-1340 > URL: https://issues.apache.org/jira/browse/AVRO-1340 > Project: Avro > Issue Type: Improvement > Components: spec > Environment: N/A >Reporter: Jim Donofrio >Priority: Minor > > The schema resolution page says: > > if both are enums: > > if the writer's symbol is not present in the reader's enum, then an > error is signalled. > This makes it difficult to use enum's because you can never add a enum value > and keep old reader's compatible. Why not use the default option to refer to > one of enum values so that when a old reader encounters a enum ordinal it > does not recognize, it can default to the optional schema provided one. If > the old schema does not provide a default then the older reader can continue > to fail as it does today. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2112) c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies
[ https://issues.apache.org/jira/browse/AVRO-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457225#comment-16457225 ] ASF GitHub Bot commented on AVRO-2112: -- blachniet commented on issue #307: AVRO-2112: Target .NET Standard/Core in C# URL: https://github.com/apache/avro/pull/307#issuecomment-385120407 Hey @sidshetye, I got the same results on my machines (both Windows and Linux) when working on this. The `dotnet` CLI attempts to build using all frameworks listed in the `.csproj`. However, you can override this with the `--framework` argument. When using `dotnet test` on either Windows or Linux, you should provide the `--framework netcoreapp2.0` argument (mentioned in the [README here](https://github.com/apache/avro/pull/307/files#diff-65d636700bfdf9f78c00d3fd22734c61R20)). For more details on this error message, see [this issue in the MSBuild project](https://github.com/Microsoft/msbuild/issues/1333#issuecomment-382332184). In short, it seems that there is no plans to add support for targeting earlier versions of the .NET Framework (3.5, 4.0) from the `dotnet` CLI. ### Side Note In this pull request I aimed for additive changes only. That's why I did not deprecate support for .NET Framework 3.5. That being said, I'm happy to deprecate support for earlier versions of the .NET Framework in this project if the rest of the community is happy with that. I'm perfectly happy to target .NET Standard 2.0 (.NET Framework 4.6.1+ according to [the chart here](https://docs.microsoft.com/en-us/dotnet/standard/net-standard#net-implementation-support)). I'm not sure who makes that kind of decision on this project. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies > > > Key: AVRO-2112 > URL: https://issues.apache.org/jira/browse/AVRO-2112 > Project: Avro > Issue Type: Improvement > Components: csharp > Environment: - Visual Studio For Mac > - Visual Studio 2017 > - mono 5.4.1.7 MacOSX > - dotnet 2.0 (MacOSX and Windows 10) >Reporter: Miljenko Cvjetko >Assignee: Miljenko Cvjetko >Priority: Minor > Original Estimate: 0h > Remaining Estimate: 0h > > Suugestion is to add .NET Standard/Core support. > In order to support modern/new .NET (both standard netfx and dotnet core) it > is necessary to convert projects to support .NET Standard Libraries. > - conversion to .NET Standard [DONE] > - added dotnet core sample (Avro.perf) [DONE] > - added netfx (standrd .NET) sample Avro.perf.netfx [DONE] > - unit testing updated to use NUnit 3 > Url for the github forked repo with branch will be added. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2172) Avro over gRPC (Java Implementation)
Srujan Narkedamalli created AVRO-2172: - Summary: Avro over gRPC (Java Implementation) Key: AVRO-2172 URL: https://issues.apache.org/jira/browse/AVRO-2172 Project: Avro Issue Type: New Feature Components: java Reporter: Srujan Narkedamalli We (wavefront/VMware) have implemented a java library/module for using RPC defined using Avro (.avdl/.avpr) over gRPC, and would like to contribute this to Apache Avro. gRPC provides ability to build streaming RPC and also in terms of Java implementation its built on recent version of Netty (4.x). Overview of our Avro-gRPC Java Implementation: gRPC by default provides support for Protobuf IDL and the APIs are tuned towards it. Following are main differences in gRPC API/Protobuf and Avro's RPC IDL that our library bridges: * Protobuf/gRPC supports only single argument for RPC request vs Avro's multiple arguments. This is handled in serialization logic. * Protobuf/gRPC does not support throwing typed exceptions over wire natively. Avro’s typed RPC exceptions are handled in the serialization logic. * Protobuf/gRPC does not support one-way RPC. We don’t avoid doing round-trip to server but respond back with null response as soon as request is received at server, and then invoke the server implementation. * gRPC provides RPC code generation for Protobuf. For Avro, we use the current minimal code generation, and provide Client and Server Invocation handlers. Can we have this as a new artifact/maven child project in Avro Java project with a name something like `avro-grpc` ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2112) c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies
[ https://issues.apache.org/jira/browse/AVRO-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456957#comment-16456957 ] ASF GitHub Bot commented on AVRO-2112: -- sidshetye commented on issue #307: AVRO-2112: Target .NET Standard/Core in C# URL: https://github.com/apache/avro/pull/307#issuecomment-385073516 hey @blachniet - changes look fantastic. But when running `dotnet test` we're seeing some issues related with `net35;net40;netcoreapp2.0` inside `Avro.test.csproj`. Basically `net35` doesn't exist on newer Windows systems and `net35` and `net40` don't exist on *nix platforms. I'm not sure if this is a limitation of current tools in .net and I don't have any definitive best path. What do you think of having `netcoreapp2.0` as the default or pass target framework over the `dotnet` CLI (if it's even possible)? Details: New windows machines lacking .net35 ``` PS D:\projects\github\avro\lang\csharp\src\apache\test> dotnet test Build started, please wait... C:\Program Files\dotnet\sdk\2.1.104\Microsoft.Common.CurrentVersion.targets(1126,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v3.5" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [D:\projects\github\avro\lang\csharp\src\apache\test\Avro.test.csproj] ``` Linux, running `dotnet test` we get ``` user@linux:/mnt/d/projects/github/avro/lang/csharp/src/apache/test$ dotnet test Build started, please wait... /usr/share/dotnet/sdk/2.1.101/Microsoft.Common.CurrentVersion.targets(1126,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.0" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/mnt/d/projects/github/avro/lang/csharp/src/apache/test/Avro.test.csproj] ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies > > > Key: AVRO-2112 > URL: https://issues.apache.org/jira/browse/AVRO-2112 > Project: Avro > Issue Type: Improvement > Components: csharp > Environment: - Visual Studio For Mac > - Visual Studio 2017 > - mono 5.4.1.7 MacOSX > - dotnet 2.0 (MacOSX and Windows 10) >Reporter: Miljenko Cvjetko >Assignee: Miljenko Cvjetko >Priority: Minor > Original Estimate: 0h > Remaining Estimate: 0h > > Suugestion is to add .NET Standard/Core support. > In order to support modern/new .NET (both standard netfx and dotnet core) it > is necessary to convert projects to support .NET Standard Libraries. > - conversion to .NET Standard [DONE] > - added dotnet core sample (Avro.perf) [DONE] > - added netfx (standrd .NET) sample Avro.perf.netfx [DONE] > - unit testing updated to use NUnit 3 > Url for the github forked repo with branch will be added. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
