Re: Avro schema evolution support in AVRO CPP

2024-01-12 Thread John McClean
psetrading.com/> >> +852 2108 7352 >> >> Follow us today on our online platforms >> [Facebook]<https://www.facebook.com/eclipsetrading/>[Linked-In]< >> https://www.linkedin.com/company/eclipse-trading>[Instagram]< >> https://www.instagram.com/ecli

Re: Avro schema evolution support in AVRO CPP

2024-01-12 Thread Andrew Marlow
"In practice, it is very rare for schema evolution to change the order of the fields." - I'll say. Since we are talking about a protocol that is deliberately not self-describing we cannot just pluck out what we want - how would such code get to it? This is why the standard

Re: Avro schema evolution support in AVRO CPP

2024-01-12 Thread Thiruvalluvan MG
y the decoder. Buffering can take a large number of allocations if the out-of-order fields is an array or map. In practice, it is very rare for schema evolution to change the order of the fields. Thanks ThiruOn Friday, 12 January, 2024 at 03:24:11 pm IST, Martin Grigorov wrote: Hi Vivek,

Re: Avro schema evolution support in AVRO CPP

2024-01-12 Thread Martin Grigorov
gt; https://www.instagram.com/eclipsetrading> > > From: Vivek Kumar > Sent: Thursday, January 11, 2024 11:07 AM > To: u...@avro.apache.org > Subject: Avro schema evolution support in AVRO CPP > > Hi Avro team, > > I am writing this e

Re: Avro schema evolution support in AVRO CPP

2024-01-10 Thread Vivek Kumar
ebook.com/eclipsetrading/>[Linked-In]<https://www.linkedin.com/company/eclipse-trading>[Instagram]<https://www.instagram.com/eclipsetrading> From: Vivek Kumar Sent: Thursday, January 11, 2024 11:07 AM To: u...@avro.apache.org Subject: Avro schema evolu

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-07-29 Thread ASF GitHub Bot (Jira)
he "rerun jobs" button disappeared! Note that the 1.11.1 RC is out, after that release, we should be WAY more reactive to attack all of these contributions! Issue Time Tracking --- Worklog Id: (was: 796382) Time Spent: 5.5h (was: 5h 20m) >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-07-29 Thread ASF GitHub Bot (Jira)
Time Spent: 5h 20m (was: 5h 10m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Project: Apache Avro >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-07-29 Thread ASF GitHub Bot (Jira)
Bot Created on: 29/Jul/22 13:07 Start Date: 29/Jul/22 13:07 Worklog Time Spent: 10m Work Description: RyanSkraba closed pull request #1584: AVRO-3408: Schema evolution with logical types URL: https://github.com/apache/avro/pull/1584 Issue Time Trac

[GitHub] [avro] RyanSkraba closed pull request #1584: AVRO-3408: Schema evolution with logical types

2022-07-29 Thread GitBox
RyanSkraba closed pull request #1584: AVRO-3408: Schema evolution with logical types URL: https://github.com/apache/avro/pull/1584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-07-28 Thread ASF GitHub Bot (Jira)
ssue Time Tracking --- Worklog Id: (was: 796193) Time Spent: 5h (was: 4h 50m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-34

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-05-15 Thread ASF GitHub Bot (Jira)
Time Spent: 4h 40m (was: 4.5h) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Project: Apache Avro >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-05-15 Thread ASF GitHub Bot (Jira)
king --- Worklog Id: (was: 770668) Time Spent: 4h 50m (was: 4h 40m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 &g

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-05-15 Thread ASF GitHub Bot (Jira)
Bot Created on: 16/May/22 06:06 Start Date: 16/May/22 06:06 Worklog Time Spent: 10m Work Description: martin-g closed pull request #1584: AVRO-3408: Schema evolution with logical types URL: https://github.com/apache/avro/pull/1584 Issue Time Trac

[GitHub] [avro] izemlyanskiy opened a new pull request, #1584: AVRO-3408: Schema evolution with logical types

2022-05-15 Thread GitBox
izemlyanskiy opened a new pull request, #1584: URL: https://github.com/apache/avro/pull/1584 Hello! Please take a look a PR for the issue https://issues.apache.org/jira/browse/AVRO-3408 I hope I fulfilled all PR requirements but feel free to point me out if I missed something.

[GitHub] [avro] martin-g closed pull request #1584: AVRO-3408: Schema evolution with logical types

2022-05-15 Thread GitBox
martin-g closed pull request #1584: AVRO-3408: Schema evolution with logical types URL: https://github.com/apache/avro/pull/1584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: AVRO-3408 Schema evolution with logical types

2022-04-04 Thread Ivan Zemlyanskiy
Hello! Sorry, I have troubles in order to subscribe to dev@avro.apache.org. .. I apologize if my answer will not fall to the thread. (if someone could help me to sign up on https://lists.apache.org/list.html?dev@avro.apache.org I'd be more than grateful) The change introduces a new representation,

[jira] [Comment Edited] (AVRO-3408) Schema evolution with logical types

2022-04-04 Thread Ivan Zemlyanskiy (Jira)
conversion for the same field should lead us to a valid result if conversion functions has no bugs. I took a look on [the spec part about logical types|https://avro.apache.org/docs/current/spec.html#Logical+Types] and I haven't found any word about conversions... IMHO, we have to put

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-04-04 Thread Ivan Zemlyanskiy (Jira)
prod environment. I'll keep you updated how it goes. Wish me luck! =)  > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Project: Apache

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-04-04 Thread Ivan Zemlyanskiy (Jira)
o a valid result if conversion functions has no bugs. I took a look on [the spec part about logical types|https://avro.apache.org/docs/current/spec.html#Logical+Types] and I haven't found any word about conversions... IMHO, we have to put those conversion limitations li

Re: AVRO-3408 Schema evolution with logical types

2022-04-04 Thread Ryan Skraba
Hello! I commented on the JIRA too, but I think I've clarified some ideas in my head. Before logical types, one way to use BigDecimal values in Java was to take advantage of the "Stringable" types in ReflectData. This would add a java-class annotation to the schema, and the data would be seriali

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-31 Thread ASF GitHub Bot (Jira)
apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 751000) Time Spent: 4h 20m (was: 4h 10m) > Schema evolution with logical types >

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-31 Thread Oscar Westra van Holthe - Kind (Jira)
olution is not guaranteed to always work the same across all languages{_}.   So the question is: do we want to allow more flexibility, or do we want to guarantee (a more limited form of) schema resolution? > Schema evolution with logical types > > >

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-31 Thread Oscar Westra van Holthe - Kind (Jira)
IMHO, logical types are only related to schema evolution. The way I understand reading Avro data is this: * The write schema describes the bytes in a binary format * The read schema describes the values to read from the data (with defaults if data is missing) * The reader object: ** reads the

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-30 Thread Micah Kornfield (Jira)
(i.e. are we saying the Java implementation really is the specification)? > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-30 Thread ASF GitHub Bot (Jira)
above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 750597) Time Spent: 4h 10m (was: 4h) >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-30 Thread ASF GitHub Bot (Jira)
rg Issue Time Tracking --- Worklog Id: (was: 750543) Time Spent: 4h (was: 3h 50m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-30 Thread ASF GitHub Bot (Jira)
che.org Issue Time Tracking --- Worklog Id: (was: 749923) Time Spent: 3h 50m (was: 3h 40m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/br

[jira] [Assigned] (AVRO-3408) Schema evolution with logical types

2022-03-30 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/AVRO-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-3408: - Assignee: Ivan Zemlyanskiy > Schema evolution with logical ty

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-30 Thread Ryan Skraba (Jira)
27;s a change to the LogicalType to interpret STRING directly as decimal character digits, I think this needs to go to 1.12.0 ! > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issu

Re: AVRO-3408 Schema evolution with logical types

2022-03-30 Thread Oscar Westra van Holthe - Kind
Hi all, As I understand, the string format is the same as the toString() representation of a BigDecimal. Thus, it has: - at least [scale] and at most [precision] digits (base 10) - the decimal separator is a dot, placed [scale] digits before the end - there is no thousands separator As for compat

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-29 Thread Micah Kornfield (Jira)
recision or scale and is >therefor not equal" > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Project: Apache Avro >

Re: AVRO-3408 Schema evolution with logical types

2022-03-29 Thread Micah Kornfield
Hi Ivan, Thank you for contributing. > that my changes requires also change specification for BigDecimal > to accept string > as an underlying format for encoding/decoding. Could you elaborate on what the string representation would be fo

AVRO-3408 Schema evolution with logical types

2022-03-29 Thread Ivan Zemlyanskiy
Hello, avro developers! First of all thank you all for your hard work and the project you move forward. The result is marvellous and it was my pleasure to make a small adjustment for it. Some time ago I opened an issue https://issues.apache.org/jira/browse/AVRO-3408 and made a pull-request with my

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-25 Thread Ivan Zemlyanskiy (Jira)
's some other way?   Thanks!  > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Project: Apache Avro > Issue Type: Improv

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-24 Thread ASF GitHub Bot (Jira)
ue Time Tracking --- Worklog Id: (was: 747364) Time Spent: 3h 40m (was: 3.5h) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-21 Thread ASF GitHub Bot (Jira)
ries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 745132) Time Spent: 3.5h (was: 3h 20m) > Schema evolution with logical types > > >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-21 Thread ASF GitHub Bot (Jira)
unsubscribe, e-mail: issues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 745045) Time Spent: 3h 20m (was: 3h 10m) > Schema evolut

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-18 Thread ASF GitHub Bot (Jira)
mail: issues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744005) Time Spent: 3h 10m (was: 3h) > Schema evolution wit

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-18 Thread ASF GitHub Bot (Jira)
ues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 743824) Time Spent: 3h (was: 2h 50m) > Schema evolution wit

[jira] [Updated] (AVRO-3408) Schema evolution with logical types

2022-03-17 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/AVRO-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3408: -- Fix Version/s: 1.12.0 > Schema evolution with logical ty

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-17 Thread Ryan Skraba (Jira)
[specification|https://avro.apache.org/docs/current/spec.html#Decimal] (which is probably something that shouldn't be included in the next minor release). To be honest, the best approach (in Java) might be to just hard-code the conversion of Stringable BigDecimal to logicalType BigDecimal w

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-17 Thread ASF GitHub Bot (Jira)
mail: issues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 743270) Time Spent: 2h 50m (was: 2h 40m) > Schema evolution wit

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-17 Thread ASF GitHub Bot (Jira)
ues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 743269) Time Spent: 2h 40m (was: 2.5h) > Schema evolution wit

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-16 Thread ASF GitHub Bot (Jira)
: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 742826) Time Spent: 2.5h (was: 2h 20m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://iss

[jira] [Updated] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread Oscar Westra van Holthe - Kind (Jira)
[ https://issues.apache.org/jira/browse/AVRO-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar Westra van Holthe - Kind updated AVRO-3408: - Fix Version/s: (was: 1.11.1) > Schema evolution with logi

[jira] [Updated] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread Oscar Westra van Holthe - Kind (Jira)
[ https://issues.apache.org/jira/browse/AVRO-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar Westra van Holthe - Kind updated AVRO-3408: - Fix Version/s: 1.11.1 > Schema evolution with logical ty

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
o.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 740230) Time Spent: 2h 20m (was: 2h 10m) > Schema evolution with logical types > -

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
the specific comment. To unsubscribe, e-mail: issues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 740202) Time Spent: 2h

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
ervice, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 740194) Time Spent: 2h (was: 1h 50m) > Schema evolution with logical types > > > Key: AVRO-3408 >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 740193) Time Spent: 1h 50m (w

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
ease contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 740189) Time Spent: 1h 40m (was: 1.5h) > Schema evolution with logical types > > > Key: AVRO-3408 >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
pent: 1.5h (was: 1h 20m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Project: Apache Avro > Issue Type: Improvem

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
740173) Time Spent: 1h 20m (was: 1h 10m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Project: Apache Avro > Issue Ty

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
rg Issue Time Tracking --- Worklog Id: (was: 740172) Time Spent: 1h 10m (was: 1h) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
ail: issues-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 740171) Time Spent: 1h (was: 50m) > Schema evolution with logical types > ---

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
d: (was: 740114) Time Spent: 50m (was: 40m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Project: Apache Avro >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
Tracking --- Worklog Id: (was: 740113) Time Spent: 40m (was: 0.5h) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-34

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
tact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 740112) Time Spent: 0.5h (was: 20m) > Schema evolution with logical types > > > Key: AVRO-3408 >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-11 Thread ASF GitHub Bot (Jira)
us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 740109) Time Spent: 20m (was: 10m) > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-34

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-08 Thread Ivan Zemlyanskiy (Jira)
we got {code:java} new BigDecimal("138474692586.50161"); {code} without any warnings,exceptions, etc. That was a remarkable day, I dare say =) No blame for Avro, of course that was our mistake and we paid our price for that. But I thought I got to do my best t

[jira] (AVRO-3408) Schema evolution with logical types

2022-03-08 Thread Ivan Zemlyanskiy (Jira)
I described above is the best luck in this scenario. Imagine this: > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 > Pr

[jira] [Commented] (AVRO-3408) Schema evolution with logical types

2022-03-08 Thread Ivan Zemlyanskiy (Jira)
this type of migration or just accidental type mismatch: the exception as I described above is the best luck in this scenario. Imagine this: > Schema evolution with logical types > > > Key: AVRO-3408 >

[jira] [Work logged] (AVRO-3408) Schema evolution with logical types

2022-03-07 Thread ASF GitHub Bot (Jira)
g Id: (was: 737841) Remaining Estimate: 0h Time Spent: 10m > Schema evolution with logical types > > > Key: AVRO-3408 > URL: https://issues.apache.org/jira/browse/AVRO-3408 >

[jira] [Updated] (AVRO-3408) Schema evolution with logical types

2022-03-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AVRO-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated AVRO-3408: - Labels: pull-request-available (was: ) > Schema evolution with logical ty

[GitHub] [avro] izemlyanskiy opened a new pull request #1584: AVRO-3408: Schema evolution with logical types

2022-03-07 Thread GitBox
izemlyanskiy opened a new pull request #1584: URL: https://github.com/apache/avro/pull/1584 Hello! Please take a look a PR for the issue https://issues.apache.org/jira/browse/AVRO-3408 I hope I fulfilled all PR requirements but feel free to point me out if I missed something.

[jira] [Updated] (AVRO-3408) Schema evolution with logical types

2022-02-18 Thread Ivan Zemlyanskiy (Jira)
pinion, `org.apache.avro.io.ResolvingDecoder` should respect logical types in _SERVER_ (_ACTUAL_) schema and use a corresponding conversion instance for reading values. In my example, I'd say it might be {code} ResolvingDecoder#readString() -> read the actual logical type -> fin

[jira] [Updated] (AVRO-3408) Schema evolution with logical types

2022-02-18 Thread Ivan Zemlyanskiy (Jira)
apache.avro.io.ResolvingDecoder` should respect logical types in _SERVER_ (_ACTUAL_) schema and use a corresponding conversion instance for reading values. In my example, I'd say it might be {code} ResolvingDecoder#readString() -> read the actual logi

[jira] [Created] (AVRO-3408) Schema evolution with logical types

2022-02-18 Thread Ivan Zemlyanskiy (Jira)
Ivan Zemlyanskiy created AVRO-3408: -- Summary: Schema evolution with logical types Key: AVRO-3408 URL: https://issues.apache.org/jira/browse/AVRO-3408 Project: Apache Avro Issue Type

[jira] [Commented] (AVRO-3235) Avro Schema Evolution with Enum – Deserialization Crashes

2021-12-01 Thread Bertram Beyer (Jira)
s to that issue. Looks like you identified the reason. The PR looks promising indeed. Regards, Bertram > Avro Schema Evolution with Enum – Deserialization Crashes > - > > Key: AVRO-3235 >

[jira] [Commented] (AVRO-3235) Avro Schema Evolution with Enum – Deserialization Crashes

2021-11-30 Thread Uwe Eisele (Jira)
dy a pull request that solves this problem and changes the check for enums from FullName to Name: [https://github.com/apache/avro/pull/1381/files]   Regards, Uwe > Avro Schema Evolution with Enum – Deserialization

[jira] [Comment Edited] (AVRO-3235) Avro Schema Evolution with Enum – Deserialization Crashes

2021-11-30 Thread Uwe Eisele (Jira)
that solves this problem and changes the check for enums from FullName to Name: [https://github.com/apache/avro/pull/1381/files]   Regards, Uwe > Avro Schema Evolution with Enum – Deserialization Crashes > - > >

[jira] [Comment Edited] (AVRO-3235) Avro Schema Evolution with Enum – Deserialization Crashes

2021-11-30 Thread Uwe Eisele (Jira)
ATCH); {code} Suggested change, which would allow deserialization with different namespace, but same name: {code:java} public static Action resolve(Schema w, Schema r, GenericData d) { if (w.getName() != null && !w.getName().equals(r.getName())) return new ErrorActio

[jira] [Commented] (AVRO-3235) Avro Schema Evolution with Enum – Deserialization Crashes

2021-11-30 Thread Uwe Eisele (Jira)
zation with different namespace, but same name: {code:java} public static Action resolve(Schema w, Schema r, GenericData d) { if (w.getName() != null && !w.getName().equals(r.getName())) return new ErrorAction(w, r, d, ErrorType.NAMES_DONT_MATCH); {code} It looks there is al

[jira] [Created] (AVRO-3235) Avro Schema Evolution with Enum – Deserialization Crashes

2021-10-15 Thread Bertram Beyer (Jira)
Bertram Beyer created AVRO-3235: --- Summary: Avro Schema Evolution with Enum – Deserialization Crashes Key: AVRO-3235 URL: https://issues.apache.org/jira/browse/AVRO-3235 Project: Apache Avro

[jira] [Commented] (AVRO-1270) Emit default values from resolved reader during schema evolution/projection

2020-10-27 Thread Diego Sogari (Jira)
? Do you have guidelines for implementing the feature, if one would be willing to do it? > Emit default values from resolved reader during schema evolution/projection > --- > > Key: AVRO-1270 >

[jira] [Updated] (AVRO-2779) Schema evolution and adding fields to nested records

2020-03-24 Thread Mateusz Mrozewski (Jira)
ma) .set("field1", 1L) .set("field2", 2L) .set("nested", nested) .build();GenericData gd1 = new GenericData(); RawMessageEncoder encoder = new RawMessageEncoder<>(gd1, newSchema); ByteBuffer encoded = encoder.encode(newRecord);GenericDat

[jira] [Created] (AVRO-2779) Schema evolution and adding fields to nested records

2020-03-24 Thread Mateusz Mrozewski (Jira)
Mateusz Mrozewski created AVRO-2779: --- Summary: Schema evolution and adding fields to nested records Key: AVRO-2779 URL: https://issues.apache.org/jira/browse/AVRO-2779 Project: Apache Avro

[jira] [Updated] (AVRO-2739) schema evolution support for case-sensitive to case-insensitive schema change

2020-02-12 Thread Jim Huang (Jira)
or value (string & enum types) of the schema.  Can this be handled by the current schema evolution API to normalize the data for schema changes that went from case-sensitive --> all UPPERCASE or lowercase?  Your opinion and views are appreciated even if this feature does not exist or sho

[jira] [Updated] (AVRO-2739) schema evolution support for case-sensitive to case-insensitive schema change

2020-02-12 Thread Jim Huang (Jira)
or value (string & enum types) of the schema.  Can this be handled by the current schema evolution API to normalize the data for schema changes that went from case-sensitive --> all UPPERCASE or lowercase?  Your opinion and views are appreciated even if this feature does not exist or sh

[jira] [Created] (AVRO-2739) schema evolution support for case-sensitive to case-insensitive schema change

2020-02-12 Thread Jim Huang (Jira)
Jim Huang created AVRO-2739: --- Summary: schema evolution support for case-sensitive to case-insensitive schema change Key: AVRO-2739 URL: https://issues.apache.org/jira/browse/AVRO-2739 Project: Apache Avro

[jira] [Resolved] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-02-07 Thread Jira
[ https://issues.apache.org/jira/browse/AVRO-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía resolved AVRO-2689. Resolution: Fixed > Avro-tools should have a way to test schema evolut

[jira] [Updated] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-02-07 Thread Jira
[ https://issues.apache.org/jira/browse/AVRO-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated AVRO-2689: --- Fix Version/s: (was: 1.10.0) > Avro-tools should have a way to test schema evolut

[jira] [Updated] (AVRO-2415) Regression: Schema evolution of union containing array fails in 1.9.0

2020-02-07 Thread Jira
[ https://issues.apache.org/jira/browse/AVRO-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated AVRO-2415: --- Fix Version/s: (was: 1.10.0) > Regression: Schema evolution of union containing array fails

[jira] [Commented] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-31 Thread ASF subversion and git services (Jira)
chema and --reader-schema-file flags to the tojson tool. * AVRO-2689: Unit test for reader schema on tojson tool. Co-authored-by: RyanSkraba > Avro-tools should have a way to test schema evolution. > -- > >

[jira] [Commented] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-31 Thread ASF subversion and git services (Jira)
chema and --reader-schema-file flags to the tojson tool. * AVRO-2689: Unit test for reader schema on tojson tool. Co-authored-by: RyanSkraba > Avro-tools should have a way to test schema evolution. > -- > >

[jira] [Commented] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-31 Thread ASF subversion and git services (Jira)
chema and --reader-schema-file flags to the tojson tool. * AVRO-2689: Unit test for reader schema on tojson tool. Co-authored-by: RyanSkraba > Avro-tools should have a way to test schema evolution. > -- > >

[jira] [Commented] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-28 Thread Hudson (Jira)
ools/src/main/java/org/apache/avro/tool/DataFileReadTool.java > Avro-tools should have a way to test schema evolution. > -- > > Key: AVRO-2689 > URL: https://issues.apache.org/ji

[jira] [Updated] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-28 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/AVRO-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-2689: -- Fix Version/s: 1.9.2 > Avro-tools should have a way to test schema evolut

[jira] [Commented] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-28 Thread ASF subversion and git services (Jira)
chema and --reader-schema-file flags to the tojson tool. * AVRO-2689: Unit test for reader schema on tojson tool. Co-authored-by: RyanSkraba > Avro-tools should have a way to test schema evolution. > -- > >

[jira] [Commented] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-28 Thread ASF subversion and git services (Jira)
chema and --reader-schema-file flags to the tojson tool. * AVRO-2689: Unit test for reader schema on tojson tool. Co-authored-by: RyanSkraba > Avro-tools should have a way to test schema evolution. > -- > >

[jira] [Commented] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-28 Thread ASF subversion and git services (Jira)
chema and --reader-schema-file flags to the tojson tool. * AVRO-2689: Unit test for reader schema on tojson tool. Co-authored-by: RyanSkraba > Avro-tools should have a way to test schema evolution. > -- > >

[jira] [Assigned] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-20 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/AVRO-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-2689: - Assignee: Roger > Avro-tools should have a way to test schema evolut

[jira] [Updated] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-20 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/AVRO-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-2689: -- Fix Version/s: 1.10.0 > Avro-tools should have a way to test schema evolut

[jira] [Updated] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-17 Thread Ryan Skraba (Jira)
[ https://issues.apache.org/jira/browse/AVRO-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-2689: -- Component/s: java > Avro-tools should have a way to test schema evolut

[jira] [Commented] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-17 Thread Ryan Skraba (Jira)
to set a different reader schema (than the one found in the file), it should be set by calling {code:java} reader.setExpected(readerSchema){code} just after the DataFileStream is created. > Avro-tools should have a way to test sche

[jira] [Created] (AVRO-2689) Avro-tools should have a way to test schema evolution.

2020-01-17 Thread Ryan Skraba (Jira)
Ryan Skraba created AVRO-2689: - Summary: Avro-tools should have a way to test schema evolution. Key: AVRO-2689 URL: https://issues.apache.org/jira/browse/AVRO-2689 Project: Apache Avro Issue

Re: Record names and schema evolution

2019-08-27 Thread Ryan Skraba
r new schemas (AddressV1, AddressV2) and just expected it to work, and it does. That being said -- in this case, there are no interoperability concerns when schema evolution is concerned, since the rules are by nature linked to one version of one implementation. If there's value in this p

Re: Record names and schema evolution

2019-08-26 Thread Doug Cutting
This may be an example of Postel's Law, where neither the implementation nor the spec are wrong. An implementation is allowed to accept more than the the strictest interpretation of the spec. Within reason, we prefer that folks can read data rather than get an error when trying. (We also want th

Record names and schema evolution

2019-08-26 Thread Ryan Skraba
Hello! I've been going through some code that should be cleaned up if https://issues.apache.org/jira/browse/AVRO-2492 is applied (removing one of the deprecated record schema constructores). In the meantime, I have a question about names in general. I noticed in the spec: https://avro.apache.or

[jira] [Commented] (AVRO-2415) Regression: Schema evolution of union containing array fails in 1.9.0

2019-06-17 Thread Hudson (JIRA)
lang/java/avro/src/test/java/org/apache/avro/TestSchemaValidation.java * (edit) lang/java/avro/src/main/java/org/apache/avro/io/parsing/ResolvingGrammarGenerator.java > Regression: Schema evolution of union containing array fails i

  1   2   >