[jira] [Commented] (AVRO-2875) avro-tools-1.10.0.jar missing slf4j binding
[
https://issues.apache.org/jira/browse/AVRO-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150611#comment-17150611
]
Sean Busbey commented on AVRO-2875:
---
looks like we don't have github -> jira notification set up properly since
https://github.com/apache/avro/pull/925 didn't get linked to this jira.
> avro-tools-1.10.0.jar missing slf4j binding
> ---
>
> Key: AVRO-2875
> URL: https://issues.apache.org/jira/browse/AVRO-2875
> Project: Apache Avro
> Issue Type: Bug
> Components: tools
>Affects Versions: 1.10.0
>Reporter: Dan Lipofsky
>Assignee: Ryan Skraba
>Priority: Minor
> Fix For: 1.11.0, 1.10.1
>
>
> {{avro-tools-1.10.0.jar}} is missing a binding for {{slf4j}}.
> This is a regression because 1.9.2 looks good but 1.10.0 generates an error.
> The tool still works so this is minor, although I wonder if we won't miss
> useful log output under some conditions.
> {noformat}
> $ java -jar ../avro-tools-1.10.0.jar idl src/main/avro/foo.avdl
> /tmp/namespaces.avpr
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> $ java -jar ../avro-tools-1.9.2.jar idl src/main/avro/foo.avdl
> /tmp/namespaces.avpr
> $
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2860) More Closely Adhere to ASF Parent POM
[
https://issues.apache.org/jira/browse/AVRO-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126323#comment-17126323
]
Sean Busbey commented on AVRO-2860:
---
It is not accurate to say jdk8 is EOL. Both the openjdk 7 updates and the
openjdk 8 updates projects are still active and there are options to purchase
commercial support for both jdks still.
> More Closely Adhere to ASF Parent POM
> -
>
> Key: AVRO-2860
> URL: https://issues.apache.org/jira/browse/AVRO-2860
> Project: Apache Avro
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
>
> So, I see that the project currently inherits from:
> {code:xml}
>
> org.apache
> apache
> 23
>
> {code}
> That's great.
> However, there are quite a few places where the values from this parent POM
> are being overrided and I'm not quite sure why. The Avro project should be
> pretty conservative here and be sticking as closely as possible to the parent
> POM in the same spirit that Avro is build with Java 1.8 (which is EOL).
> I propose reverting some plugin versions so that the project more closely
> aligns with the POM and then the project need only stay in sync with the
> parent POM releases and not each plugin individually.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2604) Artifacts were signed with a key not in KEYS
[
https://issues.apache.org/jira/browse/AVRO-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2604:
--
Fix Version/s: 1.9.1
Resolution: Fixed
Status: Resolved (was: Patch Available)
all propagated now.
> Artifacts were signed with a key not in KEYS
>
>
> Key: AVRO-2604
> URL: https://issues.apache.org/jira/browse/AVRO-2604
> Project: Apache Avro
> Issue Type: Bug
> Components: community, release
>Affects Versions: 1.9.1
>Reporter: Eric Peterson
>Assignee: Fokko Driesprong
>Priority: Critical
> Fix For: 1.9.1
>
>
> Downloads need to be checked against the KEYS obtained from the Avro project.
> Importing the current KEYS file gives:
> {noformat}
> $ gpg --import KEYS
> gpg: key 0xDBAF69BEA7239D59: public key "Doug Cutting (Lucene guy)
> " imported
> gpg: key 0xB5E0D06745472392: public key "Jeff Hammerbacher (CODE SIGNING KEY)
> " imported
> gpg: key 0x4FB955854318F669: 3 signatures not checked due to missing keys
> gpg: key 0x4FB955854318F669: public key "Tom White (CODE SIGNING KEY)
> " imported
> gpg: key 0x99CCC523E1BE8DBE: public key "Tom White (APACHE CODE SIGNING KEY)
> " imported
> gpg: key 0xFCB3CBD9D3924CCD: public key "Ryan Blue (CODE SIGNING KEY)
> " imported
> gpg: key 0x807934F7C3A8: public key "Suraj Acharya "
> imported
> gpg: Total number processed: 6
> gpg: imported: 6
> gpg: no ultimately trusted keys found
> {noformat}
> But the 1.9.1 release artifacts were not signed with any of the PGP keys in
> that file, for example:
> {noformat}
> $ for asc in *.asc; do
> gpg --verify $asc
> echo
> done
> gpg: assuming signed data in 'Avro-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-cpp-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-doc-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-js-1.9.1.tgz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-python3-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-src-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2602) Updating breaks backward compatibility by throwing AvroTypeException in some cases
[
https://issues.apache.org/jira/browse/AVRO-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969808#comment-16969808
]
Sean Busbey commented on AVRO-2602:
---
1.7.7 -> 1.8 is a major version change for Avro. AVRO-997 is marked as
backwards incompatible. What's the matter at hand [~lingchao]?
> Updating breaks backward compatibility by throwing AvroTypeException in some
> cases
> --
>
> Key: AVRO-2602
> URL: https://issues.apache.org/jira/browse/AVRO-2602
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.8.0, 1.8.1, 1.9.0, 1.8.2, 1.9.1
>Reporter: xia0c
>Priority: Major
>
> When I try to update Avro from 1.7.7 to the newer version. The following code:
> {code:java}
> @Test
> public void Demo() throws IOException{
>
> Schema schema = Schema.parse("{\"type\": \"enum\", \"name\":
> \"MyEnum\", \"symbols\": [\"A\", \"B\", \"C\"]}");
> Map expectedA = new java.util.HashMap();
> expectedA.put("", "A");
> assertEquals(expectedA, write(schema, "A"));
>
> }
>
> private Map write(Schema schema, Object datum) throws
> IOException {
> DatumWriter writer = new GenericDatumWriter(schema);
> Map out = new java.util.HashMap();
> KeyValueEncoder encoder = new KeyValueEncoder(schema, out);
> writer.write(datum, encoder);
> return out;
> }
>
> }
> class KeyValueEncoder extends ParsingEncoder implements Parser.ActionHandler {
> final Parser parser;
> protected BitSet isEmpty = new BitSet();
> private java.util.Map out;
>
> KeyValueEncoder(Schema sc, java.util.Map out) throws
> IOException {
> configure(out);
> this.parser =
> new Parser(new JsonGrammarGenerator().generate(sc), this);
> }
>
> public void flush() throws IOException {
> // Do nothing
> }
> public KeyValueEncoder configure(java.util.Map
> newOut) throws IOException {
> this.out = newOut;
> return this;
> }
>
>
> /
> @Override
> public void writeNull() throws IOException {
> parser.advance(Symbol.NULL);
> }
> @Override
> public void writeBoolean(boolean b) throws IOException {
> parser.advance(Symbol.BOOLEAN);
> out.put(getKeyPathString(), Boolean.toString(b));
> }
> @Override
> public void writeInt(int n) throws IOException {
> parser.advance(Symbol.INT);
> out.put(getKeyPathString(), Integer.toString(n));
> }
> @Override
> public void writeLong(long n) throws IOException {
> parser.advance(Symbol.LONG);
> out.put(getKeyPathString(), Long.toString(n));
> }
> @Override
> public void writeFloat(float f) throws IOException {
> parser.advance(Symbol.FLOAT);
> out.put(getKeyPathString(), Float.toString(f));
> }
> @Override
> public void writeDouble(double d) throws IOException {
> parser.advance(Symbol.DOUBLE);
> out.put(getKeyPathString(), Double.toString(d));
> }
> @Override
> public void writeString(Utf8 utf8) throws IOException {
> writeString(utf8.toString());
> }
>
> @Override
> public void writeString(String str) throws IOException {
> parser.advance(Symbol.STRING);
> trace("writeString(" + str + ")");
> if (parser.topSymbol() == Symbol.MAP_KEY_MARKER) {
> parser.advance(Symbol.MAP_KEY_MARKER);
> pushKeyPathComponent(str);
> // out.writeFieldName(str);
> } else {
> out.put(getKeyPathString(), str);
> }
> }
> @Override
> public void writeBytes(ByteBuffer bytes) throws IOException {
> if (bytes.hasArray()) {
> writeBytes(bytes.array(), bytes.position(), bytes.remaining());
> } else {
> byte[] b = new byte[bytes.remaining()];
> for (int i = 0; i < b.length; i++) {
> b[i] = bytes.get();
> }
> writeBytes(b);
> }
> }
> @Override
> public void writeBytes(byte[] bytes, int start, int len) throws
> IOException {
> parser.advance(Symbol.BYTES);
> writeByteArray(bytes, start, len);
> }
> private void writeByteArray(byte[] bytes, int start, int len) throws
> IOException {
> out.put(getKeyPathString(), new String(bytes, start, len, "
[jira] [Commented] (AVRO-2616) Do not use Hadoop FS for local files with avro-tools
[ https://issues.apache.org/jira/browse/AVRO-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968015#comment-16968015 ] Sean Busbey commented on AVRO-2616: --- I agree that it would be best to get any needed Hadoop filesystem implementation from a local Hadoop install when we're talking to a non-local filesystem. For the local case, I see a few options: 1) what if we kept a smaller Hadoop footprint for localfs? Like how we don't have all of guava 2) we could ship our own FileSystem impl of local file access that was just simple java.nio stuff 3) we can try something Clever where we look at the cli and environment and try to guess if we need to load FileSystem stuff at all or just use nio directly. What are you thinking for approach [~ryanskraba]? > Do not use Hadoop FS for local files with avro-tools > > > Key: AVRO-2616 > URL: https://issues.apache.org/jira/browse/AVRO-2616 > Project: Apache Avro > Issue Type: Bug > Components: java >Reporter: Ryan Skraba >Priority: Minor > > The avro-tools jar includes the Hadoop dependencies inside the fat jar. This > is probably so that the CLI can operate with files that are located on the > cluster (or other URIs that have Hadoop FileSystem implementations). > This is useful if the user is accessing HDFS with the same version as the > Hadoop FileSystem, and *mostly* neutral if the user is accessing other URIs, > including the local filesystem. > Hadoop doesn't currently officially support any version [after JDK > 8|https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions] > (at time of writing). We might want to access the local filesystem bypassing > the Hadoop jars to avoid requiring this dependency when it is not needed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AVRO-2604) Artifacts were signed with a key not in KEYS
[
https://issues.apache.org/jira/browse/AVRO-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2604:
--
Component/s: community
> Artifacts were signed with a key not in KEYS
>
>
> Key: AVRO-2604
> URL: https://issues.apache.org/jira/browse/AVRO-2604
> Project: Apache Avro
> Issue Type: Bug
> Components: community, release
>Affects Versions: 1.9.1
>Reporter: Eric Peterson
>Priority: Major
>
> Downloads need to be checked against the KEYS obtained from the Avro project.
> Importing the current KEYS file gives:
> {noformat}
> $ gpg --import KEYS
> gpg: key 0xDBAF69BEA7239D59: public key "Doug Cutting (Lucene guy)
> " imported
> gpg: key 0xB5E0D06745472392: public key "Jeff Hammerbacher (CODE SIGNING KEY)
> " imported
> gpg: key 0x4FB955854318F669: 3 signatures not checked due to missing keys
> gpg: key 0x4FB955854318F669: public key "Tom White (CODE SIGNING KEY)
> " imported
> gpg: key 0x99CCC523E1BE8DBE: public key "Tom White (APACHE CODE SIGNING KEY)
> " imported
> gpg: key 0xFCB3CBD9D3924CCD: public key "Ryan Blue (CODE SIGNING KEY)
> " imported
> gpg: key 0x807934F7C3A8: public key "Suraj Acharya "
> imported
> gpg: Total number processed: 6
> gpg: imported: 6
> gpg: no ultimately trusted keys found
> {noformat}
> But the 1.9.1 release artifacts were not signed with any of the PGP keys in
> that file, for example:
> {noformat}
> $ for asc in *.asc; do
> gpg --verify $asc
> echo
> done
> gpg: assuming signed data in 'Avro-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-cpp-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-doc-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-js-1.9.1.tgz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-python3-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-src-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2604) Artifacts were signed with a key not in KEYS
[
https://issues.apache.org/jira/browse/AVRO-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959414#comment-16959414
]
Sean Busbey commented on AVRO-2604:
---
thanks for catching this [~epibkr]!
Sounds like you've got the matter well in hand [~fokko], thanks for fixing it
up.
> Artifacts were signed with a key not in KEYS
>
>
> Key: AVRO-2604
> URL: https://issues.apache.org/jira/browse/AVRO-2604
> Project: Apache Avro
> Issue Type: Bug
> Components: community, release
>Affects Versions: 1.9.1
>Reporter: Eric Peterson
>Priority: Critical
>
> Downloads need to be checked against the KEYS obtained from the Avro project.
> Importing the current KEYS file gives:
> {noformat}
> $ gpg --import KEYS
> gpg: key 0xDBAF69BEA7239D59: public key "Doug Cutting (Lucene guy)
> " imported
> gpg: key 0xB5E0D06745472392: public key "Jeff Hammerbacher (CODE SIGNING KEY)
> " imported
> gpg: key 0x4FB955854318F669: 3 signatures not checked due to missing keys
> gpg: key 0x4FB955854318F669: public key "Tom White (CODE SIGNING KEY)
> " imported
> gpg: key 0x99CCC523E1BE8DBE: public key "Tom White (APACHE CODE SIGNING KEY)
> " imported
> gpg: key 0xFCB3CBD9D3924CCD: public key "Ryan Blue (CODE SIGNING KEY)
> " imported
> gpg: key 0x807934F7C3A8: public key "Suraj Acharya "
> imported
> gpg: Total number processed: 6
> gpg: imported: 6
> gpg: no ultimately trusted keys found
> {noformat}
> But the 1.9.1 release artifacts were not signed with any of the PGP keys in
> that file, for example:
> {noformat}
> $ for asc in *.asc; do
> gpg --verify $asc
> echo
> done
> gpg: assuming signed data in 'Avro-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-cpp-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-doc-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-js-1.9.1.tgz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-python3-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-src-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AVRO-2604) Artifacts were signed with a key not in KEYS
[
https://issues.apache.org/jira/browse/AVRO-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2604:
--
Priority: Critical (was: Major)
> Artifacts were signed with a key not in KEYS
>
>
> Key: AVRO-2604
> URL: https://issues.apache.org/jira/browse/AVRO-2604
> Project: Apache Avro
> Issue Type: Bug
> Components: community, release
>Affects Versions: 1.9.1
>Reporter: Eric Peterson
>Priority: Critical
>
> Downloads need to be checked against the KEYS obtained from the Avro project.
> Importing the current KEYS file gives:
> {noformat}
> $ gpg --import KEYS
> gpg: key 0xDBAF69BEA7239D59: public key "Doug Cutting (Lucene guy)
> " imported
> gpg: key 0xB5E0D06745472392: public key "Jeff Hammerbacher (CODE SIGNING KEY)
> " imported
> gpg: key 0x4FB955854318F669: 3 signatures not checked due to missing keys
> gpg: key 0x4FB955854318F669: public key "Tom White (CODE SIGNING KEY)
> " imported
> gpg: key 0x99CCC523E1BE8DBE: public key "Tom White (APACHE CODE SIGNING KEY)
> " imported
> gpg: key 0xFCB3CBD9D3924CCD: public key "Ryan Blue (CODE SIGNING KEY)
> " imported
> gpg: key 0x807934F7C3A8: public key "Suraj Acharya "
> imported
> gpg: Total number processed: 6
> gpg: imported: 6
> gpg: no ultimately trusted keys found
> {noformat}
> But the 1.9.1 release artifacts were not signed with any of the PGP keys in
> that file, for example:
> {noformat}
> $ for asc in *.asc; do
> gpg --verify $asc
> echo
> done
> gpg: assuming signed data in 'Avro-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-cpp-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-doc-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-js-1.9.1.tgz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-python3-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:13 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> gpg: assuming signed data in 'avro-src-1.9.1.tar.gz'
> gpg: Signature made Wed Aug 28 05:38:23 2019 EDT
> gpg:using RSA key CEF487F848109B4C8B8AC18DE4AE0EB72D112483
> gpg: Can't check signature: No public key
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (AVRO-1891) Generated Java code fails with union containing logical type
[
https://issues.apache.org/jira/browse/AVRO-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16945492#comment-16945492
]
Sean Busbey edited comment on AVRO-1891 at 10/7/19 12:49 AM:
-
I don't think folks know. The changes in PR #329 claimed to address this issue.
I presume no one verified that if this issue is still open.
PR#329 is in both the 1.9.0 and 1.9.1 releases, so if anyone has the time to
check reproducing this issue with one of those releases it would help a bunch
of folks out.
was (Author: busbey):
I don't think folks know. The changes in PR #329 claimed to address this issue.
I presume no one verified that if this issue is still open.
PR#329 is in both the 1.9.0 and 1.9.1 releases, so if anyone what's the time to
check reproducing this issue with one of those releases it would help a bunch
of folks out.
> Generated Java code fails with union containing logical type
>
>
> Key: AVRO-1891
> URL: https://issues.apache.org/jira/browse/AVRO-1891
> Project: Apache Avro
> Issue Type: Bug
> Components: java, logical types
>Affects Versions: 1.8.1
>Reporter: Ross Black
>Priority: Blocker
> Fix For: 1.8.3
>
> Attachments: AVRO-1891.patch, AVRO-1891.yshi.1.patch,
> AVRO-1891.yshi.2.patch, AVRO-1891.yshi.3.patch, AVRO-1891.yshi.4.patch
>
>
> Example schema:
> {code}
> {
> "type": "record",
> "name": "RecordV1",
> "namespace": "org.brasslock.event",
> "fields": [
> { "name": "first", "type": ["null", {"type": "long",
> "logicalType":"timestamp-millis"}]}
> ]
> }
> {code}
> The avro compiler generates a field using the relevant joda class:
> {code}
> public org.joda.time.DateTime first
> {code}
> Running the following code to perform encoding:
> {code}
> final RecordV1 record = new
> RecordV1(DateTime.parse("2016-07-29T10:15:30.00Z"));
> final DatumWriter datumWriter = new
> SpecificDatumWriter<>(record.getSchema());
> final ByteArrayOutputStream stream = new ByteArrayOutputStream(8192);
> final BinaryEncoder encoder =
> EncoderFactory.get().directBinaryEncoder(stream, null);
> datumWriter.write(record, encoder);
> encoder.flush();
> final byte[] bytes = stream.toByteArray();
> {code}
> fails with the exception stacktrace:
> {code}
> org.apache.avro.AvroRuntimeException: Unknown datum type
> org.joda.time.DateTime: 2016-07-29T10:15:30.000Z
> at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:741)
> at
> org.apache.avro.specific.SpecificData.getSchemaName(SpecificData.java:293)
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:706)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at
> org.brasslock.avro.compiler.GeneratedRecordTest.shouldEncodeLogicalTypeInUnion(GeneratedRecordTest.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.jun
[jira] [Commented] (AVRO-1891) Generated Java code fails with union containing logical type
[
https://issues.apache.org/jira/browse/AVRO-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16945492#comment-16945492
]
Sean Busbey commented on AVRO-1891:
---
I don't think folks know. The changes in PR #329 claimed to address this issue.
I presume no one verified that if this issue is still open.
PR#329 is in both the 1.9.0 and 1.9.1 releases, so if anyone what's the time to
check reproducing this issue with one of those releases it would help a bunch
of folks out.
> Generated Java code fails with union containing logical type
>
>
> Key: AVRO-1891
> URL: https://issues.apache.org/jira/browse/AVRO-1891
> Project: Apache Avro
> Issue Type: Bug
> Components: java, logical types
>Affects Versions: 1.8.1
>Reporter: Ross Black
>Priority: Blocker
> Fix For: 1.8.3
>
> Attachments: AVRO-1891.patch, AVRO-1891.yshi.1.patch,
> AVRO-1891.yshi.2.patch, AVRO-1891.yshi.3.patch, AVRO-1891.yshi.4.patch
>
>
> Example schema:
> {code}
> {
> "type": "record",
> "name": "RecordV1",
> "namespace": "org.brasslock.event",
> "fields": [
> { "name": "first", "type": ["null", {"type": "long",
> "logicalType":"timestamp-millis"}]}
> ]
> }
> {code}
> The avro compiler generates a field using the relevant joda class:
> {code}
> public org.joda.time.DateTime first
> {code}
> Running the following code to perform encoding:
> {code}
> final RecordV1 record = new
> RecordV1(DateTime.parse("2016-07-29T10:15:30.00Z"));
> final DatumWriter datumWriter = new
> SpecificDatumWriter<>(record.getSchema());
> final ByteArrayOutputStream stream = new ByteArrayOutputStream(8192);
> final BinaryEncoder encoder =
> EncoderFactory.get().directBinaryEncoder(stream, null);
> datumWriter.write(record, encoder);
> encoder.flush();
> final byte[] bytes = stream.toByteArray();
> {code}
> fails with the exception stacktrace:
> {code}
> org.apache.avro.AvroRuntimeException: Unknown datum type
> org.joda.time.DateTime: 2016-07-29T10:15:30.000Z
> at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:741)
> at
> org.apache.avro.specific.SpecificData.getSchemaName(SpecificData.java:293)
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:706)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at
> org.brasslock.avro.compiler.GeneratedRecordTest.shouldEncodeLogicalTypeInUnion(GeneratedRecordTest.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:
[jira] [Updated] (AVRO-2555) Update IDL docs to remove mention of avroj.jar
[ https://issues.apache.org/jira/browse/AVRO-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2555: -- Component/s: java > Update IDL docs to remove mention of avroj.jar > -- > > Key: AVRO-2555 > URL: https://issues.apache.org/jira/browse/AVRO-2555 > Project: Apache Avro > Issue Type: Improvement > Components: doc, java >Reporter: Erik Tank >Assignee: Erik Tank >Priority: Major > Attachments: AVRO-2555.patch > > > AVRO-320 renamed the avroj-tools to avro-tools.jar, however, the IDL > documentation wasn't updated. The attached patch updates the Usage section > demonstrating how to convert an .avdl to .avpr. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (AVRO-2555) Update IDL docs to remove mention of avroj.jar
[ https://issues.apache.org/jira/browse/AVRO-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned AVRO-2555: - Assignee: Erik Tank > Update IDL docs to remove mention of avroj.jar > -- > > Key: AVRO-2555 > URL: https://issues.apache.org/jira/browse/AVRO-2555 > Project: Apache Avro > Issue Type: Improvement > Components: doc >Reporter: Erik Tank >Assignee: Erik Tank >Priority: Major > Attachments: AVRO-2555.patch > > > AVRO-320 renamed the avroj-tools to avro-tools.jar, however, the IDL > documentation wasn't updated. The attached patch updates the Usage section > demonstrating how to convert an .avdl to .avpr. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (AVRO-2555) Update IDL docs to remove mention of avroj.jar
[
https://issues.apache.org/jira/browse/AVRO-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16926692#comment-16926692
]
Sean Busbey commented on AVRO-2555:
---
+1 change looks good. Can you please regenerate the patch using {{git
format-patch}} so that it includes the author information you'd like used for
merging?
> Update IDL docs to remove mention of avroj.jar
> --
>
> Key: AVRO-2555
> URL: https://issues.apache.org/jira/browse/AVRO-2555
> Project: Apache Avro
> Issue Type: Improvement
> Components: doc
>Reporter: Erik Tank
>Priority: Major
> Attachments: AVRO-2555.patch
>
>
> AVRO-320 renamed the avroj-tools to avro-tools.jar, however, the IDL
> documentation wasn't updated. The attached patch updates the Usage section
> demonstrating how to convert an .avdl to .avpr.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (AVRO-2550) branch-1.9 Build Failing When Downloading Yetus
[
https://issues.apache.org/jira/browse/AVRO-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925012#comment-16925012
]
Sean Busbey commented on AVRO-2550:
---
Yeah, if the artifact isn't on the live mirrors then it falls back to
archive.apache.org, which will still have the older releases.
> branch-1.9 Build Failing When Downloading Yetus
> ---
>
> Key: AVRO-2550
> URL: https://issues.apache.org/jira/browse/AVRO-2550
> Project: Apache Avro
> Issue Type: Bug
> Components: build
>Reporter: Brian Lachniet
>Assignee: Brian Lachniet
>Priority: Major
>
> [Build #1176|https://travis-ci.org/apache/avro/builds/579684016] is when
> {{branch-1.9}} started failing.
> {code:txt}
> Setting up python3-software-properties (0.96.20.9) ...
> Setting up software-properties-common (0.96.20.9) ...
> Processing triggers for libc-bin (2.23-0ubuntu11) ...
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 196 100 1960 0639 0 --:--:-- --:--:-- --:--:-- 638
> gzip: stdin: not in gzip format
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now
> The command "if [ -x ./.travis/before_install.sh ] ; then
> ./.travis/before_install.sh ; fi" failed and exited with 2 during .
> Your build has been stopped.
> {code}
> We upgraded to Yetus 0.10.0 on master in
> [9125e9a|https://github.com/apache/avro/commit/9125e9acff342accbb404270c09fef4ccff151d6#diff-b6fd27089de42d65941556f3c79ad672],
> alongside Ruby changes for AVRO-2482.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (AVRO-2550) branch-1.9 Build Failing When Downloading Yetus
[
https://issues.apache.org/jira/browse/AVRO-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923927#comment-16923927
]
Sean Busbey commented on AVRO-2550:
---
If we want to avoid having to update when Yetus stops considering a release
"recent" we could fork [this script the HBase project has for downloading from
the mirroring system but falling back to
archive.a.o|https://github.com/apache/hbase/blob/master/dev-support/jenkins-scripts/cache-apache-project-artifact.sh]
> branch-1.9 Build Failing When Downloading Yetus
> ---
>
> Key: AVRO-2550
> URL: https://issues.apache.org/jira/browse/AVRO-2550
> Project: Apache Avro
> Issue Type: Bug
> Components: build
>Reporter: Brian Lachniet
>Assignee: Brian Lachniet
>Priority: Major
>
> [Build #1176|https://travis-ci.org/apache/avro/builds/579684016] is when
> {{branch-1.9}} started failing.
> {code:txt}
> Setting up python3-software-properties (0.96.20.9) ...
> Setting up software-properties-common (0.96.20.9) ...
> Processing triggers for libc-bin (2.23-0ubuntu11) ...
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 196 100 1960 0639 0 --:--:-- --:--:-- --:--:-- 638
> gzip: stdin: not in gzip format
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now
> The command "if [ -x ./.travis/before_install.sh ] ; then
> ./.travis/before_install.sh ; fi" failed and exited with 2 during .
> Your build has been stopped.
> {code}
> We upgraded to Yetus 0.10.0 on master in
> [9125e9a|https://github.com/apache/avro/commit/9125e9acff342accbb404270c09fef4ccff151d6#diff-b6fd27089de42d65941556f3c79ad672],
> alongside Ruby changes for AVRO-2482.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (AVRO-2531) Update commons-compress to version 1.19
[ https://issues.apache.org/jira/browse/AVRO-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved AVRO-2531. --- Fix Version/s: 1.10.0 Resolution: Fixed > Update commons-compress to version 1.19 > --- > > Key: AVRO-2531 > URL: https://issues.apache.org/jira/browse/AVRO-2531 > Project: Apache Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.9.0 >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Critical > Fix For: 1.10.0, 1.9.1 > > > Avro's current version of commons-compress (1.18) is subject to a DoS > vulnerability so we need to upgrade it [CVE-2019-12402] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (AVRO-2522) Specific API doesn't handle schemas with IList or Nullable in name
[
https://issues.apache.org/jira/browse/AVRO-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920535#comment-16920535
]
Sean Busbey commented on AVRO-2522:
---
How close is this to landing? it's not marked as a blocker; would it be fine
waiting for 1.9.2?
> Specific API doesn't handle schemas with IList or Nullable in name
> --
>
> Key: AVRO-2522
> URL: https://issues.apache.org/jira/browse/AVRO-2522
> Project: Apache Avro
> Issue Type: Bug
> Components: csharp
>Affects Versions: 1.9.0
>Reporter: Brian Lachniet
>Assignee: Brian Lachniet
>Priority: Major
> Fix For: 1.9.1
>
>
> The Specific API in the C# bindings do not properly handle:
> * Schemas with {{IList}} in their name
> * Schemas with {{Nullable}} in their name
> * Arrays of nullables (see example schema below)
> These throw an exception from the {{ObjectCreator}} indicating that it could
> not find the type.
> This sample application on GitHub demonstrates these problems:
> [https://github.com/blachniet/AVRO-2522].
> Here's a sample schema containing an array of nullables that triggers this
> problem.
> {code:java}
> {
> "namespace": "AvroListOfLists.Records",
> "name": "MyRecord",
> "type": "record",
> "fields": [
> {
> "name": "MyList",
> "type": {
> "type": "array",
> "items": ["null", "int"]
> }
> }
> ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (AVRO-2527) Upgrade PHP version to 7.x
[ https://issues.apache.org/jira/browse/AVRO-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916724#comment-16916724 ] Sean Busbey commented on AVRO-2527: --- that sounds great. Thanks Kengo! > Upgrade PHP version to 7.x > -- > > Key: AVRO-2527 > URL: https://issues.apache.org/jira/browse/AVRO-2527 > Project: Apache Avro > Issue Type: Improvement > Components: php >Reporter: Kengo Seki >Assignee: Kengo Seki >Priority: Major > > Avro currently supports PHP 5.x, but [its support period has expired on Jan > 2019|https://www.php.net/supported-versions.php]. > We should support PHP 7.1+, on which the community support is continuing at > this time. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (AVRO-2527) Upgrade PHP version to 7.x
[ https://issues.apache.org/jira/browse/AVRO-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915956#comment-16915956 ] Sean Busbey commented on AVRO-2527: --- Does this necessitate not working with PHP 5 and PHP 6? is there any survey data about how common these versions of PHP are in use? > Upgrade PHP version to 7.x > -- > > Key: AVRO-2527 > URL: https://issues.apache.org/jira/browse/AVRO-2527 > Project: Apache Avro > Issue Type: Improvement > Components: php >Reporter: Kengo Seki >Assignee: Kengo Seki >Priority: Major > > Avro currently supports PHP 5.x, but [its support period has expired on Jan > 2019|https://www.php.net/supported-versions.php]. > We should support PHP 7.1+, on which the community support is continuing at > this time. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (AVRO-2451) Releases should be tagged in git with "rel/" prefix
Sean Busbey created AVRO-2451: - Summary: Releases should be tagged in git with "rel/" prefix Key: AVRO-2451 URL: https://issues.apache.org/jira/browse/AVRO-2451 Project: Apache Avro Issue Type: Bug Components: community Reporter: Sean Busbey ASF infra has configs in place on gitbox that make tags with the prefix "rel/" immutable. We should make sure we have such a tag for our releases, to help make sure they aren't accidentally removed or updated after the fact. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2450) log message about failure to load SnappyCodec
[ https://issues.apache.org/jira/browse/AVRO-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2450: -- Status: Patch Available (was: In Progress) > log message about failure to load SnappyCodec > - > > Key: AVRO-2450 > URL: https://issues.apache.org/jira/browse/AVRO-2450 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.9.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > We catch Throwable and then silently return null without giving any means to > find out why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AVRO-2450) log message about failure to load SnappyCodec
[ https://issues.apache.org/jira/browse/AVRO-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AVRO-2450 started by Sean Busbey. - > log message about failure to load SnappyCodec > - > > Key: AVRO-2450 > URL: https://issues.apache.org/jira/browse/AVRO-2450 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.9.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > We catch Throwable and then silently return null without giving any means to > find out why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2450) log message about failure to load SnappyCodec
Sean Busbey created AVRO-2450: - Summary: log message about failure to load SnappyCodec Key: AVRO-2450 URL: https://issues.apache.org/jira/browse/AVRO-2450 Project: Apache Avro Issue Type: Bug Components: java Affects Versions: 1.9.0 Reporter: Sean Busbey Assignee: Sean Busbey We catch Throwable and then silently return null without giving any means to find out why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2395) Stop including bare jars as convenience binary for Java
[ https://issues.apache.org/jira/browse/AVRO-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838596#comment-16838596 ] Sean Busbey commented on AVRO-2395: --- here's a walkthrough for getting the 1.8.2 avro-tools jar: * browse to https://search.maven.org * type "avro-tools" in the omni-search box * click download * click jar leads to : https://search.maven.org/remotecontent?filepath=org/apache/avro/avro-tools/1.8.2/avro-tools-1.8.2.jar another option is to ship something like a "tools" tarball that has the example schemas used for compat testing and the avro-tools jar. > Stop including bare jars as convenience binary for Java > --- > > Key: AVRO-2395 > URL: https://issues.apache.org/jira/browse/AVRO-2395 > Project: Apache Avro > Issue Type: Task > Components: community, java >Affects Versions: 1.7.7, 1.9.0, 1.8.2 >Reporter: Sean Busbey >Priority: Major > > currently as a project our "convenience binary" for java is the complete set > of jars, source jars, test jars, and javadoc jars. that means as a part of RC > vetting someone has to check all of those individual files in addition to > checking them in the staged nexus repo. > > the overwhelming majority of java users will consume our releases via maven > central. we should either > > * ship nothing for java out of dist.a.o > * ship a tarball that contains the things we currently ship > * ship a tarball that is a laid out like a repository and includes both our > artifacts and transitive dependencies > > my personal preference is for "ship nothing". The last option would have the > most potential use IMHO, since it would allow someone to easily set up a > local repository when they need to build against our artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AVRO-2395) Stop including bare jars as convenience binary for Java
Sean Busbey created AVRO-2395: - Summary: Stop including bare jars as convenience binary for Java Key: AVRO-2395 URL: https://issues.apache.org/jira/browse/AVRO-2395 Project: Apache Avro Issue Type: Task Components: community, java Affects Versions: 1.8.2, 1.7.7, 1.9.0 Reporter: Sean Busbey currently as a project our "convenience binary" for java is the complete set of jars, source jars, test jars, and javadoc jars. that means as a part of RC vetting someone has to check all of those individual files in addition to checking them in the staged nexus repo. the overwhelming majority of java users will consume our releases via maven central. we should either * ship nothing for java out of dist.a.o * ship a tarball that contains the things we currently ship * ship a tarball that is a laid out like a repository and includes both our artifacts and transitive dependencies my personal preference is for "ship nothing". The last option would have the most potential use IMHO, since it would allow someone to easily set up a local repository when they need to build against our artifacts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2385) Uppercase fields do not generate proper getter/setters in Java
[
https://issues.apache.org/jira/browse/AVRO-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2385:
--
Description:
Steps to reproduce:
Create an Avro schema with an uppercase field:
{code}
{
"type": "record",
"name": "example",
"namespace": "issue",
"fields": [
{
"name": "THERE_IS_NO_INDICATION_OF_WORDS",
"type": "string"
}
]
}
{code}
Use the avro-maven-plugin to generate Java code for this schema.
{code}
org.apache.avro
avro-maven-plugin
1.8.2
generate-sources
schema
${project.basedir}/src/main/resources
{code}
Expected result:
The generate getters and setters use camel casing ThereIsNoIndicationOfWords.
Actual result:
The generated getters and setters are in all uppercase:
{code}
/**
* Gets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
* @return The value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
*/
public java.lang.CharSequence getTHEREISNOINDICATIONOFWORDS() {
return THERE_IS_NO_INDICATION_OF_WORDS;
}
/**
* Sets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
* @param value the value to set.
*/
public void setTHEREISNOINDICATIONOFWORDS(java.lang.CharSequence value) {
this.THERE_IS_NO_INDICATION_OF_WORDS = value;
}
{code}
was:
Steps to reproduce:
Create an Avro schema with an uppercase field:
{code}
{
"type": "record",
"name": "example",
"namespace": "issue",
"fields": [
{
"name": "THERE_IS_NO_INDICATION_OF_WORDS",
"type": "string"
}
]
}
{code}
Use the avro-maven-plugin to generate Java code for this schema.
Expected result:
The generate getters and setters use camel casing ThereIsNoIndicationOfWords.
Actual result:
The generated getters and setters are in all uppercase:
{code}
/**
* Gets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
* @return The value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
*/
public java.lang.CharSequence getTHEREISNOINDICATIONOFWORDS() {
return THERE_IS_NO_INDICATION_OF_WORDS;
}
/**
* Sets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
* @param value the value to set.
*/
public void setTHEREISNOINDICATIONOFWORDS(java.lang.CharSequence value) {
this.THERE_IS_NO_INDICATION_OF_WORDS = value;
}
{code}
> Uppercase fields do not generate proper getter/setters in Java
> --
>
> Key: AVRO-2385
> URL: https://issues.apache.org/jira/browse/AVRO-2385
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
> Environment: Using Maven plugin with Java 8
>Reporter: Andrew
>Priority: Major
>
> Steps to reproduce:
> Create an Avro schema with an uppercase field:
> {code}
> {
> "type": "record",
> "name": "example",
> "namespace": "issue",
> "fields": [
> {
> "name": "THERE_IS_NO_INDICATION_OF_WORDS",
> "type": "string"
> }
> ]
> }
> {code}
> Use the avro-maven-plugin to generate Java code for this schema.
> {code}
>
> org.apache.avro
> avro-maven-plugin
> 1.8.2
>
>
> generate-sources
>
> schema
>
>
>
> ${project.basedir}/src/main/resources
>
>
>
>
> {code}
> Expected result:
> The generate getters and setters use camel casing ThereIsNoIndicationOfWords.
> Actual result:
> The generated getters and setters are in all uppercase:
> {code}
> /**
> * Gets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> * @return The value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> */
> public java.lang.CharSequence getTHEREISNOINDICATIONOFWORDS() {
> return THERE_IS_NO_INDICATION_OF_WORDS;
> }
> /**
> * Sets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> * @param value the value to set.
> */
> public void setTHEREISNOINDICATIONOFWORDS(java.lang.CharSequence value) {
> this.THERE_IS_NO_INDICATION_OF_WORDS = value;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2385) Uppercase fields do not generate proper getter/setters in Java
[
https://issues.apache.org/jira/browse/AVRO-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2385:
--
Description:
Steps to reproduce:
Create an Avro schema with an uppercase field:
{code}
{
"type": "record",
"name": "example",
"namespace": "issue",
"fields": [
{
"name": "THERE_IS_NO_INDICATION_OF_WORDS",
"type": "string"
}
]
}
{code}
Use the avro-maven-plugin to generate Java code for this schema.
Expected result:
The generate getters and setters use camel casing ThereIsNoIndicationOfWords.
Actual result:
The generated getters and setters are in all uppercase:
{code}
/**
* Gets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
* @return The value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
*/
public java.lang.CharSequence getTHEREISNOINDICATIONOFWORDS() {
return THERE_IS_NO_INDICATION_OF_WORDS;
}
/**
* Sets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
* @param value the value to set.
*/
public void setTHEREISNOINDICATIONOFWORDS(java.lang.CharSequence value) {
this.THERE_IS_NO_INDICATION_OF_WORDS = value;
}
{code}
was:
Steps to reproduce:
Create an Avro schema with an uppercase field:
{
{{ "type": "record",}}
{{ "name": "example",}}
{{ "namespace": "issue",}}
{{ "fields": [}}
{{ {}}
{{ "name": "THERE_IS_NO_INDICATION_OF_WORDS",}}
{{ "type": "string"}}
}
{{ ]}}
{{}}}
Use the avro-maven-plugin to generate Java code for this schema.
Expected result:
The generate getters and setters use camel casing ThereIsNoIndicationOfWords.
Actual result:
The generated getters and setters are in all uppercase:
/**
* Gets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
* @return The value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
*/
public java.lang.CharSequence getTHEREISNOINDICATIONOFWORDS() \{ return
THERE_IS_NO_INDICATION_OF_WORDS; }
/**
* Sets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
* @param value the value to set.
*/
public void setTHEREISNOINDICATIONOFWORDS(java.lang.CharSequence value) \{
this.THERE_IS_NO_INDICATION_OF_WORDS = value; }
> Uppercase fields do not generate proper getter/setters in Java
> --
>
> Key: AVRO-2385
> URL: https://issues.apache.org/jira/browse/AVRO-2385
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
> Environment: Using Maven plugin with Java 8:
> {{ }}
> {{ org.apache.avro}}
> {{ avro-maven-plugin}}
> {{ 1.8.2}}
> {{ }}
> {{ }}
> {{ generate-sources}}
> {{ }}
> {{ schema}}
> {{ }}
> {{ }}
> {{
> ${project.basedir}/src/main/resources}}
> {{ }}
> {{ }}
> {{ }}
> {{ }}
>Reporter: Andrew
>Priority: Major
>
> Steps to reproduce:
> Create an Avro schema with an uppercase field:
> {code}
> {
> "type": "record",
> "name": "example",
> "namespace": "issue",
> "fields": [
> {
> "name": "THERE_IS_NO_INDICATION_OF_WORDS",
> "type": "string"
> }
> ]
> }
> {code}
> Use the avro-maven-plugin to generate Java code for this schema.
> Expected result:
> The generate getters and setters use camel casing ThereIsNoIndicationOfWords.
> Actual result:
> The generated getters and setters are in all uppercase:
> {code}
> /**
> * Gets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> * @return The value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> */
> public java.lang.CharSequence getTHEREISNOINDICATIONOFWORDS() {
> return THERE_IS_NO_INDICATION_OF_WORDS;
> }
> /**
> * Sets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> * @param value the value to set.
> */
> public void setTHEREISNOINDICATIONOFWORDS(java.lang.CharSequence value) {
> this.THERE_IS_NO_INDICATION_OF_WORDS = value;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2385) Uppercase fields do not generate proper getter/setters in Java
[
https://issues.apache.org/jira/browse/AVRO-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2385:
--
Environment: Using Maven plugin with Java 8 (was: Using Maven plugin with
Java 8:
{{ }}
{{ org.apache.avro}}
{{ avro-maven-plugin}}
{{ 1.8.2}}
{{ }}
{{ }}
{{ generate-sources}}
{{ }}
{{ schema}}
{{ }}
{{ }}
{{
${project.basedir}/src/main/resources}}
{{ }}
{{ }}
{{ }}
{{ }})
> Uppercase fields do not generate proper getter/setters in Java
> --
>
> Key: AVRO-2385
> URL: https://issues.apache.org/jira/browse/AVRO-2385
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.2
> Environment: Using Maven plugin with Java 8
>Reporter: Andrew
>Priority: Major
>
> Steps to reproduce:
> Create an Avro schema with an uppercase field:
> {code}
> {
> "type": "record",
> "name": "example",
> "namespace": "issue",
> "fields": [
> {
> "name": "THERE_IS_NO_INDICATION_OF_WORDS",
> "type": "string"
> }
> ]
> }
> {code}
> Use the avro-maven-plugin to generate Java code for this schema.
> Expected result:
> The generate getters and setters use camel casing ThereIsNoIndicationOfWords.
> Actual result:
> The generated getters and setters are in all uppercase:
> {code}
> /**
> * Gets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> * @return The value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> */
> public java.lang.CharSequence getTHEREISNOINDICATIONOFWORDS() {
> return THERE_IS_NO_INDICATION_OF_WORDS;
> }
> /**
> * Sets the value of the 'THERE_IS_NO_INDICATION_OF_WORDS' field.
> * @param value the value to set.
> */
> public void setTHEREISNOINDICATIONOFWORDS(java.lang.CharSequence value) {
> this.THERE_IS_NO_INDICATION_OF_WORDS = value;
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-1989) Ruby schema validation for fixed types should use bytesize in error message
[
https://issues.apache.org/jira/browse/AVRO-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761001#comment-16761001
]
Sean Busbey commented on AVRO-1989:
---
nope. somehow I totally missed that it got a +1 two years ago. :/
In the event it still applies I think it'd be a good fix.
> Ruby schema validation for fixed types should use bytesize in error message
> ---
>
> Key: AVRO-1989
> URL: https://issues.apache.org/jira/browse/AVRO-1989
> Project: Apache Avro
> Issue Type: Bug
> Components: ruby
>Affects Versions: 1.9.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: AVRO-1989.0.patch
>
>
> From AVRO-1886:
> I'd like to get one thing improved, but it's fine as a follow-on.
> {code}
> + when :fixed
> + if datum.is_a? String
> +message = "expected fixed with size #{expected_schema.size},
> got \"#{datum}\" with size #{datum.size}"
> +result.add_error(path, message) unless datum.bytesize ==
> expected_schema.size
> + else
> +result.add_error(path, "expected fixed with size
> #{expected_schema.size}, got #{actual_value_message(datum)}")
> + end
> {code}
> the message here should use datum.bytesize instead of datum.size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (AVRO-1559) Drop support for Ruby 1.8
[ https://issues.apache.org/jira/browse/AVRO-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737323#comment-16737323 ] Sean Busbey edited comment on AVRO-1559 at 1/8/19 5:30 PM: --- I agree for Avro 1.9.0+ we should bump to Ruby 2.3+. IIRC JRuby's current versions also support those ruby versions so we should be good. was (Author: busbey): I agree for 1.9.0+ we should bump to 2.3+. IIRC JRuby's current versions also support those ruby versions so we should be good. > Drop support for Ruby 1.8 > - > > Key: AVRO-1559 > URL: https://issues.apache.org/jira/browse/AVRO-1559 > Project: Apache Avro > Issue Type: Wish > Components: ruby >Affects Versions: 1.7.7 >Reporter: Willem van Bergen >Assignee: Willem van Bergen >Priority: Major > Fix For: 1.9.0 > > Attachments: AVRO-1559.patch > > > - Ruby 1.8 is EOL, and is even security issues aren't addressed anymore. > - It is also getting hard to set up Ruby 1.8 to run the tests (e.g. on a > recent OSX, it won't compile without manual fiddling). > - Handling character encodings in Ruby 1.9 is very different than Ruby 1.8. > Supporting both at the same time adds a lot of overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1559) Drop support for Ruby 1.8
[ https://issues.apache.org/jira/browse/AVRO-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737323#comment-16737323 ] Sean Busbey commented on AVRO-1559: --- I agree for 1.9.0+ we should bump to 2.3+. IIRC JRuby's current versions also support those ruby versions so we should be good. > Drop support for Ruby 1.8 > - > > Key: AVRO-1559 > URL: https://issues.apache.org/jira/browse/AVRO-1559 > Project: Apache Avro > Issue Type: Wish > Components: ruby >Affects Versions: 1.7.7 >Reporter: Willem van Bergen >Assignee: Willem van Bergen >Priority: Major > Fix For: 1.9.0 > > Attachments: AVRO-1559.patch > > > - Ruby 1.8 is EOL, and is even security issues aren't addressed anymore. > - It is also getting hard to set up Ruby 1.8 to run the tests (e.g. on a > recent OSX, it won't compile without manual fiddling). > - Handling character encodings in Ruby 1.9 is very different than Ruby 1.8. > Supporting both at the same time adds a lot of overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1559) Drop support for Ruby 1.8
[ https://issues.apache.org/jira/browse/AVRO-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737324#comment-16737324 ] Sean Busbey commented on AVRO-1559: --- still worth a DISCUSS thread on dev@avro, since there are some more Ruby folks floating around there now. > Drop support for Ruby 1.8 > - > > Key: AVRO-1559 > URL: https://issues.apache.org/jira/browse/AVRO-1559 > Project: Apache Avro > Issue Type: Wish > Components: ruby >Affects Versions: 1.7.7 >Reporter: Willem van Bergen >Assignee: Willem van Bergen >Priority: Major > Fix For: 1.9.0 > > Attachments: AVRO-1559.patch > > > - Ruby 1.8 is EOL, and is even security issues aren't addressed anymore. > - It is also getting hard to set up Ruby 1.8 to run the tests (e.g. on a > recent OSX, it won't compile without manual fiddling). > - Handling character encodings in Ruby 1.9 is very different than Ruby 1.8. > Supporting both at the same time adds a lot of overhead. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2273) Release 1.8.3
[ https://issues.apache.org/jira/browse/AVRO-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709009#comment-16709009 ] Sean Busbey commented on AVRO-2273: --- Reminder that in Avro the second number is a major version. So asking folks to upgrade to 1.9.0 is a lot to ask. The project can certainly require that; any folks who want to still have the earlier versions can come help drive those releases. Just something to be aware of, probably worth a DISCUSS thread on dev@avro? > Release 1.8.3 > - > > Key: AVRO-2273 > URL: https://issues.apache.org/jira/browse/AVRO-2273 > Project: Apache Avro > Issue Type: Task >Reporter: Thiruvalluvan M. G. >Priority: Major > Fix For: 1.8.3 > > > This ticket is for releasing Avro 1.8.3 and discussing any topics related to > it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2250) Release 1.9.0
[
https://issues.apache.org/jira/browse/AVRO-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702154#comment-16702154
]
Sean Busbey commented on AVRO-2250:
---
Avro's use of a shaded guava purposefully only included those classes that were
used, so you can grep the jar file contents to see if the impacted class is
present.
For example, avro-tools covers most of the project:
{code}
Busbey-MBA:avro-help busbey$ jar tf avro-tools-1.8.2.jar | grep
AtomicDoubleArrays
Busbey-MBA:avro-help busbey$ echo $?
1
{code}
> Release 1.9.0
> -
>
> Key: AVRO-2250
> URL: https://issues.apache.org/jira/browse/AVRO-2250
> Project: Apache Avro
> Issue Type: Task
>Reporter: Nandor Kollar
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-1705) Set up Jenkins job to test all languages using Docker
[ https://issues.apache.org/jira/browse/AVRO-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639160#comment-16639160 ] Sean Busbey commented on AVRO-1705: --- (for reference, the jira covering Yetus integration for CI checking of contributions is AVRO-1887) > Set up Jenkins job to test all languages using Docker > - > > Key: AVRO-1705 > URL: https://issues.apache.org/jira/browse/AVRO-1705 > Project: Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.7 >Reporter: Tom White >Priority: Critical > Labels: starter > > The ASF Jenkins instance now supports Docker (BUILDS-25), so we could run all > the tests (for all languages that Avro supports) using the Avro Dockerfile. > We might also do a nightly build of the whole distribution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1705) Set up Jenkins job to test all languages using Docker
[ https://issues.apache.org/jira/browse/AVRO-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639159#comment-16639159 ] Sean Busbey commented on AVRO-1705: --- [~Fokko] you should start a discussion on the dev list. I really want us to use Yetus, because it'll get us more than just what our build tests and it'll go faster. Currently I know Yetus works well from ASF jenkins, and AFAIK there's no Travis support. > Set up Jenkins job to test all languages using Docker > - > > Key: AVRO-1705 > URL: https://issues.apache.org/jira/browse/AVRO-1705 > Project: Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.7 >Reporter: Tom White >Priority: Critical > Labels: starter > > The ASF Jenkins instance now supports Docker (BUILDS-25), so we could run all > the tests (for all languages that Avro supports) using the Avro Dockerfile. > We might also do a nightly build of the whole distribution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-1126) Upgrade to Jackson 2+
[
https://issues.apache.org/jira/browse/AVRO-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493638#comment-16493638
]
Sean Busbey commented on AVRO-1126:
---
it looks blocked on AVRO-1605
> Upgrade to Jackson 2+
> -
>
> Key: AVRO-1126
> URL: https://issues.apache.org/jira/browse/AVRO-1126
> Project: Avro
> Issue Type: Task
> Components: java
>Reporter: James Tyrrell
>Assignee: Charles Honton
>Priority: Critical
> Fix For: 1.9.0
>
>
> Quite annoyingly with Jackson 2+ the base package name has changed from
> org.codehaus.jackson to com.fasterxml.jackson so in addition to changing the
> dependencies from:
> {code:xml}
>
> org.codehaus.jackson
> jackson-core-asl
> ${jackson.version}
>
>
> org.codehaus.jackson
> jackson-mapper-asl
> ${jackson.version}
>
> {code}
> to:
> {code:xml}
>
> com.fasterxml.jackson.core
> jackson-core
> ${jackson.version}
>
>
> com.fasterxml.jackson.core
> jackson-databind
> ${jackson.version}
>
> {code}
> the base package in the code needs to be updated. More info can be found
> [here|http://wiki.fasterxml.com/JacksonUpgradeFrom19To20], I am happy to do
> the work just let me know what is preferable i.e. should I just attach a
> patch to this issue?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AVRO-2175) website refactor
Sean Busbey created AVRO-2175: - Summary: website refactor Key: AVRO-2175 URL: https://issues.apache.org/jira/browse/AVRO-2175 Project: Avro Issue Type: Task Components: community Reporter: Sean Busbey Assignee: Sean Busbey https://lists.apache.org/thread.html/4b804f4e99dc9975c42af59485225808846648b90015f04ab9787246@%3Cdev.avro.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2171) Add current-event banner to project website
[ https://issues.apache.org/jira/browse/AVRO-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2171: -- Status: In Progress (was: Patch Available) this patch has the same issue as AVRO-2170: we're currently relying on a reference to Apache Hadoop's web materials. Presumably from when we were still a sub-project under that PMC. we'll need to fork things off before we can edit them. > Add current-event banner to project website > --- > > Key: AVRO-2171 > URL: https://issues.apache.org/jira/browse/AVRO-2171 > Project: Avro > Issue Type: Improvement > Components: community >Reporter: Roy Lenferink >Assignee: Roy Lenferink >Priority: Major > Labels: website > Attachments: AVRO-2171.patch > > > As this e-mail by Rich Bowen states, every project should have an > event-banner included in its website: > [https://lists.apache.org/thread.html/d672b1849f6668c0f67ff4c71b20bbb4f56a49a1777607b12643d1dc@%3Cdev.community.apache.org%3E] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2170) Update footer of website to use Apache Avro instead of Apache Hadoop
[
https://issues.apache.org/jira/browse/AVRO-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446863#comment-16446863
]
Sean Busbey commented on AVRO-2170:
---
shoot. this won't work as-is, because author/skins is actually an svn external
reference ot part of the hadoop project. specifically:
https://svn.apache.org/repos/asf/hadoop/common/site/main/author/src/documentation/skins
And they presumably will want their website to still refer to Hadoop. So I
think we need to make a copy of skins and then alter it?
> Update footer of website to use Apache Avro instead of Apache Hadoop
>
>
> Key: AVRO-2170
> URL: https://issues.apache.org/jira/browse/AVRO-2170
> Project: Avro
> Issue Type: Improvement
> Components: community
>Reporter: Roy Lenferink
>Assignee: Roy Lenferink
>Priority: Minor
> Labels: website
> Attachments: AVRO-2170.patch
>
>
> Currently the footer of the website states:
> {quote}Apache Hadoop, Hadoop, Apache, the Apache feather logo, and the Apache
> Hadoop project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.
> {quote}
> This should be changed to:
> {quote}Apache Avro, Avro, Apache, the Apache feather logo, and the Apache
> Avro project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2170) Update footer of website to use Apache Avro instead of Apache Hadoop
[
https://issues.apache.org/jira/browse/AVRO-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2170:
--
Status: In Progress (was: Patch Available)
> Update footer of website to use Apache Avro instead of Apache Hadoop
>
>
> Key: AVRO-2170
> URL: https://issues.apache.org/jira/browse/AVRO-2170
> Project: Avro
> Issue Type: Improvement
> Components: community
>Reporter: Roy Lenferink
>Assignee: Roy Lenferink
>Priority: Minor
> Labels: website
> Attachments: AVRO-2170.patch
>
>
> Currently the footer of the website states:
> {quote}Apache Hadoop, Hadoop, Apache, the Apache feather logo, and the Apache
> Hadoop project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.
> {quote}
> This should be changed to:
> {quote}Apache Avro, Avro, Apache, the Apache feather logo, and the Apache
> Avro project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (AVRO-2171) Add current-event banner to project website
[ https://issues.apache.org/jira/browse/AVRO-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned AVRO-2171: - Assignee: Roy Lenferink > Add current-event banner to project website > --- > > Key: AVRO-2171 > URL: https://issues.apache.org/jira/browse/AVRO-2171 > Project: Avro > Issue Type: Improvement > Components: community >Reporter: Roy Lenferink >Assignee: Roy Lenferink >Priority: Major > Labels: website > Attachments: AVRO-2171.patch > > > As this e-mail by Rich Bowen states, every project should have an > event-banner included in its website: > [https://lists.apache.org/thread.html/d672b1849f6668c0f67ff4c71b20bbb4f56a49a1777607b12643d1dc@%3Cdev.community.apache.org%3E] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2170) Update footer of website to use Apache Avro instead of Apache Hadoop
[
https://issues.apache.org/jira/browse/AVRO-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2170:
--
Status: Patch Available (was: Open)
> Update footer of website to use Apache Avro instead of Apache Hadoop
>
>
> Key: AVRO-2170
> URL: https://issues.apache.org/jira/browse/AVRO-2170
> Project: Avro
> Issue Type: Improvement
> Components: community
>Reporter: Roy Lenferink
>Assignee: Roy Lenferink
>Priority: Minor
> Labels: website
> Attachments: AVRO-2170.patch
>
>
> Currently the footer of the website states:
> {quote}Apache Hadoop, Hadoop, Apache, the Apache feather logo, and the Apache
> Hadoop project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.
> {quote}
> This should be changed to:
> {quote}Apache Avro, Avro, Apache, the Apache feather logo, and the Apache
> Avro project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2170) Update footer of website to use Apache Avro instead of Apache Hadoop
[
https://issues.apache.org/jira/browse/AVRO-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2170:
--
Component/s: community
> Update footer of website to use Apache Avro instead of Apache Hadoop
>
>
> Key: AVRO-2170
> URL: https://issues.apache.org/jira/browse/AVRO-2170
> Project: Avro
> Issue Type: Improvement
> Components: community
>Reporter: Roy Lenferink
>Assignee: Roy Lenferink
>Priority: Minor
> Labels: website
> Attachments: AVRO-2170.patch
>
>
> Currently the footer of the website states:
> {quote}Apache Hadoop, Hadoop, Apache, the Apache feather logo, and the Apache
> Hadoop project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.
> {quote}
> This should be changed to:
> {quote}Apache Avro, Avro, Apache, the Apache feather logo, and the Apache
> Avro project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AVRO-2171) Add current-event banner to project website
[ https://issues.apache.org/jira/browse/AVRO-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2171: -- Component/s: community > Add current-event banner to project website > --- > > Key: AVRO-2171 > URL: https://issues.apache.org/jira/browse/AVRO-2171 > Project: Avro > Issue Type: Improvement > Components: community >Reporter: Roy Lenferink >Assignee: Roy Lenferink >Priority: Major > Labels: website > Attachments: AVRO-2171.patch > > > As this e-mail by Rich Bowen states, every project should have an > event-banner included in its website: > [https://lists.apache.org/thread.html/d672b1849f6668c0f67ff4c71b20bbb4f56a49a1777607b12643d1dc@%3Cdev.community.apache.org%3E] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2171) Add current-event banner to project website
[ https://issues.apache.org/jira/browse/AVRO-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2171: -- Status: Patch Available (was: Open) > Add current-event banner to project website > --- > > Key: AVRO-2171 > URL: https://issues.apache.org/jira/browse/AVRO-2171 > Project: Avro > Issue Type: Improvement > Components: community >Reporter: Roy Lenferink >Assignee: Roy Lenferink >Priority: Major > Labels: website > Attachments: AVRO-2171.patch > > > As this e-mail by Rich Bowen states, every project should have an > event-banner included in its website: > [https://lists.apache.org/thread.html/d672b1849f6668c0f67ff4c71b20bbb4f56a49a1777607b12643d1dc@%3Cdev.community.apache.org%3E] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-2170) Update footer of website to use Apache Avro instead of Apache Hadoop
[
https://issues.apache.org/jira/browse/AVRO-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned AVRO-2170:
-
Assignee: Roy Lenferink
> Update footer of website to use Apache Avro instead of Apache Hadoop
>
>
> Key: AVRO-2170
> URL: https://issues.apache.org/jira/browse/AVRO-2170
> Project: Avro
> Issue Type: Improvement
> Components: community
>Reporter: Roy Lenferink
>Assignee: Roy Lenferink
>Priority: Minor
> Labels: website
> Attachments: AVRO-2170.patch
>
>
> Currently the footer of the website states:
> {quote}Apache Hadoop, Hadoop, Apache, the Apache feather logo, and the Apache
> Hadoop project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.
> {quote}
> This should be changed to:
> {quote}Apache Avro, Avro, Apache, the Apache feather logo, and the Apache
> Avro project logo are either registered trademarks or trademarks of the
> Apache Software Foundation in the United States and other countries.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2162) Add Zstandard compression to avro file format
[ https://issues.apache.org/jira/browse/AVRO-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409699#comment-16409699 ] Sean Busbey commented on AVRO-2162: --- this would be great! > Add Zstandard compression to avro file format > - > > Key: AVRO-2162 > URL: https://issues.apache.org/jira/browse/AVRO-2162 > Project: Avro > Issue Type: Improvement > Components: java >Reporter: Scott Carey >Priority: Major > > I'd like to add Zstandard compression for Avro. > At compression level 1 It is almost as fast as Snappy at compression, with > compression ratios more like gzip. At higher levels of compression, it is > more compact than gzip -9 with much lower CPU when compressing and roughly 3x > faster decompression. > > Adding it to Java is fairly easy. We'll need to say something about it in > the spec however, as an 'optinal' codec. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2041) set up gitbox integration
[ https://issues.apache.org/jira/browse/AVRO-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364168#comment-16364168 ] Sean Busbey commented on AVRO-2041: --- thanks for taking this over Niels! > set up gitbox integration > - > > Key: AVRO-2041 > URL: https://issues.apache.org/jira/browse/AVRO-2041 > Project: Avro > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Niels Basjes >Priority: Major > > We got consensus back in may about [turning on gitbox > integration|https://lists.apache.org/thread.html/cdd8ba14c1bf8aca2d71d09862e9780f2dc46af414ed78b1e3fd9c56@%3Cdev.avro.apache.org%3E] > so do it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2112) c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies
[ https://issues.apache.org/jira/browse/AVRO-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2112: -- Status: Patch Available (was: Open) > c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies > > > Key: AVRO-2112 > URL: https://issues.apache.org/jira/browse/AVRO-2112 > Project: Avro > Issue Type: Improvement > Components: csharp > Environment: - Visual Studio For Mac > - Visual Studio 2017 > - mono 5.4.1.7 MacOSX > - dotnet 2.0 (MacOSX and Windows 10) >Reporter: Miljenko Cvjetko >Assignee: Miljenko Cvjetko >Priority: Minor > Original Estimate: 0h > Remaining Estimate: 0h > > Suugestion is to add .NET Standard/Core support. > In order to support modern/new .NET (both standard netfx and dotnet core) it > is necessary to convert projects to support .NET Standard Libraries. > - conversion to .NET Standard [DONE] > - added dotnet core sample (Avro.perf) [DONE] > - added netfx (standrd .NET) sample Avro.perf.netfx [DONE] > - unit testing updated to use NUnit 3 > Url for the github forked repo with branch will be added. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-2112) c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies
[ https://issues.apache.org/jira/browse/AVRO-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned AVRO-2112: - Assignee: Miljenko Cvjetko > c# (.net) port to .NET Standard 2.0 and nuget (package) dependencies > > > Key: AVRO-2112 > URL: https://issues.apache.org/jira/browse/AVRO-2112 > Project: Avro > Issue Type: Improvement > Components: csharp > Environment: - Visual Studio For Mac > - Visual Studio 2017 > - mono 5.4.1.7 MacOSX > - dotnet 2.0 (MacOSX and Windows 10) >Reporter: Miljenko Cvjetko >Assignee: Miljenko Cvjetko >Priority: Minor > Original Estimate: 0h > Remaining Estimate: 0h > > Suugestion is to add .NET Standard/Core support. > In order to support modern/new .NET (both standard netfx and dotnet core) it > is necessary to convert projects to support .NET Standard Libraries. > - conversion to .NET Standard [DONE] > - added dotnet core sample (Avro.perf) [DONE] > - added netfx (standrd .NET) sample Avro.perf.netfx [DONE] > - unit testing updated to use NUnit 3 > Url for the github forked repo with branch will be added. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2127) Throwing more specific exception if an avro file has currupted magic
[ https://issues.apache.org/jira/browse/AVRO-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2127: -- Status: Patch Available (was: Open) > Throwing more specific exception if an avro file has currupted magic > > > Key: AVRO-2127 > URL: https://issues.apache.org/jira/browse/AVRO-2127 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.8.2 >Reporter: Dmitrii Bundin >Assignee: Dmitrii Bundin >Priority: Minor > > Curently we have IOException thrown if an avro file has incorrect magic. > It seems reasonable to throw a subclass of IOException to be able to handle > incorrect magic (in case length are match) in user code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-2127) Throwing more specific exception if an avro file has currupted magic
[ https://issues.apache.org/jira/browse/AVRO-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned AVRO-2127: - Assignee: Dmitrii Bundin > Throwing more specific exception if an avro file has currupted magic > > > Key: AVRO-2127 > URL: https://issues.apache.org/jira/browse/AVRO-2127 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.8.2 >Reporter: Dmitrii Bundin >Assignee: Dmitrii Bundin >Priority: Minor > > Curently we have IOException thrown if an avro file has incorrect magic. > It seems reasonable to throw a subclass of IOException to be able to handle > incorrect magic (in case length are match) in user code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2142) SchemaBuilder Java documentation code snippet is not valid
[ https://issues.apache.org/jira/browse/AVRO-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2142: -- Status: Patch Available (was: Open) > SchemaBuilder Java documentation code snippet is not valid > -- > > Key: AVRO-2142 > URL: https://issues.apache.org/jira/browse/AVRO-2142 > Project: Avro > Issue Type: Improvement > Components: doc, java >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Trivial > > The code snippet in SchemaBuilder is invalid, it has invalid quotes and > misses one call in the builder chain. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AVRO-2142) SchemaBuilder Java documentation code snippet is not valid
[ https://issues.apache.org/jira/browse/AVRO-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned AVRO-2142: - Assignee: Ismaël Mejía > SchemaBuilder Java documentation code snippet is not valid > -- > > Key: AVRO-2142 > URL: https://issues.apache.org/jira/browse/AVRO-2142 > Project: Avro > Issue Type: Improvement > Components: doc, java >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Trivial > > The code snippet in SchemaBuilder is invalid, it has invalid quotes and > misses one call in the builder chain. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AVRO-1815) Incompatible schema change not detected when wrapped in a UNION
[
https://issues.apache.org/jira/browse/AVRO-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved AVRO-1815.
---
Resolution: Duplicate
closing as stale under the assumption that AVRO-1883 fixes this. If not, please
reopen.
> Incompatible schema change not detected when wrapped in a UNION
> ---
>
> Key: AVRO-1815
> URL: https://issues.apache.org/jira/browse/AVRO-1815
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.7.7
>Reporter: Martin Boyle
> Attachments: AVRO-1815.patch
>
>
> An incompatible schema change is not detected when it is in a UNION and the
> change is to the value type of a map e.g.
> field
> { "name": "segmentEkv", "type": ["null", {"type": "map", "values": {"type":
> "map", "values": "string"}}], "default": null},
> changes to
> { "name": "segmentEkv", "type": ["null", {"type": "map", "values": {"type":
> "array", "items": "int"}}], "default": null},
> The SchemaValidatorBuilder() will pass this as being compatible. Whereas
> SchemaCompatibility.check_reader_writer_compatibility will return an
> incompatible result. The problem for me is that the Confluent Schema
> Registry uses SchemaValidatorBuilder.
> Problem appears to be that while the ResolvingGrammerGenerator correctly
> marks the field as being an incompatible change, the check for errors on the
> Symbol object does not descend into the UnionAdjustActionField
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-1815) Incompatible schema change not detected when wrapped in a UNION
[
https://issues.apache.org/jira/browse/AVRO-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222483#comment-16222483
]
Sean Busbey commented on AVRO-1815:
---
AVRO-1883 is in branch-1.7 now. [~martinboyleie] or [~gdeschut] would you mind
seeing if current branch-1.7 fixes your issue?
> Incompatible schema change not detected when wrapped in a UNION
> ---
>
> Key: AVRO-1815
> URL: https://issues.apache.org/jira/browse/AVRO-1815
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.7.7
>Reporter: Martin Boyle
> Attachments: AVRO-1815.patch
>
>
> An incompatible schema change is not detected when it is in a UNION and the
> change is to the value type of a map e.g.
> field
> { "name": "segmentEkv", "type": ["null", {"type": "map", "values": {"type":
> "map", "values": "string"}}], "default": null},
> changes to
> { "name": "segmentEkv", "type": ["null", {"type": "map", "values": {"type":
> "array", "items": "int"}}], "default": null},
> The SchemaValidatorBuilder() will pass this as being compatible. Whereas
> SchemaCompatibility.check_reader_writer_compatibility will return an
> incompatible result. The problem for me is that the Confluent Schema
> Registry uses SchemaValidatorBuilder.
> Problem appears to be that while the ResolvingGrammerGenerator correctly
> marks the field as being an incompatible change, the check for errors on the
> Symbol object does not descend into the UnionAdjustActionField
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (AVRO-1883) Schema validator cannot find broken backwards compatibility in Union type elements
[
https://issues.apache.org/jira/browse/AVRO-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey resolved AVRO-1883.
---
Resolution: Fixed
Fix Version/s: 1.7.8
verify was clean this time. Pushed to 1.7.
> Schema validator cannot find broken backwards compatibility in Union type
> elements
> --
>
> Key: AVRO-1883
> URL: https://issues.apache.org/jira/browse/AVRO-1883
> Project: Avro
> Issue Type: Bug
>Affects Versions: 1.8.1
>Reporter: Yibing Shi
>Assignee: Yibing Shi
>Priority: Critical
> Fix For: 1.7.8, 1.9.0, 1.8.2
>
> Attachments: AVRO-1883.1.patch
>
>
> Consider below 2 schemas:
> *Schema 1*:
> {noformat}
> [
> {
> "type": "record",
> "name": "rec1",
> "fields": [
> {
> "name": "age",
> "type": "long"
> }
> ]
> },
> {
> "type": "record",
> "name": "rec2",
> "fields": [
> {
> "name": "username",
> "type": "string"
> }
> ]
> }
> ]
> {noformat}
> *Schema 2*:
> {noformat}
> [
> {
> "type": "record",
> "name": "rec1",
> "fields": [
> {
> "name": "age",
> "type": "long"
> },
> {
> "name": "address",
> "type": "string"
> }
> ]
> },
> {
> "type": "record",
> "name": "rec2",
> "fields": [
> {
> "name": "username",
> "type": "string"
> }
> ]
> }
> ]
> {noformat}
> The {{rec1}} field in these 2 unions are not compatible, because the
> {{address}} field of {{rec1}} in the second one is not nullable. However, if
> we check them with validate like below, validator doesn't return any error:
> {code}
> final SchemaValidator backwardValidator = new
> SchemaValidatorBuilder().canReadStrategy().validateLatest();
> final Schema schema1 = new Schema.Parser().parse(schema1Str);
> final Schema schema2 = new Schema.Parser().parse(schema2Str);
> backwardValidator.validate(schema2, Arrays.asList(schema1));
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-1883) Schema validator cannot find broken backwards compatibility in Union type elements
[
https://issues.apache.org/jira/browse/AVRO-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222444#comment-16222444
]
Sean Busbey commented on AVRO-1883:
---
running through maven again now, since I don't remember what the failure was.
(incidentally, I cherry-picked from branch-1.8 this time)
> Schema validator cannot find broken backwards compatibility in Union type
> elements
> --
>
> Key: AVRO-1883
> URL: https://issues.apache.org/jira/browse/AVRO-1883
> Project: Avro
> Issue Type: Bug
>Affects Versions: 1.8.1
>Reporter: Yibing Shi
>Assignee: Yibing Shi
>Priority: Critical
> Fix For: 1.9.0, 1.8.2
>
> Attachments: AVRO-1883.1.patch
>
>
> Consider below 2 schemas:
> *Schema 1*:
> {noformat}
> [
> {
> "type": "record",
> "name": "rec1",
> "fields": [
> {
> "name": "age",
> "type": "long"
> }
> ]
> },
> {
> "type": "record",
> "name": "rec2",
> "fields": [
> {
> "name": "username",
> "type": "string"
> }
> ]
> }
> ]
> {noformat}
> *Schema 2*:
> {noformat}
> [
> {
> "type": "record",
> "name": "rec1",
> "fields": [
> {
> "name": "age",
> "type": "long"
> },
> {
> "name": "address",
> "type": "string"
> }
> ]
> },
> {
> "type": "record",
> "name": "rec2",
> "fields": [
> {
> "name": "username",
> "type": "string"
> }
> ]
> }
> ]
> {noformat}
> The {{rec1}} field in these 2 unions are not compatible, because the
> {{address}} field of {{rec1}} in the second one is not nullable. However, if
> we check them with validate like below, validator doesn't return any error:
> {code}
> final SchemaValidator backwardValidator = new
> SchemaValidatorBuilder().canReadStrategy().validateLatest();
> final Schema schema1 = new Schema.Parser().parse(schema1Str);
> final Schema schema2 = new Schema.Parser().parse(schema2Str);
> backwardValidator.validate(schema2, Arrays.asList(schema1));
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (AVRO-2102) upgrade surefire plugin on maintenance branches
[ https://issues.apache.org/jira/browse/AVRO-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved AVRO-2102. --- Resolution: Fixed Fix Version/s: 1.8.3 1.7.8 > upgrade surefire plugin on maintenance branches > --- > > Key: AVRO-2102 > URL: https://issues.apache.org/jira/browse/AVRO-2102 > Project: Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.8, 1.8.3 >Reporter: Sean Busbey >Assignee: Suraj Acharya >Priority: Minor > Fix For: 1.7.8, 1.8.3 > > Attachments: AVRO-2102-branch-1.8.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2102) upgrade surefire plugin on maintenance branches
[ https://issues.apache.org/jira/browse/AVRO-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2102: -- Attachment: AVRO-2102-branch-1.8.v0.patch attaching the patch I pushed to branch-1.7 and branch-1.8 after updating Suraj's commit for message. > upgrade surefire plugin on maintenance branches > --- > > Key: AVRO-2102 > URL: https://issues.apache.org/jira/browse/AVRO-2102 > Project: Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.8, 1.8.3 >Reporter: Sean Busbey >Assignee: Suraj Acharya >Priority: Minor > Attachments: AVRO-2102-branch-1.8.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AVRO-2102) upgrade surefire plugin on maintenance branches
[ https://issues.apache.org/jira/browse/AVRO-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned AVRO-2102: - Assignee: Suraj Acharya > upgrade surefire plugin on maintenance branches > --- > > Key: AVRO-2102 > URL: https://issues.apache.org/jira/browse/AVRO-2102 > Project: Avro > Issue Type: Task > Components: build >Affects Versions: 1.7.8, 1.8.3 >Reporter: Sean Busbey >Assignee: Suraj Acharya >Priority: Minor > Attachments: AVRO-2102-branch-1.8.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AVRO-2102) upgrade surefire plugin on maintenance branches
Sean Busbey created AVRO-2102: - Summary: upgrade surefire plugin on maintenance branches Key: AVRO-2102 URL: https://issues.apache.org/jira/browse/AVRO-2102 Project: Avro Issue Type: Task Components: build Affects Versions: 1.7.8, 1.8.3 Reporter: Sean Busbey Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (AVRO-1883) Schema validator cannot find broken backwards compatibility in Union type elements
[
https://issues.apache.org/jira/browse/AVRO-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reopened AVRO-1883:
---
reopening to complete the branch-1.7 backport.
> Schema validator cannot find broken backwards compatibility in Union type
> elements
> --
>
> Key: AVRO-1883
> URL: https://issues.apache.org/jira/browse/AVRO-1883
> Project: Avro
> Issue Type: Bug
>Affects Versions: 1.8.1
>Reporter: Yibing Shi
>Assignee: Yibing Shi
>Priority: Critical
> Fix For: 1.9.0, 1.8.2
>
> Attachments: AVRO-1883.1.patch
>
>
> Consider below 2 schemas:
> *Schema 1*:
> {noformat}
> [
> {
> "type": "record",
> "name": "rec1",
> "fields": [
> {
> "name": "age",
> "type": "long"
> }
> ]
> },
> {
> "type": "record",
> "name": "rec2",
> "fields": [
> {
> "name": "username",
> "type": "string"
> }
> ]
> }
> ]
> {noformat}
> *Schema 2*:
> {noformat}
> [
> {
> "type": "record",
> "name": "rec1",
> "fields": [
> {
> "name": "age",
> "type": "long"
> },
> {
> "name": "address",
> "type": "string"
> }
> ]
> },
> {
> "type": "record",
> "name": "rec2",
> "fields": [
> {
> "name": "username",
> "type": "string"
> }
> ]
> }
> ]
> {noformat}
> The {{rec1}} field in these 2 unions are not compatible, because the
> {{address}} field of {{rec1}} in the second one is not nullable. However, if
> we check them with validate like below, validator doesn't return any error:
> {code}
> final SchemaValidator backwardValidator = new
> SchemaValidatorBuilder().canReadStrategy().validateLatest();
> final Schema schema1 = new Schema.Parser().parse(schema1Str);
> final Schema schema2 = new Schema.Parser().parse(schema2Str);
> backwardValidator.validate(schema2, Arrays.asList(schema1));
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-1883) Schema validator cannot find broken backwards compatibility in Union type elements
[
https://issues.apache.org/jira/browse/AVRO-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176633#comment-16176633
]
Sean Busbey commented on AVRO-1883:
---
I started backporting this to branch-1.7 (see AVRO-1815) but got a failure on
one of the Trevni tests. need to chase down if the failure is present without
the backport.
> Schema validator cannot find broken backwards compatibility in Union type
> elements
> --
>
> Key: AVRO-1883
> URL: https://issues.apache.org/jira/browse/AVRO-1883
> Project: Avro
> Issue Type: Bug
>Affects Versions: 1.8.1
>Reporter: Yibing Shi
>Assignee: Yibing Shi
>Priority: Critical
> Fix For: 1.9.0, 1.8.2
>
> Attachments: AVRO-1883.1.patch
>
>
> Consider below 2 schemas:
> *Schema 1*:
> {noformat}
> [
> {
> "type": "record",
> "name": "rec1",
> "fields": [
> {
> "name": "age",
> "type": "long"
> }
> ]
> },
> {
> "type": "record",
> "name": "rec2",
> "fields": [
> {
> "name": "username",
> "type": "string"
> }
> ]
> }
> ]
> {noformat}
> *Schema 2*:
> {noformat}
> [
> {
> "type": "record",
> "name": "rec1",
> "fields": [
> {
> "name": "age",
> "type": "long"
> },
> {
> "name": "address",
> "type": "string"
> }
> ]
> },
> {
> "type": "record",
> "name": "rec2",
> "fields": [
> {
> "name": "username",
> "type": "string"
> }
> ]
> }
> ]
> {noformat}
> The {{rec1}} field in these 2 unions are not compatible, because the
> {{address}} field of {{rec1}} in the second one is not nullable. However, if
> we check them with validate like below, validator doesn't return any error:
> {code}
> final SchemaValidator backwardValidator = new
> SchemaValidatorBuilder().canReadStrategy().validateLatest();
> final Schema schema1 = new Schema.Parser().parse(schema1Str);
> final Schema schema2 = new Schema.Parser().parse(schema2Str);
> backwardValidator.validate(schema2, Arrays.asList(schema1));
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2077) Avro tools should have an option to check reader-writer compatibility
[ https://issues.apache.org/jira/browse/AVRO-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176419#comment-16176419 ] Sean Busbey commented on AVRO-2077: --- this would be great! if we can make it work on earlier release lines that would be even better. > Avro tools should have an option to check reader-writer compatibility > - > > Key: AVRO-2077 > URL: https://issues.apache.org/jira/browse/AVRO-2077 > Project: Avro > Issue Type: New Feature > Components: java, tools >Reporter: Nandor Kollar >Priority: Minor > Fix For: 1.9.0 > > > It seems that avro-tools doesn't have any option to check for a given Avro > file and a given reader schema (JSON file or URL) that the reader's schema is > compatible with the writer's schema. This tool should report every > compatibility problem (if there's any). > According my knowledge, there's no such option for avro-tools as of now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2077) Avro tools should have an option to check reader-writer compatibility
[ https://issues.apache.org/jira/browse/AVRO-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2077: -- Component/s: tools > Avro tools should have an option to check reader-writer compatibility > - > > Key: AVRO-2077 > URL: https://issues.apache.org/jira/browse/AVRO-2077 > Project: Avro > Issue Type: New Feature > Components: java, tools >Reporter: Nandor Kollar >Priority: Minor > Fix For: 1.9.0 > > > It seems that avro-tools doesn't have any option to check for a given Avro > file and a given reader schema (JSON file or URL) that the reader's schema is > compatible with the writer's schema. This tool should report every > compatibility problem (if there's any). > According my knowledge, there's no such option for avro-tools as of now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2077) Avro tools should have an option to check reader-writer compatibility
[ https://issues.apache.org/jira/browse/AVRO-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2077: -- Issue Type: New Feature (was: Bug) > Avro tools should have an option to check reader-writer compatibility > - > > Key: AVRO-2077 > URL: https://issues.apache.org/jira/browse/AVRO-2077 > Project: Avro > Issue Type: New Feature > Components: java, tools >Reporter: Nandor Kollar >Priority: Minor > Fix For: 1.9.0 > > > It seems that avro-tools doesn't have any option to check for a given Avro > file and a given reader schema (JSON file or URL) that the reader's schema is > compatible with the writer's schema. This tool should report every > compatibility problem (if there's any). > According my knowledge, there's no such option for avro-tools as of now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AVRO-1815) Incompatible schema change not detected when wrapped in a UNION
[
https://issues.apache.org/jira/browse/AVRO-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169986#comment-16169986
]
Sean Busbey commented on AVRO-1815:
---
yeah I think AVRO-1883 should come back to branch-1.7 if possible.
> Incompatible schema change not detected when wrapped in a UNION
> ---
>
> Key: AVRO-1815
> URL: https://issues.apache.org/jira/browse/AVRO-1815
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.7.7
>Reporter: Martin Boyle
> Attachments: AVRO-1815.patch
>
>
> An incompatible schema change is not detected when it is in a UNION and the
> change is to the value type of a map e.g.
> field
> { "name": "segmentEkv", "type": ["null", {"type": "map", "values": {"type":
> "map", "values": "string"}}], "default": null},
> changes to
> { "name": "segmentEkv", "type": ["null", {"type": "map", "values": {"type":
> "array", "items": "int"}}], "default": null},
> The SchemaValidatorBuilder() will pass this as being compatible. Whereas
> SchemaCompatibility.check_reader_writer_compatibility will return an
> incompatible result. The problem for me is that the Confluent Schema
> Registry uses SchemaValidatorBuilder.
> Problem appears to be that while the ResolvingGrammerGenerator correctly
> marks the field as being an incompatible change, the check for errors on the
> Symbol object does not descend into the UnionAdjustActionField
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2019) Improve documentation for logical type annotations in IDL
[
https://issues.apache.org/jira/browse/AVRO-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167884#comment-16167884
]
Sean Busbey commented on AVRO-2019:
---
it looks like the timestamp related logical types are only in 1.8+. Folks have
any preference between getting them added back into the 1.7 line vs updating
this doc to use decimal (which is the only logical type defined in 1.7 AFAICT)
> Improve documentation for logical type annotations in IDL
> -
>
> Key: AVRO-2019
> URL: https://issues.apache.org/jira/browse/AVRO-2019
> Project: Avro
> Issue Type: Improvement
> Components: doc, logical types
>Reporter: Andrew Rosca
>Assignee: Andrew Rosca
>Priority: Minor
> Attachments: AVRO-2019.patch
>
>
> The IDL documentation lacks information for how annotations can be specified
> for logical types, like in the following example:
> {code}
> protocol test {
> record test {
> @logicalType("timestamp-millis")
> long time;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (AVRO-2019) Improve documentation for logical type annotations in IDL
[
https://issues.apache.org/jira/browse/AVRO-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned AVRO-2019:
-
Assignee: Andrew Rosca
> Improve documentation for logical type annotations in IDL
> -
>
> Key: AVRO-2019
> URL: https://issues.apache.org/jira/browse/AVRO-2019
> Project: Avro
> Issue Type: Improvement
> Components: doc, logical types
>Reporter: Andrew Rosca
>Assignee: Andrew Rosca
>Priority: Minor
> Attachments: AVRO-2019.patch
>
>
> The IDL documentation lacks information for how annotations can be specified
> for logical types, like in the following example:
> {code}
> protocol test {
> record test {
> @logicalType("timestamp-millis")
> long time;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2019) Improve documentation for logical type annotations in IDL
[
https://issues.apache.org/jira/browse/AVRO-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167839#comment-16167839
]
Sean Busbey commented on AVRO-2019:
---
Given the docs on logical types elsewhere, I think it's worth calling out this
particular use of annotations in IDL. Folks who rely on IDL to use Avro
shouldn't have to know the implementation details of how logical types are
implemented via properties so that they can reason out how to use them in IDL.
> Improve documentation for logical type annotations in IDL
> -
>
> Key: AVRO-2019
> URL: https://issues.apache.org/jira/browse/AVRO-2019
> Project: Avro
> Issue Type: Improvement
> Components: doc, logical types
>Reporter: Andrew Rosca
>Priority: Minor
> Attachments: AVRO-2019.patch
>
>
> The IDL documentation lacks information for how annotations can be specified
> for logical types, like in the following example:
> {code}
> protocol test {
> record test {
> @logicalType("timestamp-millis")
> long time;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2069) Use primitive fields in generated getters & setters in Java code
[ https://issues.apache.org/jira/browse/AVRO-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167822#comment-16167822 ] Sean Busbey commented on AVRO-2069: --- I like this as the approach going forward, but this will be an incompatible change. Should we make a follow on jira to make a backwards compatible version for 1.8/1.7 (something like getUnboxedFieldName in addition to getFieldName). > Use primitive fields in generated getters & setters in Java code > > > Key: AVRO-2069 > URL: https://issues.apache.org/jira/browse/AVRO-2069 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Daniil Gitelson >Assignee: Daniil Gitelson > > Currently, for primitive types (such as int, long, etc) generated getters and > setters return and accept java.lang.* boxed (while fields actually holds > primitive values). This is inefeccient and produces code boilerplate. > Changed this behaviour in pull request: > https://github.com/apache/avro/pull/243 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1810) GenericDatumWriter broken with Enum
[
https://issues.apache.org/jira/browse/AVRO-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-1810:
--
Fix Version/s: 1.9.0
1.8.4
> GenericDatumWriter broken with Enum
> ---
>
> Key: AVRO-1810
> URL: https://issues.apache.org/jira/browse/AVRO-1810
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.0
>Reporter: Ryon Day
>Priority: Blocker
> Fix For: 1.9.0, 1.8.4
>
>
> {panel:title=Description|titleBGColor=#3FA|bgColor=#DDD}
> Using the GenericDatumWriter with either Generic OR SpecificRecord will break
> if an Enum is present.
> {panel}
> {panel:title=Steps To Reproduce|titleBGColor=#8DB|bgColor=#DDD}
> I have been tracking Avro decoding oddities for a while.
> The tests for this issue can be found
> [here|https://github.com/ryonday/avroDecodingHelp/blob/master/src/test/java/com/ryonday/test/Avro180EnumFail.java]
> {panel}
> {panel:title=Notes|titleBGColor=#3AF|bgColor=#DDD}
> Due to the debacle that is the Avro "UTF8" object, we have been avoiding it
> by using the following scheme:
> * Write incoming records to a byte array using the GenericDatumWriter
> * Read back the byte array to our compiled Java domain objects using a
> SpecificDatumWriter
> This worked great with Avro 1.7.7, and this is a binary-incompatable breaking
> change with 1.8.0.
> This would appear to be caused by an addition in the
> {{GenericDatumWriter:163-164}}:
> {code}
> if (!data.isEnum(datum))
> throw new AvroTypeException("Not an enum: "+datum);
> {code}
> {panel}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-1810) GenericDatumWriter broken with Enum
[
https://issues.apache.org/jira/browse/AVRO-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166299#comment-16166299
]
Sean Busbey commented on AVRO-1810:
---
Personally I'd love to see this get fixed. I'm happy to review, but almost
certainly don't have the time to post a patch up. If anyone wants to try to
work through things please either ping here or ping me off-list and I'll help
get you up and running.
> GenericDatumWriter broken with Enum
> ---
>
> Key: AVRO-1810
> URL: https://issues.apache.org/jira/browse/AVRO-1810
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.8.0
>Reporter: Ryon Day
>Priority: Blocker
>
> {panel:title=Description|titleBGColor=#3FA|bgColor=#DDD}
> Using the GenericDatumWriter with either Generic OR SpecificRecord will break
> if an Enum is present.
> {panel}
> {panel:title=Steps To Reproduce|titleBGColor=#8DB|bgColor=#DDD}
> I have been tracking Avro decoding oddities for a while.
> The tests for this issue can be found
> [here|https://github.com/ryonday/avroDecodingHelp/blob/master/src/test/java/com/ryonday/test/Avro180EnumFail.java]
> {panel}
> {panel:title=Notes|titleBGColor=#3AF|bgColor=#DDD}
> Due to the debacle that is the Avro "UTF8" object, we have been avoiding it
> by using the following scheme:
> * Write incoming records to a byte array using the GenericDatumWriter
> * Read back the byte array to our compiled Java domain objects using a
> SpecificDatumWriter
> This worked great with Avro 1.7.7, and this is a binary-incompatable breaking
> change with 1.8.0.
> This would appear to be caused by an addition in the
> {{GenericDatumWriter:163-164}}:
> {code}
> if (!data.isEnum(datum))
> throw new AvroTypeException("Not an enum: "+datum);
> {code}
> {panel}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2066) Wrong schema file used in one TestSpecificCompiler test case
[
https://issues.apache.org/jira/browse/AVRO-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150549#comment-16150549
]
Sean Busbey commented on AVRO-2066:
---
thanks for the contribution! In the future, please generate your patches using
{{git format-patch}} so that it'll already have your authorship information and
a commit message.
> Wrong schema file used in one TestSpecificCompiler test case
>
>
> Key: AVRO-2066
> URL: https://issues.apache.org/jira/browse/AVRO-2066
> Project: Avro
> Issue Type: Bug
> Components: java
>Reporter: Nandor Kollar
>Assignee: Nandor Kollar
>Priority: Trivial
> Fix For: 1.9.0, 1.8.3
>
> Attachments: AVRO-2066.patch
>
>
> I think instead of simple_record.avsc
> {{TestSpecificCompiler#testLogicalTypesWithMultipleFields}} meant to use
> logical_types_with_multiple_fields.avsc
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AVRO-2066) Wrong schema file used in one TestSpecificCompiler test case
[
https://issues.apache.org/jira/browse/AVRO-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-2066:
--
Resolution: Fixed
Fix Version/s: 1.8.3
1.9.0
Status: Resolved (was: Patch Available)
> Wrong schema file used in one TestSpecificCompiler test case
>
>
> Key: AVRO-2066
> URL: https://issues.apache.org/jira/browse/AVRO-2066
> Project: Avro
> Issue Type: Bug
> Components: java
>Reporter: Nandor Kollar
>Assignee: Nandor Kollar
>Priority: Trivial
> Fix For: 1.9.0, 1.8.3
>
> Attachments: AVRO-2066.patch
>
>
> I think instead of simple_record.avsc
> {{TestSpecificCompiler#testLogicalTypesWithMultipleFields}} meant to use
> logical_types_with_multiple_fields.avsc
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-2069) Use primitive fields in generated getters & setters in Java code
[ https://issues.apache.org/jira/browse/AVRO-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150545#comment-16150545 ] Sean Busbey commented on AVRO-2069: --- excellent find Anna! I'll go read back through those discussions. > Use primitive fields in generated getters & setters in Java code > > > Key: AVRO-2069 > URL: https://issues.apache.org/jira/browse/AVRO-2069 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Daniil Gitelson >Assignee: Daniil Gitelson > > Currently, for primitive types (such as int, long, etc) generated getters and > setters return and accept java.lang.* boxed (while fields actually holds > primitive values). This is inefeccient and produces code boilerplate. > Changed this behaviour in pull request: > https://github.com/apache/avro/pull/243 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AVRO-2066) Wrong schema file used in one TestSpecificCompiler test case
[
https://issues.apache.org/jira/browse/AVRO-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150514#comment-16150514
]
Sean Busbey commented on AVRO-2066:
---
+1 pushing shortly.
> Wrong schema file used in one TestSpecificCompiler test case
>
>
> Key: AVRO-2066
> URL: https://issues.apache.org/jira/browse/AVRO-2066
> Project: Avro
> Issue Type: Bug
> Components: java
>Reporter: Nandor Kollar
>Assignee: Nandor Kollar
>Priority: Trivial
> Attachments: AVRO-2066.patch
>
>
> I think instead of simple_record.avsc
> {{TestSpecificCompiler#testLogicalTypesWithMultipleFields}} meant to use
> logical_types_with_multiple_fields.avsc
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-1891) Generated Java code fails with union containing logical type
[
https://issues.apache.org/jira/browse/AVRO-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150510#comment-16150510
]
Sean Busbey commented on AVRO-1891:
---
Can someone give me a synopsis of current status? do we believe this is
implemented enough for review?
> Generated Java code fails with union containing logical type
>
>
> Key: AVRO-1891
> URL: https://issues.apache.org/jira/browse/AVRO-1891
> Project: Avro
> Issue Type: Bug
> Components: java, logical types
>Affects Versions: 1.8.1
>Reporter: Ross Black
>Priority: Blocker
> Fix For: 1.8.3
>
> Attachments: AVRO-1891.patch, AVRO-1891.yshi.1.patch,
> AVRO-1891.yshi.2.patch, AVRO-1891.yshi.3.patch, AVRO-1891.yshi.4.patch
>
>
> Example schema:
> {code}
> {
> "type": "record",
> "name": "RecordV1",
> "namespace": "org.brasslock.event",
> "fields": [
> { "name": "first", "type": ["null", {"type": "long",
> "logicalType":"timestamp-millis"}]}
> ]
> }
> {code}
> The avro compiler generates a field using the relevant joda class:
> {code}
> public org.joda.time.DateTime first
> {code}
> Running the following code to perform encoding:
> {code}
> final RecordV1 record = new
> RecordV1(DateTime.parse("2016-07-29T10:15:30.00Z"));
> final DatumWriter datumWriter = new
> SpecificDatumWriter<>(record.getSchema());
> final ByteArrayOutputStream stream = new ByteArrayOutputStream(8192);
> final BinaryEncoder encoder =
> EncoderFactory.get().directBinaryEncoder(stream, null);
> datumWriter.write(record, encoder);
> encoder.flush();
> final byte[] bytes = stream.toByteArray();
> {code}
> fails with the exception stacktrace:
> {code}
> org.apache.avro.AvroRuntimeException: Unknown datum type
> org.joda.time.DateTime: 2016-07-29T10:15:30.000Z
> at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:741)
> at
> org.apache.avro.specific.SpecificData.getSchemaName(SpecificData.java:293)
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:706)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at
> org.brasslock.avro.compiler.GeneratedRecordTest.shouldEncodeLogicalTypeInUnion(GeneratedRecordTest.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:253)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
> at sun.reflect.NativeMetho
[jira] [Commented] (AVRO-2069) Use primitive fields in generated getters & setters in Java code
[ https://issues.apache.org/jira/browse/AVRO-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150481#comment-16150481 ] Sean Busbey commented on AVRO-2069: --- we'll need to flag this as an incompatible change, but I agree that it's a great improvement. Normally users who rely on the specific compiler are performance sensitive. Do we have any idea if the use of boxed object here was a discussed design decision? The only reason I can think of to use them is it allows returning null, but that won't be the case if we're directly referencing the primitives anyways. > Use primitive fields in generated getters & setters in Java code > > > Key: AVRO-2069 > URL: https://issues.apache.org/jira/browse/AVRO-2069 > Project: Avro > Issue Type: Improvement >Affects Versions: 1.8.2 >Reporter: Daniil Gitelson >Assignee: Daniil Gitelson > > Currently, for primitive types (such as int, long, etc) generated getters and > setters return and accept java.lang.* boxed (while fields actually holds > primitive values). This is inefeccient and produces code boilerplate. > Changed this behaviour in pull request: > https://github.com/apache/avro/pull/243 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2039) Ruby encoding performance improvement
[ https://issues.apache.org/jira/browse/AVRO-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2039: -- Priority: Critical (was: Major) > Ruby encoding performance improvement > - > > Key: AVRO-2039 > URL: https://issues.apache.org/jira/browse/AVRO-2039 > Project: Avro > Issue Type: Improvement > Components: ruby >Reporter: Tim Perkins >Priority: Critical > > For a use case with a few levels of nesting and unions in several places > within the schema we saw a 5x improvement in encoding performance with these > changes to encoding using Ruby. > 1. Avoid the exhaustive validation of schemas in a union > 2. Avoid the repeated validation of nested schemas > https://github.com/apache/avro/pull/230 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2039) Ruby encoding performance improvement
[ https://issues.apache.org/jira/browse/AVRO-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-2039: -- Component/s: ruby > Ruby encoding performance improvement > - > > Key: AVRO-2039 > URL: https://issues.apache.org/jira/browse/AVRO-2039 > Project: Avro > Issue Type: Improvement > Components: ruby >Reporter: Tim Perkins > > For a use case with a few levels of nesting and unions in several places > within the schema we saw a 5x improvement in encoding performance with these > changes to encoding using Ruby. > 1. Avoid the exhaustive validation of schemas in a union > 2. Avoid the repeated validation of nested schemas > https://github.com/apache/avro/pull/230 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1574) Suboptimal arraylist creation
[ https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-1574: -- Resolution: Fixed Fix Version/s: 1.8.3 1.9.0 1.7.8 Status: Resolved (was: Patch Available) pushed to all branches. Thanks Kengo! > Suboptimal arraylist creation > - > > Key: AVRO-1574 > URL: https://issues.apache.org/jira/browse/AVRO-1574 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.7 >Reporter: Zoltan Farkas >Assignee: Kengo Seki >Priority: Trivial > Labels: beginner > Fix For: 1.7.8, 1.9.0, 1.8.3 > > Attachments: AVRO-1574.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > in Schema.java: > 1231 LockableArrayList symbols = new > LockableArrayList(); > > 1232 for (JsonNode n : symbolsNode) > > 1233 symbols.add(n.getTextValue()); > should be changed to: > 1231 LockableArrayList symbols = new > LockableArrayList(symbolsNode.size()); > > 1232 for (JsonNode n : symbolsNode) > > 1233 symbols.add(n.getTextValue()); > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (AVRO-1929) SchemaCompatibility fails to recognize string<->bytes promotion
[ https://issues.apache.org/jira/browse/AVRO-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved AVRO-1929. --- Resolution: Fixed Fix Version/s: 1.7.8 Release Note: The java SchemaCompatibility class now recognizes the possible promotion between string and bytes fields. pulled this back to branch-1.7 as well. > SchemaCompatibility fails to recognize string<->bytes promotion > --- > > Key: AVRO-1929 > URL: https://issues.apache.org/jira/browse/AVRO-1929 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.7.7, 1.8.0, 1.8.1, 1.9.0 > Environment: Any Java env >Reporter: Anders Sundelin >Assignee: Anders Sundelin >Priority: Minor > Fix For: 1.7.8, 1.9.0, 1.8.2 > > Attachments: AVRO-1929.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > The class SchemaCompatibility fails to realize the change made as of > AVRO-1553, where string-to/from-bytes promotions were allowed (according to > docs, this was integrated into 1.7.7) > The submitted fix corrects this (also updating the relevant unit test) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union
[ https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-1931: -- Resolution: Fixed Fix Version/s: 1.8.3 1.9.0 1.7.8 Release Note: Reader Schemas that have a non-union field which corresponds to a union field in a Writer Schema are now recognized as compatible if the non-union field is compatible with all entries in the writer's union field. Status: Resolved (was: Patch Available) pushd to branches. Thanks for the reviews everyone! > SchemaCompatibility fails to recognize reader compatible with all branches of > a union > - > > Key: AVRO-1931 > URL: https://issues.apache.org/jira/browse/AVRO-1931 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.7.7, 1.8.1 > Environment: Java >Reporter: Anders Sundelin >Assignee: Anders Sundelin >Priority: Minor > Labels: patch > Fix For: 1.7.8, 1.9.0, 1.8.3 > > Attachments: AVRO-1931-2.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > It is stated in the Avro spec > "if writer's is a union, but reader's is not: > If the reader's schema matches the selected writer's schema, it is > recursively resolved against it. If they do not match, an error is signalled." > In case a the chosen reader is compatible with all branches of the union in > the writer, then the class SchemaCompatibility should reflect this. Currently > it does not. > The submitted patch corrects this (also added tests showing this behaviour in > Avro) > The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be > redundant, but they were very useful when exploring how Avro actually works > during de-/serialization > I will try to continue working a little bit on the SchemaCompatibility class, > adding more user-friendly error messages (suitable for deeper structures than > todays error message). Feel free to contact me if you have any ideas or > pointers to existing work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (AVRO-1929) SchemaCompatibility fails to recognize string<->bytes promotion
[ https://issues.apache.org/jira/browse/AVRO-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened AVRO-1929: --- > SchemaCompatibility fails to recognize string<->bytes promotion > --- > > Key: AVRO-1929 > URL: https://issues.apache.org/jira/browse/AVRO-1929 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.7.7, 1.8.0, 1.8.1, 1.9.0 > Environment: Any Java env >Reporter: Anders Sundelin >Assignee: Anders Sundelin >Priority: Minor > Fix For: 1.9.0, 1.8.2 > > Attachments: AVRO-1929.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > The class SchemaCompatibility fails to realize the change made as of > AVRO-1553, where string-to/from-bytes promotions were allowed (according to > docs, this was integrated into 1.7.7) > The submitted fix corrects this (also updating the relevant unit test) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AVRO-1574) Suboptimal arraylist creation
[ https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133016#comment-16133016 ] Sean Busbey commented on AVRO-1574: --- +1 pushing shortly. > Suboptimal arraylist creation > - > > Key: AVRO-1574 > URL: https://issues.apache.org/jira/browse/AVRO-1574 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.7 >Reporter: Zoltan Farkas >Assignee: Kengo Seki >Priority: Trivial > Labels: beginner > Attachments: AVRO-1574.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > in Schema.java: > 1231 LockableArrayList symbols = new > LockableArrayList(); > > 1232 for (JsonNode n : symbolsNode) > > 1233 symbols.add(n.getTextValue()); > should be changed to: > 1231 LockableArrayList symbols = new > LockableArrayList(symbolsNode.size()); > > 1232 for (JsonNode n : symbolsNode) > > 1233 symbols.add(n.getTextValue()); > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union
[ https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16132986#comment-16132986 ] Sean Busbey commented on AVRO-1931: --- +1 pushing shortly. just need an author string from one of the reviewers. > SchemaCompatibility fails to recognize reader compatible with all branches of > a union > - > > Key: AVRO-1931 > URL: https://issues.apache.org/jira/browse/AVRO-1931 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.7.7, 1.8.1 > Environment: Java >Reporter: Anders Sundelin >Assignee: Anders Sundelin >Priority: Minor > Labels: patch > Attachments: AVRO-1931-2.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > It is stated in the Avro spec > "if writer's is a union, but reader's is not: > If the reader's schema matches the selected writer's schema, it is > recursively resolved against it. If they do not match, an error is signalled." > In case a the chosen reader is compatible with all branches of the union in > the writer, then the class SchemaCompatibility should reflect this. Currently > it does not. > The submitted patch corrects this (also added tests showing this behaviour in > Avro) > The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be > redundant, but they were very useful when exploring how Avro actually works > during de-/serialization > I will try to continue working a little bit on the SchemaCompatibility class, > adding more user-friendly error messages (suitable for deeper structures than > todays error message). Feel free to contact me if you have any ideas or > pointers to existing work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1574) Suboptimal arraylist creation
[ https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-1574: -- Labels: beginner (was: ) > Suboptimal arraylist creation > - > > Key: AVRO-1574 > URL: https://issues.apache.org/jira/browse/AVRO-1574 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.7 >Reporter: Zoltan Farkas >Assignee: Kengo Seki >Priority: Trivial > Labels: beginner > Attachments: AVRO-1574.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > in Schema.java: > 1231 LockableArrayList symbols = new > LockableArrayList(); > > 1232 for (JsonNode n : symbolsNode) > > 1233 symbols.add(n.getTextValue()); > should be changed to: > 1231 LockableArrayList symbols = new > LockableArrayList(symbolsNode.size()); > > 1232 for (JsonNode n : symbolsNode) > > 1233 symbols.add(n.getTextValue()); > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AVRO-1574) Suboptimal arraylist creation
[ https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned AVRO-1574: - Assignee: Kengo Seki > Suboptimal arraylist creation > - > > Key: AVRO-1574 > URL: https://issues.apache.org/jira/browse/AVRO-1574 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.7 >Reporter: Zoltan Farkas >Assignee: Kengo Seki >Priority: Trivial > Labels: beginner > Attachments: AVRO-1574.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > in Schema.java: > 1231 LockableArrayList symbols = new > LockableArrayList(); > > 1232 for (JsonNode n : symbolsNode) > > 1233 symbols.add(n.getTextValue()); > should be changed to: > 1231 LockableArrayList symbols = new > LockableArrayList(symbolsNode.size()); > > 1232 for (JsonNode n : symbolsNode) > > 1233 symbols.add(n.getTextValue()); > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1680) Problems with code snippet in Decoder#readMapStart javadocs
[
https://issues.apache.org/jira/browse/AVRO-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-1680:
--
Labels: beginner (was: )
> Problems with code snippet in Decoder#readMapStart javadocs
> ---
>
> Key: AVRO-1680
> URL: https://issues.apache.org/jira/browse/AVRO-1680
> Project: Avro
> Issue Type: Improvement
> Components: doc, java
>Affects Versions: 1.7.7
>Reporter: Eugene Kirpichov
>Priority: Trivial
> Labels: beginner
>
> https://github.com/apache/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/io/Decoder.java
> javadocs (
> http://avro.apache.org/docs/1.7.7/api/java/org/apache/avro/io/Decoder.html#readMapStart()
> ) say:
> {code:java}
> Map m = new HashMap();
> Record reuse = new Record();
> for(long i = in.readMapStart(); i != 0; i = in.readMapNext()) {
> for (long j = 0; j < i; j++) {
> String key = in.readString();
> reuse.intField = in.readInt();
> reuse.boolField = in.readBoolean();
> m.put(key, reuse);
> }
>}
> {code}
> This can be improved in two ways:
> 1) Javadoc ate the generic arguments. This can be fixed by wrapping into
> \{@code}.
> 2) The mutable record object is being reused; as a result, the map will have
> the same shared object mapped to every key. I don't think this is likely to
> be the user's intention, so a new Record should be created on every iteration.
> Actually in 3 ways.
> 3) a much better name for "i" would be "numRecords", because otherwise it
> seems like "j" and "i" have similar roles (indices over some containers),
> which they don't - "i" is not an index into any container, only j is. Then
> "j" can be renamed to "i".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AVRO-1680) Problems with code snippet in Decoder#readMapStart javadocs
[
https://issues.apache.org/jira/browse/AVRO-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-1680:
--
Component/s: doc
> Problems with code snippet in Decoder#readMapStart javadocs
> ---
>
> Key: AVRO-1680
> URL: https://issues.apache.org/jira/browse/AVRO-1680
> Project: Avro
> Issue Type: Improvement
> Components: doc, java
>Affects Versions: 1.7.7
>Reporter: Eugene Kirpichov
>Priority: Trivial
> Labels: beginner
>
> https://github.com/apache/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/io/Decoder.java
> javadocs (
> http://avro.apache.org/docs/1.7.7/api/java/org/apache/avro/io/Decoder.html#readMapStart()
> ) say:
> {code:java}
> Map m = new HashMap();
> Record reuse = new Record();
> for(long i = in.readMapStart(); i != 0; i = in.readMapNext()) {
> for (long j = 0; j < i; j++) {
> String key = in.readString();
> reuse.intField = in.readInt();
> reuse.boolField = in.readBoolean();
> m.put(key, reuse);
> }
>}
> {code}
> This can be improved in two ways:
> 1) Javadoc ate the generic arguments. This can be fixed by wrapping into
> \{@code}.
> 2) The mutable record object is being reused; as a result, the map will have
> the same shared object mapped to every key. I don't think this is likely to
> be the user's intention, so a new Record should be created on every iteration.
> Actually in 3 ways.
> 3) a much better name for "i" would be "numRecords", because otherwise it
> seems like "j" and "i" have similar roles (indices over some containers),
> which they don't - "i" is not an index into any container, only j is. Then
> "j" can be renamed to "i".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AVRO-1213) Revisit dependencies on Jetty, servlet-api, and Netty
[ https://issues.apache.org/jira/browse/AVRO-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-1213: -- Summary: Revisit dependencies on Jetty, servlet-api, and Netty (was: Dependency on Jetty Servlet API in IPC) > Revisit dependencies on Jetty, servlet-api, and Netty > - > > Key: AVRO-1213 > URL: https://issues.apache.org/jira/browse/AVRO-1213 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.2 >Reporter: Sharmarke Aden >Priority: Blocker > Fix For: 1.9.0 > > > The compile scoped dependency on jetty servlet-api in the IPC pom file can be > problematic if using Avro in a webapp environment. Would it be possible to > make this dependency either optional or provided? Or maybe Avro modularize > into sub-modules in such a way that desired features can be assembled > piecemeal? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1213) Dependency on Jetty Servlet API in IPC
[ https://issues.apache.org/jira/browse/AVRO-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-1213: -- Priority: Blocker (was: Minor) > Dependency on Jetty Servlet API in IPC > -- > > Key: AVRO-1213 > URL: https://issues.apache.org/jira/browse/AVRO-1213 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.2 >Reporter: Sharmarke Aden >Priority: Blocker > Fix For: 1.9.0 > > > The compile scoped dependency on jetty servlet-api in the IPC pom file can be > problematic if using Avro in a webapp environment. Would it be possible to > make this dependency either optional or provided? Or maybe Avro modularize > into sub-modules in such a way that desired features can be assembled > piecemeal? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1213) Dependency on Jetty Servlet API in IPC
[ https://issues.apache.org/jira/browse/AVRO-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-1213: -- Fix Version/s: 1.9.0 > Dependency on Jetty Servlet API in IPC > -- > > Key: AVRO-1213 > URL: https://issues.apache.org/jira/browse/AVRO-1213 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.7.2 >Reporter: Sharmarke Aden >Priority: Blocker > Fix For: 1.9.0 > > > The compile scoped dependency on jetty servlet-api in the IPC pom file can be > problematic if using Avro in a webapp environment. Would it be possible to > make this dependency either optional or provided? Or maybe Avro modularize > into sub-modules in such a way that desired features can be assembled > piecemeal? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1640) Does SpecificData.schemaCache actually do anything?
[ https://issues.apache.org/jira/browse/AVRO-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated AVRO-1640: -- Labels: should_be_dev_discussion (was: ) > Does SpecificData.schemaCache actually do anything? > --- > > Key: AVRO-1640 > URL: https://issues.apache.org/jira/browse/AVRO-1640 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.7.7 >Reporter: Silas Davis >Priority: Minor > Labels: should_be_dev_discussion > > I've been looking at the source code for SpecificData. On line 182 a > schemaCache is defined as a WeakHashMap. > Having looked into how WeakHashMap works - it stores a WeakReference for the > keys of the map but not the values, I can't see how the schemaCache actually > provides a cache. I think it just stores all Schema objects it has ever seen, > and never expires any. This is because it stores keys of type > java.lang.reflect.Type key, typlically a Class, which typically never fall > out of reference. The only way the Class (a static singleton object) will be > garbage collected is if the ClassLoader that holds a reference to it is > garbage collected. This mostly doesn't happen, so the schemaCache is really > just behaving as a HashMap. > It seems like WeakHashMap is commonly mistakenly used as a cache: > - > http://stackoverflow.com/questions/1802809/javas-weakhashmap-and-caching-why-is-it-referencing-the-keys-not-the-values > - > http://www.codeinstructions.com/2008/09/weakhashmap-is-not-cache-understanding.html > Perhaps I am missing something, like Avro's use of ClassLoaders meaning that > Class objects do get collected, apologies if so. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-1310) Avro Maven project can't be built from scratch
[
https://issues.apache.org/jira/browse/AVRO-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated AVRO-1310:
--
Labels: should_be_dev_discussion (was: )
> Avro Maven project can't be built from scratch
> --
>
> Key: AVRO-1310
> URL: https://issues.apache.org/jira/browse/AVRO-1310
> Project: Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.7.4
> Environment: Maven on Eclipse
>Reporter: Nir Zamir
>Priority: Minor
> Labels: should_be_dev_discussion
>
> When getting the Java 'trunk' from SVN and trying to use Maven Install ('mvn
> install') there are errors.
> Most of the errors are in tests so I tried skipping the tests but it still
> fails.
> See more details in my post on Avro Users:
> http://apache-avro.679487.n3.nabble.com/help-with-Avro-compilation-td4026946.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (AVRO-1640) Does SpecificData.schemaCache actually do anything?
[ https://issues.apache.org/jira/browse/AVRO-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16132166#comment-16132166 ] Sean Busbey commented on AVRO-1640: --- this would be much better as a DISCUSS thread on dev@, could someone please bring it up there? > Does SpecificData.schemaCache actually do anything? > --- > > Key: AVRO-1640 > URL: https://issues.apache.org/jira/browse/AVRO-1640 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.7.7 >Reporter: Silas Davis >Priority: Minor > Labels: should_be_dev_discussion > > I've been looking at the source code for SpecificData. On line 182 a > schemaCache is defined as a WeakHashMap. > Having looked into how WeakHashMap works - it stores a WeakReference for the > keys of the map but not the values, I can't see how the schemaCache actually > provides a cache. I think it just stores all Schema objects it has ever seen, > and never expires any. This is because it stores keys of type > java.lang.reflect.Type key, typlically a Class, which typically never fall > out of reference. The only way the Class (a static singleton object) will be > garbage collected is if the ClassLoader that holds a reference to it is > garbage collected. This mostly doesn't happen, so the schemaCache is really > just behaving as a HashMap. > It seems like WeakHashMap is commonly mistakenly used as a cache: > - > http://stackoverflow.com/questions/1802809/javas-weakhashmap-and-caching-why-is-it-referencing-the-keys-not-the-values > - > http://www.codeinstructions.com/2008/09/weakhashmap-is-not-cache-understanding.html > Perhaps I am missing something, like Avro's use of ClassLoaders meaning that > Class objects do get collected, apologies if so. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AVRO-1580) Use newer version of surefire, 2.17 instead of 2.12
[ https://issues.apache.org/jira/browse/AVRO-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16132165#comment-16132165 ] Sean Busbey commented on AVRO-1580: --- as a part of this, let's upgrade the apache parent pom (atleast in 1.9+) since we're relying on version 10, which is very very out of date. > Use newer version of surefire, 2.17 instead of 2.12 > --- > > Key: AVRO-1580 > URL: https://issues.apache.org/jira/browse/AVRO-1580 > Project: Avro > Issue Type: Improvement > Components: build, java >Affects Versions: 1.7.7 >Reporter: Zoltan Farkas >Priority: Minor > Labels: beginner, build > > version 2.12 does not work well with Netbeans 8.0 and makes development > cumbersome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
