Re: Branch missing for 1.6
We branch lazily. If this is work locally and you do not need to commit, then the 1.6.3 tag is the correct location. Once a committer needs a branch in order to commit a change that that does not overlap with trunk, we create the branch. In this case, if you need to commit a patch that follows 1.6.3 and trunk is not the appropriate place, we can create a 1.6.x branch off of tag 1.6.3 to collect changes in preparation for a possible 1.6.4 release. Another option is that trunk becomes 1.6.4, but only if it does not contain incompatible API changes from 1.6.3 (which I believe it now does). -Scott On 5/7/12 10:24 PM, Harsh J ha...@cloudera.com wrote: Hi, I'm unable to locate a branch for 1.6 tip under http://svn.apache.org/repos/asf/avro/branches/. Can this please be published remotely? Or have we decided to stop maintaining branches and cut new releases off of trunk with tags alone? The tags on http://svn.apache.org/repos/asf/avro/tags/ appear perfect, and am currently relying on tags/release-1.6.3 for my 1.6 tip reference. Thanks, -- Harsh J
[jira] [Commented] (AVRO-1066) ArrayIndexOutOfBoundsException in ParsingEncoder when trying to use a json encoder to serialize a deep object graph
[
https://issues.apache.org/jira/browse/AVRO-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270806#comment-13270806
]
Doug Cutting commented on AVRO-1066:
Why the addition of the Accessor interface? A comment explaining this would
probably be good to add too.
ArrayIndexOutOfBoundsException in ParsingEncoder when trying to use a json
encoder to serialize a deep object graph
---
Key: AVRO-1066
URL: https://issues.apache.org/jira/browse/AVRO-1066
Project: Avro
Issue Type: Bug
Affects Versions: 1.6.3
Reporter: Daniel Lord
Assignee: Thiruvalluvan M. G.
Fix For: 1.7.0
Attachments: AVRO-1066.patch
I have an avro record that can have an array of children records of the same
type. If this structure gets to be too deep then I continually get an
ArrayOutOfBoundsException in ParsingEncoder when trying to use a json
encoder. This works fine when using a binary encoder. My schema looks
something like this:
{
name : MyRecord,
type : record,
fields : [
{
name : fooField,
type : int
},
{
name : childRecords,
type : [ null, { type : array, items : MyRecord } ]
}
]
}
The code I'm using to capture a JSON string for debugging looks like this:
ByteArrayOutputStream os = null;
try {
os = new ByteArrayOutputStream(2048);
final Encoder jsonEncoder =
EncoderFactory.get().jsonEncoder(MyRecord.SCHEMA$, os);
final DatumWriterMyRecord datumWriter = new
SpecificDatumWriterMyRecord(MyRecord.class);
datumWriter.write(record, jsonEncoder);
jsonEncoder.flush();
return new String(os.toByteArray(), Charset.defaultCharset());
} catch (IOException e) {
return null;
} finally {
if (os != null) {
try {
os.close();
} catch (IOException e) {
}
}
}
The error I get looks like this:
java.lang.ArrayIndexOutOfBoundsException: 10
at org.apache.avro.io.ParsingEncoder.push(ParsingEncoder.java:55)
at org.apache.avro.io.JsonEncoder.writeArrayStart(JsonEncoder.java:231)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:125)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
[jira] [Commented] (AVRO-1070) AvroSequenceFileOutputFormat is in wrong package.
[
https://issues.apache.org/jira/browse/AVRO-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270811#comment-13270811
]
Doug Cutting commented on AVRO-1070:
+1 This is the right fix. It's not something primarily related to
org.apache.hadoop.io, but rather to org.apache.hadoop.mapreduce.
AvroSequenceFileOutputFormat is in wrong package.
-
Key: AVRO-1070
URL: https://issues.apache.org/jira/browse/AVRO-1070
Project: Avro
Issue Type: Bug
Components: java
Reporter: Thiruvalluvan M. G.
Assignee: Thiruvalluvan M. G.
Attachments: AVRO-1070.patch
The source directory {{org/apache/avro/mapreduce}} and package name
{{org.apache.avro.hadoop.io}} do not match.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1070) AvroSequenceFileOutputFormat is in wrong package.
[
https://issues.apache.org/jira/browse/AVRO-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Cutting updated AVRO-1070:
---
Fix Version/s: 1.7.0
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
AvroSequenceFileOutputFormat is in wrong package.
-
Key: AVRO-1070
URL: https://issues.apache.org/jira/browse/AVRO-1070
Project: Avro
Issue Type: Bug
Components: java
Reporter: Thiruvalluvan M. G.
Assignee: Thiruvalluvan M. G.
Fix For: 1.7.0
Attachments: AVRO-1070.patch
The source directory {{org/apache/avro/mapreduce}} and package name
{{org.apache.avro.hadoop.io}} do not match.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1071) Fix CHANGES.txt reference to AVRO-656 rather than AVRO-246.
[ https://issues.apache.org/jira/browse/AVRO-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated AVRO-1071: --- Resolution: Fixed Status: Resolved (was: Patch Available) I committed this. Thanks, Jakob! Fix CHANGES.txt reference to AVRO-656 rather than AVRO-246. --- Key: AVRO-1071 URL: https://issues.apache.org/jira/browse/AVRO-1071 Project: Avro Issue Type: Bug Components: java Affects Versions: 1.5.0 Reporter: Jakob Homan Assignee: Jakob Homan Priority: Minor Fix For: 1.7.0 Attachments: AVRO-1071.patch While tracking down an incompatibilty between versions I noticed that the referenced JIRA for AVRO-656 says 246. The commit log is also wrong, but this fix will at least help anyone looking into the same issues that I am. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1073) JSON binary encoding doesn't work
[ https://issues.apache.org/jira/browse/AVRO-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270848#comment-13270848 ] Doug Cutting commented on AVRO-1073: Can you please provide a test case that fails? Thanks! JSON binary encoding doesn't work - Key: AVRO-1073 URL: https://issues.apache.org/jira/browse/AVRO-1073 Project: Avro Issue Type: Improvement Components: java Affects Versions: 1.6.3 Environment: uname -a Darwin zmac 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386 java -version java version 1.6.0_29 Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527) Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode) Reporter: Zhihong Zhang The JSON encoder doesn't encode binary data (bytes or ByteBuffer) properly. It treats the data as Latin-1 and encode it as text. But there is a bug in text encoding of non-Roman characters so the binary encoding doesn't work at all. The binary should be encoded into ASCII (Base64 or 85) before it's encoded in JSON. The text encoding of binary data is very inefficient even it works. For example, the 0 is encoded as \\u in JSON (7 bytes). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1072) The JSON encoder doesn't handle non-ASCII character properly
[ https://issues.apache.org/jira/browse/AVRO-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270857#comment-13270857 ] Doug Cutting commented on AVRO-1072: Avro's JsonEncoder.java specifies the UTF-8 encoding, so I don't see how this is happening. Can you please provide a test that fails in your environment? Thanks! The JSON encoder doesn't handle non-ASCII character properly Key: AVRO-1072 URL: https://issues.apache.org/jira/browse/AVRO-1072 Project: Avro Issue Type: Bug Components: java Affects Versions: 1.6.3 Environment: uname -a Darwin zmac 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386 java -version java version 1.6.0_29 Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527) Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode) Reporter: Zhihong Zhang The JSON encoder uses default encoding of the platform. It should always use UTF-8. This causes multiple problems for us, 1. The text is mangled if sending/receiving machine has different encoding. 2. Some encodings (like Latin-1 or MacRoman) can't handle all characters (like Chinese) and we get ? in the text. 3. The binary encoder (ByteBuffer) doesn't work due to this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1075) Missing entries in .gitignore
[ https://issues.apache.org/jira/browse/AVRO-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated AVRO-1075: --- Resolution: Fixed Assignee: Karthik K Status: Resolved (was: Patch Available) I committed this. Thanks, Karthik. Missing entries in .gitignore - Key: AVRO-1075 URL: https://issues.apache.org/jira/browse/AVRO-1075 Project: Avro Issue Type: Improvement Components: build Affects Versions: 1.6.3 Reporter: Karthik K Assignee: Karthik K Priority: Minor Fix For: 1.7.0 Attachments: AVRO-1075.patch Missing entries in .gitignore, that are best not added to the source control. ( includes target / .project / .classpath among others) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1076) Bug - o.a.a.Protocol equals : props field
[ https://issues.apache.org/jira/browse/AVRO-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270863#comment-13270863 ] Doug Cutting commented on AVRO-1076: Good catch. Can you please add a test case that checks this? There are some related tests in TestSchema#testRecord(). Thanks! Bug - o.a.a.Protocol equals : props field - Key: AVRO-1076 URL: https://issues.apache.org/jira/browse/AVRO-1076 Project: Avro Issue Type: Bug Components: java Affects Versions: 1.6.3 Reporter: Karthik K Fix For: 1.7.0 Attachments: AVRO-1076.patch Bug w.r.t a self comparison. ( thanks to findbugs ) . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1074) Suggested optimization for Utf8.length()
[
https://issues.apache.org/jira/browse/AVRO-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Cutting updated AVRO-1074:
---
Attachment: AVRO-1074.patch
Here's a patch that implements Wolfgang's suggestion. Tests pass.
Suggested optimization for Utf8.length()
Key: AVRO-1074
URL: https://issues.apache.org/jira/browse/AVRO-1074
Project: Avro
Issue Type: Improvement
Components: java
Affects Versions: 1.6.3
Reporter: Hernan Otero
Fix For: 1.7.0
Attachments: AVRO-1074.patch
Wanted to suggest a small optimization. It is quite common for applications
to do a quick check (along with null-ness) for 0-length of string-typed
fields. Today's Utf8.length() implementation is:
{code}return toString().length;{code}
I believe this could easily get optimized to return this instead:
{code}return length == 0 ? 0 : toString().length();{code}
Thus avoiding the need to create the string representation for this common
use case (and scenario).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1066) ArrayIndexOutOfBoundsException in ParsingEncoder when trying to use a json encoder to serialize a deep object graph
[
https://issues.apache.org/jira/browse/AVRO-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271032#comment-13271032
]
Thiruvalluvan M. G. commented on AVRO-1066:
---
Oops! That was a mistake in the patch. I'll send a right patch.
ArrayIndexOutOfBoundsException in ParsingEncoder when trying to use a json
encoder to serialize a deep object graph
---
Key: AVRO-1066
URL: https://issues.apache.org/jira/browse/AVRO-1066
Project: Avro
Issue Type: Bug
Affects Versions: 1.6.3
Reporter: Daniel Lord
Assignee: Thiruvalluvan M. G.
Fix For: 1.7.0
Attachments: AVRO-1066.patch
I have an avro record that can have an array of children records of the same
type. If this structure gets to be too deep then I continually get an
ArrayOutOfBoundsException in ParsingEncoder when trying to use a json
encoder. This works fine when using a binary encoder. My schema looks
something like this:
{
name : MyRecord,
type : record,
fields : [
{
name : fooField,
type : int
},
{
name : childRecords,
type : [ null, { type : array, items : MyRecord } ]
}
]
}
The code I'm using to capture a JSON string for debugging looks like this:
ByteArrayOutputStream os = null;
try {
os = new ByteArrayOutputStream(2048);
final Encoder jsonEncoder =
EncoderFactory.get().jsonEncoder(MyRecord.SCHEMA$, os);
final DatumWriterMyRecord datumWriter = new
SpecificDatumWriterMyRecord(MyRecord.class);
datumWriter.write(record, jsonEncoder);
jsonEncoder.flush();
return new String(os.toByteArray(), Charset.defaultCharset());
} catch (IOException e) {
return null;
} finally {
if (os != null) {
try {
os.close();
} catch (IOException e) {
}
}
}
The error I get looks like this:
java.lang.ArrayIndexOutOfBoundsException: 10
at org.apache.avro.io.ParsingEncoder.push(ParsingEncoder.java:55)
at org.apache.avro.io.JsonEncoder.writeArrayStart(JsonEncoder.java:231)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:125)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
at
