[jira] [Updated] (AVRO-1080) JsonIO.cc should allow \u escape sequence in string
[
https://issues.apache.org/jira/browse/AVRO-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1080:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 1335985.
Thank you Keh-Li Sheng.
> JsonIO.cc should allow \u escape sequence in string
> ---
>
> Key: AVRO-1080
> URL: https://issues.apache.org/jira/browse/AVRO-1080
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.6.3
> Environment: C++
>Reporter: Keh-Li Sheng
> Attachments: AVRO-1080.patch
>
>
> If an avro string contains a unicode escape sequence that begins with "\u"
> instead of "\U" an exception is thrown by the parser. The problematic code is
> at JsonIO.cc line 269.
> {code}
> JsonParser::Token JsonParser::tryString()
> {
> sv.clear();
> for ( ; ;) {
> char ch = in_.read();
> if (ch == '"') {
> return tkString;
> } else if (ch == '\\') {
> ch = in_.read();
> switch (ch) {
> case '"':
> case '\\':
> case '/':
> sv.push_back(ch);
> continue;
> case 'b':
> sv.push_back('\b');
> continue;
> case 'f':
> sv.push_back('\f');
> continue;
> case 'n':
> sv.push_back('\n');
> continue;
> case 'r':
> sv.push_back('\r');
> continue;
> case 't':
> sv.push_back('\t');
> continue;
> case 'U':
> {
> unsigned int n = 0;
> char e[4];
> in_.readBytes(reinterpret_cast(e), 4);
> for (int i = 0; i < 4; i++) {
> n *= 16;
> char c = e[i];
> if (isdigit(c)) {
> n += c - '0';
> } else if (c >= 'a' && c <= 'f') {
> n += c - 'a' + 10;
> } else if (c >= 'A' && c <= 'F') {
> n += c - 'A' + 10;
> } else {
> unexpected(c);
> }
> }
> sv.push_back(n);
> }
> break;
> default:
> unexpected(ch);
> }
> } else {
> sv.push_back(ch);
> }
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1066) ArrayIndexOutOfBoundsException in ParsingEncoder when trying to use a json encoder to serialize a deep object graph
[
https://issues.apache.org/jira/browse/AVRO-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1066:
--
Attachment: AVRO-1066-2.patch
Here is the correct patch.
> ArrayIndexOutOfBoundsException in ParsingEncoder when trying to use a json
> encoder to serialize a deep object graph
> ---
>
> Key: AVRO-1066
> URL: https://issues.apache.org/jira/browse/AVRO-1066
> Project: Avro
> Issue Type: Bug
>Affects Versions: 1.6.3
>Reporter: Daniel Lord
>Assignee: Thiruvalluvan M. G.
> Fix For: 1.7.0
>
> Attachments: AVRO-1066-2.patch, AVRO-1066.patch
>
>
> I have an avro record that can have an array of children records of the same
> type. If this structure gets to be too deep then I continually get an
> ArrayOutOfBoundsException in ParsingEncoder when trying to use a json
> encoder. This works fine when using a binary encoder. My schema looks
> something like this:
> {
> "name" : "MyRecord",
> "type" : "record",
> "fields" : [
> {
> "name" : "fooField",
> "type" : "int"
> },
> {
> "name" : "childRecords",
> "type" : [ "null", { "type" : "array", "items" : "MyRecord" } ]
> }
> ]
> }
> The code I'm using to capture a JSON string for debugging looks like this:
> ByteArrayOutputStream os = null;
> try {
> os = new ByteArrayOutputStream(2048);
> final Encoder jsonEncoder =
> EncoderFactory.get().jsonEncoder(MyRecord.SCHEMA$, os);
> final DatumWriter datumWriter = new
> SpecificDatumWriter(MyRecord.class);
> datumWriter.write(record, jsonEncoder);
> jsonEncoder.flush();
> return new String(os.toByteArray(), Charset.defaultCharset());
> } catch (IOException e) {
> return null;
> } finally {
> if (os != null) {
> try {
> os.close();
> } catch (IOException e) {
> }
> }
> }
> The error I get looks like this:
> java.lang.ArrayIndexOutOfBoundsException: 10
> at org.apache.avro.io.ParsingEncoder.push(ParsingEncoder.java:55)
> at org.apache.avro.io.JsonEncoder.writeArrayStart(JsonEncoder.java:231)
> at
> org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:125)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
> at
> org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
> at
> org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
> at
> org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
> at
> org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatum
[jira] [Updated] (AVRO-1070) AvroSequenceFileOutputFormat is in wrong package.
[
https://issues.apache.org/jira/browse/AVRO-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiruvalluvan M. G. updated AVRO-1070:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 1335954.
Thank you Doug for reviewing.
> AvroSequenceFileOutputFormat is in wrong package.
> -
>
> Key: AVRO-1070
> URL: https://issues.apache.org/jira/browse/AVRO-1070
> Project: Avro
> Issue Type: Bug
> Components: java
>Reporter: Thiruvalluvan M. G.
>Assignee: Thiruvalluvan M. G.
> Fix For: 1.7.0
>
> Attachments: AVRO-1070.patch
>
>
> The source directory {{org/apache/avro/mapreduce}} and package name
> {{org.apache.avro.hadoop.io}} do not match.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1065) NodeRecord::isValid() treats records with no fields as invalid
[
https://issues.apache.org/jira/browse/AVRO-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271177#comment-13271177
]
Thiruvalluvan M. G. commented on AVRO-1065:
---
I'll commit this soon if there are no objections.
> NodeRecord::isValid() treats records with no fields as invalid
> --
>
> Key: AVRO-1065
> URL: https://issues.apache.org/jira/browse/AVRO-1065
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.6.3
>Reporter: Keh-Li Sheng
>Assignee: Thiruvalluvan M. G.
> Attachments: AVRO-1065.patch
>
>
> Implementation of NodeRecord in NodeImpl.hh requires that a record have at
> least 1 field in it but this is not required as per the specification at
> http://avro.apache.org/docs/1.6.3/spec.html#schema_record. Either remove the
> condition or update the spec. This only seems to be a quirk in c++.
> {code:title=NodeImpl.hh}
> class AVRO_DECL NodeRecord : public NodeImplRecord
> {
> ...
> bool isValid() const {
> return (
> (nameAttribute_.size() == 1) &&
> (leafAttributes_.size() > 0) &&
> (leafAttributes_.size() == leafNameAttributes_.size())
>);
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
