[jira] Updated: (AVRO-504) ruby impl could stand better error messages on schema parsing
[ https://issues.apache.org/jira/browse/AVRO-504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Hodges updated AVRO-504: - Resolution: Fixed Fix Version/s: 1.4.0 Status: Resolved (was: Patch Available) Committed as r931437. > ruby impl could stand better error messages on schema parsing > - > > Key: AVRO-504 > URL: https://issues.apache.org/jira/browse/AVRO-504 > Project: Avro > Issue Type: Improvement > Components: ruby >Affects Versions: 1.3.2 >Reporter: Jeff Hodges >Assignee: Jeff Hodges > Fix For: 1.4.0 > > Attachments: AVRO-504.patch > > > The current schema parsing error code is janky and wet. Let's correct that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-494) the genavro format should support defaults
[ https://issues.apache.org/jira/browse/AVRO-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854336#action_12854336 ] Todd Lipcon commented on AVRO-494: -- I spent a few hours trying to figure out how to use Jackson and JavaCC together in such a way that I could put inline JSON in certain places in genavro. Unfortunately it turned out to be really rather difficult because of lookahead, etc. Any JavaCC or Jackson gurus out there? > the genavro format should support defaults > -- > > Key: AVRO-494 > URL: https://issues.apache.org/jira/browse/AVRO-494 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.3.2 >Reporter: Eric Evans >Priority: Minor > Attachments: > v1-0001-AVRO-494.-teach-genavro-about-defaults-and-fix-order.txt > > > It would be great if the genavro format also supported assigning default > values. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-494) the genavro format should support defaults
[
https://issues.apache.org/jira/browse/AVRO-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854312#action_12854312
]
Eric Evans commented on AVRO-494:
-
The parser was written to handle arbitrary attributes using the @identifer
annotation, but this isn't working as expected; it's actually adding them as
properties to the attributes type object.
So in other words you end up with:
{noformat}
{name: aname, type: {type: string, order: ignore}}
{noformat}
... instead of:
{noformat}
{name: aname, type: string, order: ignore}
{noformat}
The patch attached here parses the attributes for default and order into
JsonNode and Field.Order instances respectively and passes them into the Field
constructor. There is probably a better way. For example, this requires you to
supply an attribute that parses as valid JSON. For example:
{noformat}
record FooRecord {
@default(false)
boolean yesno;
@default(100)
int number;
/* Fails; parses to the bareword ignore */
@order("ignore")
string badstring;
/* Succeeds; parses to "ignore" */
@order("\"ignore\"")
string goodstring;
}
{noformat}
Thoughts?
> the genavro format should support defaults
> --
>
> Key: AVRO-494
> URL: https://issues.apache.org/jira/browse/AVRO-494
> Project: Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.3.2
>Reporter: Eric Evans
>Priority: Minor
> Attachments:
> v1-0001-AVRO-494.-teach-genavro-about-defaults-and-fix-order.txt
>
>
> It would be great if the genavro format also supported assigning default
> values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-494) the genavro format should support defaults
[ https://issues.apache.org/jira/browse/AVRO-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Evans updated AVRO-494: Attachment: v1-0001-AVRO-494.-teach-genavro-about-defaults-and-fix-order.txt > the genavro format should support defaults > -- > > Key: AVRO-494 > URL: https://issues.apache.org/jira/browse/AVRO-494 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.3.2 >Reporter: Eric Evans >Priority: Minor > Attachments: > v1-0001-AVRO-494.-teach-genavro-about-defaults-and-fix-order.txt > > > It would be great if the genavro format also supported assigning default > values. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Re: new Avro committer: Bruce Mitchener
Congratulatons~ 2010-04-07 ajiu.009 发件人： Matt Massie 发送时间： 2010-04-07 06:03:05 收件人： avro-dev@hadoop.apache.org 抄送： 主题： Re: new Avro committer: Bruce Mitchener Congratulations, Bruce! You really deserve this. You've contributed so much that I was getting tired of typing "Contributed by Bruce Mitchener" in all the commit messages and CHANGES.txt. -Matt On Tue, Apr 6, 2010 at 1:50 PM, Doug Cutting wrote: > The Hadoop PMC has voted to add Bruce Mitchener as an Avro committer. > > Bruce, can you please file an Individual Contributor License Agreement > (ICLA) with Apache? Instructions are at: > > http://www.apache.org/licenses/#clas > > Once this is on file I can ask for your account to be created. > > Congratulations, Bruce! > > Doug >
Re: new Avro committer: Bruce Mitchener
Congratulations, Bruce! You really deserve this. You've contributed so much that I was getting tired of typing "Contributed by Bruce Mitchener" in all the commit messages and CHANGES.txt. -Matt On Tue, Apr 6, 2010 at 1:50 PM, Doug Cutting wrote: > The Hadoop PMC has voted to add Bruce Mitchener as an Avro committer. > > Bruce, can you please file an Individual Contributor License Agreement > (ICLA) with Apache? Instructions are at: > > http://www.apache.org/licenses/#clas > > Once this is on file I can ask for your account to be created. > > Congratulations, Bruce! > > Doug >
[jira] Resolved: (AVRO-505) Java impl is spitting any ol' error text on to the wire
[ https://issues.apache.org/jira/browse/AVRO-505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Hodges resolved AVRO-505. -- Resolution: Invalid While there is a problem with our error handling, it's not the one described here. > Java impl is spitting any ol' error text on to the wire > --- > > Key: AVRO-505 > URL: https://issues.apache.org/jira/browse/AVRO-505 > Project: Avro > Issue Type: Bug > Components: java >Affects Versions: 1.3.2 >Reporter: Jeff Hodges >Priority: Critical > > The current java implementation is sending the error text of unhandled errors > back as RPC responses. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: new Avro committer: Bruce Mitchener
Congrats! -- Jeff On Tue, Apr 6, 2010 at 1:50 PM, Doug Cutting wrote: > The Hadoop PMC has voted to add Bruce Mitchener as an Avro committer. > > Bruce, can you please file an Individual Contributor License Agreement > (ICLA) with Apache? Instructions are at: > > http://www.apache.org/licenses/#clas > > Once this is on file I can ask for your account to be created. > > Congratulations, Bruce! > > Doug >
Re: new Avro committer: Bruce Mitchener
Thank you! I look forward to continuing to contribute. I filed an ICLA last week and got an email that it had been received and processed. That said, I do not yet see my name on http://people.apache.org/~jim/committers.html - Bruce On Tue, Apr 6, 2010 at 2:50 PM, Doug Cutting wrote: > The Hadoop PMC has voted to add Bruce Mitchener as an Avro committer. > > Bruce, can you please file an Individual Contributor License Agreement > (ICLA) with Apache? Instructions are at: > > http://www.apache.org/licenses/#clas > > Once this is on file I can ask for your account to be created. > > Congratulations, Bruce! > > Doug >
new Avro committer: Bruce Mitchener
The Hadoop PMC has voted to add Bruce Mitchener as an Avro committer. Bruce, can you please file an Individual Contributor License Agreement (ICLA) with Apache? Instructions are at: http://www.apache.org/licenses/#clas Once this is on file I can ask for your account to be created. Congratulations, Bruce! Doug
[jira] Created: (AVRO-512) define and implement mapreduce connector protocol
define and implement mapreduce connector protocol - Key: AVRO-512 URL: https://issues.apache.org/jira/browse/AVRO-512 Project: Avro Issue Type: New Feature Components: java Reporter: Doug Cutting Avro should provide Hadoop Mapper and Reducer implementations that connect to a subprocess in another programming language, transmitting raw binary values to and from that process. This should be modeled after Hadoop Pipes. It would allow one to easily write efficient mapreduce programs in non-Java languages that process Avro-format data. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Avro-456/457/458 in Google Summer of Code 2010
I have very little knowledge about GSOC and do not know what is considered a project there. Avro doesn't define 'projects'. Its JIRA tickets can be items that take an hour to do or a month. Some of the tasks related to conversion of csv, xml, or json data into or out of avro are very simple, and some are not. The scope of all three, including a unified tool for conversion and testing, is a fairly large overall project. JSON would be the easiest, since Avro already supports serialization to and from it. XML has the potential to be the most complete and flexible. It is a lot of work but would use standard APIs and tools. CSV has the most restrictions -- it can't support recursive schemas or unions well. Additionally it can have some tricky corner cases since it is not a strict standard. It will likely be the most popular 'import into avro' variation, however. The tools for conversion (command line or other) and the testing required for all of the above are non-trivial, mostly because such a tool has to be very good at clear error handling and reporting or it won't be very useful in the real world. -Scott On Apr 5, 2010, at 6:03 PM, Zheng Yang wrote: > Hi, Scott, > > as you can see, these three tools can be eventually combined into one > multifunctional tool which accepts different format(csv,xml,json..). > since I'm also going to submit a proposal for this , may i know are they > considered to be one project or three? > > Yang > > On Tue, Apr 6, 2010 at 7:43 AM, Scott Carey wrote: > >> FYI, it seems as though at least one other person has chosen a similar task >> for GSOC: >> >> >> >> http://mail-archives.apache.org/mod_mbox/hadoop-avro-dev/201003.mbox/%3c179519d11003212231y4537eb03i6f89eb3f6f745...@mail.gmail.com%3e >> >> On Apr 5, 2010, at 4:31 PM, Jasintha Dasanayaka wrote: >> >>> Hey..! >>> I am going to submit proposal for AVRO-457 can you do it AVRO-458 only >>> >>> On Tue, Apr 6, 2010 at 4:00 AM, Hua Huang wrote: >>> Hi all, This is Hua Huang, a CS master student from Simon Fraser University, Canada. I am going to participate in the Google Summer of Code 2010 and I also >> find out that several projects of AVRO are quite interesting, especially AVRO-456(add tools that read/write json records from/to avro data files) together with AVRO-457 and AVRO-458. I plan to submit a proposal for these projects which would produce a >> C/C++ command-line tool to support transformation between AVRO data and other types of data, like CSV, Json or XML. My key idea is to use parallel bit stream technology to speed up the parsing procedure in order to build a high performance tool which will be very useful in practical, especially in >> the large-scale dataset. I sent an email to Doug Cutting(cutt...@apache.org) who is the reporter >> of these projects, but I haven't received any reply yet. So I am wondering, >> is there anybody who can communicate with me for the details of the >> projects, or even suggest me a person so that I could contact with him/her for the details? Any feedback is really appreciated. Thank you very much. Yours Sincerely, Hua >>> >>> >>> -- >>> Jasintha Dasanayaka >>> +94 772 916 596 >>> +94 472 232 139 >>> http://www.jasintha.info >>> jasint...@gmail.com >> >> > > > -- > School of Computing / Computing / Year 2 > National University of Singapore
[jira] Commented: (AVRO-494) the genavro format should support defaults
[ https://issues.apache.org/jira/browse/AVRO-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853923#action_12853923 ] Hudson commented on AVRO-494: - Integrated in Cassandra #399 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/399/]) genavro schema file (not to be used until ) Patch by eevans > the genavro format should support defaults > -- > > Key: AVRO-494 > URL: https://issues.apache.org/jira/browse/AVRO-494 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.3.2 >Reporter: Eric Evans >Priority: Minor > > It would be great if the genavro format also supported assigning default > values. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-511) ruby side should pass the rpc interop tests
[ https://issues.apache.org/jira/browse/AVRO-511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Hodges updated AVRO-511: - Fix Version/s: 1.4.0 Assignee: Jeff Hodges Status: Patch Available (was: Open) > ruby side should pass the rpc interop tests > --- > > Key: AVRO-511 > URL: https://issues.apache.org/jira/browse/AVRO-511 > Project: Avro > Issue Type: Improvement > Components: ruby >Affects Versions: 1.3.2 >Reporter: Jeff Hodges >Assignee: Jeff Hodges > Fix For: 1.4.0 > > Attachments: AVRO-511.patch > > > The ruby side should definitely pass the RPC interop tests. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (AVRO-511) ruby side should pass the rpc interop tests
ruby side should pass the rpc interop tests --- Key: AVRO-511 URL: https://issues.apache.org/jira/browse/AVRO-511 Project: Avro Issue Type: Improvement Components: ruby Affects Versions: 1.3.2 Reporter: Jeff Hodges Attachments: AVRO-511.patch The ruby side should definitely pass the RPC interop tests. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-511) ruby side should pass the rpc interop tests
[ https://issues.apache.org/jira/browse/AVRO-511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Hodges updated AVRO-511: - Attachment: AVRO-511.patch Oh, hey, look at that. A patch for adding and passing RPC interop tests on the ruby side. > ruby side should pass the rpc interop tests > --- > > Key: AVRO-511 > URL: https://issues.apache.org/jira/browse/AVRO-511 > Project: Avro > Issue Type: Improvement > Components: ruby >Affects Versions: 1.3.2 >Reporter: Jeff Hodges > Attachments: AVRO-511.patch > > > The ruby side should definitely pass the RPC interop tests. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
