[jira] Commented: (AVRO-530) allow for mutual recursion in type definitions
[
https://issues.apache.org/jira/browse/AVRO-530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12861717#action_12861717
]
Jeff Hodges commented on AVRO-530:
--
(It could be done in one pass, of course. That was just an off-the-cuff idea
that I thought would be easier to write but might not be.)
allow for mutual recursion in type definitions
--
Key: AVRO-530
URL: https://issues.apache.org/jira/browse/AVRO-530
Project: Avro
Issue Type: Improvement
Components: spec
Affects Versions: 1.3.2
Reporter: Jeff Hodges
Suppose you have these two types in your protocol:
{code}
{name: User, type: record, fields: [{name: current_status,
type: Status}]}
{name: Status, type: record, fields: [{name: author, type:
User}]}
{code}
This will raise an error! The current workaround is to define one of them at
their first usage. Like:
{code}
{name: User, type: record, fields: [{name: current_status,
type: {name: Status, type: record, fields: [.. lots of fields
...]}]}
{code}
But this is incredibly unwieldy. It would be really nice for the spec to
require all the parsers to allow for mutual recursion, instead. It could be
done by implementing a two-pass parser. One pass to acquire names referenced,
and a second to fill in those names with their appropriate references.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-530) allow for mutual recursion in type definitions
[
https://issues.apache.org/jira/browse/AVRO-530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12861897#action_12861897
]
Jeff Hodges commented on AVRO-530:
--
Yeah, but now might be the time to change that! This is a known,
well-understood, and solved parsing problem. We can totally do it!
At my company, we're building lots of little services and its really, really
handy to have the schema right up next to the code. We deal with a lot of
thrift and while we understand the compilation step, it turns out to really
nice to have the protocol Right There while you're working. Especially, if
you're still tweaking it. We're hitting the desire/need for mutual recursion
way before we're hitting a spot where genavro makes real sense for us. (Maybe
others have different experiences?)
And making mutual recursion would be a huge boon for anyone who doesn't want to
deal with the genavro step, too. I know a lot of folks that have come on board
just because we make it easy to play with Avro and push it to its limits
quickly. I wouldn't want to take that away and say use this thing that you'll
have to make part of your build and then get all the file path stuff right and
have to think when all they want to do is see how far they can push it.
In any case, I'm going to give this a go in the ruby implementation, and put up
a patch. Maybe someone with a similar desire can do it in java or python and
see how well it works?
As the apps grow, genavro makes total sense! But I'd hate to limit wonderful,
powerful ideas to it that are, while not trivial, totally doable at every
level. How nice it is to know your idea will always work, no matter how you
interact with avro.
allow for mutual recursion in type definitions
--
Key: AVRO-530
URL: https://issues.apache.org/jira/browse/AVRO-530
Project: Avro
Issue Type: Improvement
Components: spec
Affects Versions: 1.3.2
Reporter: Jeff Hodges
Suppose you have these two types in your protocol:
{code}
{name: User, type: record, fields: [{name: current_status,
type: Status}]}
{name: Status, type: record, fields: [{name: author, type:
User}]}
{code}
This will raise an error! The current workaround is to define one of them at
their first usage. Like:
{code}
{name: User, type: record, fields: [{name: current_status,
type: {name: Status, type: record, fields: [.. lots of fields
...]}]}
{code}
But this is incredibly unwieldy. It would be really nice for the spec to
require all the parsers to allow for mutual recursion, instead. It could be
done by implementing a two-pass parser. One pass to acquire names referenced,
and a second to fill in those names with their appropriate references.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-530) allow for mutual recursion in type definitions
[
https://issues.apache.org/jira/browse/AVRO-530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12861899#action_12861899
]
Jeff Hodges commented on AVRO-530:
--
Oh, and I guess I screwed up my searches earlier. Where did we have this
discussion at? Maybe I'm missing something important.
allow for mutual recursion in type definitions
--
Key: AVRO-530
URL: https://issues.apache.org/jira/browse/AVRO-530
Project: Avro
Issue Type: Improvement
Components: spec
Affects Versions: 1.3.2
Reporter: Jeff Hodges
Suppose you have these two types in your protocol:
{code}
{name: User, type: record, fields: [{name: current_status,
type: Status}]}
{name: Status, type: record, fields: [{name: author, type:
User}]}
{code}
This will raise an error! The current workaround is to define one of them at
their first usage. Like:
{code}
{name: User, type: record, fields: [{name: current_status,
type: {name: Status, type: record, fields: [.. lots of fields
...]}]}
{code}
But this is incredibly unwieldy. It would be really nice for the spec to
require all the parsers to allow for mutual recursion, instead. It could be
done by implementing a two-pass parser. One pass to acquire names referenced,
and a second to fill in those names with their appropriate references.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-530) allow for mutual recursion in type definitions
[
https://issues.apache.org/jira/browse/AVRO-530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12861935#action_12861935
]
Scott Carey commented on AVRO-530:
--
Some of it was in this conversation:
http://www.mail-archive.com/avro-dev@hadoop.apache.org/msg02397.html
I agree the JSON could certainly be easier to use in several ways. But those
are spec changes that are not backwards compatible. Maybe Avro 2.0?
allow for mutual recursion in type definitions
--
Key: AVRO-530
URL: https://issues.apache.org/jira/browse/AVRO-530
Project: Avro
Issue Type: Improvement
Components: spec
Affects Versions: 1.3.2
Reporter: Jeff Hodges
Suppose you have these two types in your protocol:
{code}
{name: User, type: record, fields: [{name: current_status,
type: Status}]}
{name: Status, type: record, fields: [{name: author, type:
User}]}
{code}
This will raise an error! The current workaround is to define one of them at
their first usage. Like:
{code}
{name: User, type: record, fields: [{name: current_status,
type: {name: Status, type: record, fields: [.. lots of fields
...]}]}
{code}
But this is incredibly unwieldy. It would be really nice for the spec to
require all the parsers to allow for mutual recursion, instead. It could be
done by implementing a two-pass parser. One pass to acquire names referenced,
and a second to fill in those names with their appropriate references.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-530) allow for mutual recursion in type definitions
[
https://issues.apache.org/jira/browse/AVRO-530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12861979#action_12861979
]
Jeff Hodges commented on AVRO-530:
--
Yeah, I suppose. I'm happy to have a ticket for this.
allow for mutual recursion in type definitions
--
Key: AVRO-530
URL: https://issues.apache.org/jira/browse/AVRO-530
Project: Avro
Issue Type: Improvement
Components: spec
Affects Versions: 1.3.2
Reporter: Jeff Hodges
Suppose you have these two types in your protocol:
{code}
{name: User, type: record, fields: [{name: current_status,
type: Status}]}
{name: Status, type: record, fields: [{name: author, type:
User}]}
{code}
This will raise an error! The current workaround is to define one of them at
their first usage. Like:
{code}
{name: User, type: record, fields: [{name: current_status,
type: {name: Status, type: record, fields: [.. lots of fields
...]}]}
{code}
But this is incredibly unwieldy. It would be really nice for the spec to
require all the parsers to allow for mutual recursion, instead. It could be
done by implementing a two-pass parser. One pass to acquire names referenced,
and a second to fill in those names with their appropriate references.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
