[ 
https://issues.apache.org/jira/browse/AVRO-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831111#action_12831111
 ] 

Todd Lipcon commented on AVRO-406:
----------------------------------

bq. A way of doing this without changing anything in the spec is to transmit an 
array of chunks, since arrays are already blocked and may be arbitrarily long
bq. To my thinking it is only applicable when the final parameter of a method 
is an array type and/or when the return type is an array type

Agreed on both fronts. Having multiple streamed parameters would be very tricky 
to understand since you can't start streaming the second parameter until the 
first is complete. The same goes for streamed record members - I don't think we 
should allow it since we can't ensure that the client will consume them in the 
correct order.

bq. One could annotate the schema somehow, so that the compiler generates this 
alternate API

This was my "streamed array<Chunk>" notation above. In JSON syntax it might 
look something like
{code}
{ "type": "array",
  "items": "Chunk",
  "streamed: "true" }

This would modify the interfaces for both client and server to generate the 
streaming-style APIs. This is a good example usecase for mixin annotations 
(AVRO-404) since some users might want the "simple API" instead. The call would 
be completely wire-compatible between streamed and non-streamed 
implementations, of course.



> Support streaming RPC calls
> ---------------------------
>
>                 Key: AVRO-406
>                 URL: https://issues.apache.org/jira/browse/AVRO-406
>             Project: Avro
>          Issue Type: New Feature
>          Components: java, spec
>            Reporter: Todd Lipcon
>
> Avro nicely supports chunking of container types into multiple frames. We 
> need to expose this to RPC layer to facilitate use cases like the Hadoop 
> Datanode where a single "RPC" can yield far more data than should be buffered 
> in memory.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to