[
https://issues.apache.org/jira/browse/AVRO-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530314#comment-16530314
]
Srujan Narkedamalli commented on AVRO-2187:
-------------------------------------------
Thanks for your response and very sorry for the delay in getting back.
Avro's one-way is quite useful and unique, but the streaming that I was
referring for to is a bit different and more in line with what gRPC offers.
What I mentioned as server streaming rpc is a single rpc request by client that
yeilds a repeated response from server (of defined type). An example of this be
a request by client for fetching a list of files from a storage like s3 and
each of file is sent as response of for rpc request. Similarly for
client-streaming rpc, multiple requests (of defined type) can be sent for
single requests and sever sends back a single response. In bi-directional
streaming both multiple requests and responses can be sent but all in context
of single RPC. The difference/improvement from one-way calls is being able to
receive multiple responses or send multiple requests in context of a single RPC.
In order to use such streaming calls provided by gRPC in Avro's IDL, we are
currently annotating the calls with our custom annotations and modified the
velocity templates for stub code generation for these. Instead of these
annotations we would prefer to have this information specified as a Avro
keyword to make sure its compatible for different languages and info is in the
schema.
One way I see for this change to be backward compatible is, every rpc marked as
streaming yeilds two rpcs: regular rpc call and in addition a streaming call
with call name appended streaming probably and has the property streaming.
Generating the additional streaming rpc could be an option.
Our avro avdl file for a server streaming RPC looks like this:
{code:java}
@(io.grpc.ServerStreaming)
File searchByNameAndStreamFiles(string name);
//our custom modified generated code is as below, (StreamObserver is gRPC API):
void searchByNameAndStreamFiles(string name, StreamObserver<File> name); {code}
We would prefer to have the above schema be represented as:
{code:java}
File searchByNameAndStreamFiles(string name) serverstreaming;
{code}
For comparison: gRPC's preferred IDL- Protobuf, represent a similar server
streaming call as:
{code:java}
rpc searchByNameAndStreamFiles (SearchRequest) returns (stream File) {}
{code}
> Add RPC Streaming constructs/keywords to Avro IDL or schema
> -----------------------------------------------------------
>
> Key: AVRO-2187
> URL: https://issues.apache.org/jira/browse/AVRO-2187
> Project: Avro
> Issue Type: New Feature
> Reporter: Srujan Narkedamalli
> Priority: Major
>
> Motivation:
> We recently added support for transporting Avro serialization and IDL over
> gRPC for Java. In order to use the streaming features of gRPC or any other
> transport that supports streaming we need to be able to specify them IDL and
> schema.
> Details:
> Currently, gRPC supports 3 types of streaming calls:
> # server streaming (server can send multiple responses for a single request)
> # client streaming (client can multiple requests and server sends a single
> response)
> # bi-directional streaming call (on going rpc with multiple requests and
> responses)
> We would want a way to represent these types on calls in Avro's IDL similar
> to one-way calls using a keywords. Usually in gRPC with other IDLs a
> streaming request or response is repeated payload of same type. For client
> streaming and bi-directional streaming it would be simpler to have a single
> request argument when representing their type in callbacks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
