[
https://issues.apache.org/jira/browse/THRIFT-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648929#comment-13648929
]
Carl Yeksigian commented on THRIFT-1948:
----------------------------------------
[~chet...@gmail.com]: I guess your comments catch me a little off-guard. Can
you detail the poor experience that you have been having? I see that the only
bug that you have filed is to include the ocaml tutorial; are there other
places that you have been struggling?
I think that if we are restricting Thrift to contain the types that have
already been defined, we are severely limiting the future potential that Thrift
has. As for your first point, I have also used the tricks that you have laid
out in my own code to get around the issue of Thrift not allowing state to be
kept between transactions. However, I do not think that we should be aiming to
have every developer reimplement the wheel when they want to transfer state; if
we can add some smarts to the protocol, I think that's a net win. I also don't
see this as a limiting feature: if someone includes streams, then they know
that they are taking on the extra protocol overhead. If they use no streams,
then the generated code will be the same as it was before.
For the other points: I don't think that the answer should be yet another
protocol; it would be better to be a part of the language that is defined. My
justification is that the other way doesn't allow a realistic two way
communication; the client sends their part (fully), then the server waits and
sends their part (fully). Although language fidelity is a real issue, there are
two gaping problems with that: 1. I'm not going to be working on language
fidelity in lieu of this issue (I'd rather just work on some other project),
and 2. Unless actual bugs are going to be filed, no one else knows of the
issues that pop up regarding this.
I know that Cassandra has implemented their own protocol, with streaming being
one of the benefits of not using Thrift. This ticket is meant to address this
particular limitation of Thrift.
> Add a stream type
> -----------------
>
> Key: THRIFT-1948
> URL: https://issues.apache.org/jira/browse/THRIFT-1948
> Project: Thrift
> Issue Type: New Feature
> Components: AS3 - Compiler, AS3 - Library, C glib - Compiler, C glib
> - Library, C# - Compiler, C# - Library, C++ - Compiler, C++ - Library, Cocoa
> - Compiler, Cocoa - Library, Compiler (General), D - Compiler, D - Library,
> Delphi - Compiler, Delphi - Library, Erlang - Compiler, Erlang - Library, Go
> - Compiler, Go - Library, Haskell - Compiler, Haskell - Library, Java -
> Compiler, Java - Library, JavaME - Compiler, JavaME - Library, JavaScript -
> Compiler, JavaScript - Library, Node.js - Compiler, Node.js - Library, OCaml
> - Compiler, OCaml - Library, Perl - Compiler, Perl - Library, PHP - Compiler,
> PHP - Library, Python - Compiler, Python - Library, Ruby - Compiler, Ruby -
> Library, Smalltalk - Compiler, Smalltalk - Library
> Reporter: Carl Yeksigian
> Assignee: Carl Yeksigian
>
> This is a proposal for an addition to the Thrift IDL, which allows for
> sending chunks of data between the server and the client without having the
> whole message in memory at the start of the communication.
> Here are two use cases where I have been thinking about the possibility of
> using streams.
> LockServer.thrift:
> {code}
> struct Update {
> 1: required string lock_handle,
> 2: required i64 owner
> }
> service LockService {
> stream<Update> updates_for(1: string prefix)
> }
> {code}
> This would allow the LockServer to push out updates that happen based on the
> prefix the client has specified, rather than the constant polling that would
> currently be required to imitate this interface.
> ManyResults.thrift:
> {code}
> service QueryProvider {
> stream<Result> run_query()
> }
> {code}
> This allows the query provider to run the query and send back the results as
> they come in, rather than having to bunch them up, or provide a way to page
> through the results to the client.
> The new keyword, "stream<T>", would indicate that there is a series of values
> typed T which would be communicated between client and server. Stream would
> have three primitives:
> {code}
> next(T)
> error(TException)
> end()
> {code}
> Protocols would be enhanced with the following methods:
> {code}
> writeStreamBegin(etype, streamid)
> writeStreamNext(streamid, streamMessageType)
> writeStreamNextEnd()
> writeStreamErrorEnd()
> etype, streamid = readStreamBegin()
> streamid, streamMessageType = readStreamNext()
> readStreamNextEnd()
> readStreamErrorEnd()
> {code}
> streamMessageType is one of the following:
> # next
> This means that the message will be of the element type.
> # error
> An exception was thrown during materialization of the stream.
> The stream is now closed.
> # end
> This means that the stream is finished.
> The stream is now closed.
> Once all streams are closed, readMessageEnd should be called. Before the
> first writeStreamNext() could be called, the message should otherwise be
> complete. Otherwise, an exception should be raised.
> It is possible that an exception will be thrown while the stream is being
> materialized; however, this can only occur inside of a service. In this case,
> error() will be called; the exception should be one of the exceptions that
> the service call would have thrown. The values that were generated before the
> exception will generally be valid, but may only have meaning if the stream is
> ended. All streams which are currently open may get the same exception.
> If the following service was defined:
> {code}
> stream<i64> random_numbers(stream<i64> max)
> {code}
> A sample session from client to server would be:
> {code}
> writeMessageBegin()
> writeStreamBegin(I64, 0)
> writeStreamNext(0, next)
> writeI64(10)
> writeStreamNextEnd()
> writeStreamNext(0, end)
> writeMessageEnd()
> {code}
> A sample session from server to client would be:
> {code}
> writeMessageBegin()
> writeStreamBegin(i64, 0)
> writeStreamNext(0, next)
> writeI64(3)
> writeStreamNextEnd()
> writeStreamNext(0, end)
> writeMessageEnd()
> {code}
> This change would not be compatible with previous versions of Thrift. Also,
> for languages which do not support this type of streaming, it could be
> translated into a list.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
