[ https://issues.apache.org/jira/browse/MESOS-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306291#comment-15306291 ]
Vinod Kone edited comment on MESOS-3067 at 5/30/16 6:30 AM: ------------------------------------------------------------ We chose RecordIO format because it can be applied consistently to both JSON and serialized protobufs. As for the content type, agree that application/json is incorrect but that is what Twitter streaming API and other APIs I have looked around that does RecordIO streaming call it :( Maybe we should call it application/x-json-recordio and application/x-protobuf-recordio respectively? was (Author: vinodkone): We chose RecordIO format because it can be applied consistently to both JSON and serialized protobufs. As for the content type, agree that application/json is incorrect but that is what Twitter streaming API and other APIs I have looked around that does RecordIO streaming call it :( Maybe we should call it application/x-json-recordio? > Implement a streaming response decoder for events stream > -------------------------------------------------------- > > Key: MESOS-3067 > URL: https://issues.apache.org/jira/browse/MESOS-3067 > Project: Mesos > Issue Type: Task > Reporter: Anand Mazumdar > Assignee: Benjamin Mahler > Fix For: 0.24.0 > > > We need a streaming response decoder to de-serialize chunks sent from the > master on the events stream. > From the HTTP API design doc: > Master encodes each Event in RecordIO format, i.e. a string representation of > length of the event in bytes followed by JSON or binary Protobuf (possibly > compressed) encoded event. > As of now for getting the basic features right , this is being done in the > test-cases: > {code} > auto reader = response.get().reader; > ASSERT_SOME(reader); > Future<std::string> eventFuture = reader.get().read(); > AWAIT_READY(eventFuture); > Event event; > event.ParseFromString(eventFuture.get()); > {code} > Two things need to happen: > - We need master to emit events in RecordIO format i.e. event size followed > by the serialized event instead of just the serialized events as is the case > now. > - The decoder class should then abstract away the logic of reading the > response and de-serializing events from the stream. > Ideally, the decoder should work with both "json" and "protobuf" responses. -- This message was sent by Atlassian JIRA (v6.3.4#6332)