[jira] [Commented] (MESOS-3601) Formalize all headers and metadata for HTTP API Event Stream
[
https://issues.apache.org/jira/browse/MESOS-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947395#comment-14947395
]
Benjamin Mahler commented on MESOS-3601:
Hm... the {{Connection: keep-alive}} header is for HTTP 1.0; all connections in
HTTP 1.1 are persistent by default.
It would actually be more appropriate for us to use HTTP 1.1's {{Connection:
close}} header to prevent the caller from trying to pipeline on this
connection, since the stream response is infinite. IIUC the {{Connection}}
header has to do with whether multiple requests can occur on the same
connection, not sure how you're interpreting its meaning..?
> Formalize all headers and metadata for HTTP API Event Stream
>
>
> Key: MESOS-3601
> URL: https://issues.apache.org/jira/browse/MESOS-3601
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.24.0
> Environment: Mesos 0.24.0
>Reporter: Ben Whitehead
> Labels: api, http, wireprotocol
>
> From and HTTP standpoint the current set of headers returned when connecting
> to the HTTP scheduler API are insufficient.
> {code:title=current headers}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> Since the response from mesos is intended to function as a stream
> {{Connection: keep-alive}} should be specified so that the connection can
> remain open.
> If RecordIO is going to be applied to the messages, the headers should
> include the information necessary for a client to be able to detect RecordIO
> and setup it response handlers appropriately.
> How RecordIO is expressed will come down to the semantics of what is actually
> "Returned" as the response from {{POST /api/v1/scheduler}}.
> h4. Proposal
> One approach would be to leverage http as much as possible, having a client
> specify an {{Accept-Encoding}} along with the {{Accept}} header to indicate
> that it can handle RecordIO {{Content-Encoding}} of {{Content-Type}}
> messages. (This approach allows for things like gzip to be woven in fairly
> easily in the future)
> For this approach I would expect the following:
> {code:title=Request}
> POST /api/v1/scheduler HTTP/1.1
> Host: localhost:5050
> Accept: application/x-protobuf
> Accept-Encoding: recordio
> Content-Type: application/x-protobuf
> Content-Length: 35
> User-Agent: RxNetty Client
> {code}
> {code:title=Response}
> HTTP/1.1 200 OK
> Connection: keep-alive
> Transfer-Encoding: chunked
> Content-Type: application/x-protobuf
> Content-Encoding: recordio
> Cache-Control: no-transform
> {code}
> When Content-Encoding is used it is recommended to set {{Cache-Control:
> no-transform}} to signal to any proxies that no transformation should be
> applied to the the content encoding [Section 14.11 RFC
> 2616|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3601) Formalize all headers and metadata for HTTP API Event Stream
[
https://issues.apache.org/jira/browse/MESOS-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947516#comment-14947516
]
Ben Whitehead commented on MESOS-3601:
--
As I understand it, part of the motivation for RecordIO was to be tolerant of
proxies (1.0 or 1.1) between the scheduler and mesos (Since these proxies
according to the http spec can change chunk sizes). Being tolerant of a 1.0
proxy would require the connection semantics to be explicitly defined rather
than depending on default handling.
You are correct that keep-alive can be used to express intent for handling
request pipelining which we probably don't want for the specific socket the
event stream is flowing over, but it could be useful when other Calls are being
sent to the server.
In the case of the event stream itself {{Connection: close}} is probably more
appropriate.
> Formalize all headers and metadata for HTTP API Event Stream
>
>
> Key: MESOS-3601
> URL: https://issues.apache.org/jira/browse/MESOS-3601
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.24.0
> Environment: Mesos 0.24.0
>Reporter: Ben Whitehead
> Labels: api, http, wireprotocol
>
> From and HTTP standpoint the current set of headers returned when connecting
> to the HTTP scheduler API are insufficient.
> {code:title=current headers}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> Since the response from mesos is intended to function as a stream
> {{Connection: keep-alive}} should be specified so that the connection can
> remain open.
> If RecordIO is going to be applied to the messages, the headers should
> include the information necessary for a client to be able to detect RecordIO
> and setup it response handlers appropriately.
> How RecordIO is expressed will come down to the semantics of what is actually
> "Returned" as the response from {{POST /api/v1/scheduler}}.
> h4. Proposal
> One approach would be to leverage http as much as possible, having a client
> specify an {{Accept-Encoding}} along with the {{Accept}} header to indicate
> that it can handle RecordIO {{Content-Encoding}} of {{Content-Type}}
> messages. (This approach allows for things like gzip to be woven in fairly
> easily in the future)
> For this approach I would expect the following:
> {code:title=Request}
> POST /api/v1/scheduler HTTP/1.1
> Host: localhost:5050
> Accept: application/x-protobuf
> Accept-Encoding: recordio
> Content-Type: application/x-protobuf
> Content-Length: 35
> User-Agent: RxNetty Client
> {code}
> {code:title=Response}
> HTTP/1.1 200 OK
> Connection: keep-alive
> Transfer-Encoding: chunked
> Content-Type: application/x-protobuf
> Content-Encoding: recordio
> Cache-Control: no-transform
> {code}
> When Content-Encoding is used it is recommended to set {{Cache-Control:
> no-transform}} to signal to any proxies that no transformation should be
> applied to the the content encoding [Section 14.11 RFC
> 2616|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3601) Formalize all headers and metadata for HTTP API Event Stream
[
https://issues.apache.org/jira/browse/MESOS-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15655531#comment-15655531
]
James DeFelice commented on MESOS-3601:
---
what about something like this?
{code}
Content-type: application/json; streamFormat=record-io
{code}
(a) maintains backwards compat; and
(b) provides a way for clients to determine if it's a a series/stream of json
objects (and how they're packaged) vs a single object (would not include a
streamFormat parameter here)
there's already the concept of a "stream" in the via (via the Mesos-Stream-Id
header)
> Formalize all headers and metadata for HTTP API Event Stream
>
>
> Key: MESOS-3601
> URL: https://issues.apache.org/jira/browse/MESOS-3601
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.24.0
> Environment: Mesos 0.24.0
>Reporter: Ben Whitehead
> Labels: api, http, mesosphere, wireprotocol
>
> From an HTTP standpoint the current set of headers returned when connecting
> to the HTTP scheduler API are insufficient.
> {code:title=current headers}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> Since the response from mesos is intended to function as a stream
> {{Connection: keep-alive}} should be specified so that the connection can
> remain open.
> If RecordIO is going to be applied to the messages, the headers should
> include the information necessary for a client to be able to detect RecordIO
> and setup it response handlers appropriately.
> How RecordIO is expressed will come down to the semantics of what is actually
> "Returned" as the response from {{POST /api/v1/scheduler}}.
> h4. Proposal
> One approach would be to leverage http as much as possible, having a client
> specify an {{Accept-Encoding}} along with the {{Accept}} header to indicate
> that it can handle RecordIO {{Content-Encoding}} of {{Content-Type}}
> messages. (This approach allows for things like gzip to be woven in fairly
> easily in the future)
> For this approach I would expect the following:
> {code:title=Request}
> POST /api/v1/scheduler HTTP/1.1
> Host: localhost:5050
> Accept: application/x-protobuf
> Accept-Encoding: recordio
> Content-Type: application/x-protobuf
> Content-Length: 35
> User-Agent: RxNetty Client
> {code}
> {code:title=Response}
> HTTP/1.1 200 OK
> Connection: keep-alive
> Transfer-Encoding: chunked
> Content-Type: application/x-protobuf
> Content-Encoding: recordio
> Cache-Control: no-transform
> {code}
> When Content-Encoding is used it is recommended to set {{Cache-Control:
> no-transform}} to signal to any proxies that no transformation should be
> applied to the the content encoding [Section 14.11 RFC
> 2616|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3601) Formalize all headers and metadata for HTTP API Event Stream
[
https://issues.apache.org/jira/browse/MESOS-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15808310#comment-15808310
]
Vinod Kone commented on MESOS-3601:
---
Commented on the doc.
Posting here just in case:
FWICT, a media-type of "foo/bar+baz" should be used when "bar" is a specialized
format based on the more general "baz" format. See
https://tools.ietf.org/html/rfc3023#section-8.16
"+baz" is for those parsers who understand the generic "baz" format but not
necessarily the "bar" format.
The corollary is that if a parser understands the specific format "foo/bar" it
doesn't care about the suffix "baz".
In our case "recordio" is not a specialized format based on "json" or
"protobuf", it is a completely different format. A parser that understands
"application/recordio" cannot process the data without knowing additional
information.
> Formalize all headers and metadata for HTTP API Event Stream
>
>
> Key: MESOS-3601
> URL: https://issues.apache.org/jira/browse/MESOS-3601
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.24.0
> Environment: Mesos 0.24.0
>Reporter: Ben Whitehead
>Assignee: Anand Mazumdar
>Priority: Blocker
> Labels: api, http, mesosphere, wireprotocol
>
> From an HTTP standpoint the current set of headers returned when connecting
> to the HTTP scheduler API are insufficient.
> {code:title=current headers}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> Since the response from mesos is intended to function as a stream
> {{Connection: keep-alive}} should be specified so that the connection can
> remain open.
> If RecordIO is going to be applied to the messages, the headers should
> include the information necessary for a client to be able to detect RecordIO
> and setup it response handlers appropriately.
> How RecordIO is expressed will come down to the semantics of what is actually
> "Returned" as the response from {{POST /api/v1/scheduler}}.
> h4. Proposal
> One approach would be to leverage http as much as possible, having a client
> specify an {{Accept-Encoding}} along with the {{Accept}} header to indicate
> that it can handle RecordIO {{Content-Encoding}} of {{Content-Type}}
> messages. (This approach allows for things like gzip to be woven in fairly
> easily in the future)
> For this approach I would expect the following:
> {code:title=Request}
> POST /api/v1/scheduler HTTP/1.1
> Host: localhost:5050
> Accept: application/x-protobuf
> Accept-Encoding: recordio
> Content-Type: application/x-protobuf
> Content-Length: 35
> User-Agent: RxNetty Client
> {code}
> {code:title=Response}
> HTTP/1.1 200 OK
> Connection: keep-alive
> Transfer-Encoding: chunked
> Content-Type: application/x-protobuf
> Content-Encoding: recordio
> Cache-Control: no-transform
> {code}
> When Content-Encoding is used it is recommended to set {{Cache-Control:
> no-transform}} to signal to any proxies that no transformation should be
> applied to the the content encoding [Section 14.11 RFC
> 2616|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3601) Formalize all headers and metadata for HTTP API Event Stream
[
https://issues.apache.org/jira/browse/MESOS-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824759#comment-15824759
]
Adam B commented on MESOS-3601:
---
Any progress to report, [~anandmazumdar]? Should we add this to the Mesosphere
sprint 49 if you're working on it?
> Formalize all headers and metadata for HTTP API Event Stream
>
>
> Key: MESOS-3601
> URL: https://issues.apache.org/jira/browse/MESOS-3601
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.24.0
> Environment: Mesos 0.24.0
>Reporter: Ben Whitehead
>Assignee: Anand Mazumdar
>Priority: Blocker
> Labels: api, http, mesosphere, wireprotocol
>
> From an HTTP standpoint the current set of headers returned when connecting
> to the HTTP scheduler API are insufficient.
> {code:title=current headers}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> Since the response from mesos is intended to function as a stream
> {{Connection: keep-alive}} should be specified so that the connection can
> remain open.
> If RecordIO is going to be applied to the messages, the headers should
> include the information necessary for a client to be able to detect RecordIO
> and setup it response handlers appropriately.
> How RecordIO is expressed will come down to the semantics of what is actually
> "Returned" as the response from {{POST /api/v1/scheduler}}.
> h4. Proposal
> One approach would be to leverage http as much as possible, having a client
> specify an {{Accept-Encoding}} along with the {{Accept}} header to indicate
> that it can handle RecordIO {{Content-Encoding}} of {{Content-Type}}
> messages. (This approach allows for things like gzip to be woven in fairly
> easily in the future)
> For this approach I would expect the following:
> {code:title=Request}
> POST /api/v1/scheduler HTTP/1.1
> Host: localhost:5050
> Accept: application/x-protobuf
> Accept-Encoding: recordio
> Content-Type: application/x-protobuf
> Content-Length: 35
> User-Agent: RxNetty Client
> {code}
> {code:title=Response}
> HTTP/1.1 200 OK
> Connection: keep-alive
> Transfer-Encoding: chunked
> Content-Type: application/x-protobuf
> Content-Encoding: recordio
> Cache-Control: no-transform
> {code}
> When Content-Encoding is used it is recommended to set {{Cache-Control:
> no-transform}} to signal to any proxies that no transformation should be
> applied to the the content encoding [Section 14.11 RFC
> 2616|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3601) Formalize all headers and metadata for HTTP API Event Stream
[
https://issues.apache.org/jira/browse/MESOS-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824773#comment-15824773
]
Anand Mazumdar commented on MESOS-3601:
---
[~adam-mesos] Not much implementation wise. Though, we have agreed on the
approach to take based on the two proposed solutions mentioned in the design
doc. I would update the ticket/design doc shortly with more details on the
solution and start the implementation from tomorrow!
> Formalize all headers and metadata for HTTP API Event Stream
>
>
> Key: MESOS-3601
> URL: https://issues.apache.org/jira/browse/MESOS-3601
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.24.0
> Environment: Mesos 0.24.0
>Reporter: Ben Whitehead
>Assignee: Anand Mazumdar
>Priority: Blocker
> Labels: api, http, mesosphere, wireprotocol
>
> From an HTTP standpoint the current set of headers returned when connecting
> to the HTTP scheduler API are insufficient.
> {code:title=current headers}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> Since the response from mesos is intended to function as a stream
> {{Connection: keep-alive}} should be specified so that the connection can
> remain open.
> If RecordIO is going to be applied to the messages, the headers should
> include the information necessary for a client to be able to detect RecordIO
> and setup it response handlers appropriately.
> How RecordIO is expressed will come down to the semantics of what is actually
> "Returned" as the response from {{POST /api/v1/scheduler}}.
> h4. Proposal
> One approach would be to leverage http as much as possible, having a client
> specify an {{Accept-Encoding}} along with the {{Accept}} header to indicate
> that it can handle RecordIO {{Content-Encoding}} of {{Content-Type}}
> messages. (This approach allows for things like gzip to be woven in fairly
> easily in the future)
> For this approach I would expect the following:
> {code:title=Request}
> POST /api/v1/scheduler HTTP/1.1
> Host: localhost:5050
> Accept: application/x-protobuf
> Accept-Encoding: recordio
> Content-Type: application/x-protobuf
> Content-Length: 35
> User-Agent: RxNetty Client
> {code}
> {code:title=Response}
> HTTP/1.1 200 OK
> Connection: keep-alive
> Transfer-Encoding: chunked
> Content-Type: application/x-protobuf
> Content-Encoding: recordio
> Cache-Control: no-transform
> {code}
> When Content-Encoding is used it is recommended to set {{Cache-Control:
> no-transform}} to signal to any proxies that no transformation should be
> applied to the the content encoding [Section 14.11 RFC
> 2616|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3601) Formalize all headers and metadata for HTTP API Event Stream
[
https://issues.apache.org/jira/browse/MESOS-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826427#comment-15826427
]
Anand Mazumdar commented on MESOS-3601:
---
The implementation of the headers proposed in the design doc is being tracked
on MESOS-6936
Resolving this for now.
> Formalize all headers and metadata for HTTP API Event Stream
>
>
> Key: MESOS-3601
> URL: https://issues.apache.org/jira/browse/MESOS-3601
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 0.24.0
> Environment: Mesos 0.24.0
>Reporter: Ben Whitehead
>Assignee: Anand Mazumdar
>Priority: Blocker
> Labels: api, http, mesosphere, wireprotocol
> Fix For: 1.2.0
>
>
> From an HTTP standpoint the current set of headers returned when connecting
> to the HTTP scheduler API are insufficient.
> {code:title=current headers}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> Since the response from mesos is intended to function as a stream
> {{Connection: keep-alive}} should be specified so that the connection can
> remain open.
> If RecordIO is going to be applied to the messages, the headers should
> include the information necessary for a client to be able to detect RecordIO
> and setup it response handlers appropriately.
> How RecordIO is expressed will come down to the semantics of what is actually
> "Returned" as the response from {{POST /api/v1/scheduler}}.
> h4. Proposal
> One approach would be to leverage http as much as possible, having a client
> specify an {{Accept-Encoding}} along with the {{Accept}} header to indicate
> that it can handle RecordIO {{Content-Encoding}} of {{Content-Type}}
> messages. (This approach allows for things like gzip to be woven in fairly
> easily in the future)
> For this approach I would expect the following:
> {code:title=Request}
> POST /api/v1/scheduler HTTP/1.1
> Host: localhost:5050
> Accept: application/x-protobuf
> Accept-Encoding: recordio
> Content-Type: application/x-protobuf
> Content-Length: 35
> User-Agent: RxNetty Client
> {code}
> {code:title=Response}
> HTTP/1.1 200 OK
> Connection: keep-alive
> Transfer-Encoding: chunked
> Content-Type: application/x-protobuf
> Content-Encoding: recordio
> Cache-Control: no-transform
> {code}
> When Content-Encoding is used it is recommended to set {{Cache-Control:
> no-transform}} to signal to any proxies that no transformation should be
> applied to the the content encoding [Section 14.11 RFC
> 2616|http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
