[jira] [Updated] (CASSANDRA-2478) Custom CQL protocol/transport
[ https://issues.apache.org/jira/browse/CASSANDRA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2478: -- Reviewer: yukim > Custom CQL protocol/transport > - > > Key: CASSANDRA-2478 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2478 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Eric Evans >Assignee: Sylvain Lebresne >Priority: Minor > Labels: cql > Attachments: cql_binary_protocol, cql_binary_protocol-v2 > > > A custom wire protocol would give us the flexibility to optimize for our > specific use-cases, and eliminate a troublesome dependency (I'm referring to > Thrift, but none of the others would be significantly better). Additionally, > RPC is bad fit here, and we'd do better to move in the direction of something > that natively supports streaming. > I don't think this is as daunting as it might seem initially. Utilizing an > existing server framework like Netty, combined with some copy-and-paste of > bits from other FLOSS projects would probably get us 80% of the way there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2478) Custom CQL protocol/transport
[ https://issues.apache.org/jira/browse/CASSANDRA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2478: Attachment: cql_binary_protocol-v2 I've pushed at https://github.com/pcmanus/cassandra/commits/2478-2 an updated version of this patch. I'm also attaching the v2 document describing this new version. This version adds to the previous one: * keyspace and table names to the metadata returned as response to SELECT and prepared queries. The protocol supports having different ks/cf names for each column, but have a more compact form when all columns are from the same ks/cf (which is always the case for SELECT, but not for prepared queries). * it removes (mostly from the document) the few tidbits about the sketched auto-paging. I turns out this is more complicated than I though and may require a bit more discussion. So I prefer living that to a follow up ticket. * a few other cleanups In this form, this is a "complete" first version in that it is on par with the thrift API. So imo it's a good first step and I'd like trying to get that in and then adds new features/improve it in separate tickets (SSL, auto-paging, consider SASL, etc...). I think the sooner we get a working version in, the sooner we'll have people playing with it and get feedback. > Custom CQL protocol/transport > - > > Key: CASSANDRA-2478 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2478 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Eric Evans >Assignee: Sylvain Lebresne >Priority: Minor > Labels: cql > Attachments: cql_binary_protocol, cql_binary_protocol-v2 > > > A custom wire protocol would give us the flexibility to optimize for our > specific use-cases, and eliminate a troublesome dependency (I'm referring to > Thrift, but none of the others would be significantly better). Additionally, > RPC is bad fit here, and we'd do better to move in the direction of something > that natively supports streaming. > I don't think this is as daunting as it might seem initially. Utilizing an > existing server framework like Netty, combined with some copy-and-paste of > bits from other FLOSS projects would probably get us 80% of the way there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2478) Custom CQL protocol/transport
[ https://issues.apache.org/jira/browse/CASSANDRA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2478: Attachment: cql_binary_protocol Attaching cql_binary_protocol as a draft for such a custom binary protocol (CQL3 only). The protocol follows more or less Rick's outline above. It is frame based, each frame has a small header indicating amongst other the opcode for the message that defines what the body of said message must contain. A typical communication start by a small handshake/authentication phase, after which queries (and preparation/execution) can be performed. The protocol support a form of 'cursor' api for queries. Given a select query, the client can ask the server to return only a handful of rows first, and then the client can fetch more rows at his own rate using a NEXT message. Outside of the cursor thing, the protocol as described here pretty much expose the same things than the thrift transport (as far as CQL is concerned) but not much more (a small exception is that CASSANDRA-3707 is included). I plan on experimenting next with a few additional features, like allowing clients to register to events like 'a new node joined' and be notified when such event happen, but I'll leave that to follow up tickets. I've push at https://github.com/pcmanus/cassandra/commits/2478 an initial implementation of this protocol (using netty). Almost all of the protocol is implemented except for the NEXT message, but I'll get to that. There is currently mainly 4 patches: * the first one is the bulk of the new server * the second one is a simple client, along with a small "debug" console (whose code is ugly), that allow to send message to test the server. This does not necessarily have to make it in the final commit, but it is useful for testing. * the third one replace the use of CqlResult in the CQL3 code to directly build the new messages. And for the thrift interface, it simply translate those message to CqlResult. I've done it that way (instead of generating CqlResult and convert that to messages of the native protocol) because I think that is the direction we want to go. However currently that means that there is a few info that don't make it anymore in the CqlResult, namely the timestamp of the columns. Anywa, imo CASSANDRA-4217 is a much better way to access the timestamp and I'm not sure existing client were exposing the timestamp, but if there is complaints, that can be fixe is a much better way to access the timestamp and I'm not sure existing client were exposing the timestamp, but if there is complaints, that can be fixed. * the last one changes our CassandraDaemon business so that we can run a server with both the thrift and native protocol server running cleanly. Other than that, I have really benchmarked this (but that should be done). I meant to update stress to use the new server but realized that stress doesn't work for CQL3 at all, so that will be a separate ticket probably. > Custom CQL protocol/transport > - > > Key: CASSANDRA-2478 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2478 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Eric Evans >Assignee: Sylvain Lebresne >Priority: Minor > Labels: cql > Attachments: cql_binary_protocol > > > A custom wire protocol would give us the flexibility to optimize for our > specific use-cases, and eliminate a troublesome dependency (I'm referring to > Thrift, but none of the others would be significantly better). Additionally, > RPC is bad fit here, and we'd do better to move in the direction of something > that natively supports streaming. > I don't think this is as daunting as it might seem initially. Utilizing an > existing server framework like Netty, combined with some copy-and-paste of > bits from other FLOSS projects would probably get us 80% of the way there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2478) Custom CQL protocol/transport
[ https://issues.apache.org/jira/browse/CASSANDRA-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2478: -- Labels: cql (was: ) Summary: Custom CQL protocol/transport (was: Custom protocol/transport) > Custom CQL protocol/transport > - > > Key: CASSANDRA-2478 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2478 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Eric Evans >Priority: Minor > Labels: cql > > A custom wire protocol would give us the flexibility to optimize for our > specific use-cases, and eliminate a troublesome dependency (I'm referring to > Thrift, but none of the others would be significantly better). Additionally, > RPC is bad fit here, and we'd do better to move in the direction of something > that natively supports streaming. > I don't think this is as daunting as it might seem initially. Utilizing an > existing server framework like Netty, combined with some copy-and-paste of > bits from other FLOSS projects would probably get us 80% of the way there. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
