Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread bened...@apache.org
so that failures to behave in an appropriate manner cannot unduly prevent progress. From: Mick Semb Wever Date: Friday, 15 October 2021 at 16:33 To: dev@cassandra.apache.org Subject: Re: Tradeoffs for Cassandra transaction management > > I have reviewed CEP-15 and I must say, I'm excited t

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Henrik Ingo
On Fri, Oct 15, 2021 at 5:54 PM Dinesh Joshi wrote: > Thank you for clarifying the terminology. I haven’t honestly heard anybody > call these as interactive transactions. Therefore it is very crucial that > we lay out things systematically so everyone is on the same page. You’re > talking about

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Bowen Song
I'm worried that by the time a consensus is reached, the people who originally purposed the CEP may have long lost their passion about it and may no longer willing to contribute. On 15/10/2021 16:55, Benjamin Lerer wrote: Reaching consensus is hard but we will get there :-) Le ven. 15 oct.

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Benjamin Lerer
Reaching consensus is hard but we will get there :-) Le ven. 15 oct. 2021 à 17:33, Mick Semb Wever a écrit : > > > > I have reviewed CEP-15 and I must say, I'm excited to see its inclusion > > into mainline Cassandra, and I'm disheartened to see what appears to be > an > > unsubstantiated veto

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Mick Semb Wever
> > I have reviewed CEP-15 and I must say, I'm excited to see its inclusion > into mainline Cassandra, and I'm disheartened to see what appears to be an > unsubstantiated veto of the proposal from the committee's leadership. > Leif, the Accord paper and CEP-15 has indeed generated a lot of

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Dinesh Joshi
Thank you for clarifying the terminology. I haven’t honestly heard anybody call these as interactive transactions. Therefore it is very crucial that we lay out things systematically so everyone is on the same page. You’re talking about bundling several statements into a single SQL transaction

RE: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Leif Walsh
Hi all, I'm not an active member of the c* developer community, but I'm a user of c* at my day job, and I have a healthy background in distributed storage systems and consensus protocols (my previous job and university training). I have reviewed CEP-15 and I must say, I'm excited to see its

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Henrik Ingo
On Fri, Oct 15, 2021 at 3:37 AM Dinesh Joshi wrote: > On 10/14/21 6:54 AM, Jonathan Ellis wrote: > > > I think I've also been clear that I want a path to supporting (1) local > > latencies (SLOG is a more elegant solution but "let's just let people > give > > up global serializability like LWT"

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread Jordan West
Hi All, First off, thank you for the very interesting technical discussions on this topic. It's been great to see some back and forth on it. I haven't been involved mainly because my research on this topic is relatively stale. I did however want to chime in to encourage us to step back and take a

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread Dinesh Joshi
On 10/14/21 6:54 AM, Jonathan Ellis wrote: > I think I've also been clear that I want a path to supporting (1) local > latencies (SLOG is a more elegant solution but "let's just let people give > up global serializability like LWT" is also reasonable) and (2) SQL with > interactive transactions.

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread Jonathan Ellis
On Thu, Oct 14, 2021 at 4:01 PM bened...@apache.org wrote: > The only TPC-C New Order transaction I recall you linking was interactive, > which as far as I am aware is not supported by Calvin. > The SQLite version I linked was interactive, but it can be implemented non-interactively, which is

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread bened...@apache.org
explanation of the problem case will help me better understand what needs to be stated in response to your query. From: Jonathan Ellis Date: Thursday, 14 October 2021 at 21:47 To: dev Subject: Re: Tradeoffs for Cassandra transaction management I already linked a description of the TPC-C New Order

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread Jonathan Ellis
> conflicting demands. > > 3. Why do you think Accord cannot support your preferred semantics? > > 4. Will you accept a video call so we may discuss this with you in > detail, > > so we may understand your difficulty understanding these points I keep > > repeat

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread bened...@apache.org
disproves this statement. From: Jonathan Ellis Date: Thursday, 14 October 2021 at 14:55 To: dev Subject: Re: Tradeoffs for Cassandra transaction management Hi Benedict, I'm not sure how to reconcile your statement that "your request to separate consensus from execution is [nonsensical]" with

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread bened...@apache.org
: Thursday, 14 October 2021 at 14:55 To: dev Subject: Re: Tradeoffs for Cassandra transaction management Hi Benedict, I'm not sure how to reconcile your statement that "your request to separate consensus from execution is [nonsensical]" with your earlier claims that we could build whatever

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread Jeff Jirsa
preferred semantics? > > 4. Will you accept a video call so we may discuss this with you in > detail, > > so we may understand your difficulty understanding these points I keep > > repeating? > > > > After several weeks of back and forth you should already be able to &g

Re: Tradeoffs for Cassandra transaction management

2021-10-14 Thread Jonathan Ellis
r them now, I > perceive this as obstructive and I will escalate this to a PMC vote to > break the deadlock. > > > > From: Jonathan Ellis > Date: Wednesday, 13 October 2021 at 04:21 > To: dev > Subject: Re: Tradeoffs for Cassandra transaction management > Blake (and Bened

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Henrik Ingo
Sorry Jonathan, didn't see this reply earlier today. That would be common behaviour for many MVCC databases, including MongoDB, MySQL Galera Cluster, PostgreSQL... https://www.postgresql.org/docs/9.5/transaction-iso.html *"Applications using this level must be prepared to retry transactions due

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Alex Miller
On Wed, Oct 13, 2021 at 3:52 AM Henrik Ingo wrote: > Aren't you actually pointing out a limitation in any "single shot" > transactional algorithm? Including Accord itself, without any interactive > part? > > What you are saying is that an Accord transaction is limited by the need > for both the

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread bened...@apache.org
: Tradeoffs for Cassandra transaction management Hi Alex, I hugely value and respect your input here, but I think in this case you may be mistaken. Postgres[1] makes explicit that subsequent SELECT statements may see different data, and SQL Server[2] does the same. I believe the Oracle documents you

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Henrik Ingo
Thank you Alex for your feedback, I greatly value these thoughts and always enjoy learning new details in this space. On Wed, Oct 13, 2021 at 10:07 AM Alex Miller wrote: > These two pieces together seem to imply that your claim is that Read > Committed may read whatever the most recently

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread bened...@apache.org
to answer these questions. If you cannot invest the time to answer them now, I perceive this as obstructive and I will escalate this to a PMC vote to break the deadlock. From: Jonathan Ellis Date: Wednesday, 13 October 2021 at 04:21 To: dev Subject: Re: Tradeoffs for Cassandra transaction

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread bened...@apache.org
-committed.html [2] https://docs.microsoft.com/en-us/sql/connect/jdbc/understanding-isolation-levels?view=sql-server-ver15 From: Alex Miller Date: Wednesday, 13 October 2021 at 08:07 To: dev@cassandra.apache.org Subject: Re: Tradeoffs for Cassandra transaction management On Tue, Oct 12, 2021

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Alex Miller
On Tue, Oct 12, 2021 at 3:55 PM Henrik Ingo wrote: > We define READ COMMITTED as "whatever is returned by Cassandra when > executing the query (with QUORUM consistency)". In other words, this > functionality doesn't require any changes to the storage engine or other > fundamental changes to

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Jonathan Ellis
Blake (and Benedict), I’ll ask for your patience here. We don’t have a precedent of pushing through major initiatives in this project in a matter of weeks. We [members of the PMC that weren’t involved in creating Accord] need time to do thorough research and make sure both that we understand

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Jonathan Ellis
Hi Henrik, I don't see how this resolves the fundamental problem that I outlined to start with, namely, that without having the entire logic of the transaction available to it, the server cannot retry the transaction when concurrent changes are found to have been applied after the reconnaissance

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Blake Eggleston
Hi Henrik, I would agree that the local serial experience for valid use cases should be supported in some form before legacy LWT is replaced by Accord. Regarding your read committed proposal, I think this CEP discussion has already spent too much time talking about hypothetical SQL

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Henrik Ingo
On Tue, Oct 12, 2021 at 11:54 PM Henrik Ingo wrote: > Secondary indexes are supported without any additional work needed. > > Correction: The "transaction reads its own writes" feature would require to also store secondary index keys in the transaction state. These of course needn't be part of

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Henrik Ingo
Hi all I was expecting to stay out of the way while a vote on CEP-15 seemed imminent. But discussing this tradeoffs thread with Jonathan, he encouraged me to say these points in my own words, so here we are. On Sun, Oct 10, 2021 at 7:17 AM Blake Eggleston wrote: > 1. Is it worth giving up

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread bened...@apache.org
for Cassandra transaction management On Mon, Oct 11, 2021 at 5:11 PM bened...@apache.org wrote: > If we want to fully unpack this particular point, as far as I can tell > claiming ANSI SQL would indeed require interactive transactions in which > arbitrary conditional work may be performed by

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Blake Eggleston
Let’s get back on topic. Jonathan, in your opening email you stated that, in your view, the 2 main areas of tradeoff were: > 1. Is it worth giving up local latencies to get full global consistency? Now we’ve established that we don’t need to give up local latencies with Accord, which leaves:

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Jonathan Ellis
On Mon, Oct 11, 2021 at 5:11 PM bened...@apache.org wrote: > If we want to fully unpack this particular point, as far as I can tell > claiming ANSI SQL would indeed require interactive transactions in which > arbitrary conditional work may be performed by a client within a > transaction in

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread bened...@apache.org
at 22:20 To: dev Subject: Re: Tradeoffs for Cassandra transaction management Hi Benedict, Yes, interactive transactions are a necessary part of SQL support (as opposed to a tiny subset of SQL that matches CQL semantics, I don't know any other way to make sense of your claim that "SQL

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Jonathan Ellis
bort-free interactive transactions, not SQL. SQL does not > necessitate support for interactive transactions, let alone abort-free > ones. The technique you mention can support SQL scripts, and also > interactive client transactions that may be aborted by the server. However, > see [1] whi

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Jonathan Ellis
On Mon, Oct 11, 2021 at 12:31 PM Blake Eggleston wrote: > > Come on Blake, you have all been developing software long enough to know > > that "there's nothing about Accord that prevents this" is close to > > meaningless. > > > > If it's so easy to address an overwhelmingly popular use case, then

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Blake Eggleston
> Come on Blake, you have all been developing software long enough to know > that "there's nothing about Accord that prevents this" is close to > meaningless. > > If it's so easy to address an overwhelmingly popular use case, then let's > add it to the initial work. This is moving the goal

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Jonathan Ellis
Thanks for flagging that, Alex. Here it is without trying to include the inline table: *After calling several times for a broader discussion of goals and tradeoffs around transaction management in the CEP-15 thread, I’ve put together a short analysis to kick that off.Here is

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Oleksandr Petrov
I realise this is not contributing to this discussion, but this email is very difficult to read because it seems like something has happened with formatting. For me it gets displayed as a single paragraph with no line breaks. There seems to be some overlap between the image uploaded to imgur and

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Jonathan Ellis
On Sat, Oct 9, 2021 at 11:23 PM Blake Eggleston wrote: > 1. Is it worth giving up local latencies to get full global consistency? > Most LWT use cases use > LOCAL_SERIAL. > > This isn’t a tradeoff that needs to be made. There’s nothing about Accord > that prevents performing consensus in one DC

Re: Tradeoffs for Cassandra transaction management

2021-10-11 Thread Jonathan Ellis
On Sat, Oct 9, 2021 at 7:20 PM Jeff Jirsa wrote: > Most LWT use cases use LOCAL_SERIAL because the difference in latency is > huge today (given the 4x RTTs) AND almost none of the users actually > understand how cassandra replication or consistency works, so they > misunderstand the guarantees

Re: Tradeoffs for Cassandra transaction management

2021-10-10 Thread bened...@apache.org
can support SQL scripts, and also interactive client transactions that may be aborted by the server. However, see [1] which may support all of these properties. From: Blake Eggleston Date: Sunday, 10 October 2021 at 05:17 To: dev@cassandra.apache.org Subject: Re: Tradeoffs for Cassandra

Re: Tradeoffs for Cassandra transaction management

2021-10-09 Thread Blake Eggleston
1. Is it worth giving up local latencies to get full global consistency? Most LWT use cases use LOCAL_SERIAL. This isn’t a tradeoff that needs to be made. There’s nothing about Accord that prevents performing consensus in one DC and replicating the writes to others. That’s not in scope for the

Re: Tradeoffs for Cassandra transaction management

2021-10-09 Thread Jeff Jirsa
I'll read more of this in a bit, I want to make sure I fully digest it before commenting on the rest, but this block here deserves a few words: On Sat, Oct 9, 2021 at 9:54 AM Jonathan Ellis wrote: > After putting the above together it seems to > me that the two main areas of tradeoff are, 1.

Tradeoffs for Cassandra transaction management

2021-10-09 Thread Jonathan Ellis
* Hi all,After calling several times for a broader discussion of goals and tradeoffs around transaction management in the CEP-15 thread, I’ve put together a short analysis to kick that off.Here is a table that summarizes the state of the art for distributed transactions that offer serializability,