Re: [HACKERS] incoherent view of serializable transactions

2009-01-21 Thread Kevin Grittner
Here's a shot at a more radical revision, to try to address concerns raised over my failure in the previous (very minimal) suggested patch to address PostgreSQL behavior close to where the spec's behavior is described, and my dragging in of language directly from the spec in a confusing context. I

Re: [HACKERS] incoherent view of serializable transactions

2009-01-08 Thread Kevin Grittner
>>> I wrote: Peter Eisentraut wrote: >> Kevin Grittner wrote: "is a natural consequence of the fact" --- There is nothing natural about any of this. Why is it a consequence and how? >>> >>> How could you possibly get any of those phenomena if there are no >>> concurrent transac

Re: [HACKERS] incoherent view of serializable transactions

2009-01-07 Thread Kevin Grittner
>>> Peter Eisentraut wrote: > Kevin Grittner wrote: >>> "is a natural consequence of the fact" --- There is nothing >>> natural about any of this. Why is it a consequence and how? >> >> How could you possibly get any of those phenomena if there are no >> concurrent transactions? > > I see wha

Re: [HACKERS] incoherent view of serializable transactions

2009-01-07 Thread Peter Eisentraut
Kevin Grittner wrote: "is a natural consequence of the fact" --- There is nothing natural about any of this. Why is it a consequence and how? How could you possibly get any of those phenomena if there are no concurrent transactions? I see what you mean now, but you could write out that logi

Re: [HACKERS] incoherent view of serializable transactions

2009-01-06 Thread Kevin Grittner
>>> Paul Schlie wrote: >> Kevin Grittner wrote: >> There is a related thread on which I'm attempting to come up with >> documentation to assist those familiar with true serializable >> behavior who are attempting to recognize application coding >> patterns where the differences between that and

Re: [HACKERS] incoherent view of serializable transactions

2009-01-06 Thread Paul Schlie
> Kevin Grittner wrote: Paul Schlie wrote: >> Sorry if I'm restating the obvious, however I don't understand the >> confusion, as it seems the standard's definition isn't mysterious; >> it simply requires that the resulting state from the concurrent >> execution of transactions (and implicit

Re: [HACKERS] incoherent view of serializable transactions

2009-01-06 Thread Kevin Grittner
>>> Paul Schlie wrote: > Sorry if I'm restating the obvious, however I don't understand the > confusion, as it seems the standard's definition isn't mysterious; > it simply requires that the resulting state from the concurrent > execution of transactions (and implicitly any subset) designated to

Re: [HACKERS] incoherent view of serializable transactions

2009-01-05 Thread Paul Schlie
Sorry if I'm restating the obvious, however I don't understand the confusion, as it seems the standard's definition isn't mysterious; it simply requires that the resulting state from the concurrent execution of transactions (and implicitly any subset) designated to occur at the isolation level SERI

Re: [HACKERS] incoherent view of serializable transactions

2009-01-05 Thread Kevin Grittner
>>> Peter Eisentraut wrote: > A language lawyer might also point out that the note that contains > the "explicitness" isn't actually part of the formal standard. The only > thing that the standard formally defines are the excluded phenomena. Previously quoted, from the standard: "The execut

Re: [HACKERS] incoherent view of serializable transactions

2009-01-03 Thread Peter Eisentraut
On Sunday 04 January 2009 00:53:51 Gregory Stark wrote: > Peter Eisentraut writes: > > On Wednesday 31 December 2008 02:33:26 Kevin Grittner wrote: > >> I'm still working on section "Serializable Isolation versus True > >> Serializability", but here are all the changes I can see which precede > >>

Re: [HACKERS] incoherent view of serializable transactions

2009-01-03 Thread Gregory Stark
Peter Eisentraut writes: > On Wednesday 31 December 2008 02:33:26 Kevin Grittner wrote: >> I'm still working on section "Serializable Isolation versus True >> Serializability", but here are all the changes I can see which precede >> it. Has the review of the SQL specs convinced everyone that thi

Re: [HACKERS] incoherent view of serializable transactions

2009-01-03 Thread Peter Eisentraut
On Tuesday 30 December 2008 19:28:01 Kevin Grittner wrote: > Here we go. I've labeled the transactions consistently with new > thread I started trying to characterize the full scope of issues and > workarounds. OK, I believe it now. :-) The missing predicate locking is again at fault here. Beca

Re: [HACKERS] incoherent view of serializable transactions

2009-01-03 Thread Peter Eisentraut
On Wednesday 31 December 2008 02:33:26 Kevin Grittner wrote: > I'm still working on section "Serializable Isolation versus True > Serializability", but here are all the changes I can see which precede > it. Has the review of the SQL specs convinced everyone that this much > is appropriate? I don'

Re: [HACKERS] incoherent view of serializable transactions

2008-12-30 Thread Kevin Grittner
>>> Heikki Linnakangas wrote: > If you have a concrete suggestion (= patch) for the documentation, I'm > all ears. I'm still working on section "Serializable Isolation versus True Serializability", but here are all the changes I can see which precede it. Has the review of the SQL specs convin

Re: [HACKERS] incoherent view of serializable transactions

2008-12-30 Thread Kevin Grittner
>>> Peter Eisentraut wrote: > Greg Stark wrote: >> And I don't see why you discard "visibility" as unimportant. All >> the transaction isolations are defined in terms of the results if >> the transactions. Those results include both the database state and >> the data returned by the queries. Othe

Re: [HACKERS] incoherent view of serializable transactions

2008-12-30 Thread Peter Eisentraut
Greg Stark wrote: And I don't see why you discard "visibility" as unimportant. All the transaction isolations are defined in terms of the results if the transactions. Those results include both the database state and the data returned by the queries. Otherwise "phantom read" is a meaningless co

Re: [HACKERS] incoherent view of serializable transactions

2008-12-29 Thread Greg Stark
Sorry for top posting -- damn phone. The standard defines all the *other* transaction isolations in terms of phantom reads, dirty reads, and, uh one other phenomenon. But it defines serializability literally, as the transactions having the same effect as ift they were run serially. It expli

Re: [HACKERS] incoherent view of serializable transactions

2008-12-29 Thread Kevin Grittner
>>> Peter Eisentraut wrote: > Serializability is not defined in terms of what is visible, but what the > state of the database is. A belated thought on this: It would be easy enough to add a daily_receipt_total table to the example to capture the result of the query, instead of assuming that it

Re: [HACKERS] incoherent view of serializable transactions

2008-12-29 Thread Kevin Grittner
>>> Peter Eisentraut wrote: > On Tuesday 23 December 2008 16:51:03 Kevin Grittner wrote: >> If you look at the serializable queries in the original post for this >> thread, it's not hard to see that this standard is not met. The >> insert of receipt 3 appears to happen before the update of the

Re: [HACKERS] incoherent view of serializable transactions

2008-12-29 Thread Peter Eisentraut
On Tuesday 23 December 2008 16:51:03 Kevin Grittner wrote: > If you look at the serializable queries in the original post for this > thread, it's not hard to see that this standard is not met.  The > insert of receipt 3 appears to happen before the update of the control > record, since it has the o

Re: [HACKERS] incoherent view of serializable transactions

2008-12-26 Thread Kevin Grittner
>>> Emmanuel Cecchet wrote: > There was an interesting comment by Oracle folks: Oracle does > not provide serializability but only snapshot isolation, and still users > prefer to 'downgrade' to read committed for better performance. The > Oracle guys experience seemed to indicate that there wa

Re: [HACKERS] incoherent view of serializable transactions

2008-12-24 Thread Robert Haas
On Dec 24, 2008, at 6:46 PM, Ron Mayer wrote: Robert Haas wrote: ... serializable transaction ... If we were to construct a database that had one giant lock for the entire database so that only a single query could execute at one time, transactions would be serializable (because they'd i

Re: [HACKERS] incoherent view of serializable transactions

2008-12-24 Thread Ron Mayer
Robert Haas wrote: ... serializable transaction ... If we were to construct a database that had one giant lock for the entire database so that only a single query could execute at one time, transactions would be serializable (because they'd in fact be serialized). However, performance would suc

Re: [HACKERS] incoherent view of serializable transactions

2008-12-24 Thread Emmanuel Cecchet
Hi Kevin, The link didn't seem to work for me, but I think I found the article you meant: "Serializable Isolation for Snapshot Databases" by Michael J. Cahill, et al An interesting work. If nothing else, it will help flesh out the documentation of anomalies. If the PostgreSQL community ever

Re: [HACKERS] incoherent view of serializable transactions

2008-12-24 Thread Kevin Grittner
>>> Emmanuel Cecchet 12/22/08 11:42 PM >>> > If you want to know how to build SERIALIZABLE with a database that > provides SI (Snapshot Isolation), read > http://portal.acm.org/citation.cfm?doid=1376616.137669 The link didn't seem to work for me, but I think I found the article you meant: "Se

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Emmanuel Cecchet
Jeff Davis wrote: On Tue, 2008-12-23 at 00:42 -0500, Emmanuel Cecchet wrote: I still don't get why people would use SERIALIZABLE since there is no efficient implementation of it. If they need true serializable transactions, and don't care about efficiency. Is there such people who

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Kevin Grittner
>>> Simon Riggs wrote: > On Tue, 2008-12-23 at 10:10 -0600, Kevin Grittner wrote: > > I think the current docs make too much of a deal about how hard it is to > do predicate locking in databases. Most RDBMS use predicate locking via > indexes, ie the locking happens in the index. One might also

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Kevin Grittner
>>> "Robert Haas" wrote: > For most users, the artifacts that have been > introduced by these fine-grained locks are outweighed by the > performance benefits of greater concurrency - but, as you point out, > not necessarily always. That's what I don't understand. I never did point that out.

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Simon Riggs
On Tue, 2008-12-23 at 10:10 -0600, Kevin Grittner wrote: > Well, I figured I should try to get a consensus here before submitting > a patch. Last time I tried submitting a simple patch to remove the > line about the PostgreSQL community not knowing about any other > databases which use predicate

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Robert Haas
> The page locking provides this because every index page or data page > the serializable transaction looks at is locked against updates until > the end of the transaction. If it can see all the COLUMN=0 rows > through an index, the index locks protect the transaction. If a table > scan is requir

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Jeff Davis
On Tue, 2008-12-23 at 00:42 -0500, Emmanuel Cecchet wrote: > I still don't > get why people would use SERIALIZABLE since there is no efficient > implementation of it. If they need true serializable transactions, and don't care about efficiency. Regards, Jeff Davis -- Sent via pgsql-h

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Kevin Grittner
>>> Stephan Szabo wrote: > On Tue, 23 Dec 2008, Kevin Grittner wrote: > >> The page locking provides this because every index page or data page >> the serializable transaction looks at is locked against updates until >> the end of the transaction. If it can see all the COLUMN=0 rows >> through

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Kevin Grittner
>>> Gregory Stark wrote: > "Kevin Grittner" writes: > Gregory Stark wrote: >>> Afaict doing a few google searches Sybase doesn't do predicate locking > >>> either. >> >> The page locking provides this because every index page or data page >> the serializable transaction looks at is locke

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Stephan Szabo
On Tue, 23 Dec 2008, Kevin Grittner wrote: > The page locking provides this because every index page or data page > the serializable transaction looks at is locked against updates until > the end of the transaction. If it can see all the COLUMN=0 rows > through an index, the index locks protect

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Gregory Stark
Reading the spec it does seem our initial statement "The SQL standard defines four levels of transaction isolation in terms of three phenomena that must be prevented between concurrent transactions" is not accurate. The spec defines the first three modes in terms of P1,P2,P3 but serializable is d

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Kevin Grittner
>>> Gregory Stark wrote: > "Kevin Grittner" writes: > Emmanuel Cecchet 12/23/08 8:59 AM >>> >>> Have you ever used serializable transactions with Sybase? >> >> Every day for over 15 years. > > Afaict doing a few google searches Sybase doesn't do predicate locking > either. > It would v

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Gregory Stark
"Kevin Grittner" writes: Emmanuel Cecchet 12/23/08 8:59 AM >>> >> Have you ever used serializable transactions with Sybase? > > Every day for over 15 years. Afaict doing a few google searches Sybase doesn't do predicate locking either. It would very much surprise me if they did since th

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Kevin Grittner
>>> Heikki Linnakangas 12/23/08 9:47 AM >>> > Kevin Grittner wrote: > Emmanuel Cecchet 12/23/08 8:59 AM >>> >>> Why don't users program the application to >>> deal with a lower isolation (actually I think they do)? >> >> There really are good reasons. I'm not up to going through that now

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Heikki Linnakangas
Kevin Grittner wrote: Emmanuel Cecchet 12/23/08 8:59 AM >>> I am somewhat mystified by the interest some people still have in serializable transactions. Why don't users program the application to deal with a lower isolation (actually I think they do)? There really are good reasons. I'm no

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Kevin Grittner
>>> Emmanuel Cecchet 12/23/08 8:59 AM >>> > Kevin Grittner wrote: >> This isn't some hypothetical "maybe some day some product might >> implement this, but it'll never catch on" sort of thing -- Microsoft >> and Sybase SQL Server had this from version 1. I used it from 1990 >> until the conversi

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Emmanuel Cecchet
Kevin Grittner wrote: This isn't some hypothetical "maybe some day some product might implement this, but it'll never catch on" sort of thing -- Microsoft and Sybase SQL Server had this from version 1. I used it from 1990 until the conversion to PostgreSQL over the last couple years. Have yo

Re: [HACKERS] incoherent view of serializable transactions

2008-12-23 Thread Kevin Grittner
>>> Tom Lane wrote: > "Kevin Grittner" writes: >> At this point, SERIALIZABLE transactions appear to have worked, with >> receipt 3 happening before the update of deposit_date; however, there >> was a window of time when the update to deposit_date was visible and >> receipt 3 was not. > >> Thi

Re: [HACKERS] incoherent view of serializable transactions

2008-12-22 Thread Emmanuel Cecchet
Kevin, If you want to know how to build SERIALIZABLE with a database that provides SI (Snapshot Isolation), read http://portal.acm.org/citation.cfm?doid=1376616.137669 Note that in practice, READ COMMITTED is the most largely used isolation level and its limitations are relatively well underst

Re: [HACKERS] incoherent view of serializable transactions

2008-12-22 Thread Tom Lane
"Kevin Grittner" writes: > At this point, SERIALIZABLE transactions appear to have worked, with > receipt 3 happening before the update of deposit_date; however, there > was a window of time when the update to deposit_date was visible and > receipt 3 was not. > This absolutely can't happen in a

Re: [HACKERS] incoherent view of serializable transactions

2008-12-22 Thread Kevin Grittner
>>> Martijn van Oosterhout wrote: > On Mon, Dec 22, 2008 at 11:00:53AM -0600, Kevin Grittner wrote: >> As I've understood limitations of the PostgreSQL implementation of >> SERIALIZABLE transactions, at least the only example given in the >> documentation, revolve around a rather unlikely situatio

Re: [HACKERS] incoherent view of serializable transactions

2008-12-22 Thread Martijn van Oosterhout
On Mon, Dec 22, 2008 at 11:00:53AM -0600, Kevin Grittner wrote: > As I've understood limitations of the PostgreSQL implementation of > SERIALIZABLE transactions, at least the only example given in the > documentation, revolve around a rather unlikely situation: > > Given concurrent transactions T

[HACKERS] incoherent view of serializable transactions

2008-12-22 Thread Kevin Grittner
As I've understood limitations of the PostgreSQL implementation of SERIALIZABLE transactions, at least the only example given in the documentation, revolve around a rather unlikely situation: Given concurrent transactions T1 and T2 and non-overlapping sets of data A and B, T1 reads data including