Re: [HACKERS] Visibility Groups

2008-08-11 Thread Decibel!
On Aug 7, 2008, at 9:49 AM, Robert Haas wrote: This proposal sounds like it would target batch jobs, because those are the kinds of jobs that where you can predict in advance what tables will be needed. I don't know whether my personal set of problems with MVCC syncs up with anyone else's, but

Re: [HACKERS] Visibility Groups

2008-08-08 Thread daveg
On Thu, Aug 07, 2008 at 01:30:27PM +0100, Gregory Stark wrote: Simon Riggs [EMAIL PROTECTED] writes: Currently, we calculate a single OldestXmin across all snapshots on the assumption that any transaction might access any table. I propose creating Visibility Groups that *explicitly*

[HACKERS] Visibility Groups

2008-08-07 Thread Simon Riggs
Currently, we calculate a single OldestXmin across all snapshots on the assumption that any transaction might access any table. I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). By default, visibility_groups

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Jochem van Dieten
On Thu, Aug 7, 2008 at 12:38 PM, Simon Riggs wrote: I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). Doesn't every transaction need to access data from the catalogs? Wouldn't the inclusion of a catalogs

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Simon Riggs
On Thu, 2008-08-07 at 12:55 +0200, Jochem van Dieten wrote: On Thu, Aug 7, 2008 at 12:38 PM, Simon Riggs wrote: I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). Doesn't every transaction need to

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Gregory Stark
Simon Riggs [EMAIL PROTECTED] writes: Currently, we calculate a single OldestXmin across all snapshots on the assumption that any transaction might access any table. I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Richard Huxton
Simon Riggs wrote: On Thu, 2008-08-07 at 12:55 +0200, Jochem van Dieten wrote: On Thu, Aug 7, 2008 at 12:38 PM, Simon Riggs wrote: I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). Doesn't every

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Simon Riggs
On Thu, 2008-08-07 at 13:30 +0100, Gregory Stark wrote: Hm, so backing up a bit from the specific proposed interface, the key here is being able to explicitly mark which tables your transaction will need in the future? Think of it as a promise to touch nothing except a specific set of

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Simon Riggs
On Thu, 2008-08-07 at 14:18 +0100, Richard Huxton wrote: Simon Riggs wrote: On Thu, 2008-08-07 at 12:55 +0200, Jochem van Dieten wrote: On Thu, Aug 7, 2008 at 12:38 PM, Simon Riggs wrote: I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Kevin Grittner
Simon Riggs wrote: Currently, we calculate a single OldestXmin across all snapshots on the assumption that any transaction might access any table. I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). By

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Richard Huxton
Simon Riggs wrote: On Thu, 2008-08-07 at 14:18 +0100, Richard Huxton wrote: An attempt to write to user_emails by T0 will fail with an error. All above correct The point of doing this is that *if* T0 becomes the oldest transaction it will *not* interfere with removal of rows on user_emails.

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Alvaro Herrera
Simon Riggs wrote: Currently, we calculate a single OldestXmin across all snapshots on the assumption that any transaction might access any table. I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). By

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Simon Riggs
On Thu, 2008-08-07 at 10:20 -0400, Alvaro Herrera wrote: Simon Riggs wrote: Currently, we calculate a single OldestXmin across all snapshots on the assumption that any transaction might access any table. I propose creating Visibility Groups that *explicitly* limit the ability of a

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). By default, visibility_groups would be NULL, implying potential access to all tables. I think this would be a lot of

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Robert Haas
I think this would be a lot of mechanism and complication that will go completely unused in the field. It'll be impossible even to explain let alone to use effectively, for anyone who's not intensely steeped in the details of MVCC. +1. This proposal sounds like it would target batch jobs,

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Simon Riggs [EMAIL PROTECTED] writes: I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). By default, visibility_groups would be NULL, implying potential access to all

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Gregory Stark
Gregory Stark [EMAIL PROTECTED] writes: I think Simon's interface was overly complex but if we can simplify it then it could be useful. As Grittner mentioned implicit queries could make use of it automatically. Also pg_dump or Slony could make use of it automatically. Sorry implicit

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Alvaro Herrera
Gregory Stark wrote: I think Simon's interface was overly complex but if we can simplify it then it could be useful. As Grittner mentioned implicit queries could make use of it automatically. Also pg_dump or Slony could make use of it automatically. Hmm, what use would it have for pg_dump?

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Simon Riggs
On Thu, 2008-08-07 at 10:28 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I propose creating Visibility Groups that *explicitly* limit the ability of a transaction to access data outside its visibility group(s). By default, visibility_groups would be NULL, implying potential

Re: [HACKERS] Visibility Groups

2008-08-07 Thread Gregory Stark
Alvaro Herrera [EMAIL PROTECTED] writes: Gregory Stark wrote: I think Simon's interface was overly complex but if we can simplify it then it could be useful. As Grittner mentioned implicit queries could make use of it automatically. Also pg_dump or Slony could make use of it automatically.