Re: [Fwd: Re: [HACKERS] Transactions and temp tables]

2009-01-22 Thread Heikki Linnakangas
Bruce Momjian wrote: Heikki Linnakangas wrote: IMHO, this is just getting too kludgey. We came up with pretty good ideas on how to handle temp tables properly, by treating the same as non-temp tables. That should eliminate all the problems the latest patch did, and also the issues with

Re: [Fwd: Re: [HACKERS] Transactions and temp tables]

2009-01-21 Thread Bruce Momjian
Heikki Linnakangas wrote: Emmanuel Cecchet wrote: I just saw that this new patch was not considered because the previous version ended being rejected. Note that this version of the patch aims at supporting ONLY temp tables that are created AND dropped in the same transaction. We need to

Re: [Fwd: Re: [HACKERS] Transactions and temp tables]

2008-12-24 Thread Emmanuel Cecchet
Hi Heikki, The point of using temp tables was performance. Using regular tables in our case would hurt performance too much. Well if we cannot get a temporary fix in 8.4, we will maintain a separate patch to get that functionality just for temp tables that are created and dropped in the same

Re: [Fwd: Re: [HACKERS] Transactions and temp tables]

2008-12-23 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: I just saw that this new patch was not considered because the previous version ended being rejected. Note that this version of the patch aims at supporting ONLY temp tables that are created AND dropped in the same transaction. We need to be able to use temp tables in

Re: [HACKERS] Transactions and temp tables

2008-12-03 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: I think that the Assert in is_temp_rel(Oid) in tablecmds.c should be replaced by if (on_commits == NULL) return false; As the use case below shows, a regular table can be created and hold a LOCKTAG_RELATION lock that will trigger the call to is_temp_rel in

Re: [HACKERS] Transactions and temp tables

2008-12-03 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: There is a problem with temp tables with on delete rows that are created inside a transaction. Take the 2pc_on_delete_rows_transaction.sql test case and change the creation statement, instead of create temp table foo(x int) on commit delete rows; try create temp table

Re: [HACKERS] Transactions and temp tables

2008-12-03 Thread Emmanuel Cecchet
I would really like to have support for temp tables at least for the case where the table is created and dropped in the same transaction. But I guess that the other limitations on index, sequences and views would still hold, right? manu Heikki Linnakangas wrote: Emmanuel Cecchet wrote:

Re: [HACKERS] Transactions and temp tables

2008-12-01 Thread Emmanuel Cecchet
Heikki, I think that the Assert in is_temp_rel(Oid) in tablecmds.c should be replaced by if (on_commits == NULL) return false; As the use case below shows, a regular table can be created and hold a LOCKTAG_RELATION lock that will trigger the call to is_temp_rel in is_preparable_locktag. The

Re: [HACKERS] Transactions and temp tables

2008-11-24 Thread Emmanuel Cecchet
Heikki, The following test fails with your patch on my system. Could you check if you can reproduce? psql (8.4devel) Type help for help. test=# begin; BEGIN test=# create table paul(x int); CREATE TABLE test=# insert into paul values(1); INSERT 0 1 test=# prepare transaction

Re: [HACKERS] Transactions and temp tables

2008-11-21 Thread Emmanuel Cecchet
Heikki Linnakangas wrote: Emmanuel Cecchet wrote: I still quite did not get what the big deal was if an ON COMMIT DELETE ROWS temp table was created inside a transaction. In case the transaction that created a temp table rolls back, the table needs to be removed. Removing a temporary table

Re: [HACKERS] Transactions and temp tables

2008-11-20 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: I still quite did not get what the big deal was if an ON COMMIT DELETE ROWS temp table was created inside a transaction. In case the transaction that created a temp table rolls back, the table needs to be removed. Removing a temporary table belonging to another

Re: [HACKERS] Transactions and temp tables

2008-11-19 Thread Emmanuel Cecchet
Hi Heikki, I will try to make more tests. I still quite did not get what the big deal was if an ON COMMIT DELETE ROWS temp table was created inside a transaction. Why the new checks you are doing in lock.c would not work with dropped temp tables? Could it be possible to drop the lock as soon

Re: [HACKERS] Transactions and temp tables

2008-11-18 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: Emmanuel Cecchet wrote: As I have not found yet an elegant solution to deal with the DROP CASCADE issue, here is a simpler patch that handles temp tables that are dropped at commit time. This is a good first step and we will try to elaborate further to support ON

Re: [HACKERS] Transactions and temp tables

2008-11-18 Thread Heikki Linnakangas
Heikki Linnakangas wrote: Somehow this feels pretty baroque, though. Perhaps a better approach would be to add a new AtPrepare_OnCommitActions function to tablecmds.c, that gets called before AtPrepare_Locks. It would scan through the on_commits list, and release all locks for the PREPARE-safe

Re: [HACKERS] Transactions and temp tables

2008-11-17 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: As I have not found yet an elegant solution to deal with the DROP CASCADE issue, here is a simpler patch that handles temp tables that are dropped at commit time. This is a good first step and we will try to elaborate further to support ON COMMIT DELETE ROWS. The

Re: [HACKERS] Transactions and temp tables

2008-11-17 Thread Emmanuel Cecchet
Hi Heikki, Emmanuel Cecchet wrote: As I have not found yet an elegant solution to deal with the DROP CASCADE issue, here is a simpler patch that handles temp tables that are dropped at commit time. This is a good first step and we will try to elaborate further to support ON COMMIT DELETE

Re: [HACKERS] Transactions and temp tables

2008-11-07 Thread Emmanuel Cecchet
Hi, As I have not found yet an elegant solution to deal with the DROP CASCADE issue, here is a simpler patch that handles temp tables that are dropped at commit time. This is a good first step and we will try to elaborate further to support ON COMMIT DELETE ROWS. I have also added a

Re: [HACKERS] Transactions and temp tables

2008-11-04 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: What's the purpose of checking that a table is empty on prepare? I think I'd feel more comfortable with the approach of only accepting PREPARE TRANSACTIOn if the accessed temp tables have been created and destroyed in the same transaction, to avoid possibly surprising

Re: [HACKERS] Transactions and temp tables

2008-11-04 Thread Emmanuel Cecchet
Heikki Linnakangas wrote: Yes, I was trying to allow also ON COMMIT DROP and ON COMMIT DELETE ROW. An empty temp table at PREPARE time would be similar to an ON COMMIT DELETE ROW table. I think you'll want to check explicitly that the table is defined with ON COMMIT DELETE ROWS, instead of

Re: [HACKERS] Transactions and temp tables

2008-11-03 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: Here is the latest patch and the regression tests for the temp tables and 2PC issue. This fails: postgres=# begin; BEGIN postgres=# CREATE TEMPORARY TABLE temp1 (id int4); CREATE TABLE postgres=# PREPARE TRANSACTION 'foo'; PREPARE TRANSACTION postgres=# CREATE

Re: [HACKERS] Transactions and temp tables

2008-11-03 Thread Emmanuel Cecchet
Hi Heikki, Emmanuel Cecchet wrote: Here is the latest patch and the regression tests for the temp tables and 2PC issue. This fails: postgres=# begin; BEGIN postgres=# CREATE TEMPORARY TABLE temp1 (id int4); CREATE TABLE postgres=# PREPARE TRANSACTION 'foo'; PREPARE TRANSACTION postgres=#

Re: [HACKERS] Transactions and temp tables

2008-10-14 Thread Emmanuel Cecchet
Emmanuel Cecchet wrote: Here is the latest patch and the regression tests for the temp tables and 2PC issue. Is there a way to stop and restart postmaster between 2 tests? Thanks for your feedback, Emmanuel I did not get any comment on that one. How should I proceed so that the patch

Re: [HACKERS] Transactions and temp tables

2008-10-14 Thread Alvaro Herrera
Emmanuel Cecchet wrote: Emmanuel Cecchet wrote: Here is the latest patch and the regression tests for the temp tables and 2PC issue. Is there a way to stop and restart postmaster between 2 tests? Thanks for your feedback, Emmanuel I did not get any comment on that one. How should I

Re: [HACKERS] Transactions and temp tables

2008-10-10 Thread Emmanuel Cecchet
Hi all, Here is the latest patch and the regression tests for the temp tables and 2PC issue. Is there a way to stop and restart postmaster between 2 tests? Thanks for your feedback, Emmanuel -- Emmanuel Cecchet FTO @ Frog Thinker Open Source Development Consulting -- Web:

Re: [HACKERS] Transactions and temp tables

2008-10-09 Thread Emmanuel Cecchet
Hi, I am attaching a new patch that deals with the issue of the locks on temporary tables that have been accessed in transactions that have been prepared but not committed. I have added another list that keeps track of temp tables accessed by transactions that are prepared but not committed.

Re: [HACKERS] Transactions and temp tables

2008-10-08 Thread Emmanuel Cecchet
Heikki Linnakangas wrote: I was thinking of a transaction that's just prepared (1st phase), but not committed or rolled back: postgres=# CREATE TEMP TABLE foo (bar int); CREATE TABLE postgres=# BEGIN; BEGIN postgres=# DROP TABLE foo; DROP TABLE postgres=# PREPARE TRANSACTION '2pc'; PREPARE

Re: [HACKERS] Transactions and temp tables

2008-10-08 Thread Tom Lane
Emmanuel Cecchet [EMAIL PROTECTED] writes: Thanks for the example, I get it now. Does it make sense to allow any request execution between PREPARE TRANSACTION and the subsequent COMMIT or ROLLBACK? Yes. Don't even think of trying to disallow that. The COMMIT doesn't even have to happen in

Re: [HACKERS] Transactions and temp tables

2008-10-08 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: Also, even if the table is created and dropped in the same transaction, a subsequent transaction that tries to create and drop the table gets blocked on the lock. I suppose we could just say that that's the way it works, but I'm afraid it will come as a nasty surprise,

Re: [HACKERS] Transactions and temp tables

2008-10-08 Thread Emmanuel Cecchet
Tom Lane wrote: Emmanuel Cecchet [EMAIL PROTECTED] writes: Thanks for the example, I get it now. Does it make sense to allow any request execution between PREPARE TRANSACTION and the subsequent COMMIT or ROLLBACK? Yes. Don't even think of trying to disallow that. The COMMIT doesn't

Re: [HACKERS] Transactions and temp tables

2008-10-08 Thread Tom Lane
Emmanuel Cecchet [EMAIL PROTECTED] writes: Ok, so actually I don't see any different behavior between a temp table or a regular table. The locking happens the same way and as long as the commit prepared happens (potentially in another session), the lock is released at commit time which

Re: [HACKERS] Transactions and temp tables

2008-10-07 Thread Heikki Linnakangas
Emmanuel Cecchet wrote: Instead of relying on a boolean that tells if a temp table was accessed, I keep a list of the Oid for the temp tables accessed in the transaction and at prepare commit time, I check if the relations are still valid. I also added a check to allow empty temp tables at

Re: [HACKERS] Transactions and temp tables

2008-10-07 Thread Emmanuel Cecchet
Hi Heikki, The patch allows preparing any transaction that has dropped the temp table, even if it wasn't created in the same transaction. Is that sane? If you have a temp table created with an 'on commit delete rows' option in another transaction, it would be fine to drop it in another

Re: [HACKERS] Transactions and temp tables

2008-10-07 Thread Emmanuel Cecchet
Heikki, Here is a new version of the patch using a hash table as you suggested. I also include the tests that I have added to the regression test suite to test the various scenarios. All patches are based on Postgres 8.3.3, let me know if you want me to generate patch for 8.4. Thanks in

Re: [HACKERS] Transactions and temp tables

2008-10-07 Thread David Fetter
On Tue, Oct 07, 2008 at 04:57:37PM -0400, Emmanuel Cecchet wrote: Heikki, Here is a new version of the patch using a hash table as you suggested. I also include the tests that I have added to the regression test suite to test the various scenarios. All patches are based on Postgres 8.3.3,

Re: [HACKERS] Transactions and temp tables

2008-10-07 Thread Emmanuel Cecchet
Hi, On Tue, Oct 07, 2008 at 04:57:37PM -0400, Emmanuel Cecchet wrote: Heikki, Here is a new version of the patch using a hash table as you suggested. I also include the tests that I have added to the regression test suite to test the various scenarios. All patches are based on Postgres

Re: [HACKERS] Transactions and temp tables

2008-10-07 Thread David Fetter
On Tue, Oct 07, 2008 at 06:12:14PM -0400, Emmanuel Cecchet wrote: Hi, On Tue, Oct 07, 2008 at 04:57:37PM -0400, Emmanuel Cecchet wrote: Heikki, Here is a new version of the patch using a hash table as you suggested. I also include the tests that I have added to the regression test

Re: [HACKERS] Transactions and temp tables

2008-10-07 Thread Emmanuel Cecchet
Hi, Here are the patches for 8.4 (1 patch for the code and 1 patch for the regression tests). Thanks in advance for your feedback, Emmanuel David Fetter wrote: On Tue, Oct 07, 2008 at 06:12:14PM -0400, Emmanuel Cecchet wrote: Hi, On Tue, Oct 07, 2008 at 04:57:37PM -0400, Emmanuel