Re: [HACKERS] DO INSTEAD and conditional rules

2005-04-25 Thread Neil Conway
Tom Lane wrote: Here I've got to differ. The alphabetical-order rule was introduced to nail down the order of execution of operations that were going to happen in any case, but would otherwise have happened in an unspecified order. You are proposing to let it define what gets executed and what doe

Re: [HACKERS] DO INSTEAD and conditional rules

2005-04-25 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> DO INSTEAD means that the *original* query will not execute; it does >> not suppress actions produced by other rules. > I think DO INSTEAD should control whether the original query is added to > the set of query trees produced by the ru

Re: [HACKERS] simplify register_dirty_segment()

2005-04-25 Thread Qingqing Zhou
"Tom Lane" <[EMAIL PROTECTED]> writes > It would be a reasonable > bet for a kernel designer to assume that fsync() is generally going to > have to wait for some I/O and so a bit of CPU overhead isn't really > going to matter. Reasonable. > > Adjusting such a global refcount would require global

Re: [HACKERS] DO INSTEAD and conditional rules

2005-04-25 Thread Neil Conway
Tom Lane wrote: DO INSTEAD means that the *original* query will not execute; it does not suppress actions produced by other rules. If we did not define it that way, I think your example would have to error out --- how would you choose which INSTEAD rule wins? I think DO INSTEAD should control whet

Re: [HACKERS] DO INSTEAD and conditional rules

2005-04-25 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > Note that although both rules are DO INSTEAD, they both get fired for > the insertion. DO INSTEAD means that the *original* query will not execute; it does not suppress actions produced by other rules. If we did not define it that way, I think your examp

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Mark Kirkwood
Bruce Momjian wrote: Mark Kirkwood wrote: Personally I favor 1), so would prefer: enable it by default only for interactive sessions, like AUTOCOMMIT We are going for 'off' by default and only interactive. Sweet. The fact that interactive and non-interactive behavior is different is something th

Re: [HACKERS] simplify register_dirty_segment()

2005-04-25 Thread Tom Lane
"Qingqing Zhou" <[EMAIL PROTECTED]> writes: > I can't figure out why it tooks so long time in windows and SunOS for clean > files - I told you why: they don't maintain bookkeeping information that allows them to quickly identify dirty buffers belonging to a particular file. Linux does ... but I'm

[HACKERS] DO INSTEAD and conditional rules

2005-04-25 Thread Neil Conway
I find the following behavior confusing: neilc=# create table t1 (a int, b int); CREATE TABLE neilc=# create table t2 (a int, b int); CREATE TABLE neilc=# create table t3 (a int, b int); CREATE TABLE neilc=# create rule t1_rule1 as on insert to t1 where NEW.a > 100 do instead insert into t2 values

Re: [HACKERS] simplify register_dirty_segment()

2005-04-25 Thread Qingqing Zhou
"Tom Lane" <[EMAIL PROTECTED]> writes > On platforms that I'm familiar with, an fsync call causes the kernel > to spend a significant amount of time groveling through its buffers > to see if any are dirty. We shouldn't incur that cost to buy marginal > speedups at the application level. (In other

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Bruce Momjian
Mark Kirkwood wrote: > Bruce Momjian wrote: > > > > disable it by default for all sessions (current patch) > > enable it by default only for interactive sessions, like AUTOCOMMIT > > These two seem like the best contenders. > > The choice comes down to whether we should: > > 1) Be like

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Mark Kirkwood
Bruce Momjian wrote: disable it by default for all sessions (current patch) enable it by default only for interactive sessions, like AUTOCOMMIT These two seem like the best contenders. The choice comes down to whether we should: 1) Be like most other dbms's (e.g. Oracle, Firebird) + fol

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Christopher Kings-Lynne
enable it by default only for interactive sessions, like AUTOCOMMIT And that is what other dbms' do. The current version controls only interactive sessions, and is off by default. I am willing to change that, but I need to hear from more people on this to change it. It's good with me. Chris

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Bruce Momjian
Christopher Kings-Lynne wrote: > > disable it by default for all sessions (current patch) > > That is the most backwards non-surprising solution. > > > enable it by default only for interactive sessions, like AUTOCOMMIT > > And that is what other dbms' do. The current version controls o

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Christopher Kings-Lynne
disable it by default for all sessions (current patch) That is the most backwards non-surprising solution. enable it by default only for interactive sessions, like AUTOCOMMIT And that is what other dbms' do. Chris ---(end of broadcast)--- TIP 3: if

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Dave Held
> -Original Message- > From: Andrew Dunstan [mailto:[EMAIL PROTECTED] > Sent: Monday, April 25, 2005 3:43 PM > To: josh@agliodbs.com > Cc: pgsql-perform; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks > suggested? > > Josh Berkus wrote: >

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Tom Lane
Bruce Momjian writes: > I think everyone agrees this should only work in interactive mode. I > think the only unknown is if it should be 'on' by default in interactive > mode? Does it make sense to follow the standard in interactive mode if > we don't follow it in non-interative mode? I doubt i

Re: [HACKERS] [proposal] protocol extension to support loadable stream filters

2005-04-25 Thread Tom Lane
Brent Verner <[EMAIL PROTECTED]> writes: > I'd like to introduce the concept of (dynamically loaded) stream > filters that would be used to wrap calls to send/recv by the FE/BE > protocol. I think the "dynamically loaded" part of that is going to be way more headache than it's worth. You certa

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Bruce Momjian
I think everyone agrees this should only work in interactive mode. I think the only unknown is if it should be 'on' by default in interactive mode? Does it make sense to follow the standard in interactive mode if we don't follow it in non-interative mode? ---

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2005-04-25 at 11:23 -0400, Tom Lane wrote: >> It's not just the scan --- you also have to sort, or something like >> that, if you want to count distinct values. I doubt anyone is really >> going to consider this a feasible answer for large tables.

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Dave Held
> -Original Message- > From: Josh Berkus [mailto:[EMAIL PROTECTED] > Sent: Sunday, April 24, 2005 2:08 PM > To: Andrew Dunstan > Cc: Tom Lane; Greg Stark; Marko Ristola; pgsql-perform; > pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks > sugge

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Andrew Dunstan
Josh Berkus wrote: Simon, Tom: While it's not possible to get accurate estimates from a fixed size sample, I think it would be possible from a small but scalable sample: say, 0.1% of all data pages on large tables, up to the limit of maintenance_work_mem. Setting up these samples as a % of da

Re: [HACKERS] Constant WAL replay

2005-04-25 Thread Joshua D. Drake
What really interests me here: Where is you code located in order to make sure that this things can work reliably? We are integrated into the PostgreSQL backend. So they have to get a new PostgreSQL release from you for every minor upgrade, I assume at no cost? Yep :) Sincerely, Joshua D. Drake

Re: [HACKERS] Constant WAL replay

2005-04-25 Thread =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?=
Joshua D. Drake wrote: Joshua, This sounds interesting. If you don't use the WAL but a transaction log shipping - what does it mean in terms of PostgreSQL? Do you create your own transaction log? Yes. What really interests me here: Where is you code located in order to make sure that this thing

Re: [HACKERS] Constant WAL replay

2005-04-25 Thread Bruce Momjian
Joshua D. Drake wrote: > > > > > > Joshua, > > > > This sounds interesting. If you don't use the WAL but a transaction log > > shipping - what does it mean in terms of PostgreSQL? Do you create your > > own transaction log? > > Yes. > > > What really interests me here: Where is you code loca

Re: [HACKERS] Constant WAL replay

2005-04-25 Thread Joshua D. Drake
Joshua, This sounds interesting. If you don't use the WAL but a transaction log shipping - what does it mean in terms of PostgreSQL? Do you create your own transaction log? Yes. What really interests me here: Where is you code located in order to make sure that this things can work reliably? We

Re: [HACKERS] Constant WAL replay

2005-04-25 Thread =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?=
Neil Conway wrote: Hans-Jürgen Schönig wrote: The idea: We are looking for a way to implement a synchronous single-master / multiple slaves systems. Meanwhile we are able to serialize / deserialize WAL records and send them to a group communication system which transports those records to the sl

Re: [HACKERS] Constant WAL replay

2005-04-25 Thread =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?=
Joshua D. Drake wrote: Alvaro Herrera wrote: On Sun, Apr 24, 2005 at 08:10:34AM +0200, Hans-Jürgen Schönig wrote: The idea: We are looking for a way to implement a synchronous single-master / multiple slaves systems. Meanwhile we are able to serialize / deserialize WAL records and send them to a

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Josh Berkus
Simon, Tom: While it's not possible to get accurate estimates from a fixed size sample, I think it would be possible from a small but scalable sample: say, 0.1% of all data pages on large tables, up to the limit of maintenance_work_mem. Setting up these samples as a % of data pages, rather th

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Josh Berkus
Guys, > While it's not possible to get accurate estimates from a fixed size sample, > I think it would be possible from a small but scalable sample: say, 0.1% of > all data pages on large tables, up to the limit of maintenance_work_mem. BTW, when I say "accurate estimates" here, I'm talking about

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Simon Riggs
On Mon, 2005-04-25 at 11:23 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > My suggested hack for PostgreSQL is to have an option to *not* sample, > > just to scan the whole table and find n_distinct accurately. > > ... > > What price a single scan of a table, however large, wh

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Michael Paesold
Bruce Momjian wrote: Greg Sabino Mullane wrote: > The SQL-Standard itself says that errors inside transactions should only > rollback the last statement, if possible. So why is that not > implemented in > PostgreSQL? What I read from past discussions here, is because it's > just > unsave and wi

[HACKERS] [proposal] protocol extension to support loadable stream filters

2005-04-25 Thread Brent Verner
Hackers, I'd like to introduce the concept of (dynamically loaded) stream filters that would be used to wrap calls to send/recv by the FE/BE protocol. The initial "StreamFilter" will be a zlib compression filter. Yeah, I know it could just be added along-side (in the same way as) the SSL cod

Re: [HACKERS] [PATCHES] Continue transactions after errors in psql

2005-04-25 Thread Bruce Momjian
Greg Sabino Mullane wrote: > > The SQL-Standard itself says that errors inside transactions should only > > rollback the last statement, if possible. So why is that not implemented in > > PostgreSQL? What I read from past discussions here, is because it's just > > unsave and will lead to data-garb

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Dave Held
> -Original Message- > From: Tom Lane [mailto:[EMAIL PROTECTED] > Sent: Monday, April 25, 2005 10:23 AM > To: Simon Riggs > Cc: josh@agliodbs.com; Greg Stark; Marko Ristola; pgsql-perform; > pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks > s

Re: [HACKERS] possible TODO: read-only tables, select from indexes

2005-04-25 Thread Hannu Krosing
I send it now the 3rd time because I'm not sure my mail works, as it has not appeared on pgsql-hackers On L, 2005-04-23 at 18:27 -0400, Tom Lane wrote: > Ron Mayer <[EMAIL PROTECTED]> writes: > > Is this a fair summary of the potential benefits of READ-ONLY > > tables? (from both this thread and

[HACKERS] How to make lazy VACUUM of one table run in several transactions ?

2005-04-25 Thread Hannu Krosing
I send it now the 3rd time because I'm not sure my mail works, as it has not appeared on pgsql-hackers Hi Tom, I've got the impression that you have worked most actively on VACUUM and so I ask you this directly instead of adressing pgsql-hackers list in general. Feel free to correct me :) I h

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > My suggested hack for PostgreSQL is to have an option to *not* sample, > just to scan the whole table and find n_distinct accurately. > ... > What price a single scan of a table, however large, when incorrect > statistics could force scans and sorts to occu

Re: [HACKERS] How to make lazy VACUUM of one table run in several transactions ?

2005-04-25 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > On Sun, Apr 24, 2005 at 12:02:37PM +0300, Hannu Krosing wrote: >> Must some locks also be released an reaquired inside this loop, or is >> there something else I should keep in mind when trying to do this ? > There is "session lock" on the table. You m

Re: [HACKERS] How to make lazy VACUUM of one table run in several transactions ?

2005-04-25 Thread Alvaro Herrera
On Sun, Apr 24, 2005 at 12:02:37PM +0300, Hannu Krosing wrote: > to check for some time/page_cnt limit after each heap page (near the > start of main loop in lazy_scan_heap() ), and if it is reached then > stop, clean up indexes, and return the blkno of next page needing to > be vacuumed, and repl

Re: [HACKERS] possible TODO: read-only tables, select from indexes only.

2005-04-25 Thread Jochem van Dieten
On 4/24/05, Tom Lane wrote: > > What you are talking about is not a "read only" table, it is a > "non-MVCC" table. This is a much greater assault on the fundamental > semantics of Postgres than it's being painted to be in this thread. > In particular, how is such a table going to come into being?

Re: [HACKERS] Bitmap scans vs. the statistics views

2005-04-25 Thread Hannu Krosing
On R, 2005-04-22 at 13:46 -0700, Josh Berkus wrote: > Tom, > > Hmmm ... we need to flag *something* in pg_stat_*_indexes, whether it is a > new > column or the tuplefetch column. People use that view to find indexes they > can drop. I think that "idx_scan" and "idx_tup_read" can have the sam

Re: [HACKERS] possible TODO: read-only tables, select from indexes

2005-04-25 Thread Hannu Krosing
On L, 2005-04-23 at 00:02 -0400, Bruce Momjian wrote: > Russell Smith wrote: ... > > I know especially for some tables, I would choose this index with > > visibility as it would increase performance by not looking at the table > > at all for that information (count being a good example). However f

[HACKERS] How to make lazy VACUUM of one table run in several transactions ?

2005-04-25 Thread Hannu Krosing
Hi Tom, I've got the impression that you have worked most actively on VACUUM and so I ask you this directly instead of adressing pgsql-hackers list in general. Feel free to correct me :) I have a problem, that I think can be solved by splitting the vacuum up to run in several transactions, eac

[HACKERS] How to make lazy VACUUM of one table run in several transactions ?

2005-04-25 Thread Hannu Krosing
Hi Tom, I've got the impression that you have worked most actively on VACUUM and so I ask you this directly instead of adressing pgsql-hackers list in general. Feel free to correct me :) I have a problem, that I think can be solved by splitting the vacuum up to run in several transactions, each

Re: [HACKERS] possible TODO: read-only tables, select from indexes

2005-04-25 Thread Hannu Krosing
On R, 2005-04-22 at 13:14 -0400, Bruce Momjian wrote: > > This can be prohibitively pricey for big tables with lots of indexes, as > > marking the tuple means alsn finding and possibly writing to all index > > enytries pointing to this tuple. > > Yep, it could be very ugly, but it would help with

Re: [HACKERS] possible TODO: read-only tables, select from indexes

2005-04-25 Thread Hannu Krosing
On L, 2005-04-23 at 18:27 -0400, Tom Lane wrote: > Ron Mayer <[EMAIL PROTECTED]> writes: > > Is this a fair summary of the potential benefits of READ-ONLY > > tables? (from both this thread and the archives): > > > 1. Index-only scans are made possible fairly easily because > > you wouldn

Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?

2005-04-25 Thread Simon Riggs
On Sat, 2005-04-23 at 16:39 -0700, Josh Berkus wrote: Greg Stark wrote > > I looked into this a while back when we were talking about changing the > > sampling method. The conclusions were discouraging. Fundamentally, using > > constant sized samples of data for n_distinct is bogus. Constant sized

Re: [HACKERS] simplify register_dirty_segment()

2005-04-25 Thread Tom Lane
"Qingqing Zhou" <[EMAIL PROTECTED]> writes: > That is, we don't care if a segment is dirty or not, if someone opened it, > then we will fsync it at checkpoint time. On platforms that I'm familiar with, an fsync call causes the kernel to spend a significant amount of time groveling through its buff

[HACKERS] simplify register_dirty_segment()

2005-04-25 Thread Qingqing Zhou
The basic idea is to change register_dirty_segment() to register_opened_segment(). That is, we don't care if a segment is dirty or not, if someone opened it, then we will fsync it at checkpoint time. Currently, register_dirty_segment() is called in mdextend(), mdwrite() and mdtruncate(), this is c

Re: [HACKERS] Passwords in PSQL

2005-04-25 Thread Gavin Sherry
On Mon, 25 Apr 2005, Rafaqat Ali wrote: > Hello All > I am using > \c [database] [user-name] >to connect to any database. >Any one can connect to any database. If any one knows user > name, he/she connect to db. I want to provide some security that n

Re: [HACKERS] Passwords in PSQL

2005-04-25 Thread Christopher Kings-Lynne
Any one can connect to any database. If any one knows user name, he/she connect to db. I want to provide some security that no one can connect without providing passwords. postgres uses a function do_coonect() for this perpose. I provide it passwords but it let me connect to db with any pas