Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: array_lower() and array_upper() are returning NULL for a non-null input, the empty array, even though lower and upper bounds are known just as well as they are for any other sized array. They are behaving as if there's something unknown about the empty

Re: [HACKERS] pre-loading a user table.

2004-04-02 Thread Zeugswetter Andreas SB SD
Why do you think it useful to preload something during InitPostgres, anyway? Any heavily used table will certainly be present in shared buffers already, and even more surely present in kernel buffers. And if you really want it preloaded you can issue dummy selects with a client right after

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Hm. My idea of the index lower bound is the smallest index for which there is an array member --- so I agree with Joe that it's not very well defined for an empty array. Hm. The problem is that they have other implications. Like the upper bound is one

[HACKERS] GiST future

2004-04-02 Thread ohp
Hi every one. I've played a bit with contrib/intarray and very much liked being able to index a whole array. Now I wonder why 7.4 array queries using SOME/ANY/ALL don't make use of this king of indexes. Is there any plan to integrate Oleg's job into the core as GiST indexes were? it would be

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: Well that breaks other things. Then lots of functions have to become non-strict to work properly because they should have valid output when passed null values. Ick. ... I'm leaning towards suggesting that postgres should follow sql-99 here and normalize

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Joe Conway
Greg Stark wrote: array_lower() and array_upper() are returning NULL for a non-null input, the empty array, even though lower and upper bounds are known just as well as they are for any other sized array. They are behaving as if there's something unknown about the empty array that makes it hard to

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Joe Conway
Tom Lane wrote: I think that maybe we ought to question these two properties: * empty array is different from NULL ... really? Why? I think this makes sense, similar to the difference between '' and NULL. * storing a value into an element of a NULL array yields a NULL

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Jim Seymour
I had written: [snip] The problem is that attempts to vacuum these tables resulted in NNN dead row versions cannot be removed yet. Went through a lot of analysis (e.g.: Any hanging txns?) and trying different things with folks on the #PostgreSQL IRC channel, all to no avail. [snip] Okay,

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Joe Conway
Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: I'm leaning towards suggesting that postgres should follow sql-99 here and normalize all array indexes to have a lower bound of 1. That would break even more things, no? On the other hand, it'd get rid of the problem that we presently face with

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Joe Conway
Greg Stark wrote: I'm leaning towards suggesting that postgres should follow sql-99 here and normalize all array indexes to have a lower bound of 1. Then array_lower and array_upper become entirely unnecessary. Instead we just have array_length which is exactly equivalent to my idea of

Re: [HACKERS] PITR for replication?

2004-04-02 Thread Josh Berkus
Andrew, I may be completely missing the point here, but it looks to me as though the PITR archival mechanism is also most of a native replication facility. Is there anyone reason this couldn't be extended to replication, and if so, is anyone planning on using it as such? I believe that

[HACKERS] Prepared select

2004-04-02 Thread Robert Turnbull
How can I use a prepared select statement asmentioned in the documentation on SQL PREPARE.Preparing the statement is easy, the problem is using the plan to get a cursor.My assumption is the SQL OPEN command is not documented or there is some otherlibpq API to make this happen. Thanks

Re: [HACKERS] Better support for whole-row operations and composite

2004-04-02 Thread Joe Conway
Tom Lane wrote: We have a number of issues revolving around the fact that composite types (row types) aren't first-class objects. I think it's past time to fix that. Here are some notes about doing it. I am not sure all these ideas are fully-baked ... comments appreciated. [Sorry for the delay

Re: [HACKERS] PITR for replication?

2004-04-02 Thread Alvaro Herrera
On Fri, Apr 02, 2004 at 08:47:09AM -0800, Josh Berkus wrote: I believe that Command Prompt's Mammoth Replicator works on something like log shipping.There are both advantages and disadvantages to the approach, meaning that we will still need other replication strategies, like Slony and

Re: [HACKERS] PITR for replication?

2004-04-02 Thread Simon Riggs
Greg Stark J. Andrew Rogers [EMAIL PROTECTED] writes: I may be completely missing the point here, but it looks to me as though the PITR archival mechanism is also most of a native replication facility. Is there anyone reason this couldn't be extended to replication, and if so, is

Re: [HACKERS] GiST future

2004-04-02 Thread Joe Conway
[EMAIL PROTECTED] wrote: Now I wonder why 7.4 array queries using SOME/ANY/ALL don't make use of this king of indexes. Is there any plan to integrate Oleg's job into the core as GiST indexes were? I hope to get to it someday, but have higher priorities if I can find some time between now and the

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Alvaro Herrera
On Fri, Apr 02, 2004 at 12:02:22PM -0500, Jim Seymour wrote: [...] Which is all well-and-good (tho, my ignorance readily conceded, four minutes seems a mite... long), *except*: If I shut-down the WebObjects application which, again, never accesses the db in question, much-less any of its

Re: [HACKERS] GiST future

2004-04-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: [EMAIL PROTECTED] wrote: Now I wonder why 7.4 array queries using SOME/ANY/ALL don't make use of this king of indexes. Is there any plan to integrate Oleg's job into the core as GiST indexes were? I hope to get to it someday, but have higher priorities

Re: [HACKERS] GiST future

2004-04-02 Thread Oleg Bartunov
On Fri, 2 Apr 2004, Joe Conway wrote: [EMAIL PROTECTED] wrote: Now I wonder why 7.4 array queries using SOME/ANY/ALL don't make use of this king of indexes. Is there any plan to integrate Oleg's job into the core as GiST indexes were? I hope to get to it someday, but have higher

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Tom Lane
Manfred Koizar [EMAIL PROTECTED] writes: What I have in mind is a kind of Double Vitter algorithm. Whatever we do to get our sample of rows, in the end the sampled rows come from no more than sample_size different blocks. So my idea is to first create a random sample of sample_size block

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Jim Seymour
On Fri, Apr 02, 2004 at 12:02:22PM -0500, Jim Seymour wrote: [...] Which is all well-and-good (tho, my ignorance readily conceded, four minutes seems a mite... long), *except*: If I shut-down the WebObjects application which, again, never accesses the db in question, much-less any

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: That would break even more things, no? On the other hand, it'd get rid of the problem that we presently face with dump/restore of arrays that don't have lower bound 1. Because pg_dump doesn't do anything to mark such values, they'll end up with lower

Re: [HACKERS] Better support for whole-row operations and composite types

2004-04-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Tom Lane wrote: We will be able to make generic I/O routines for composite types, comparable to those used now for arrays. Not sure what a convenient external format would look like. (Possibly use the same conventions as for a 1-D array?) So you mean

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Alvaro Herrera
On Fri, Apr 02, 2004 at 02:51:30PM -0500, Jim Seymour wrote: On Fri, Apr 02, 2004 at 12:02:22PM -0500, Jim Seymour wrote: Ok, so the WebObjects app keeps an idle open transaction? (i.e. it issues a BEGIN as soon as the previous transaction is finished.) I'm not sure I read the code

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Manfred Koizar
On Fri, 02 Apr 2004 14:48:13 -0500, Tom Lane [EMAIL PROTECTED] wrote: Manfred Koizar [EMAIL PROTECTED] writes: What I have in mind is a kind of Double Vitter algorithm. [...] random sample of sample_size block numbers, and then to sample the rows out of this pool of blocks. That assumption is

[HACKERS] Function to kill backend

2004-04-02 Thread Magnus Hagander
Hi! When debugging on win32, I've created myself a little function that I feel should be added to the backend proper. While it adds a lot of vlaue on win32, I think it adds quite a bit of value on non-win32 platforms as well... The function is pg_kill_backend(pid,signal). superuser-only, of

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Andrew Dunstan
Magnus Hagander wrote: Hi! When debugging on win32, I've created myself a little function that I feel should be added to the backend proper. While it adds a lot of vlaue on win32, I think it adds quite a bit of value on non-win32 platforms as well... The function is pg_kill_backend(pid,signal).

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Magnus Hagander wrote: The function is pg_kill_backend(pid,signal). superuser-only, of course. Which simply sends a signal to the specified backend - querycancel, terminate, etc. If' we're going to have this shouldn't it be a proper command? And maybe

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Magnus Hagander
Hi! When debugging on win32, I've created myself a little function that I feel should be added to the backend proper. While it adds a lot of vlaue on win32, I think it adds quite a bit of value on non-win32 platforms as well... The function is pg_kill_backend(pid,signal). superuser-only, of

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Dann Corbit
-Original Message- From: Magnus Hagander [mailto:[EMAIL PROTECTED] Sent: Friday, April 02, 2004 2:34 PM To: Andrew Dunstan; [EMAIL PROTECTED] Subject: Re: [HACKERS] Function to kill backend Hi! When debugging on win32, I've created myself a little function that I feel

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Tom Lane
Manfred Koizar [EMAIL PROTECTED] writes: On Fri, 02 Apr 2004 14:48:13 -0500, Tom Lane [EMAIL PROTECTED] wrote: A bigger problem is that this makes the sampling quite nonuniform, because rows that are on relatively low-density pages would be more likely to become part of the final sample than

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Greg Stark
Joe Conway [EMAIL PROTECTED] writes: Greg Stark wrote: I'm leaning towards suggesting that postgres should follow sql-99 here and normalize all array indexes to have a lower bound of 1. Then array_lower and array_upper become entirely unnecessary. Instead we just have array_length which

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Greg Stark
Joe Conway [EMAIL PROTECTED] writes: Greg Stark wrote: array_lower() and array_upper() are returning NULL for a non-null input, the empty array, even though lower and upper bounds are known just as well as they are for any other sized array. They are behaving as if there's something

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Jim Seymour
Alvaro Herrera [EMAIL PROTECTED] wrote: [snip] Turn on query logging and see if the BEGIN is issued right after the COMMIT/ROLLBACK, or whether it waits and issues it right before SELECT/CREATE TEMP TABLE. It doesn't matter if it's only doing queries; if it does them inside a

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Magnus Hagander wrote: The function is pg_kill_backend(pid,signal). superuser-only, of course. Which simply sends a signal to the specified backend - querycancel, terminate, etc. If' we're going to have this shouldn't it

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Tom Lane
[EMAIL PROTECTED] (Jim Seymour) writes: Again the difference: With WebObjects running, deleting rows and trying to vacuum immediately, even full, fails. Shut-down WebObjects and I can. WebObjects is evidently holding an open transaction. Ergo, anything deleted after the start of that

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Manfred Koizar
On Fri, 02 Apr 2004 18:06:12 -0500, Tom Lane [EMAIL PROTECTED] wrote: You should not need to use the Vitter algorithm for the block-level selection, since you can know the number of blocks in the table in advance. You can just use the traditional method of choosing each block or not with

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Jim Seymour
Tom Lane [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] (Jim Seymour) writes: Again the difference: With WebObjects running, deleting rows and trying to vacuum immediately, even full, fails. Shut-down WebObjects and I can. WebObjects is evidently holding an open transaction. Ergo,

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Tom Lane
Manfred Koizar [EMAIL PROTECTED] writes: You'd run the Vitter algorithm separately to decide whether to keep or discard each live row you find in the blocks you read. You mean once a block is sampled we inspect it in any case? This was not the way I had planned to do it, but I'll keep this

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: it would definitely need to be a lot more constrained than send-any-signal-to-any-postgres-process ... even for a superuser, that's a mighty fat-gauge foot-gun. What sort of constraints do you have in mind? I'd limit it to SIGINT

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Manfred Koizar
On Fri, 02 Apr 2004 19:57:47 -0500, Tom Lane [EMAIL PROTECTED] wrote: If you like I can send you the Vitter paper off-list (I have a PDF of it). The comments in the code are not really intended to teach someone what it's good for ... Yes, please. [Would have sent this off-list. But I'm

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Tom Lane
[EMAIL PROTECTED] (Jim Seymour) writes: Tom Lane [EMAIL PROTECTED] wrote: WebObjects is evidently holding an open transaction. It certainly isn't holding open a transaction in the database I'm working with. Which database the transaction is in isn't real relevant... the logic is done

Re: [HACKERS] Better support for whole-row operations and composite

2004-04-02 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: So you mean like an array, but with possibly mixed datatypes? '{1 , abc def, 2.3}' Seems to make sense. The unresolved question in my mind is how to represent NULL elements. However, we have to solve that sooner or later for arrays too. Any

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Alvaro Herrera
On Fri, Apr 02, 2004 at 07:35:20PM -0500, Tom Lane wrote: [EMAIL PROTECTED] (Jim Seymour) writes: Again the difference: With WebObjects running, deleting rows and trying to vacuum immediately, even full, fails. Shut-down WebObjects and I can. WebObjects is evidently holding an open

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Bruce Momjian
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: it would definitely need to be a lot more constrained than send-any-signal-to-any-postgres-process ... even for a superuser, that's a mighty fat-gauge foot-gun. What sort of constraints do you have in mind?

Re: [HACKERS] Inconsistent behavior on Array Is Null?

2004-04-02 Thread Joe Conway
Greg Stark wrote: This really ought to work, it obviously shouldn't allow you to set a[5] and then surreptitiously move it to a[1]. But nor should it generate an error, since I may well have a specific meaning for a[5] and may be planning to fill in a[1]..a[4] later. The logical thing to do, I

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Stephan Szabo
On Fri, 2 Apr 2004, Alvaro Herrera wrote: On Fri, Apr 02, 2004 at 07:35:20PM -0500, Tom Lane wrote: [EMAIL PROTECTED] (Jim Seymour) writes: Again the difference: With WebObjects running, deleting rows and trying to vacuum immediately, even full, fails. Shut-down WebObjects and I

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Jim Seymour
Tom Lane [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] (Jim Seymour) writes: Tom Lane [EMAIL PROTECTED] wrote: WebObjects is evidently holding an open transaction. It certainly isn't holding open a transaction in the database I'm working with. Which database the transaction is in

Re: [HACKERS] Better support for whole-row operations and composite types

2004-04-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Tom Lane wrote: ... I believe that CVS tip is source-code-compatible with existing SRFs, even though I adjusted all the ones in the distribution to stop using the TupleTableSlot stuff. Almost compatible. I found that, to my surprise, PL/R compiles with

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: But, if I read the code correctly, the oldest xmin vacuum cares about for a non-shared relation should be local to the database, shouldn't it? It's the oldest xmin of any transaction that's local to your database, but those xmin values themselves were

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Seems like useful functionality. Right now, how does an administrator kill another backend from psql? They can't. The question to ask is should they be able to? I think any such facility is inherently a security risk, since it means that a remote

Re: [HACKERS] Problems Vacuum'ing

2004-04-02 Thread Tom Lane
[EMAIL PROTECTED] (Jim Seymour) writes: But I see entries like that if I just *start* *up* psql, without doing anything: Sure. You are doing something when you execute select from pg_locks ... that command executes inside a transaction, just like any other Postgres operation. The problem

Re: [HACKERS] Function to kill backend

2004-04-02 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Seems like useful functionality. Right now, how does an administrator kill another backend from psql? They can't. The question to ask is should they be able to? I think any such facility is inherently a security risk, since it

Re: [HACKERS] Better support for whole-row operations and composite

2004-04-02 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: Almost compatible. I found that, to my surprise, PL/R compiles with no changes after your commit. However it no segfaults (as I expected) on composite type arguments. Should be easy to fix though (I think, really haven't looked at it hard

Re: [HACKERS] Better support for whole-row operations and composite types

2004-04-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: I still haven't had time to look closely, and well may have been doing something non-standard all along, but in any case this is the current failing code: /* for tuple args, convert to a one row data.frame */ TupleTableSlot *slot =

Re: [HACKERS] Better support for whole-row operations and composite

2004-04-02 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: /* for tuple args, convert to a one row data.frame */ TupleTableSlot *slot = (TupleTableSlot *) arg[i]; HeapTuple tuples = slot-val; TupleDesc tupdesc = slot-ttc_tupleDescriptor; Um. Well, the arg is not a TupleTableSlot * anymore, so this

Re: [HACKERS] Better support for whole-row operations and composite types

2004-04-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: ... The SRF API was for user defined functions, not procedural languages anyway. I'll look at how the other procedural languages handle tuple arguments. It was a dozen-or-so-lines change in each of the PL languages AFAIR. You will probably also want to look

Re: [HACKERS] Better support for whole-row operations and composite

2004-04-02 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: ... The SRF API was for user defined functions, not procedural languages anyway. I'll look at how the other procedural languages handle tuple arguments. It was a dozen-or-so-lines change in each of the PL languages AFAIR. You will probably also

Re: [HACKERS] Better support for whole-row operations and composite types

2004-04-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Just for reference, what is arg[i] if it isn't a (TupleTableSlot *) anymore -- is it just a HeapTuple? No, it's a HeapTupleHeader pointer. You need to reconstruct a HeapTuple on top of that to work with heap_getattr and most other core backend routines.

Re: [HACKERS] [GENERAL] Large DB

2004-04-02 Thread Manfred Koizar
[time to move this to -hackers] On Fri, 02 Apr 2004 11:16:21 -0500, Tom Lane [EMAIL PROTECTED] wrote: Manfred Koizar [EMAIL PROTECTED] writes: The first step, however, (acquire_sample_rows() in analyze.c) has to read more rows than finally end up in the sample. It visits less than O(nblocks)