Re: [HACKERS] trivial DoS on char recoding

2006-06-20 Thread Martijn van Oosterhout
On Tue, Jun 20, 2006 at 06:10:38PM -0400, Tom Lane wrote: > > Should we get firmer in rejecting invalid configurations? > > The question is how sure are we whether a configuration is "invalid". > AFAIK there's not a really portable way to determine which encoding > matches a locale. initdb has a

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Qingqing Zhou
"Alvaro Herrera" <[EMAIL PROTECTED]> wrote > > But the problem (or at last a part of the problem) is not what context > each chunk is allocated in, but where did a given chunk come from (where > was it allocated), Which is why saving __FILE__/__LINE__ is useful. > Agreed. Maybe we should not clut

[HACKERS] My new email address

2006-06-20 Thread Bruce Momjian
After twelve years of using the domain candle.pha.pa.us, I have moved to a new domain that is easier to remember, momjian.us. New email address: [EMAIL PROTECTED] New web site: http://momjian.us The domain candle.pha.pa.us will continue to function indefinitely, but if you

Re: [HACKERS] trivial DoS on char recoding

2006-06-20 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Note that the PO file for the spanish translation is written in Latin1, > not UTF8. So I can adventure that the server is trying to recode a > string which is originally in Latin1, but assuming it is UTF-8, to > Win1250. Yeah, this is a known problem -

Re: [HACKERS] union all bug?

2006-06-20 Thread Joe Conway
Tom Lane wrote: Joe Conway <[EMAIL PROTECTED]> writes: I was trying to work around limitations with "partitioning" of tables using constraint exclusion, when I ran across this little oddity: I think you're under a misimpression about the syntax behavior of ORDER BY and UNION. Per spec, ORDER

Re: [HACKERS] trivial DoS on char recoding

2006-06-20 Thread Alvaro Herrera
Alvaro Herrera wrote: > To reproduce, you using a non-C locale is (es_ES works for me). *blush* Sorry, I rewrote this phrase and obviously didn't reread it very carefully :-) It means that you must use a non-C locale. -- Alvaro Herrerahttp://www.CommandPrompt.

Re: [HACKERS] UPDATE crash in HEAD and 8.1

2006-06-20 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> I'm fairly sure this query is illegal per spec. There are ancient > >> discussions in the archives about whether aggregates in an UPDATE target > >> list can have a consistent interpretation or not. We never fou

Re: [HACKERS] UPDATE crash in HEAD and 8.1

2006-06-20 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I'm fairly sure this query is illegal per spec. There are ancient >> discussions in the archives about whether aggregates in an UPDATE target >> list can have a consistent interpretation or not. We never found one, >> but never got a

[HACKERS] trivial DoS on char recoding

2006-06-20 Thread Alvaro Herrera
Oswaldo Hernandez just reported this in the pgsql-es-ayuda list. Basically, a conversion between UTF8 and windows_1250 can crash the server. I recall a bug around this general code but I don't recall it being able to provoke a PANIC. To reproduce, create a cluster with UTF-8 encoding and locale e

Re: [HACKERS] Slightly bogus regression test for contrib/dblink

2006-06-20 Thread Joe Conway
Joe Conway wrote: ...the "BEGIN" statement returned successfully as usual, but for some reason left (PQtransactionStatus(conn) != PQTRANS_IDLE), causing dblink_open() to start a transaction and later complete it on line 454. Oops, I meant "... some reason left (PQtransactionStatus(conn) ==

Re: [HACKERS] Slightly bogus regression test for contrib/dblink

2006-06-20 Thread Joe Conway
Tom Lane wrote: What's even more interesting is that there are now three later runs of HEAD on osprey, and none of them failed. So unless Remi's been fooling with the environment on that machine, this was a one-shot irreproducible failure. That's disturbing in a different way ... http://www.pg

Re: [HACKERS] Slightly bogus regression test for contrib/dblink

2006-06-20 Thread Tom Lane
Tom Lane <[EMAIL PROTECTED]> writes: > Joe Conway <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >>> It looks to me like the diffs are consistent with the idea that the >>> test is using a copy of dblink that predates this patch ... >> I would think that the diffs would be significantly larger if

Re: [HACKERS] Slightly bogus regression test for contrib/dblink

2006-06-20 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >>> It looks to me like the diffs are consistent with the idea that the >>> test is using a copy of dblink that predates this patch ... > I would think that the diffs would be significantly larger if that were > the case. In fact, when was

Re: [HACKERS] Slightly bogus regression test for contrib/dblink

2006-06-20 Thread Joe Conway
Tom Lane wrote: Joe Conway <[EMAIL PROTECTED]> writes: I haven't really looked at the buildfarm before -- I might be blind, but I couldn't figure out how to see the regression.diff file. It's on the cited page, if you scroll down far enough. OK, I'm officially blind (so much for that lasik

Re: [HACKERS] Some small code-restructuring issues

2006-06-20 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > A workaround of sorts would be to mention the origin of the files being > moved, so that an interested person can look it up via the Attic. Yeah, that should be sufficient. The history is actually still there, just attached to the old file location.

Re: [HACKERS] Some small code-restructuring issues

2006-06-20 Thread Alvaro Herrera
Stephen Frost wrote: > * Tom Lane ([EMAIL PROTECTED]) wrote: > > libpgport (ie, move 'em to src/port). Moving them would lose some CVS > > history but would probably be the cleanest thing in the long run. > > Comments? > > Time to consider something other than CVS...? In the end, personally > I'

Re: [HACKERS] UPDATE crash in HEAD and 8.1

2006-06-20 Thread Joshua D. Drake
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: update pk set id = max(id) + 2; I'm fairly sure this query is illegal per spec. There are ancient discussions in the archives about whether aggregates in an UPDATE target list can have a consistent interpretation or not. We never fou

Re: [HACKERS] Initdb segfaults during "initializing pg_authid"

2006-06-20 Thread Wade Klaver
Looks like the distclean may have done it. I thought I had already, but who knows. Thanks. -Wade On Tuesday 20 June 2006 09:51, Tom Lane wrote: > Wade Klaver <[EMAIL PROTECTED]> writes: > > Initdb seems to barf on me during the pg_authid bit. Below are the > > specifics. Please ask if you need

Re: [HACKERS] UPDATE crash in HEAD and 8.1

2006-06-20 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > update pk set id = max(id) + 2; > > I'm fairly sure this query is illegal per spec. There are ancient > discussions in the archives about whether aggregates in an UPDATE target > list can have a consistent interpretation or not. We

Re: [HACKERS] UPDATE crash in HEAD and 8.1

2006-06-20 Thread Andrew Dunstan
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: update pk set id = max(id) + 2; I'm fairly sure this query is illegal per spec. There are ancient discussions in the archives about whether aggregates in an UPDATE target list can have a consistent interpretation or not. We n

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-20 Thread Robert Lor
Greg Stark wrote: It seems pointless to me to expose things like lwlock_acuire that map 1-1 to C function calls like LWLockAcquire. They're useless except to people who understand what's going on and if people know the low level implementation details of Postgres they can already trace those cal

Re: [HACKERS] Some small code-restructuring issues

2006-06-20 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > libpgport (ie, move 'em to src/port). Moving them would lose some CVS > history but would probably be the cleanest thing in the long run. > Comments? Time to consider something other than CVS...? In the end, personally I'd rather have it be cleaner than th

Re: [HACKERS] UPDATE crash in HEAD and 8.1

2006-06-20 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > update pk set id = max(id) + 2; I'm fairly sure this query is illegal per spec. There are ancient discussions in the archives about whether aggregates in an UPDATE target list can have a consistent interpretation or not. We never found one, but never

Re: [HACKERS] Slightly bogus regression test for contrib/dblink

2006-06-20 Thread Andrew Dunstan
Tom Lane wrote: Note to Andrew: would it make sense for the larger log files to be split out as linked pages in a buildfarm report? regards, tom lane I will put it on the TODO list. cheers andrew ---(end of broadcast)--

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-20 Thread Robert Lor
Simon Riggs wrote: This needs to work on Linux and Windows, minimum, also. The proposed solution will work on Linux & Windows if they similar facility that the macros can map to. Otherwise, the macros stay as no-ops and will not affect those platforms at all. It's obviously impossible to

Re: [HACKERS] Slightly bogus regression test for contrib/dblink

2006-06-20 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> BTW, I was led to notice this while examining the current buildfarm >> failure report from osprey, >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=osprey&dt=2006-06-17%2004:00:16 > I haven't really looked at the buildfarm before -- I

[HACKERS] Some small code-restructuring issues

2006-06-20 Thread Tom Lane
In pursuit of eliminating some redundant gettimeofday() calls, I just tried to change struct Port's session_start field to TimestampTz, which necessitated including utils/timestamp.h in libpq/libpq-be.h. That caused things to blow up real good :-(. The problem is that backend/libpq/md5.c includes

Re: [HACKERS] pltcl -- solved

2006-06-20 Thread ohp
Hi Tom, On Tue, 20 Jun 2006, Tom Lane wrote: > Date: Tue, 20 Jun 2006 12:42:24 -0400 > From: Tom Lane <[EMAIL PROTECTED]> > To: ohp@pyrenet.fr > Cc: pgsql-hackers list > Subject: Re: [HACKERS] pltcl -- solved > > ohp@pyrenet.fr writes: > > Just a note that pltcl is now passing regression tests on

Re: [HACKERS] Slightly bogus regression test for contrib/dblink

2006-06-20 Thread Joe Conway
Tom Lane wrote: Lines 509-512 of contrib/dblink/expected/dblink.out read: -- this should fail because there is no open transaction SELECT dblink_exec('myconn','DECLARE xact_test CURSOR FOR SELECT * FROM foo'); ERROR: sql error DETAIL: ERROR: cursor "xact_test" already exists The error messag

[HACKERS] UPDATE crash in HEAD and 8.1

2006-06-20 Thread Alvaro Herrera
A bug reported by Josh Drake, crashes 8.1 and CVS HEAD: Test case is: create table pk (id bigserial primary key); insert into pk values (DEFAULT); insert into pk values (DEFAULT); insert into pk values (DEFAULT); update pk set id = max(id) + 2; -- SEGV simple eh? :-) The backtrace on HEAD looks

Re: [HACKERS] Initdb segfaults during "initializing pg_authid"

2006-06-20 Thread Tom Lane
Wade Klaver <[EMAIL PROTECTED]> writes: > Initdb seems to barf on me during the pg_authid bit. Below are the > specifics. > Please ask if you need anything else. The build is CVS -HEAD. Are you sure it's a clean build? "make distclean" and trying again is often the first thing to try when se

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Simon Riggs wrote: >> Could we set that as an option for each memory context when we create >> it? All or nothing seems too extreme for me for most cases. > What most cases? There is only one case -- there is a big leak and you > want to find out where

Re: [HACKERS] pltcl -- solved

2006-06-20 Thread Tom Lane
ohp@pyrenet.fr writes: > Just a note that pltcl is now passing regression tests on Unixware. > For some unexplained reason, it did'nt pass with tcl-8.5 but is ok with > tcl -8.4.13. AFAICT there is no "tcl 8.5" yet; there is an alpha release tcl8.5a4 which is stated to still be under active featur

[HACKERS] Initdb segfaults during "initializing pg_authid"

2006-06-20 Thread Wade Klaver
Hello folks, Initdb seems to barf on me during the pg_authid bit. Below are the specifics. Please ask if you need anything else. The build is CVS -HEAD. Initdb output: [EMAIL PROTECTED]:bin/initdb The files belonging to this database system will be owned by user "pgsql". This user must also o

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-20 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Greg Stark <[EMAIL PROTECTED]> writes: > > What would be useful is instrumenting high level calls that can't be traced > > without application guidance. For example, inserting a dtrace probe for each > > SQL and each plan node. That way someone could get the

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-20 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > What would be useful is instrumenting high level calls that can't be traced > without application guidance. For example, inserting a dtrace probe for each > SQL and each plan node. That way someone could get the same info as EXPLAIN > ANALYZE from a producti

Re: [HACKERS] sync_file_range()

2006-06-20 Thread Zeugswetter Andreas DCP SD
"Tom Lane" <[EMAIL PROTECTED]> writes: > > Indeed, I've been wondering lately if we shouldn't resurrect > > LET_OS_MANAGE_FILESIZE and make that the default on systems with > > largefile support. If nothing else it would cut down on open/close > > overhead on very large relations. > > > I'd s

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-20 Thread Greg Stark
Robert Lor <[EMAIL PROTECTED]> writes: > Yes, I'm proposing user-space probes (aka User Statically-Defined Tracing - > USDT). USDT provides a high-level abstraction so the application can expose > well defined probes without the user having to know the detailed > implementation. For example, ins

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Another thing to consider is that the proximate location of the palloc > is frequently *not* very useful. For instance, if your memory is > getting eaten by lists, all the palloc traces will point at > new_tail_cell(). Not much help. I don't know what to d

Re: [HACKERS] sync_file_range()

2006-06-20 Thread Tom Lane
"Zeugswetter Andreas DCP SD" <[EMAIL PROTECTED]> writes: > Indeed, I've been wondering lately if we shouldn't resurrect > LET_OS_MANAGE_FILESIZE and make that the default on systems with > largefile support. If nothing else it would cut down on open/close > overhead on very large relations. >

Re: [HACKERS] sync_file_range()

2006-06-20 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > So we would use the async properties of sync, but not the file range > support? That's the part of it that looked potentially useful to me, anyway. I don't see any value for us in syncing just part of a file, because we don't have enough disk layout knowle

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Martijn van Oosterhout
On Tue, Jun 20, 2006 at 12:18:32AM -0400, Tom Lane wrote: > Another thing to consider is that the proximate location of the palloc > is frequently *not* very useful. For instance, if your memory is > getting eaten by lists, all the palloc traces will point at > new_tail_cell(). Not much help. I

Re: [HACKERS] PAM auth

2006-06-20 Thread Alvaro Herrera
Satoshi Nagayasu wrote: > Andrew Dunstan wrote: > >don't use system auth. PAM can authenticate from many sources, not just > >the system password files. LDAP is a commonly used source. > > The reason why I'm trying to use PAM, is I need a feature > to account lock-out after N-times login failures

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Alvaro Herrera
Andrew Dunstan wrote: > Alvaro Herrera said: > > > > >> That seems mostly the hard way to me, because our memory management > >> scheme is *not* based around "thou shalt free() what thou malloc()ed". > >> You'd need a tool that understood about resetting memory contexts > >> (recursively) to get a

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-20 Thread Martijn van Oosterhout
On Mon, Jun 19, 2006 at 05:14:15PM -0400, Chris Browne wrote: > [EMAIL PROTECTED] (Robert Lor) writes: > > For DTrace, probes can be enabled using a D script. When the probes > > are not enabled, there is absolutely no performance hit whatsoever. > > That seems inconceivable. > > In order to have

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Andrew Dunstan
Alvaro Herrera said: > >> That seems mostly the hard way to me, because our memory management >> scheme is *not* based around "thou shalt free() what thou malloc()ed". >> You'd need a tool that understood about resetting memory contexts >> (recursively) to get anywhere at all in analyzing such a t

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Alvaro Herrera
Simon Riggs wrote: > On Tue, 2006-06-20 at 00:18 -0400, Tom Lane wrote: > > > One idea that comes to mind is to have a compile time option to record > > the palloc __FILE__ and _LINE__ in every AllocChunk header. Then it > > would not be so hard to identify the culprit while trawling through > >

Re: [HACKERS] CVS HEAD busted on Windows?

2006-06-20 Thread Dave Page
> -Original Message- > From: Andrew Dunstan [mailto:[EMAIL PROTECTED] > Sent: 20 June 2006 12:12 > To: Dave Page > Cc: Tom Lane; Peter Eisentraut; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] CVS HEAD busted on Windows? > > > 1. it's nothing to do with upgrading - I saw this

Re: [HACKERS] PAM auth

2006-06-20 Thread Satoshi Nagayasu
Andrew Dunstan wrote: don't use system auth. PAM can authenticate from many sources, not just the system password files. LDAP is a commonly used source. The reason why I'm trying to use PAM, is I need a feature to account lock-out after N-times login failures on PG, like pam_tally module. I'm

Re: [HACKERS] CVS HEAD busted on Windows?

2006-06-20 Thread Andrew Dunstan
Dave Page wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom Lane Sent: 20 June 2006 00:02 To: Peter Eisentraut; pgsql-hackers@postgresql.org Subject: [HACKERS] CVS HEAD busted on Windows? I notice buildfarm member snake is unhappy: The

Re: [HACKERS] PAM auth

2006-06-20 Thread Andrew Dunstan
Satoshi Nagayasu wrote: >Albe, > >Albe Laurenz wrote: > > >>/etc/pam.d/system-auth probably uses pam_unix.so to authenticate. >> >>Does the user exist on the machine and have the password you try? >> >> > >Yes, I have same user name on my linux box and postgresql, >and they have same passw

[HACKERS] pltcl -- solved

2006-06-20 Thread ohp
Hi all Just a note that pltcl is now passing regression tests on Unixware. For some unexplained reason, it did'nt pass with tcl-8.5 but is ok with tcl -8.4.13. build farm build script updated accordingly. My next try will be python. Regards -- Olivier PRENANT Tel: +33-5-61-50-

Re: [HACKERS] sync_file_range()

2006-06-20 Thread Zeugswetter Andreas DCP SD
> > Indeed, I've been wondering lately if we shouldn't resurrect > > LET_OS_MANAGE_FILESIZE and make that the default on systems with > > largefile support. If nothing else it would cut down on open/close > > overhead on very large relations. I'd still put some limit on the filesize, else you

Re: [HACKERS] Generic Monitoring Framework Proposal

2006-06-20 Thread Simon Riggs
On Mon, 2006-06-19 at 19:36 -0400, Theo Schlossnagle wrote: > The idea of having intelligently placed dtrace probes in Postrgres > would allow us ... > to troubleshoot[ing] obtuse production > problems. That, to me, is exciting stuff. [paraphrased by SR] I very much agree with the requireme

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Simon Riggs
On Tue, 2006-06-20 at 00:18 -0400, Tom Lane wrote: > One idea that comes to mind is to have a compile time option to record > the palloc __FILE__ and _LINE__ in every AllocChunk header. Then it > would not be so hard to identify the culprit while trawling through > memory. The overhead costs wou

Re: [HACKERS] sync_file_range()

2006-06-20 Thread Simon Riggs
On Mon, 2006-06-19 at 21:35 -0400, Tom Lane wrote: > Greg Stark <[EMAIL PROTECTED]> writes: > > Come to think of it I wonder whether there's anything to be gained by using > > smaller files for tables. Instead of 1G files maybe 256M files or something > > like that to reduce the hit of fsyncing a f

Re: [HACKERS] PAM auth

2006-06-20 Thread Satoshi Nagayasu
Albe, Albe Laurenz wrote: > /etc/pam.d/system-auth probably uses pam_unix.so to authenticate. > > Does the user exist on the machine and have the password you try? Yes, I have same user name on my linux box and postgresql, and they have same password (now). > You could add 'debug' to the pam_un

Re: [HACKERS] CVS HEAD busted on Windows?

2006-06-20 Thread Dave Page
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Tom Lane > Sent: 20 June 2006 00:02 > To: Peter Eisentraut; pgsql-hackers@postgresql.org > Subject: [HACKERS] CVS HEAD busted on Windows? > > I notice buildfarm member snake is unhappy: > > The pro

Re: [HACKERS] shall we have a TRACE_MEMORY mode

2006-06-20 Thread Gurjeet Singh
On 6/20/06, Tom Lane <[EMAIL PROTECTED]> wrote: One idea that comes to mind is to have a compile time option to record the palloc __FILE__ and _LINE__ in every AllocChunk header. Then it would not be so hard to identify the culprit while trawling through memory. The overhead costs would be so h

Re: [HACKERS] PAM auth

2006-06-20 Thread Albe Laurenz
satoshi nagayasu wrote: > I'm trying to use PAM auth on PostgreSQL, but I still cannot > get success on PAM auth (with PG813 and RHEL3). > > pg_hba.conf has > > hostpamtest all 0.0.0.0/0 pam > > /etc/pam.d/postgresql is > > #%PAM-1.0 > > auth required pam_stac