Re: [HACKERS] pg_dump feature

2008-09-21 Thread David Fetter
On Mon, Sep 22, 2008 at 03:25:35AM +1000, Naz wrote: > Hi all, > I brought this up a few years ago in the 7.4 days, and since there > is still no satisfactory solution to this I thought I'd raise it > again. When dumping a schema, it is often necessary to dump the > tables separately to the const

[HACKERS] pg_dump feature

2008-09-21 Thread Naz
Hi all, I brought this up a few years ago in the 7.4 days, and since there is still no satisfactory solution to this I thought I'd raise it again. When dumping a schema, it is often necessary to dump the tables separately to the constraints and other non-structural metadata. The most obvious us

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: What do you think about getting rid of the password_from_string state variable? It was always a bit of a kluge, and we don't seem to need it anymore with this approach. It is still used in PQconnectionUsedPassword(). That is still needed to prevent a non-superuser from loggin

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Maybe better: > static PQconninfoOption * > conninfo_parse(const char *conninfo, PQExpBuffer errorMessage, > bool fill_defaults, bool *password_from_string) I'm thinking a separate conninfo_fill_defaults function is better, though it's not

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Joe Conway wrote: Tom Lane wrote: Refactoring doesn't seem like an easy way to fix this, because of the problem that the behavior of pulling up defaults is part of the API specification for PQconndefaults(). Thoughts? Hmm, I could have sworn I looked for that, and saw it elsewhere. Anyway, y

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Refactoring doesn't seem like an easy way to fix this, because of the >> problem that the behavior of pulling up defaults is part of the API >> specification for PQconndefaults(). > conninfo_parse() is presently only called from a few pla

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Joe Conway <[EMAIL PROTECTED]> writes: New patch attached. erm ... wait a minute. This approach doesn't actually solve the problem at all, because conninfo_parse is responsible for filling in various sorts of default values. In particular it would happily pull a password from

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > New patch attached. erm ... wait a minute. This approach doesn't actually solve the problem at all, because conninfo_parse is responsible for filling in various sorts of default values. In particular it would happily pull a password from the services file

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > New patch attached. This is close, but you're failing to guard against a few out-of-memory corner cases (and now that I look, PQconndefaults() is too). The libpq documentation needs more work than this, too. I'll make a cleanup pass and commit. BTW, I'm

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > If we were to accept the pg_attrdef approach, why aren't we > > doing a pg_attracl table instead of adding a column to pg_attribute? > > That's actually not an unreasonable question. If you were to do that > the

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Yeah. We could make one further refinement: callers that don't care about acquiring an error string can pass NULL for the errmsg parameter. That tells PQconninfoParse to throw away the errmsg string anyway. With that, the minimal case isn't much uglier than your original: just ne

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
Stephen Frost <[EMAIL PROTECTED]> writes: > Honestly, I really disliked the code which assumed pg_attribute had no > NULLable/toastable columns and used what seemed like pretty gruesome > hacks to create pg_attribute structures. Agreed, but that seems orthogonal to the point here, which is that a

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > I can think of a way around that: represent a default expression using > classid = OID of pg_attribute, objid = OID of table, objsubid = column > attnum. This is distinct from the column itself, which is represented > with classid = OID of pg_class. It seem

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Uh, you're confusing the backend environment with libpq's much more >> spartan lifestyle. errmsg will be malloc'd and it will *not* go away >> unless the caller free()s it. > Yup, just figured that out. Otherwise OK with it? Yeah. We c

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Joe Conway <[EMAIL PROTECTED]> writes: If the return value is NULL, use errmsg if you'd like. I'd guess in most instances you don't even need to bother freeing errmsg as it is in a limited life memory context. Uh, you're confusing the backend environment with libpq's much more

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > If the return value is NULL, use errmsg if you'd like. I'd guess in most > instances you don't even need to bother freeing errmsg as it is in a > limited life memory context. Uh, you're confusing the backend environment with libpq's much more spartan life

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: Hmm ... one problem with this is that the caller can't tell failure-because-out-of-memory from failure-because-string-is-bogus. Is it worth having the PQconninfoParse function pass back the error message to avoid this corner case? I thought briefly about it, and wasn't sure

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> So that seems to tilt the decision towards exposing the conninfo_parse >> function. Joe, do you want to have a go at it, or shall I? > Here's a first shot. Hmm ... one problem with this is that the caller can't tell failure-because-out-

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: "Marko Kreen" <[EMAIL PROTECTED]> writes: On 9/21/08, Joe Conway <[EMAIL PROTECTED]> wrote: Why? pg_service does not appear to support wildcards, so what is the attack vector? "service=foo host=custom" The proposal to require a password = foo entry in the conn string seems

Re: [HACKERS] Predictable order of SQL commands in pg_dump

2008-09-21 Thread Dmitry Koterov
Great! Would it be implemented in a next version? Seems it would be very helpful, especially for people who commit database structure to CVS/SVN once per minute to track changes history (or similar)... On Sun, Sep 21, 2008 at 11:57 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Dmitry Koterov" <[EMAI

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Tom Lane
"=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?=" <[EMAIL PROTECTED]> writes: >> ... And implementing it would require introducing weird >> corner cases into the tuple toaster, because it might now come across >> TOAST pointers that point to a no-longer-existent table, and have to >> consider that to be

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > If we push the responsibility back to dblink, we might as well export > conninfo_parse() or some wrapper thereof and let dblink simply check for > a non-null password from the very beginning. That's not totally unreasonable, since we already export the PQ

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Simon Riggs
On Sun, 2008-09-21 at 15:07 -0400, Andrew Dunstan wrote: > > Simon Riggs wrote: > > No, its not possible. Need a trigger. > > > > I think we should support it though. If we extend the relational model > > with arrays then it would be sensible if we support this aspect as > > well. > > > > Implem

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Alex Hunsaker
On Sun, Sep 21, 2008 at 11:09 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > A possible objection to this plan is that if the column-level privileges > patch doesn't get in, then we're left with a useless column in > pg_attribute. But an always-null column doesn't cost much of anything, > and we know t

Re: [HACKERS] Predictable order of SQL commands in pg_dump

2008-09-21 Thread Tom Lane
"Dmitry Koterov" <[EMAIL PROTECTED]> writes: > CREATE TRIGGER t000_set_id > -BEFORE INSERT OR DELETE OR UPDATE ON a > +BEFORE INSERT OR DELETE OR UPDATE ON b > FOR EACH ROW > EXECUTE PROCEDURE i_trg(); > CREATE TRIGGER t000_set_id > -BEFORE INSERT OR DELETE OR UPDATE ON b >

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Dmitry Koterov
> I strongly suspect you'd benefit a lot more by learning database best > practices rather than assuming, as you appear to be doing, that you > are dealing with a new field and that you know it best. Neither is true. Of course, you absolutely right. I venerate you! O! :-) -- Sent via pgsql-h

Re: [HACKERS] parallel pg_restore

2008-09-21 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > I am working on getting parallel pg_restore working. I'm currently > getting all the scaffolding working, and hope to have a naive prototype > posted within about a week. > The major question is how to choose the restoration order so as to > maximize

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Tom Lane
"Marko Kreen" <[EMAIL PROTECTED]> writes: > On 9/21/08, Joe Conway <[EMAIL PROTECTED]> wrote: >> Why? pg_service does not appear to support wildcards, so what is the attack >> vector? > "service=foo host=custom" The proposal to require a password = foo entry in the conn string seems to resolve al

Re: [HACKERS] parallel pg_restore

2008-09-21 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: I am working on getting parallel pg_restore working. I'm currently getting all the scaffolding working, and hope to have a naive prototype posted within about a week. The major question is how to choose the restoration or

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
"Alex Hunsaker" <[EMAIL PROTECTED]> writes: > Hrm, I thought if anything we wanted to put them in pg_constraints (at > least inherited ones). Now maybe I have defaults confused with NOT > NULLs... But don't we want to be able to give defaults names and and > such? No, I think you're thinking of N

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: BTW, a possible hole in this scheme would be if a user could supply a conninfo string that was intentionally malformed in a way that would cause a tacked-on pgpassfile option to be ignored by libpq. We might need to add some validity checks to dblink, or tighten libpq's own check

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Tom Lane wrote: "Marko Kreen" <[EMAIL PROTECTED]> writes: On 9/21/08, Joe Conway <[EMAIL PROTECTED]> wrote: Why? pg_service does not appear to support wildcards, so what is the attack vector? "service=foo host=custom" The proposal to require a password = foo entry in the conn string seems

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Hans-Jürgen Schönig
*snip* Judging from that, the toasted table cleanup may be part of ALTER TABLE DROP COLUMN. That would only help if you were dropping the last potentially- toastable column of a table. And implementing it would require introducing weird corner cases into the tuple toaster, because it m

Re: [HACKERS] Assert Levels

2008-09-21 Thread Greg Smith
On Fri, 19 Sep 2008, Tom Lane wrote: Well, there are certain things that --enable-cassert turns on that are outrageously expensive...I don't think anyone knows what the performance impact of just the regular Asserts is; it's been too long since these other things were stuck in there. The nex

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Marko Kreen
On 9/21/08, Joe Conway <[EMAIL PROTECTED]> wrote: > Marko Kreen wrote: > > You need to ignore pg_service also. (And PGPASSWORD) > > Why? pg_service does not appear to support wildcards, so what is the attack > vector? "service=foo host=custom" > And on PGPASSWORD, the fine manual says the foll

Re: [HACKERS] Assert Levels

2008-09-21 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > The next time I'm doing some performance testing I'll try to quantify how > much damage the expensive ones do by playing with pg_config_manual.h. > Normally I'm testing with 1GB+ of shared_buffers which makes the current > assert scheme unusable. There i

[HACKERS] parallel pg_restore

2008-09-21 Thread Andrew Dunstan
I am working on getting parallel pg_restore working. I'm currently getting all the scaffolding working, and hope to have a naive prototype posted within about a week. The major question is how to choose the restoration order so as to maximize efficiency both on the server and in reading the ar

Re: [HACKERS] Predictable order of SQL commands in pg_dump

2008-09-21 Thread Dmitry Koterov
Unfortunately, I cannot reproduce this with 100% effect. But, time to time I execute diff utility for a database and notice that two or more trigger or constraint definitions (or something else) are permuted. Something like this: +ALTER TABLE ONLY a +ADD CONSTRAINT "fk_b_Id" FOREIGN KEY (b_i

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread David Fetter
On Sun, Sep 21, 2008 at 10:49:56PM +0400, Dmitry Koterov wrote: > Normalization is not a panacea here. Sometimes such normalization > creates too much overeat and a lot of additional code (especially if > there are a lot of such dependencies). Array support in Postgres is > quite handy; in my pra

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Andrew Dunstan
Simon Riggs wrote: No, its not possible. Need a trigger. I think we should support it though. If we extend the relational model with arrays then it would be sensible if we support this aspect as well. Implementation would be fairly straightforward. ri_triggers currently assumes a non-array

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Dmitry Koterov
Normalization is not a panacea here. Sometimes such normalization creates too much overeat and a lot of additional code (especially if there are a lot of such dependencies). Array support in Postgres is quite handy; in my practive, moving from a_b_map to arrays economizes hundreds of lines of store

Re: [HACKERS] pg_settings.sourcefile patch is a security breach

2008-09-21 Thread Magnus Hagander
Tom Lane wrote: > We go to some lengths to prevent non-superusers from examining > data_directory and other values that would tell them exactly where the > PG data directory is in the server's filesystem. The recently applied > patch to expose full pathnames of GUC variables' source files blows a

[HACKERS] pg_settings.sourcefile patch is a security breach

2008-09-21 Thread Tom Lane
We go to some lengths to prevent non-superusers from examining data_directory and other values that would tell them exactly where the PG data directory is in the server's filesystem. The recently applied patch to expose full pathnames of GUC variables' source files blows a hole a mile wide in that

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
[EMAIL PROTECTED] writes: > pgadmin has some umm, interesting queries over pg_depends. It sounds > like this change could complicate those. I doubt it's an > insurmountable problem of course. Yeah. But the only real point of the change is cleanliness, and if it's injecting ugliness into clients t

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread dpage
pgadmin has some umm, interesting queries over pg_depends. It sounds like this change could complicate those. I doubt it's an insurmountable problem of course. On 9/21/08, Tom Lane <[EMAIL PROTECTED]> wrote: > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >>> A possible objectio

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> A possible objection to this plan is that if the column-level privileges >> patch doesn't get in, then we're left with a useless column in >> pg_attribute. But an always-null column doesn't cost much of anything, >> and we know tha

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Joe Conway
Marko Kreen wrote: On 9/21/08, Tom Lane <[EMAIL PROTECTED]> wrote: Joe Conway <[EMAIL PROTECTED]> writes: Good point -- I'll look into that and post something tomorrow. How does > "requirepassword" sound for the option? It is consistent with > "requiressl" but a bit long and hard to read. Ma

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Joshua D. Drake
Tom Lane wrote: A possible objection to this plan is that if the column-level privileges patch doesn't get in, then we're left with a useless column in pg_attribute. But an always-null column doesn't cost much of anything, and we know that sooner or later we will support per-column ACLs: they

Re: [HACKERS] Assert Levels

2008-09-21 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Simon Riggs wrote: >> Well, we don't. That's why I'd suggest to do it slowly and classify >> everything as medium weight until proven otherwise. > Once you have classified all asserts, what do we do with the result? > What would be the practical impa

[HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-21 Thread Tom Lane
I had a thought while looking over the column-level privileges patch that Stephen Frost is working on. To wit, that the only reason that column default expressions are stored in a separate catalog pg_attrdef is the historical assumption in some parts of the code that pg_attribute rows are fixed-wi

Re: [HACKERS] Assert Levels

2008-09-21 Thread Peter Eisentraut
Simon Riggs wrote: Well, we don't. That's why I'd suggest to do it slowly and classify everything as medium weight until proven otherwise. Once you have classified all asserts, what do we do with the result? What would be the practical impact? What would be your recommendation about who runs

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Tom Lane
Zoltan Boszormenyi <[EMAIL PROTECTED]> writes: > we came across a database where a table had a toasted table, > keeping huge amounts of disk space allocated. However, > the table's current definition didn't explain why there was > a toasted table. Then upon some experiments, it struck me. > There _

[HACKERS] Toasted table not deleted when no out of line columns left

2008-09-21 Thread Zoltan Boszormenyi
Hi, we came across a database where a table had a toasted table, keeping huge amounts of disk space allocated. However, the table's current definition didn't explain why there was a toasted table. Then upon some experiments, it struck me. There _was_ a toasted field but as the schema was modified,

Re: [HACKERS] Foreign key constraint for array-field?

2008-09-21 Thread Simon Riggs
On Sun, 2008-09-21 at 04:38 +0400, Dmitry Koterov wrote: > Is it possible to create a foreign key constraint for ALL elements of > an array field? > > CREATE TABLE a(id INTEGER); > CREATE TABLE b(id INTEGER, a_ids INTEGER[]); > > Field b.a_ids contains a list of ID's of "a" table. I want to ens

Re: [HACKERS] [patch] fix dblink security hole

2008-09-21 Thread Marko Kreen
On 9/21/08, Tom Lane <[EMAIL PROTECTED]> wrote: > Joe Conway <[EMAIL PROTECTED]> writes: > > Good point -- I'll look into that and post something tomorrow. How does > > "requirepassword" sound for the option? It is consistent with > > "requiressl" but a bit long and hard to read. Maybe "require_p