Re: [HACKERS] Hacking on PostgreSQL via GIT
Martin Langhoff <[EMAIL PROTECTED]> writes: > Aidan Van Dyk wrote: >> And remember the warning I gave that my conversion is *not* a direct CVS >> import - I intentionally *unexpand* all Keywords before stuffing them >> into GIT so that merging and branching can ignore all the Keyword >> conflicts... > My import is unexpanding those as well to support rebasing and merging > better. Um ... why do either of you feel there's an issue there? We switched over to $PostgreSQL$ a few years ago specifically to avoid creating merge problems for downstream repositories. If there are any other keyword expansions left in the source text I'd vote to remove them. If you have a problem with $PostgreSQL$, why? regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Autovacuum vs statement_timeout
Alvaro Herrera wrote: > I think that is too strong an assumption, which is why I'm planning to > back-patch the change to reset statement_timeout to 0 on autovacuum till > 8.0, as discussed. I think I should also backpatch the change to set > zero_damaged_pages as well (which is not on 8.0 AFAIR). > > It's very very easy to change things in postgresql.conf. Actually > knowing what you are doing (i.e. thinking on the consequences on VACUUM > and such) is a whole another matter. Frankly, setting statement_timeout in postgresql.conf seems so risky in so many ways, perhaps we just need to document that the parameter probably should not be set in postgresql.conf, and why. -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] Re: [COMMITTERS] pgsql: Update error message for COPY with a multi-byte delimiter.
Tom Lane wrote:
> [EMAIL PROTECTED] (Bruce Momjian) writes:
> > Update error message for COPY with a multi-byte delimiter.
>
> You forgot the CSV strings.
OK, I wasn't clear on those but I am now. I backpatched the doc change
to 8.2.X.
--
Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/ref/copy.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/ref/copy.sgml,v
retrieving revision 1.79
diff -c -c -r1.79 copy.sgml
*** doc/src/sgml/ref/copy.sgml 18 Apr 2007 00:17:56 - 1.79
--- doc/src/sgml/ref/copy.sgml 18 Apr 2007 02:26:36 -
***
*** 225,231
quote
! Specifies the quotation character in CSV mode.
The default is double-quote.
--- 225,231
quote
! Specifies the ASCII quotation character in CSV mode.
The default is double-quote.
***
*** 235,241
escape
! Specifies the character that should appear before a
QUOTE data character value in CSV mode.
The default is the QUOTE value (usually double-quote).
--- 235,241
escape
! Specifies the ASCII character that should appear before a
QUOTE data character value in CSV mode.
The default is the QUOTE value (usually double-quote).
Index: src/backend/commands/copy.c
===
RCS file: /cvsroot/pgsql/src/backend/commands/copy.c,v
retrieving revision 1.281
diff -c -c -r1.281 copy.c
*** src/backend/commands/copy.c 18 Apr 2007 00:38:57 - 1.281
--- src/backend/commands/copy.c 18 Apr 2007 02:26:37 -
***
*** 893,899
if (cstate->csv_mode && strlen(cstate->quote) != 1)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("COPY quote must be a single character")));
/* Check escape */
if (!cstate->csv_mode && cstate->escape != NULL)
--- 893,899
if (cstate->csv_mode && strlen(cstate->quote) != 1)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("COPY quote must be a single ASCII character")));
/* Check escape */
if (!cstate->csv_mode && cstate->escape != NULL)
***
*** 904,910
if (cstate->csv_mode && strlen(cstate->escape) != 1)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("COPY escape must be a single character")));
/* Check force_quote */
if (!cstate->csv_mode && force_quote != NIL)
--- 904,910
if (cstate->csv_mode && strlen(cstate->escape) != 1)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("COPY escape must be a single ASCII character")));
/* Check force_quote */
if (!cstate->csv_mode && force_quote != NIL)
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] Autovacuum vs statement_timeout
Joshua D. Drake wrote: > Tom Lane wrote: > >Robert Treat <[EMAIL PROTECTED]> writes: > >>I'm with Joshua on this one. Statement_timeout is often used as a means > >>for protection from long running statements due to server load and > >>locking and all of the above commands can certainly fall into that area. > >>If people feel strongly that the command line programs need a way to > >>circumvent it, add a --ignore-statement-timeout option or similar > >>mechanism. > > > >The worst-case scenario here is that your server fails and you discover > >that all your backups are corrupt because you didn't notice pg_dump was > >failing due to statement_timeout. (Maybe it just recently started to > >fail because your biggest table grew past the point at which the COPY > >command exceeded statement_timeout.) > > > >I'm not excited about the other ones but I can see the argument for > >making pg_dump force the timeout to 0. > > I guess my point is, if you are knowledgeable enough to actually set a > statement_timeout, you are likely knowledgeable enough to know how to > turn it off for programs like pg_dump. I think that is too strong an assumption, which is why I'm planning to back-patch the change to reset statement_timeout to 0 on autovacuum till 8.0, as discussed. I think I should also backpatch the change to set zero_damaged_pages as well (which is not on 8.0 AFAIR). It's very very easy to change things in postgresql.conf. Actually knowing what you are doing (i.e. thinking on the consequences on VACUUM and such) is a whole another matter. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] utf8 COPY DELIMITER?
> Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > The message in question should be something like: > > "COPY delimiter must be a single ASCII character" > > If we phrase it like that we should enforce it like that --- ie, reject > high-bit-set characters. > > But I'm a bit hesitant to do so, because it actually does work fine to > use a high-bit-set character as a delimiter as long as client and server > encodings are the same LATINx set. We'd be taking away functionality > for European users for no very good reason. > > Is it worth going to the trouble of distinguish same-encoding and > different-encoding cases and applying a looser check for the former > case? I think yes. Seems a good idea. Even better, however, is fixing the CVS escaping and quoting I think. Clearly it's a bug. -- Tatsuo Ishii SRA OSS, Inc. Japan ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Autovacuum vs statement_timeout
Tom Lane wrote: Robert Treat <[EMAIL PROTECTED]> writes: I'm with Joshua on this one. Statement_timeout is often used as a means for protection from long running statements due to server load and locking and all of the above commands can certainly fall into that area. If people feel strongly that the command line programs need a way to circumvent it, add a --ignore-statement-timeout option or similar mechanism. The worst-case scenario here is that your server fails and you discover that all your backups are corrupt because you didn't notice pg_dump was failing due to statement_timeout. (Maybe it just recently started to fail because your biggest table grew past the point at which the COPY command exceeded statement_timeout.) I'm not excited about the other ones but I can see the argument for making pg_dump force the timeout to 0. I guess my point is, if you are knowledgeable enough to actually set a statement_timeout, you are likely knowledgeable enough to know how to turn it off for programs like pg_dump. Sincerely, Joshua D. Drake regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Hacking on PostgreSQL via GIT
* Florian G. Pflug <[EMAIL PROTECTED]> [070417 20:30]: > >So - if you are committed to providing your gateway long term to > >Florian, I'm happy to drop my gateway in favour of yours. > > There seem to be other people than me who are interested in a git > mirror. Maybe we could declare one of those mirrors the > "official" one - I guess things would be easier if all people > interested in using git would use the same mirror... > > What do you guys think? I'll provide that gateway as long as I have access to hardware and access that can keep up with PostgreSQL CVS... Of course, the beauty of a DVCS is that we don't really need an official one... And with GIT, you can even "graft" history in if you want. So you could even "start" your GIT work from a cvs checkout of whenever, and "graft" any commit from any of the CVS->GIT conversion history as a parent to your starting point. a. -- Aidan Van Dyk Create like a god, [EMAIL PROTECTED] command like a king, http://www.highrise.ca/ work like a slave. signature.asc Description: Digital signature
Re: [HACKERS] Hacking on PostgreSQL via GIT
Aidan Van Dyk wrote: > I'm an unknown here, I know - I've used PostgreSQL for years, but only > recently started following the development community... And at this I'm probably unknown here as well. Hi everyone ;-) > And remember the warning I gave that my conversion is *not* a direct CVS > import - I intentionally *unexpand* all Keywords before stuffing them > into GIT so that merging and branching can ignore all the Keyword > conflicts... My import is unexpanding those as well to support rebasing and merging better. So - if you are committed to providing your gateway long term to Florian, I'm happy to drop my gateway in favour of yours. (Florian, before basing your code on either you should get a checkout of Aidan's and mine and check that the tips of the branches you are working on match the cvs branches -- the cvsimport code is good but whereever CVS is involved, there's a lot of interpretation at play, a sanity check is always good). cheers, m -- --- Martin @ Catalyst .Net .NZ Ltd, PO Box 11-053, Manners St, Wellington WEB: http://catalyst.net.nz/ PHYS: Level 2, 150-154 Willis St OFFICE: +64(4)916-7224 UK: 0845 868 5733 ext 7224 MOB: +64(21)364-017 Make things as simple as possible, but no simpler - Einstein --- ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [COMMITTERS] pgsql: Update error message for COPY with a multi-byte delimiter.
[EMAIL PROTECTED] (Bruce Momjian) writes: > Update error message for COPY with a multi-byte delimiter. You forgot the CSV strings. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Autovacuum vs statement_timeout
Robert Treat <[EMAIL PROTECTED]> writes: > I'm with Joshua on this one. Statement_timeout is often used as a means for > protection from long running statements due to server load and locking and > all of the above commands can certainly fall into that area. If people feel > strongly that the command line programs need a way to circumvent it, add > a --ignore-statement-timeout option or similar mechanism. The worst-case scenario here is that your server fails and you discover that all your backups are corrupt because you didn't notice pg_dump was failing due to statement_timeout. (Maybe it just recently started to fail because your biggest table grew past the point at which the COPY command exceeded statement_timeout.) I'm not excited about the other ones but I can see the argument for making pg_dump force the timeout to 0. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Autovacuum vs statement_timeout
On Tuesday 17 April 2007 18:38, Jim C. Nasby wrote:
> On Tue, Apr 17, 2007 at 12:51:51PM -0700, Joshua D. Drake wrote:
> > Jim C. Nasby wrote:
> > >On Sun, Apr 01, 2007 at 12:36:01AM +0200, Peter Eisentraut wrote:
> > >>Tom Lane wrote:
> > >>>I seem to remember that we'd agreed that autovacuum should ignore any
> > >>>globally set statement_timeout, on the grounds that a poorly chosen
> > >>>setting could indefinitely prevent large tables from being vacuumed.
> > >>
> > >>On a vaguely related matter, should programs such as pg_dump, vacuumdb,
> > >>and reindexdb disable statement_timeout?
> > >
> > >Youch... yes, they should IMO. Add clusterdb, pg_dumpall and pg_restore
> > >to that list as well (really, pg_dump(all) should output a command to
> > >disable statement_timeout).
> >
> > I don't know if that should be a default or not. It is certainly easy
> > enough to disable it should you want to.
>
> How would you disable it for those command-line utilities? Or are you
> referring to disabling it via an ALTER ROLE SET ... for superusers?
>
> ISTM current behavior is a bit of a foot-gun. These are administrative
> shell commands that aren't going to be run by Joe-user.
I'm with Joshua on this one. Statement_timeout is often used as a means for
protection from long running statements due to server load and locking and
all of the above commands can certainly fall into that area. If people feel
strongly that the command line programs need a way to circumvent it, add
a --ignore-statement-timeout option or similar mechanism.
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Hacking on PostgreSQL via GIT
Martin Langhoff wrote: Aidan Van Dyk wrote: And remember the warning I gave that my conversion is *not* a direct CVS import - I intentionally *unexpand* all Keywords before stuffing them into GIT so that merging and branching can ignore all the Keyword conflicts... My import is unexpanding those as well to support rebasing and merging better. So - if you are committed to providing your gateway long term to Florian, I'm happy to drop my gateway in favour of yours. There seem to be other people than me who are interested in a git mirror. Maybe we could declare one of those mirrors the "official" one - I guess things would be easier if all people interested in using git would use the same mirror... What do you guys think? (Florian, before basing your code on either you should get a checkout of Aidan's and mine and check that the tips of the branches you are working on match the cvs branches -- the cvsimport code is good but whereever CVS is involved, there's a lot of interpretation at play, a sanity check is always good). I actually hoped that I could just take my current git repo, and rebase my branch onto one of those two repos - or does rebase only work from an ancestor to a descendant? greetings, Florian Pflug ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] utf8 COPY DELIMITER?
Tatsuo Ishii wrote: > > On looking at the code, there's another issue: the CSV escape and quote > > characters are assumed to be the same in client and server encodings, > > because they're checked for before we do transcoding. This pretty much > > restricts them to be ASCII. > > > > regards, tom lane > > +1. > > The message in question should be something like: > > "COPY delimiter must be a single ASCII character" New text is: The single ASCII character that separates columns within each row Backpatched to 8.2.X. -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Hacking on PostgreSQL via GIT
Aidan Van Dyk <[EMAIL PROTECTED]> writes: > I have my CVS->GIT conversion running hourly from the anon-rsync of the > cvsroot. I don't know the specifics of the PostgreSQL rsync/mirror > setup, so I may be pulling it more frequently than it's actually > published, but until I hear from someone that tells me I'm taxing to > many rsync resources, I'll just leave it that way... The anoncvs mirror updates once an hour, so you're fine. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [COMMITTERS] pgsql: Also done for PL/pgSQL: < o Add support for WITH HOLD and
Tom Lane wrote: > [EMAIL PROTECTED] (Bruce Momjian) writes: > > Also done for PL/pgSQL: > > > < o Add support for WITH HOLD and SCROLL cursors > > > o -Add support for WITH HOLD and SCROLL cursors > > WITH HOLD support is not there. Thanks, I was wondering about that. TODO item split in two: o Add support for WITH HOLD cursors -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?
On Tue, 17 Apr 2007, Tom Lane wrote: > A recent discussion led me to the idea that FK triggers are fired > unnecessarily during an UPDATE if the foreign-key column(s) contain > any NULLs, because ri_KeysEqual() treats two nulls as unequal, > and therefore we conclude the row has changed when it has not. > I claim that both ri_KeysEqual() and ri_OneKeyEqual() could consider > two nulls to be equal. For ri_KeysEqual, I think so, since we actually aren't testing equality as much as difference between the rows that might invalidate the constraint. And, it does seem like with the code in trigger.c that the other checks in the _upd functions in ri_triggers.c are redundant, but I'm vaguely afraid I've forgotten something. For ri_OneKeyEqual, I think like ri_AllKeysUnequal we know that the old row doesn't have NULLs in the places it's currently called (although I don't think this is commented). It seems like it should stay consistent with ri_KeysEqual and that not putting the foo = NULL or foo = DEFAULT seems better for the current calling cases besides. > Furthermore it seems like ri_AllKeysUnequal() should do so too; the case > can't arise at the moment because the sole caller already knows that one > of the key sets contains no nulls, but if this weren't so, the > optimization would be actively wrong if we concluded that two nulls were > unequal. Hmm, probably so, although at least this does appear to be commented at the calling site to mention that it's depending on the fact that there are no NULLs. ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)
Tom, Andrew, KaiGai, "Andrew Dunstan" <[EMAIL PROTECTED]> writes: What's more, we have a SoC project for column level access controls. I don't see the SE stuff as a replacement for that, since it apparently exists outside the standard SQL security model. ... which presumably wouldn't involve any added dependency on outside code. For people who are already using SELinux or Trusted Solaris, making the database dependent on that infrastructure might be seen as a plus, but I'm not sure the rest of the world would be pleased. Yes, I was thinking that this should be a compile-time option with a lot of warnings in the Docs. Give the team some credit, though; they've managed to come up with a system that integrates OS-level ACLs for both SElinux and TxSol, are not asking us to incorporate two different sets, and are coming to us with a serious proposal that has a lot of work behind it. Please don't blow them off like they were undergrads submitting a semester project. If they need to come back after 8.3 beta so we can properly pay attention to the proposal, then say so. There are also some interesting questions about SQL spec compliance and whether a database that silently hides some rows from you will give semantically consistent results. Yeah -- that's a potentially serious issue; KaiGai, have you looked into it? --Josh Berkus ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] SoC Students/Projects selected
Jim, Will all discussions take place here or in -students? The idea is that discussions about the SoC *program* and "where do I submit ..." questions will go on -students; any questions about actual code will go here or on the appropriate dev list. Probably worth posting about that to -announce and/or -general... Actually, I have a couple possible candidates. Now to discuss funding ... --Josh ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Hacking on PostgreSQL via GIT
* Martin Langhoff <[EMAIL PROTECTED]> [070417 17:32]: > > Having a git mirror of the pgsql CVS would be great. > > BTW, I've just check out repo.or.cz, and noticed that there is already a > > git mirror of the pgsql CVS: http://repo.or.cz/w/PostgreSQL.git > > Yes, I've seen it, but I don't know the guy. I can ensure you have a > CVS->GIT gateway updated daily or twice daily. I'm an unknown here, I know - I've used PostgreSQL for years, but only recently started following the development community... And at this point I'm still pretty much just following, hence my interest in getting a GIT repot of PostgreSQL. GIT is *very* helful at a "new" code-base. I have my CVS->GIT conversion running hourly from the anon-rsync of the cvsroot. I don't know the specifics of the PostgreSQL rsync/mirror setup, so I may be pulling it more frequently than it's actually published, but until I hear from someone that tells me I'm taxing to many rsync resources, I'll just leave it that way... The CVS->GIT conversion is cheap - it's the rsync that takes most of the time... I can run it more frequently if people think it would be valuble and the rsync-admins don't care... And remember the warning I gave that my conversion is *not* a direct CVS import - I intentionally *unexpand* all Keywords before stuffing them into GIT so that merging and branching can ignore all the Keyword conflicts... a. -- Aidan Van Dyk Create like a god, [EMAIL PROTECTED] command like a king, http://www.highrise.ca/ work like a slave. signature.asc Description: Digital signature
Re: [HACKERS] utf8 COPY DELIMITER?
Tatsuo Ishii <[EMAIL PROTECTED]> writes: > The message in question should be something like: > "COPY delimiter must be a single ASCII character" If we phrase it like that we should enforce it like that --- ie, reject high-bit-set characters. But I'm a bit hesitant to do so, because it actually does work fine to use a high-bit-set character as a delimiter as long as client and server encodings are the same LATINx set. We'd be taking away functionality for European users for no very good reason. Is it worth going to the trouble of distinguish same-encoding and different-encoding cases and applying a looser check for the former case? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] utf8 COPY DELIMITER?
> On looking at the code, there's another issue: the CSV escape and quote > characters are assumed to be the same in client and server encodings, > because they're checked for before we do transcoding. This pretty much > restricts them to be ASCII. > > regards, tom lane +1. The message in question should be something like: "COPY delimiter must be a single ASCII character" -- Tatsuo Ishii SRA OSS, Inc. Japan ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Autovacuum vs statement_timeout
On Tue, Apr 17, 2007 at 12:51:51PM -0700, Joshua D. Drake wrote: > Jim C. Nasby wrote: > >On Sun, Apr 01, 2007 at 12:36:01AM +0200, Peter Eisentraut wrote: > >>Tom Lane wrote: > >>>I seem to remember that we'd agreed that autovacuum should ignore any > >>>globally set statement_timeout, on the grounds that a poorly chosen > >>>setting could indefinitely prevent large tables from being vacuumed. > >>On a vaguely related matter, should programs such as pg_dump, vacuumdb, > >>and reindexdb disable statement_timeout? > > > >Youch... yes, they should IMO. Add clusterdb, pg_dumpall and pg_restore > >to that list as well (really, pg_dump(all) should output a command to > >disable statement_timeout). > > I don't know if that should be a default or not. It is certainly easy > enough to disable it should you want to. How would you disable it for those command-line utilities? Or are you referring to disabling it via an ALTER ROLE SET ... for superusers? ISTM current behavior is a bit of a foot-gun. These are administrative shell commands that aren't going to be run by Joe-user. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] SoC Students/Projects selected
On Sun, Apr 15, 2007 at 08:00:23PM -0300, Josh Berkus wrote: > Now, while each of these students has an assigned mentor, that doesn't > mean other people shouldn't help. If you're interested in their work, > please pitch in. > > Note that we'll also be using the pgsql-students mailing list for > discussions about SoC itself. Will all discussions take place here or in -students? > Unfortunately, we only got ONE proposal to work on the buildfarm, and > that one was snagged by another project. So I'm going to be proposing > that we use SPI funds to get the Buildfarm work done; if anyone knows a > likely candidate, speak up. Probably worth posting about that to -announce and/or -general... -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql: Also done for PL/pgSQL: < o Add support for WITH HOLD and
[EMAIL PROTECTED] (Bruce Momjian) writes: > Also done for PL/pgSQL: > < o Add support for WITH HOLD and SCROLL cursors > > o -Add support for WITH HOLD and SCROLL cursors WITH HOLD support is not there. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] where to write small reusable functions ?
On Fri, Apr 13, 2007 at 03:02:28PM +0200, Dany DeBontridder wrote: > On 4/13/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > > > >Dany DeBontridder wrote: > >> I'm working to implement a new feature to pg_dump: the ability to dump > >> objects like function, indexes... > > > >pg_dump already dumps functions and indexes. > > Right but you can't dump only one or two functions or only the functions and > nothing else. (the same for index, triggers...) You should make sure and read past discussion about this, as well as propose a design to the community before getting too far into a patch. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Hacking on PostgreSQL via GIT
Florian G. Pflug wrote: > Cool - I'm new to git, so I really appreciate any help that I can get. Great - I am a SoC mentor for 2 other projects (git and moodle) so I've got some time set aside for SoC stuff. You might as well take advantage of it :-) >> For the kind of work you'll be doing (writing patches that you'll want >> to be rebasing onto the latest HEAD for merging later) git is probably >> the best tool. That's what I use it for... tracking my experimental / >> custom branches of projects that use CVS or SVN :-) > > Thats how I figured I'd work - though I don't yet understand what > the advantage of "rebase" is over "merge". Probably during your development cycle you'll want to merge the changes from cvshead into your dev branch - that's what you seem to be doing. Great. Later when you are getting things ready for actual merging into CVS you'll want to prepare a series of patches that apply to the top of cvshead. That's where the rebase tools become useful. > Currently, I've setup a git repo that pulls in the changes from the SVN > repo, and pushed them to my main soc git repo. On that main repo I have > two branches, master and pgsql-head, and I call "cg-merge pgsql-head" > if I want to merge with CVS HEAD. You are doing the right thing. If possible, I'd suggest that you use git instead of cogito. Recent git is as user-friendly as cogito. The main difference is that you'll need to learn a bit about the index, and that'll be useful. >> Initially, I'll post it on http://git.catalyst.net.nz/ and I can run a >> daily import for you - once that's in place you can probably get a repo >> with your work on http://repo.or.cz/ > > Having a git mirror of the pgsql CVS would be great. > BTW, I've just check out repo.or.cz, and noticed that there is already a > git mirror of the pgsql CVS: http://repo.or.cz/w/PostgreSQL.git Yes, I've seen it, but I don't know the guy. I can ensure you have a CVS->GIT gateway updated daily or twice daily. cheers, martin -- --- Martin @ Catalyst .Net .NZ Ltd, PO Box 11-053, Manners St, Wellington WEB: http://catalyst.net.nz/ PHYS: Level 2, 150-154 Willis St OFFICE: +64(4)916-7224 UK: 0845 868 5733 ext 7224 MOB: +64(21)364-017 Make things as simple as possible, but no simpler - Einstein --- ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?
On Tue, 2007-04-17 at 17:16 -0400, Tom Lane wrote: > A recent discussion led me to the idea that FK triggers are fired > unnecessarily during an UPDATE if the foreign-key column(s) contain > any NULLs, because ri_KeysEqual() treats two nulls as unequal, > and therefore we conclude the row has changed when it has not. FK trigger *can be optimised away* is true. No need to have a discussion about whether NULL == NULL, but the critical test is: if I overwrote it, would you be able to tell? The answer is No, so away it goes. > I claim that both ri_KeysEqual() and ri_OneKeyEqual() could consider > two nulls to be equal. Furthermore it seems like ri_AllKeysUnequal() > should do so too; the case can't arise at the moment because the sole > caller already knows that one of the key sets contains no nulls, but > if this weren't so, the optimization would be actively wrong if we > concluded that two nulls were unequal. > > Comments? > > Also, I am wondering to what extent the ri_KeysEqual() calls in > ri_triggers.c are redundant, given that commands/trigger.c now has > the smarts to not even queue the trigger when those cases apply. Would be good to document that behaviour in ri_triggers.c - I was completely misled when I looked at the code a while back. Probably more than one thing can go. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Hacking on PostgreSQL via GIT
Florian G. Pflug wrote: >> Initially, I'll post it on http://git.catalyst.net.nz/ and I can run a >> daily import for you - once that's in place you can probably get a repo >> with your work on http://repo.or.cz/ Ok - you can now clone from http://git.catalyst.net.nz/postgresql.git viewable from http://git.catalyst.net.nz/gitweb too. It's 24hs behind, and I'm sorting the updating scripts that will run daily. The HEAD of CVS is renamed to cvshead there. All the other branches and tags are untouched. Please DO check that the tip of cvshead matches a CVS checkout with -kk. I've had limited time to sanitycheck the import ;-) cheers, m -- --- Martin @ Catalyst .Net .NZ Ltd, PO Box 11-053, Manners St, Wellington WEB: http://catalyst.net.nz/ PHYS: Level 2, 150-154 Willis St OFFICE: +64(4)916-7224 UK: 0845 868 5733 ext 7224 MOB: +64(21)364-017 Make things as simple as possible, but no simpler - Einstein --- ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] RESET command seems pretty disjointed now
Tom Lane wrote:
Mark Kirkwood <[EMAIL PROTECTED]> writes:
Tom Lane wrote:
The current documentation for RESET exhibits a certain lack of, um,
intellectual cohesiveness:
Synopsis
RESET configuration_parameter
RESET ALL
RESET { PLANS | SESSION | TEMP | TEMPORARY }
Maybe DISCARD for the plans etc might be more intuitive than extending
RESET?
DISCARD PLANS and DISCARD TEMP seem pretty reasonable, but DISCARD SESSION
sounds a bit odd --- it seems like it might mean "disconnect", which of
course is exactly what we're trying to avoid. But possibly we could
rename RESET SESSION as DISCARD ALL.
Leastwise I haven't got any better ideas. Anyone have another proposal?
What about
RESET parameter
RESET { PLANS | TEMP | TEMPORARY }
RESET ALL { PARAMETERS | STATE }
RESET ALL would become an abbreviation of RESET ALL PARAMETERS (for backwards
compatibility), while RESET SESSION would be renamed to RESET ALL STATE.
greetings, Florian Pflug
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
[HACKERS] Can't ri_KeysEqual() consider two nulls as equal?
A recent discussion led me to the idea that FK triggers are fired unnecessarily during an UPDATE if the foreign-key column(s) contain any NULLs, because ri_KeysEqual() treats two nulls as unequal, and therefore we conclude the row has changed when it has not. I claim that both ri_KeysEqual() and ri_OneKeyEqual() could consider two nulls to be equal. Furthermore it seems like ri_AllKeysUnequal() should do so too; the case can't arise at the moment because the sole caller already knows that one of the key sets contains no nulls, but if this weren't so, the optimization would be actively wrong if we concluded that two nulls were unequal. Comments? Also, I am wondering to what extent the ri_KeysEqual() calls in ri_triggers.c are redundant, given that commands/trigger.c now has the smarts to not even queue the trigger when those cases apply. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Unhelpful debug tools on OS X :-(
Tom Lane wrote: Heikki Linnakangas <[EMAIL PROTECTED]> writes: But I did notice that we're not fsyncing the newly written relation like we should. Good point, but doesn't the analogy to copy_relation_data say that we should sync if not rd_istemp? Right. I don't know why I didn't copy that if-statement. It seems to be bedtime for me... -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Unhelpful debug tools on OS X :-(
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > But I did notice that we're not fsyncing the newly written relation like > we should. Good point, but doesn't the analogy to copy_relation_data say that we should sync if not rd_istemp? (This is my fault BTW; your original patch kept the data in shared buffers, so it wasn't subject to the problem.) > BTW: In tablecmds.c the new relation is fsynced with smgrimmedsync, not > heap_sync. That's okay since that routine is just copying the one table. TOAST is handled via recursion of ATExecSetTableSpace. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Unhelpful debug tools on OS X :-(
I wrote: > Thread 0 Crashed: > 0 postmaster 0x001af4ef smgrextend + 12 (smgr.c:485) > 1 postmaster 0x00029044 end_heap_rewrite + 208 (rewriteheap.c:278) > 2 postmaster 0x000bdc22 cluster_rel + 850 (cluster.c:806) > 3 postmaster 0x000be119 cluster + 160 (cluster.c:220) AFAICS, a crash inside smgrextend must indicate it was passed a junk SMgrRelation, which implies that the problem has to trace back to the new heap's rd_smgr getting closed since the rewriteheap operation started. I think the only possible path for that is an sinval queue overflow and consequent SI reset during the test. That would explain why it's such an intermittent failure, but it is curious that Darwin seems much more prone to this than other platforms. I wonder if Apple has tweaked the scheduler in a way that allows backends to not be given any cycles for long periods ... Anyway, patch committed; we'll see if things don't get more stable. Memo to self: run some tests with CLOBBER_CACHE_ALWAYS after the remaining major patches have landed. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Unhelpful debug tools on OS X :-(
Tom Lane wrote: Heikki Linnakangas <[EMAIL PROTECTED]> writes: Tom Lane wrote: Any suggestions how to extract some info out of this? Does OS X have the catchsegv tool? No, but I suddenly remembered about CrashReporter, and sure enough it's catching these crashes: Exception: EXC_BAD_ACCESS (0x0001) Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x0010 Thread 0 Crashed: 0 postmaster 0x001af4ef smgrextend + 12 (smgr.c:485) 1 postmaster 0x00029044 end_heap_rewrite + 208 (rewriteheap.c:278) 2 postmaster 0x000bdc22 cluster_rel + 850 (cluster.c:806) 3 postmaster 0x000be119 cluster + 160 (cluster.c:220) 4 postmaster 0x001b74a8 PortalRunUtility + 233 (palloc.h:84) 5 postmaster 0x001b7784 PortalRunMulti + 237 (pquery.c:1271) 6 postmaster 0x001b80ae PortalRun + 918 (pquery.c:813) 7 postmaster 0x001b2afd exec_simple_query + 656 (postgres.c:965) 8 postmaster 0x001b4b0c PostgresMain + 5628 (postgres.c:3507) 9 postmaster 0x00183973 ServerLoop + 2828 (postmaster.c:2614) 10 postmaster 0x00184b1e PostmasterMain + 2794 (postmaster.c:972) 11 postmaster 0x00130f8e main + 1236 (main.c:188) 12 postmaster 0x1e86 _start + 216 13 postmaster 0x1dad start + 41 So it looks like this has got something to do with the MVCC-safe cluster changes, which is not too surprising considering it started happening around about then. Off to have a look ... I've been looking at the code for a few minutes as well, but haven't found an explanation for that yet. But I did notice that we're not fsyncing the newly written relation like we should. There's a comment raw_heap_insert: /* * Now write the page. We say isTemp = true even if it's not a * temp table, because there's no need for smgr to schedule an * fsync for this write; we'll do it ourselves before committing. */ smgrextend(state->rs_new_rel->rd_smgr, state->rs_blockno, (char *) page, true); That's copy-pasted from tablecmds.c. But unlike in tablecmds.c, end_heap_rewrite only fsyncs the new file if we're not WAL-logging. Proposed fix: Index: src/backend/access/heap/rewriteheap.c === RCS file: /home/hlinnaka/pgcvsrepository/pgsql/src/backend/access/heap/rewriteheap.c,v retrieving revision 1.1 diff -c -r1.1 rewriteheap.c *** src/backend/access/heap/rewriteheap.c 8 Apr 2007 01:26:27 - 1.1 --- src/backend/access/heap/rewriteheap.c 17 Apr 2007 20:50:05 - *** *** 272,282 } /* !* If not WAL-logging, must fsync before commit. We use heap_sync !* to ensure that the toast table gets fsync'd too. */ ! if (!state->rs_use_wal) ! heap_sync(state->rs_new_rel); /* Deleting the context frees everything */ MemoryContextDelete(state->rs_cxt); --- 272,284 } /* !* Must fsync before commit, even if we've WAL-logged the changes, !* because we've written pages outside the buffer manager. See comments!* in copy_relation_data in commands/tablecmds.c for more information. !* !* We use heap_sync to ensure that the toast table gets fsync'd too. */ ! heap_sync(state->rs_new_rel); /* Deleting the context frees everything */ MemoryContextDelete(state->rs_cxt); BTW: In tablecmds.c the new relation is fsynced with smgrimmedsync, not heap_sync. How about the toast table, it goes through shared buffers as usual, right? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] CREATE DATABASE foo OWNER bar
Tom Lane wrote: > Larry Rosenman <[EMAIL PROTECTED]> writes: > > I guess the issue is that I'd expect public to be owned by the DB Owner > > after > > a CREATE DATABASE foo OWNER bar, > > Why? Do you expect the system catalogs to be owned by the DB owner? > What about other random objects that might have been created in the > template database? If the DBA has installed nondefault permission > settings on the public schema or other objects, how do you expect those > to be transformed? > > I do not actually agree with that TODO item, as I think it requires > AI-completeness to guess what sorts of changes to apply, and getting > ownership/permissions wrong would create a significant risk of security > issues. Caution added to TODO item: * Set proper permissions on non-system schemas during db creation Currently all schemas are owned by the super-user because they are copied from the template1 database. However, since all objects are inherited from the template database, it is not clear that setting schemas to the db owner is correct. -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] CREATE DATABASE foo OWNER bar
Alvaro Herrera wrote: > Larry Rosenman wrote: > > Greetings, > > I think I found a bug, or at least a POLA violation. At work, I created > > a user that is NOT a superuser, nor can that user create databases. When I > > did a create database foo owner bar, all the schemas are set to be owned by > > the superuser that created the database, not the database owner. > > > > Shouldn't everything that is in the DB be owned by the purported owner? > > Right. This is on TODO: > > %Set proper permissions on non-system schemas during db creation > > Currently all schemas are owned by the super-user because they are copied from > the template1 database. > > > I note it is marked with a %, but it's clearly not easy at all. '%' removed. -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Unhelpful debug tools on OS X :-(
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Any suggestions how to extract some info out of this? > Does OS X have the catchsegv tool? No, but I suddenly remembered about CrashReporter, and sure enough it's catching these crashes: Exception: EXC_BAD_ACCESS (0x0001) Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x0010 Thread 0 Crashed: 0 postmaster 0x001af4ef smgrextend + 12 (smgr.c:485) 1 postmaster 0x00029044 end_heap_rewrite + 208 (rewriteheap.c:278) 2 postmaster 0x000bdc22 cluster_rel + 850 (cluster.c:806) 3 postmaster 0x000be119 cluster + 160 (cluster.c:220) 4 postmaster 0x001b74a8 PortalRunUtility + 233 (palloc.h:84) 5 postmaster 0x001b7784 PortalRunMulti + 237 (pquery.c:1271) 6 postmaster 0x001b80ae PortalRun + 918 (pquery.c:813) 7 postmaster 0x001b2afd exec_simple_query + 656 (postgres.c:965) 8 postmaster 0x001b4b0c PostgresMain + 5628 (postgres.c:3507) 9 postmaster 0x00183973 ServerLoop + 2828 (postmaster.c:2614) 10 postmaster 0x00184b1e PostmasterMain + 2794 (postmaster.c:972) 11 postmaster 0x00130f8e main + 1236 (main.c:188) 12 postmaster 0x1e86 _start + 216 13 postmaster 0x1dad start + 41 So it looks like this has got something to do with the MVCC-safe cluster changes, which is not too surprising considering it started happening around about then. Off to have a look ... regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Unhelpful debug tools on OS X :-(
Tom Lane wrote: Any suggestions how to extract some info out of this? Does OS X have the catchsegv tool? If you can run postmaster with that, you should get a backtrace when it crashes. Unless it has the same problem as gdb, of course.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] schema creation during initdb
sharath kumar wrote: How can I create a schema during initdb time? For example pg_catalog, pg_toast, information_schema are created during initdb time. Likewise I want to create my own schema at initdb time. How can i do it? Also how to create a table into this schema at that time itself? This question really does not belong on -hackers, which is about postgres development, not usage. Next time use pgsql-general. You have misunderstood the purpose of initdb, which is to prepare a location for running a postmaster against. After initdb has run there are no user databases yet created (unless you count the postgres database). So you would need to do something like this: initdb ... pg_ctl start ... createdb mytemplate psql -c 'create schema foo' mytemplate psql -c 'create table foo.bar ...' mytemplate After that you can do this: createdb --template mytemplate newdb and the newdb will have your schema and table. HTH andrew ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
[HACKERS] schema creation during initdb
How can I create a schema during initdb time? For example pg_catalog, pg_toast, information_schema are created during initdb time. Likewise I want to create my own schema at initdb time. How can i do it? Also how to create a table into this schema at that time itself? Thanks Sharat.
Re: [HACKERS] Autovacuum vs statement_timeout
Jim C. Nasby wrote: On Sun, Apr 01, 2007 at 12:36:01AM +0200, Peter Eisentraut wrote: Tom Lane wrote: I seem to remember that we'd agreed that autovacuum should ignore any globally set statement_timeout, on the grounds that a poorly chosen setting could indefinitely prevent large tables from being vacuumed. On a vaguely related matter, should programs such as pg_dump, vacuumdb, and reindexdb disable statement_timeout? Youch... yes, they should IMO. Add clusterdb, pg_dumpall and pg_restore to that list as well (really, pg_dump(all) should output a command to disable statement_timeout). I don't know if that should be a default or not. It is certainly easy enough to disable it should you want to. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Autovacuum vs statement_timeout
On Sun, Apr 01, 2007 at 12:36:01AM +0200, Peter Eisentraut wrote: > Tom Lane wrote: > > I seem to remember that we'd agreed that autovacuum should ignore any > > globally set statement_timeout, on the grounds that a poorly chosen > > setting could indefinitely prevent large tables from being vacuumed. > > On a vaguely related matter, should programs such as pg_dump, vacuumdb, > and reindexdb disable statement_timeout? Youch... yes, they should IMO. Add clusterdb, pg_dumpall and pg_restore to that list as well (really, pg_dump(all) should output a command to disable statement_timeout). -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] utf8 COPY DELIMITER?
On looking at the code, there's another issue: the CSV escape and quote characters are assumed to be the same in client and server encodings, because they're checked for before we do transcoding. This pretty much restricts them to be ASCII. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] utf8 COPY DELIMITER?
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Mark Dilger wrote: >> I'm working on fixing bugs relating to multibyte character encodings. >> I wasn't sure whether this was a bug or not. I don't think we should >> use the phrasing "COPY delimiter must be a single character" when, in >> utf8 land, I did in fact use a single character. We might say "a >> single byte", or we might extend the functionality to handle multibyte >> characters. > Doing the latter would be a feature, and so is of course right off the > table for this release. Changing the error messages to be clearer should > be fine. +1 on changing the message: "character" is clearly less correct than "byte" here. I doubt that supporting a single multibyte character would be an interesting extension --- if we wanted to do anything at all there, we'd just generalize the delimiter to be an arbitrary string. But it would certainly slow down COPY by some amount, which is an area where you'll get push-back for performance losses, so you'd need to make a convincing use-case for it. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] utf8 COPY DELIMITER?
Mark Dilger wrote: Andrew Dunstan wrote: Mark Dilger wrote: The \COPY command rejects multibyte delimiters. Is this intentional behavior? It is certainly a known limitation, and I suspect removing it could add non-trivial overhead to the input processing. What is the use case for using such a delimiter? I'm working on fixing bugs relating to multibyte character encodings. I wasn't sure whether this was a bug or not. I don't think we should use the phrasing "COPY delimiter must be a single character" when, in utf8 land, I did in fact use a single character. We might say "a single byte", or we might extend the functionality to handle multibyte characters. Doing the latter would be a feature, and so is of course right off the table for this release. Changing the error messages to be clearer should be fine. cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] utf8 COPY DELIMITER?
Andrew Dunstan wrote: Mark Dilger wrote: The \COPY command rejects multibyte delimiters. Is this intentional behavior? It is certainly a known limitation, and I suspect removing it could add non-trivial overhead to the input processing. What is the use case for using such a delimiter? I'm working on fixing bugs relating to multibyte character encodings. I wasn't sure whether this was a bug or not. I don't think we should use the phrasing "COPY delimiter must be a single character" when, in utf8 land, I did in fact use a single character. We might say "a single byte", or we might extend the functionality to handle multibyte characters. mark cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] utf8 COPY DELIMITER?
Mark Dilger wrote: The \COPY command rejects multibyte delimiters. Is this intentional behavior? It is certainly a known limitation, and I suspect removing it could add non-trivial overhead to the input processing. What is the use case for using such a delimiter? cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] utf8 COPY DELIMITER?
The \COPY command rejects multibyte delimiters. Is this intentional behavior? Here is an example of the behavior: [EMAIL PROTECTED] ~ $ touch foo [EMAIL PROTECTED] ~ $ psql -p Welcome to psql 8.3devel, the PostgreSQL interactive terminal. Type: \copyright for distribution terms \h for help with SQL commands \? for help with psql commands \g or terminate with semicolon to execute query \q to quit pgsql=# create table foo (a integer); CREATE TABLE pgsql=# \copy foo from foo delimiter '標' ERROR: COPY delimiter must be a single character \copy: ERROR: COPY delimiter must be a single character If your email/news reader doesn't render that properly, I'm using a pictogram character for the delimiter. I checked out a new copy of the sources from cvs this morning. It behaves the same way on 8.2.3. mark ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Unhelpful debug tools on OS X :-(
Tom Lane wrote: > Dunno if anyone's noticed, but buildfarm member jackal has been crashing > intermittently in the "cluster" regression test. I found out that I can > reproduce that here on a Mac Mini (apparently the same or nearly same > hardware/software as jackal) --- just repeat the parallel tests often > enough, and it'll fail once in awhile. I've got a core file but gdb > is being no help at all: > > mini:~/pgsql/src/test/regress tgl$ gdb /Users/tgl/testversion/bin/postmaster > /cores/core.23609 > GNU gdb 6.3.50-20050815 (Apple version gdb-563) (Wed Jul 19 05:10:58 GMT 2006) > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-apple-darwin"...Reading symbols for shared > libraries .. done > > Core was generated by `/Users/tgl/testversion/bin/postmaster'. > Core file contained no thread-specific data > > (gdb) bt > #0 0x in ?? () > (gdb) > > Any suggestions how to extract some info out of this? no idea on that one - but clownfish managed to fail the cluster test once too lately: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=clownfish&dt=2007-04-09%2023:03:03 Stefan ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)
For people who are already using SELinux or Trusted Solaris, making the database dependent on that infrastructure might be seen as a plus, but I'm not sure the rest of the world would be pleased. Even where SELinux is available it has had mixed reviews - I habitually disable it. The relationship between your works and SE-PostgreSQL effort is similar to the relationship between UNIX-DAC/Posix-ACL and SELinux on operating systems. I believe those are not conflicted efforts. Thanks, -- KaiGai Kohei <[EMAIL PROTECTED]> ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)
Tom Lane wrote: > "Andrew Dunstan" <[EMAIL PROTECTED]> writes: >> What's more, we have a SoC project for column level access controls. > > ... which presumably wouldn't involve any added dependency on outside code. > For people who are already using SELinux or Trusted Solaris, making the > database dependent on that infrastructure might be seen as a plus, but > I'm not sure the rest of the world would be pleased. The most significant purpose of PGACE and SE-PostgreSQL is integration between database and operating system security policy, on mandatory access controlled (MAC) system especially. Thus, not to provide any regression is the most desired behavior of PGACE on non-MAC system like SELinux disabled Linux, I think. PGACE without using SELinux or Trusted Solaris does not give any effect, because it is defined as static inline function with no operation mostly. > There are also > some interesting questions about SQL spec compliance and whether a > database that silently hides some rows from you will give semantically > consistent results. Any violated tuples are always filtered from result set, before using them, as if an additional condition is appended onto WHERE or JOIN ON clause. The condition means whether client has enough permission on this tuple, or not. All those processes are done after rewriting phase, so result set is constant even if a table is referred via views. Thus, client can deal a table as if it does not contain any violated tuples. # If a result set contains any violated tuple, it's a bug of SE-PostgreSQL. An exception is foreign key implementation. It internally uses UPDATE query to support 'ON UPDATE CASCADE' rule and so on. If violated tuples are filtered, the FK constraint is not kept. For example, when there are five tuples to be cascaded but client does not have enough permission onto two of them, the two tuple will be remained without cascading. In SE-PostgreSQL, whole of transaction will be aborted, if client does not have enough permissions on all the tuples cascaded under FK processing. One more exception is TRUNCATE statement. In SE-PostgreSQL, TRUNCATE statement is nonsense, because it is translated into unconditional DELETE statement to avoid removing tuples without enough permission. Currently, I have not gotten any inconsistency in the access control model. But any pointed out is welcome. Thanks, -- KaiGai Kohei <[EMAIL PROTECTED]> ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)
Tom Lane wrote: > Josh Berkus <[EMAIL PROTECTED]> writes: >> Column level? We don't currently support that, except through VIEWs. >> How is it implemented? > > It wasn't clear to me how much of this is actually working today and how > much is a paper design --- one thing in particular that stood out as > probable handwaving was the bit about being able to assign to a system > column in INSERT or UPDATE. I'm fairly sure that that would take some > *significant* redesign of querytree and plan targetlist representation > :-( ... I looked at it once for OIDs and decided it wasn't worth the > trouble. Currently, writable system column support is already included as a part of PGACE, and it works fine to make setting up SE-PostgreSQL. The implementation uses TargetEntry->resjunk effectively to make it simplified. When a targetlist contains "security_context" column in a UPDATE or INSERT query, SE-PostgreSQL marks the TargetEntry as a junk. Then, the value explicitly given as "security_context" is computed in the normal way. It is fetched at ExecutePlan() just before calling ExecUpdate() or ExecInsert(), and stored into HeapTupleHeader->t_security. Maybe, a part of the patch to implement them is less than 100L, without any significant redesign, Is there any oversight? If so, please tell me. Thanks, -- KaiGai Kohei <[EMAIL PROTECTED]> ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)
Josh Berkus wrote: > KaiGai, > >> It provides database users fine grained mandatory access control >> including row and column level one, and integration with operating >> system security policy. > > Column level? We don't currently support that, except through VIEWs. > How is it implemented? PGACE provides a hook just after query rewriting phase. SE-PostgreSQL walks on the query tree to check any required references onto columns, as the implementation of the hook. If a client does not have enough permissions onto the column, SE-PostgreSQL abort the current transaction via ereport(). Thanks, -- KaiGai Kohei <[EMAIL PROTECTED]> ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
[HACKERS] Unhelpful debug tools on OS X :-(
Dunno if anyone's noticed, but buildfarm member jackal has been crashing intermittently in the "cluster" regression test. I found out that I can reproduce that here on a Mac Mini (apparently the same or nearly same hardware/software as jackal) --- just repeat the parallel tests often enough, and it'll fail once in awhile. I've got a core file but gdb is being no help at all: mini:~/pgsql/src/test/regress tgl$ gdb /Users/tgl/testversion/bin/postmaster /cores/core.23609 GNU gdb 6.3.50-20050815 (Apple version gdb-563) (Wed Jul 19 05:10:58 GMT 2006) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .. done Core was generated by `/Users/tgl/testversion/bin/postmaster'. Core file contained no thread-specific data (gdb) bt #0 0x in ?? () (gdb) Any suggestions how to extract some info out of this? regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Buildfarm member Narwhal: Python 2.5/8.1
Tom Lane wrote: Dave Page <[EMAIL PROTECTED]> writes: Andrew Dunstan wrote: The question in my mind is this: how much do we back-patch to cover new and incompatible releases of software we depend on? I guess that depends on the invasiveness - in this case it's a couple of simple updates to the regression tests so I think it's probably worth doing. It's not just the regression tests; there are at least two rounds of patches in the C code --- plpython.c r1.90, r1.97, maybe r1.100. Only the first of these has seen any testing "in the wild". Ahh - missed that bit. Another objection to patching 8.1 is why stop there ... why not 8.0, etc? 8.0 didn't have the PL regression tests and as it appeared to be a regression test issue... I'll disable python on < 8.2. Regards, Dave. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Buildfarm member Narwhal: Python 2.5/8.1
Dave Page <[EMAIL PROTECTED]> writes: > Andrew Dunstan wrote: >> The question in my mind is this: how much do we back-patch to cover new >> and incompatible releases of software we depend on? > I guess that depends on the invasiveness - in this case it's a couple of > simple updates to the regression tests so I think it's probably worth doing. It's not just the regression tests; there are at least two rounds of patches in the C code --- plpython.c r1.90, r1.97, maybe r1.100. Only the first of these has seen any testing "in the wild". Another objection to patching 8.1 is why stop there ... why not 8.0, etc? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Buildfarm member Narwhal: Python 2.5/8.1
Dave Page wrote: New buildfarm member Narwhal is failing the PL regression tests for Python on REL8_1_STABLE. This appears to be because it's running Python 2.5 (the causes being a deprecated module - whrandom - and some changed messages). The former problem was fixed by Peter, and the latter by Tom but both only for 8.2+`- any reason this wasn't backported to 8.1? I couldn't find anything in the archives. http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=narwhal&dt=2007-04-17%20085153&stg=pl-install-check The question in my mind is this: how much do we back-patch to cover new and incompatible releases of software we depend on? Python 2.5 was released on 19 Sept 2006, long after Postgres 8.1. I guess you could make a case to say that we should back-patch to the release immediately before the library change. (BTW, it is possible to include python only in certain branches in your buildfarm client - examples are in the config file). cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] RESET command seems pretty disjointed now
On 4/17/07, Marko Kreen <[EMAIL PROTECTED]> wrote: If DISCARD is the final word, I start to prepare a patch. Attached patch does following conversions: RESET PLANS -> DISCARD PLANS RESET TEMP -> DISCARD TEMP RESET SESSION -> DISCARD ALL -- marko discard.diff.gz Description: GNU Zip compressed data ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Buildfarm member Narwhal: Python 2.5/8.1
Andrew Dunstan wrote: The question in my mind is this: how much do we back-patch to cover new and incompatible releases of software we depend on? Python 2.5 was released on 19 Sept 2006, long after Postgres 8.1. I guess you could make a case to say that we should back-patch to the release immediately before the library change. I guess that depends on the invasiveness - in this case it's a couple of simple updates to the regression tests so I think it's probably worth doing. (BTW, it is possible to include python only in certain branches in your buildfarm client - examples are in the config file). Yeah, I already use that functionality to handle the features that we've added over past releases to the windows port (ldap, thread safety etc). Handy :-) /D ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Python test failures
Hello!
I'm trying to get the buildfarm to run on my XP x64 machine (still in
32-bit mode), and it's acting weird on me :-(
The plpython tests because of:
+ NOTICE: ('import failed -- No module named bisect',)
and
! ERROR: plpython: function "import_test_two" failed
! DETAIL: : No module named sha
All other plpython tests work fine. It appears to load all other modules
fine.
Running the following little python script directly on the
commandline works fine:
import sha
print sha.new('foo').hexdigest()
in c:\python25\lib I have sha.py and sha.pyc. I also have random, array etc
- the other modules that the plpython tests run, and that work.
Permissions are the same on all these files, the buildfarm user can read
the files fine. And the fact that caling python directly seems to indicate
that the path shuold be fine...
I know very little about python... :( Any pointers?
//Magnus
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Buildfarm member Narwhal: Python 2.5/8.1
On 4/17/07, Dave Page <[EMAIL PROTECTED]> wrote: New buildfarm member Narwhal is failing the PL regression tests for Python on REL8_1_STABLE. This appears to be because it's running Python 2.5 (the causes being a deprecated module - whrandom - and some changed messages). I also suggest backporting following fix to 8.2 and 8.1: http://archives.postgresql.org/pgsql-hackers/2007-04/msg00127.php because otherwise python2.5 on 64bit platform will crash. -- marko ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
[HACKERS] Buildfarm member Narwhal: Python 2.5/8.1
New buildfarm member Narwhal is failing the PL regression tests for Python on REL8_1_STABLE. This appears to be because it's running Python 2.5 (the causes being a deprecated module - whrandom - and some changed messages). The former problem was fixed by Peter, and the latter by Tom but both only for 8.2+`- any reason this wasn't backported to 8.1? I couldn't find anything in the archives. http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=narwhal&dt=2007-04-17%20085153&stg=pl-install-check Regards, Dave. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Hacking on PostgreSQL via GIT
Chris Browne wrote: This strikes me as being a really super thing, having both Subversion and Git repositories publicly available that are tracking the PostgreSQL sources. Stepping back to the SCM discussion, people were interested in finding out what merits there were in having these sorts of SCMs, and in finding out what glitches people might discover (e.g. - like the files where the CVS repository is a bit schizophrenic as to whether they are still there or not...). Having these repositories should allow some of this experimentation to take place now. Yep. It'd be nice to have official GIT and SVN etc. mirrors of the main CVS repository. There's no pressing reason for the PostgreSQL project to switch from CVS, but we could provide alternatives to developers. As long as you can create a diff to send to pgsql-patches, it doesn't matter which version control system you use. I'm interested in trying GIT or Monotone myself, presumably they would be good for managing unapplied, work-in-progress patches. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] modifying the table function
"Islam Hegazy" <[EMAIL PROTECTED]> writes: > My question is how to inform the client that there is a tuple to display = > and return back to the backend to continue the query execution? I'd suggest you start by reading http://developer.postgresql.org/pgdocs/postgres/protocol.html and then develop a clear specification at that level of what you think should happen. Perhaps after that exercise it will be clearer how to change the code. "Think first, program later." regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] modifying the table function
Hi there I made some changes in postgresql source code to let the table function work in iteration fashion rather than materialization fashion. My main modifications are in 'exec_simple_query' function, I changed 'portalRunSelect' to return just one tuple instead of 'FETCH_ALL'. I made other variables static as the 'portal', 'receiver', etc. so that the program returns to them to continue execution from the last point. Then I added more calls to 'exec_simple_query' and it worked correctly. For example, if I added 4 calls to 'exec_simple_query' the 4th tuple is returned. My problem lies now in how to display the 1st, 2nd, 3rd tuples not just the 4th. I tried to call 'pq_flush' after each 'exec_simple_query' but nothing is displayed. I replaced 'pq_flush' with 'ReadyForQuery' but it gives the following error 'error 0x54 message received while system is idle'. My question is how to inform the client that there is a tuple to display and return back to the backend to continue the query execution? Regards Islam Hegazy
Re: [HACKERS] RESET command seems pretty disjointed now
On 4/17/07, Tom Lane <[EMAIL PROTECTED]> wrote:
Florian Pflug <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>>> The current documentation for RESET exhibits a certain lack of, um,
>>> intellectual cohesiveness:
> What about
> RESET parameter
> RESET { PLANS | TEMP | TEMPORARY }
> RESET ALL { PARAMETERS | STATE }
> RESET ALL would become an abbreviation of RESET ALL PARAMETERS (for backwards
> compatibility), while RESET SESSION would be renamed to RESET ALL STATE.
This doesn't do anything to address the lack of coherence. It's not
only that backward compatibility forces us to break the clear meaning of
ALL; another problem is that we break the symmetry between SET, RESET,
and SHOW. If you can RESET SESSION, what does it mean to SET SESSION?
Or SHOW SESSION?
Given the precedent that RESET ALL only resets GUC variables, I think
it's probably best if we just say that RESET only affects GUC variables,
period. The new functionality should go by another name entirely.
I'm not wedded to DISCARD by any means, but I do not believe that
changing some words after RESET is going to fix my complaint.
Can't argue with that. Also I don't have better proposals.
If DISCARD is the final word, I start to prepare a patch.
--
marko
---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at
http://www.postgresql.org/about/donate
