[HACKERS] Tablespaces

2004-02-26 Thread Gavin Sherry
Hi all, I've been looking at implementing table spaces for 7.5. Some notes and implementation details follow. -- Type of table space: There are many different table space implementations in relational database management systems. In my implementation, a table space in PostgreSQL will be

Re: [HACKERS] Tablespaces

2004-02-26 Thread Shridhar Daithankar
On Thursday 26 February 2004 15:37, Gavin Sherry wrote: Tying it all together: The catalogs pg_database, pg_namespace, and pg_class will have a 'tblspc' field. This will be the OID of the table space the object resides in, or 0 (default table space). Since we can then resolve relid/relname,

Re: [HACKERS] Tablespaces

2004-02-26 Thread Hans-Jürgen Schönig
Gavin Sherry wrote: Hi all, I've been looking at implementing table spaces for 7.5. Some notes and implementation details follow. -- Type of table space: There are many different table space implementations in relational database management systems. In my implementation, a table space in

Re: [HACKERS] Tablespaces

2004-02-26 Thread Richard Huxton
On Thursday 26 February 2004 10:07, Gavin Sherry wrote: CREATE TABLESPACE tbl1 LOCATION '/var/' which will result in something like '/var/123443' is a bad idea. Then again, the user should know better. Comments? The LOCATION should have the same owner and permissions as $PGDATA - that

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Josh Berkus
Tom, Our first try at a bug tracking system, several years ago, was open to anybody to create entries, and we found that the signal-to-noise ratio went to zero in no time. Too many not-a-bugs, too many support requests, too few actual bugs. We went back to using the pgsql-bugs mailing

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: I actually sort of agree with Tom, although I don't want to raise the barrier too high. I'd suggest allowing all registered users to submit bugs. Possibly workable, but what's your definition of registered user? I'd hope that anyone subscribed to any of

Re: [HACKERS] Tablespaces

2004-02-26 Thread Gavin Sherry
Is it possible to put WALs and CLOGs into different tablespaces? (maybe different RAID systems). Some companies want that ... I wasn't going to look at that just yet. There is of course the temporary hack of symlinking WAL else where. I'd be interested to see the performance difference

Re: [HACKERS] Tablespaces

2004-02-26 Thread Joshua D. Drake
Is it possible to put WALs and CLOGs into different tablespaces? (maybe different RAID systems). Some companies want that ... You can do this now, but it would be nice to be able to have it actually configurable versus the hacked linked method. J -- Co-Founder Command Prompt, Inc.

[HACKERS] ORDER BY different locales

2004-02-26 Thread Karel Zak
Hi, a lot of people sometimes need order same data in same DB by more different locales. For example multi-language web application with DB in UTF-8. It's problem in PostgreSQL, because PostgreSQL require set LC_COLLATE by initdb. I think possible solution is special

Re: [HACKERS] Check Constraints and pg_dump

2004-02-26 Thread Tom Lane
Jonathan Scott [EMAIL PROTECTED] writes: The functions and tables create just fine, but when it gets to the COPY part of the sql script, it tries to load tables in what really is the wrong order. The check constraint is making sure there is a plan before there is a contract, yet pg_dump is

Re: [HACKERS] ORDER BY different locales

2004-02-26 Thread Tom Lane
Karel Zak [EMAIL PROTECTED] writes: I think possible solution is special function used ORDER BY clause which knows to switch by safe way to wanted locales, convert string by strxfrm() and switch back to backend locales. This function breaks the whole backend if an elog() failure

Re: [HACKERS] ORDER BY different locales

2004-02-26 Thread Karel Zak
On Thu, Feb 26, 2004 at 09:16:03AM -0500, Tom Lane wrote: Karel Zak [EMAIL PROTECTED] writes: I think possible solution is special function used ORDER BY clause which knows to switch by safe way to wanted locales, convert string by strxfrm() and switch back to backend locales.

Re: [HACKERS] Tablespaces

2004-02-26 Thread Alex J. Avriette
On Thu, Feb 26, 2004 at 11:22:28PM +1100, Gavin Sherry wrote: Certainly, table spaces are used in many ways in oracle, db2, etc. You can mirror data across them, have different buffer sizes for example. In some implementations, they can be raw disk partitions (no file system). I don't intend

[HACKERS] Collaboration Tool Proposal

2004-02-26 Thread Josh Berkus
Folks, Discuss/vote/object/screamshout: PROPOSAL: GBorg -- GForge Migration Why do we want a full-service collaboration tool? PostgreSQL is no longer a monolithic project, but rather a collection of closely related projects. Some of these projects are official, some are unofficial, some are

Re: [HACKERS] Collaboration Tool Proposal

2004-02-26 Thread Joseph Tate
Josh Berkus wrote: Folks, Discuss: Has anyone talked to the people at collabnet (http://www.collab.net)? I wonder if they'd be willing to put something together for the PostgreSQL team? They run the tigris.org site, which is one of the nicest OSS collaboration sites I've worked with.

Re: [HACKERS] bgwriter never dies

2004-02-26 Thread Josh Berkus
Tom, ... In the case of a postmaster crash, I think something in the system is so wrong that I'd prefer an immediate shutdown. Surely some other people have opinions on this?  Hello out there? Well, my opinion is based on the question, can we restart the postmaster if it dies and the

Re: [HACKERS] Collaboration Tool Proposal

2004-02-26 Thread Josh Berkus
Joseph, Thanks for feedback. Has anyone talked to the people at collabnet (http://www.collab.net)? I wonder if they'd be willing to put something together for the PostgreSQL team? They run the tigris.org site, which is one of the nicest OSS collaboration sites I've worked with. GForge

Re: [pgsql-www] [HACKERS] Collaboration Tool Proposal

2004-02-26 Thread Josh Berkus
Robert, Josh, are you still in favor of this move if the larger community does not want to move the main project to a gforge based system? or vice versa? Not sure. Depends on what the leads of the associated projects think. Obviously, if everyone's dead set against it, we won't do it.

Re: [HACKERS] bgwriter never dies

2004-02-26 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Well, my opinion is based on the question, can we restart the postmaster if it dies and the other backends are still running? You can't start a fresh postmaster until the last of the old backends is gone (and yes, there's an interlock to prevent you from

Re: [pgsql-www] [HACKERS] Collaboration Tool Proposal

2004-02-26 Thread Jeroen T. Vermeulen
On Thu, Feb 26, 2004 at 10:49:46AM -0800, Josh Berkus wrote: Not sure. Depends on what the leads of the associated projects think. Obviously, if everyone's dead set against it, we won't do it. I for one am willing to try this in the near term. I've got an external domain (pqxx.tk)

Re: [pgsql-www] [HACKERS] Collaboration Tool Proposal

2004-02-26 Thread Josh Berkus
Jeroen, I for one am willing to try this in the near term. Great! I've got an external domain (pqxx.tk) pointing to the libpqxx page on GBorg, and moving it over to a new URL is child's play. My main worry is transition management: - How will mailing list subscribers be affected? -

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Peter Eisentraut
Josh Berkus wrote: PROPOSAL: GBorg -- GForge Migration Why do we want a full-service collaboration tool? In terms of improving the hosting infrastructure, this would surely be a step forward, but the problem with collaboration is not that the tools are missing, it's that people are unwilling

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Jeroen T. Vermeulen
On Thu, Feb 26, 2004 at 09:16:38PM +0100, Peter Eisentraut wrote: In terms of improving the hosting infrastructure, this would surely be a step forward, but the problem with collaboration is not that the tools are missing, it's that people are unwilling to use any tools for issue

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Andrew Dunstan
Josh Berkus wrote: Peter, So yes, I think this is a reasonable plan, just don't expect collaboration to suddenly appear out of nowhere. Yeah. As my grandfather used to say, You can lead a horse to water, but you can't make him shrink. (granddad is under care, now). Everyone:

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Robert Treat
On Thu, 2004-02-26 at 15:41, Andrew Dunstan wrote: Josh Berkus wrote: Peter, So yes, I think this is a reasonable plan, just don't expect collaboration to suddenly appear out of nowhere. Yeah. As my grandfather used to say, You can lead a horse to water, but you

Re: [HACKERS] Tablespaces

2004-02-26 Thread Barry Lind
Gavin, After creating a tablespace what (if any) changes can be done to it. Can you DROP a tablespace, or once created will it always exist? Can you RENAME a tablespace? Can you change the location of a tablespace (i.e you did a disk reorg and move the contents to a different location and

Re: [HACKERS] Tablespaces

2004-02-26 Thread Gavin Sherry
On Thu, 26 Feb 2004, Alex J. Avriette wrote: On Thu, Feb 26, 2004 at 11:22:28PM +1100, Gavin Sherry wrote: Certainly, table spaces are used in many ways in oracle, db2, etc. You can mirror data across them, have different buffer sizes for example. In some implementations, they can be raw

Re: [HACKERS] CVS HEAD compile warning

2004-02-26 Thread Gaetano Mendola
Tom Lane wrote: Red Hat's still shipping 2.5.4a according to a quick look... Well Red Hat's still ship Postgres 7.3.4 ... I'm not considering anymore RH to be up to date with various versions :-( Gaetano ---(end of broadcast)--- TIP 9: the

Re: [HACKERS] Tablespaces

2004-02-26 Thread Simon Riggs
Gavin Sherry The creation of table spaces will need to be recorded in xlog in the same way that files are in heap_create() with the corresponding delete logic incase of ABORT. Overall, sounds very cool. Please could we record the OID of the tablespace in the WAL logs, not the path to the

Re: [HACKERS] Tablespaces

2004-02-26 Thread Tom Lane
Gavin Sherry [EMAIL PROTECTED] writes: Speaking of locking, can we do anything to prevent people from shooting themselves in the foot by changing active tablespaces? Are we even going to have a DROP TABLESPACE command, and if so what would it do? Ahh. I forgot to detail my ideas on this. It

Re: [HACKERS] bgwriter never dies

2004-02-26 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Should we have a pgmon process that watches the postmaster and restarts it if required? I doubt it; in practice the postmaster is *very* reliable (because it doesn't do much), and so I'm not sure that adding a watchdog is going to increase the net

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Tom Lane
Robert Treat [EMAIL PROTECTED] writes: On Thu, 2004-02-26 at 15:41, Andrew Dunstan wrote: Perhaps when BZ supports PG - some progress is being made on that front, but it's not a done deal yet. I can't imagine the BZ plugin for Gforge would require you to use a second database system would

Re: [HACKERS] Tablespaces

2004-02-26 Thread Tom Lane
James Rogers [EMAIL PROTECTED] writes: With the implementation of much smarter and more adaptive cache replacement algorithm i.e. ARC, I would expect the benefit of using the kernel file system cache to diminish significantly. It appears to me, and I could be wrong, that the reason Postgres

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Peter Eisentraut
Tom Lane wrote: Yeah, I looked into that when core started discussing this whole thing awhile back. The Red Hat port of BZ to Postgres is perfectly usable. Is it available anywhere? ---(end of broadcast)--- TIP 8: explain analyze is your

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Josh Berkus
People, The question is, do we need BZ right off or should we try GForge's lightweight tool first?Personally I find that BZ is a little intimidating to new users, particularly for searching on issues; as a result it tends to lead to a lot of duplicate filings. -- -Josh Berkus Aglio

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Peter Eisentraut
Josh Berkus wrote: The question is, do we need BZ right off or should we try GForge's lightweight tool first?Personally I find that BZ is a little intimidating to new users, particularly for searching on issues; as a result it tends to lead to a lot of duplicate filings. I think we had

Re: [HACKERS] Tablespaces

2004-02-26 Thread James Rogers
On Thu, 2004-02-26 at 13:22, Gavin Sherry wrote: Postgres benefits a lot from kernel file system cache at the moment. With the implementation of much smarter and more adaptive cache replacement algorithm i.e. ARC, I would expect the benefit of using the kernel file system cache to diminish

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Neil Conway
Peter Eisentraut [EMAIL PROTECTED] writes: I think we had previously decided that we will not allow a random user off the street to file bug reports into whatever system we end up using. Uh, why not? (And more to the point, why raise the barrier to entry on reporting bugs?) Individuals can

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Cott Lang
On Thu, 2004-02-26 at 15:45, Peter Eisentraut wrote: Tom Lane wrote: Yeah, I looked into that when core started discussing this whole thing awhile back. The Red Hat port of BZ to Postgres is perfectly usable. Is it available anywhere?

Re: [HACKERS] Tablespaces

2004-02-26 Thread Greg Stark
I am expecting to hear some bleating about this from people whose preferred platforms don't support symlinks ;-). However, if we don't Well, one option would be to have the low level filesystem storage (md.c?) routines implement a kind of symlink themselves. Just a file with a special magic

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: Peter Eisentraut [EMAIL PROTECTED] writes: I think we had previously decided that we will not allow a random user off the street to file bug reports into whatever system we end up using. Uh, why not? (And more to the point, why raise the barrier to

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: I would favor using Bugzilla over anything else just because I'm used to it (have to use it internally at Red Hat anyway). I might suggest again RT. It's open source and has serious commercial traction. The postgres port needs a lot of work for it to really

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Cott Lang
On Thu, 2004-02-26 at 13:41, Andrew Dunstan wrote: Perhaps when BZ supports PG - some progress is being made on that front, but it's not a done deal yet. Redhat puts out a PG version of Bugzilla. It works pretty well. However, we just dropped it in favor of Jira. Jira is a lot friendlier

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Andrew Dunstan
Peter Eisentraut wrote: Josh Berkus wrote: The question is, do we need BZ right off or should we try GForge's lightweight tool first?Personally I find that BZ is a little intimidating to new users, particularly for searching on issues; as a result it tends to lead to a lot of duplicate

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Josh Berkus
Peter, So yes, I think this is a reasonable plan, just don't expect collaboration to suddenly appear out of nowhere. Yeah. As my grandfather used to say, You can lead a horse to water, but you can't make him shrink. (granddad is under care, now). Everyone: Further data: if we prefer

Re: [HACKERS] Tablespaces

2004-02-26 Thread Alex J. Avriette
On Fri, Feb 27, 2004 at 08:22:25AM +1100, Gavin Sherry wrote: interested if anyone could provide some real world benchmarking of file system vs. raw disk. Postgres benefits a lot from kernel file system cache at the moment. Also, I believe that database designers have traditionally made bad

Re: [HACKERS] Tablespaces

2004-02-26 Thread Gavin Sherry
On Thu, 26 Feb 2004, Tom Lane wrote: Gavin Sherry [EMAIL PROTECTED] writes: A table space is a directory structure. The directory structure is as follows: [EMAIL PROTECTED] /path/to/tblspc]$ ls OID1/ OID2/ OID1 and OID2 are the OIDs of databases which have created a table space

Re: [HACKERS] ORDER BY different locales

2004-02-26 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: This function breaks the whole backend if an elog() failure occurs while it's got the wrong locale set. I believe it would also be remarkably slow --- doesn't setlocale() involve reading a new locale definition file from whereever those are stored? I

Re: [pgsql-www] [HACKERS] Collaboration Tool Proposal

2004-02-26 Thread Robert Treat
On Thu, 2004-02-26 at 13:19, Josh Berkus wrote: What does the Apache project run? Not sure. Anyone? Apache uses a home-brew collection of OSS tools. I think they have the advantage of a larger community of web developers to help out than we have ;-) Josh, are you still in favor of this

Re: [HACKERS] Tablespaces

2004-02-26 Thread Tom Lane
Gavin Sherry [EMAIL PROTECTED] writes: A table space is a directory structure. The directory structure is as follows: [EMAIL PROTECTED] /path/to/tblspc]$ ls OID1/ OID2/ OID1 and OID2 are the OIDs of databases which have created a table space against this file system location. In this

[HACKERS] log_line_info

2004-02-26 Thread Andrew Dunstan
I haven't had any other feedback on this patch that I posted. However, I'm a bit dissatisfied with it for a couple of reasons: . when a connection is logged we don't yet know the user and database, because we haven't processed the initial packet yet. That causes %U and %D to produce empty

Re: [HACKERS] Tablespaces

2004-02-26 Thread Hans-Jürgen Schönig
Gavin Sherry wrote: Is it possible to put WALs and CLOGs into different tablespaces? (maybe different RAID systems). Some companies want that ... I wasn't going to look at that just yet. There is of course the temporary hack of symlinking WAL else where. that's what we do now. we symlink

Re: [HACKERS] Tablespaces

2004-02-26 Thread Gavin Sherry
On Thu, 26 Feb 2004, Dennis Bjorklund wrote: On Thu, 26 Feb 2004, Gavin Sherry wrote: Comments? Questions? Suggestions? Is that plan that in the future one can split a single table into different table spaces? Like storing all rows with year 1999 in one tablespace and the rest in

Re: [HACKERS] [pgsql-www] Collaboration Tool Proposal

2004-02-26 Thread Josh Berkus
Tom, Possibly workable, but what's your definition of registered user? Signing up via a webform, getting an e-mailed password back, logging in. I'd hope that anyone subscribed to any of the mailing lists would be considered registered, for instance. Not sure if we can do that with either

Re: [HACKERS] Thread safe connection-name mapping in ECPG. Is it required?

2004-02-26 Thread Michael Meskes
On Mon, Feb 23, 2004 at 09:27:40PM +0530, Shridhar Daithankar wrote: It looks like the mutex protects the connections list in connection.c. I do not like that from a application developers perspective. If I am developing an application and using multiple connections in multiple threads, I

Re: [HACKERS] CVS HEAD compile warning

2004-02-26 Thread Michael Meskes
On Thu, Feb 26, 2004 at 02:44:52AM -0500, Tom Lane wrote: I knew at the time that ecpg was the only one of our lexers in which echo-to-stdout could conceivably be a reasonable default rule. But since flex 2.5.4 did not complain, I went ahead and committed the addition in ecpg as well as