[HACKERS] Another pg_dump bug
If you ALTER USER the cluster owner (eg. 'postgres') to set a user GUC variable, it is not dumped in any way. I see that we're really trying to avoid referring to the cluster owner by name, but shall I just go ahead and fix it by making it output an ALTER USER for the cluster owner in this case? Chris ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] bug in pg_dump ALTER DATABASE
As part of my testing, I noticed this bug. My database has a search_path set in the database vars. It dumps lik ethis: DROP DATABASE usa; CREATE DATABASE usa WITH TEMPLATE = template0 OWNER = usadmin ENCODING = 'LATIN1'; ALTER DATABASE usa SET search_path TO 'public, contrib'; Notice the single quotes around the TO bit? That's completely broken. Those '' must not be there. Is a fix for this required for only search_path, or is it a more general problem? So what are we going to do about this problem? The pg_settings view does not have enough information to determine it generically. (It only says 'string', not 'list'...) I propose that we modify pg_dumpall to hard-code the set of list-type GUC variables for each backend version. The current (CVS) list of such GUCs is: * DateStyle * preload_libraries * search_path * log_destination * custom_variable_classes (probably doesn't need to be worried about) Shall I go ahead and do this? Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Is trust really a good default?
No, but none of the others are better. See previous discussions in the archives. I don't think the situation has changed any since the last time we hashed this out. If they supply a password to initdb, shouldn't we then require a password in pg_hba.conf. This is further to my previous suggestion that we output the encoding that is being defaulted to. NEW USERS DO NOT KNOW THAT -W EXISTS! Even the majority of experienced users don't! Exactly... How about requiring them to put in *either* -W (or --pwfile of course) *or* a switch that *explicitly* enables trust. And if they don't put in either of these parameters, refuse to initdb. (are other params required?) That will at least require a concious decision to enable the unsafe stuff. And packagers/distributions can add that trust switch if it's necessary by their packaging system (which arguably is not very flexible if it does, but I assume this is the reason why the default wasn't changed - can't find the old discussions in the archives) This will require initdb to edit pg_hba.conf on the fly and not just copy it in, but that shuoldn't be too hard. //Magnus ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] bug in pg_dump ALTER DATABASE
So what are we going to do about this problem? The pg_settings view does not have enough information to determine it generically. (It only says 'string', not 'list'...) I propose that we modify pg_dumpall to hard-code the set of list-type GUC variables for each backend version. The current (CVS) list of such GUCs is: * DateStyle * preload_libraries * search_path * log_destination * custom_variable_classes (probably doesn't need to be worried about) Shall I go ahead and do this? Oh, and we'll need to fix the pg_settings view for the future, because otherwise it will make life difficult for GUI writies (like me)... Chris ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Is trust really a good default?
IMO, forcing su password at initdb time (allowing blank password with a very stern warning) and bumping localhost to auth is the right way to go. This isn't happening for a number of reasons, the most obvious being that we cannot require initdb to be run interactively. (That stern warning will not impress /dev/null.) This is the very reason --pwfile was added. It's not just a win32 fix, it's a any packager that needs to run without interactivity fix. Yes, you can stick a blank password in there, but again, this is a choice and not a default in that case. //Magnus ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Is trust really a good default?
It has probably be said before, but new users need to be able to get up and running on their *development* system quickly. Throwing new users for a loop with the password configuration issues would be a problem. This is exactly the argument that was thrown out when people wanted to be able to run their development backends as an admin account on Windows.. These users are thrown into setting up new accounts etc, which is a lot more work than just setting a superuser password in the db. Most people would put up a test server first anyway in order to check things out and configure as necessary. See above. The only difference is that one lets yuo root the machine, the other one lets you root the database. Sure, the machine is worse, but not *that* much worse. //Magnus ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Is trust really a good default?
Magnus Hagander wrote: This is the very reason --pwfile was added. It's not just a win32 fix, it's a any packager that needs to run without interactivity fix. Yes, you can stick a blank password in there, but again, this is a choice and not a default in that case. No, that's not what it was added for. It was added for catering to packaging mechanisms that have other interfaces for interactivity. But just hardcoding a default password into a package gains no security at all. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Anoncvs down?
On Tue, 2004-07-13 at 02:44, Marc G. Fournier wrote: temporarily while I figure out what I screwed up that allowed a hacker to make use of he anoncvs account :( and, no, anoncvs doesn't have access to the core cvsroot ... On Tue, 13 Jul 2004, Christopher Kings-Lynne wrote: -bash-2.05b$ cvs up cvs update: authorization failed: server anoncvs.postgresql.org rejected access to /projects/cvsroot for user anoncvs Still down ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Point in Time Recovery
On Tue, 2004-07-06 at 22:40, Simon Riggs wrote: On Mon, 2004-07-05 at 22:46, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: - when we stop, keep reading records until EOF, just don't apply them. When we write a checkpoint at end of recovery, the unapplied transactions are buried alive, never to return. - stop where we stop, then force zeros to EOF, so that no possible record remains of previous transactions. Go with plan B; it's best not to destroy data (what if you chose the wrong restart point the first time)? Actually this now reminds me of a discussion I had with Patrick Macdonald some time ago. The DB2 practice in this connection is that you *never* overwrite existing logfile data when recovering. Instead you start a brand new xlog segment file, which is given a new branch number so it can be distinguished from the future-time xlog segments that you chose not to apply. I don't recall what the DB2 terminology was exactly --- not branch number I don't think --- but anyway the idea is that when you restart the database after an incomplete recovery, you are now in a sort of parallel universe that has its own history after the branch point (PITR stop point). You need to be able to distinguish archived log segments of this parallel universe from those of previous and subsequent incarnations. I'm not sure whether Vadim intended our StartUpID to serve this purpose, but it could perhaps be used that way, if we reflected it in the WAL file names. Some more thoughts...focusing on the what do we do after we've finished recovering. The objectives, as I see them, are to put the system into a state, that preserves these features: 1. we never overwrite files, in case we want to re-run recovery 2. we never write files that MIGHT have been written previously 3. we need to ensure that any xlog records skipped at admins request (in PITR mode) are never in a position to be re-applied to this timeline. 4. ensure we can re-recover, if we need to, without further problems Tom's concept above, I'm going to call timelines. A timeline is the sequence of logs created by the execution of a server. If you recover the database, you create a new timeline. [This is because, if you've invoked PITR you absolutely definitely want log records written to, say, xlog15 to be different to those that were written to xlog15 in a previous timeline that you have chosen not to reapply.] Objective (1) is complex. When we are restoring, we always start with archived copies of the xlog, to make sure we don't finish too soon. We roll forward until we either reach PITR stop point, or we hit end of archived logs. If we hit end of logs on archive, then we switch to a local copy, if one exists that is higher than those, we carry on rolling forward until either we reach PITR stop point, or we hit end of that log. (Hopefully, there isn't more than one local xlog higher than the archive, but its possible). If we are rolling forward on local copies, then they are our only copies. We'd really like to archive them ASAP, but the archiver's not running yet - we don't want to force that situation in case the archive device (say a tape) is the one being used to recover right now. So we write an archive_status of .ready for that file, ensuring that the checkpoint won't remove it until it gets copied to archive, whenever that starts working again. Objective (1) met. When we have finished recovering we: - create a new xlog at the start of a new ++timeline - copy the last applied xlog record to it as the first record - set the record pointer so that it matches That way, when we come up and begin running, we never overwrite files that might have been written previously. Objective (2) met. We do the other stuff because recovery finishes up by pointing to the last applied record...which is what was causing all of this extra work in the first place. At this point, we also reset the secondary checkpoint record, so that should recovery be required again before next checkpoint AND the shutdown checkpoint record written after recovery completes is wrong/damaged, the recovery will not autorewind back past the PITR stop point and attempt to recover the records we have just tried so hard to reverse/ignore. Objective (3) met. (Clearly, that situation seems unlikely, but I feel we must deal with it...a newly restored system is actually very fragile, so a crash again within 3 minutes or so is very commonplace, as far as these things go). Should we need to re-recover, we can do so because the new timeline xlogs are further forward than the old timeline, so never get seen by any processes (all of which look backwards). Re-recovery is possible without problems, if required. This means you're a lot safer from some of the mistakes you might of made, such as deciding you need to go into recovery, then realising it wasn't required (or some other
Re: [HACKERS] Point in Time Recovery
The starting a new timeline thought works for xlogs, but not for clogs. No matter how far you go into the future, there is a small (yet vanishing) possibility that there is a yet undiscovered committed transaction in the future. (Because transactions are ordered in the clog because xids are assigned sequentially at txn start, but not ordered in the xlog where they are recorded in the order the txns complete). Won't ExtendCLOG take care of this with ZeroCLOGPage ? Else the same problem would arise at xid wraparound, no ? Andreas ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Point in Time Recovery
On Tue, 2004-07-13 at 13:18, Zeugswetter Andreas SB SD wrote: The starting a new timeline thought works for xlogs, but not for clogs. No matter how far you go into the future, there is a small (yet vanishing) possibility that there is a yet undiscovered committed transaction in the future. (Because transactions are ordered in the clog because xids are assigned sequentially at txn start, but not ordered in the xlog where they are recorded in the order the txns complete). Won't ExtendCLOG take care of this with ZeroCLOGPage ? Else the same problem would arise at xid wraparound, no ? Sounds like a good start... When PITR ends, we need to stop mid-way through a file. Does that handle that situation? Simon Riggs ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] possibly updating techdocs; mysql2pgsql on gborg
Josh Berkus wrote: Robert, Bruce, If anybody still has access to that page, the project has moved to gborg specifically over to http://gborg.postgresql.org/project/mysql2psql/projdisplay.php where a new version of the perl conversion script is located. There's one of these on GBorg too? I started on on pgFoundry because I made a bunch of changes to the script (which now need debugging). Since this one needs nothing but CVS, any objections to just migrating it? Makes sense. Please be sure they haven't made additions themselves. Also send me the new URL so I can update contrib. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] Assisting developers
One failing that has appeared during the 7.5 development cycle is that we as a community haven't been able to provide timely feedback to developers working on large feature additions. I am particularly thinking of Alvaro (nested transactions) and Simon (PITR), where we haven't been able to give them sufficient feedback to make them fully productive. I am not sure what can be done to solve this in the future. There are only a limited number of us who have the experience and time to review and comment on very complex patches. Hopefully this is just growing pains and the community will grow to the point where we can have more people focused on assisting developers adding complex features. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Is trust really a good default?
Magnus Hagander [EMAIL PROTECTED] writes: This isn't happening for a number of reasons, the most obvious being that we cannot require initdb to be run interactively. (That stern warning will not impress /dev/null.) This is the very reason --pwfile was added. No, that does not address the objection in the least. That simply allows a level of indirection. There still has to be user interaction going on somewhere to supply the password. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Assisting developers
One failing that has appeared during the 7.5 development cycle is that we as a community haven't been able to provide timely feedback to developers working on large feature additions. I am particularly thinking of Alvaro (nested transactions) and Simon (PITR), where we haven't been able to give them sufficient feedback to make them fully productive. I am not sure what can be done to solve this in the future. There are only a limited number of us who have the experience and time to review and comment on very complex patches. Hopefully this is just growing pains and the community will grow to the point where we can have more people focused on assisting developers adding complex features. Well, I note (and I'm not being unkind or anything here) that a lot of the high level committers we have haven't been so active this release. Peter and Joe haven't been around much and Jan has been busy with Slony. We also lost Thomas Lockhart. Neil's also away on holidays. You and Tom have basically been doing all the reviewing - a great job - but I can't believe Tom hasn't cracked yet :P I've been around for years, but I've never really gotten into the depths of things, so I'm not much use for checking complex patches, but I'm willing to review simpler stuff :) Maybe you should promote a new committer? (Not me!) Chris ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Point in Time Recovery
Simon Riggs [EMAIL PROTECTED] writes: Please tell me that we can ignore the state of the clog, We can. The reason that keeping track of timelines is interesting for xlog is simply to take pity on the poor DBA who needs to distinguish the various archived xlog files he's got laying about, and so that we can detect errors like supplying inconsistent sets of xlog segments during restore. This does not apply to clog because it's not archived. It's no more than a data file. If you think you have trouble recreating clog then you have the same issues recreating data files. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Is trust really a good default?
This isn't happening for a number of reasons, the most obvious being that we cannot require initdb to be run interactively. (That stern warning will not impress /dev/null.) This is the very reason --pwfile was added. No, that does not address the objection in the least. That simply allows a level of indirection. There still has to be user interaction going on somewhere to supply the password. Right. I realised that after posting. I still think it would be a good move to add a switch you have to explicitly put in there to make it use trust auth by default. This can spit out some warnings, which will of course be ignored by RPM and such packagers. But for those installs the package creator made the decisions, and we will still get most of the install-from-source-but-don't-know-about-the-switches people. It would, of course, be better if the RPMs could do this as well. Don'tk now how they work, though, but I assume this is the part that has been discussed before by people who do. Or we could initdb with requiring password, and just refuse logins with blank passwords. That way you get a system that can be installed, but you cannot actually connect to it until you have set a password. We'd need to supply a tool that could change the password without connecting (since you can't connect with no password, catch-22) but that should be fairly straightforward with a standalone backend, right? For reference, does anybody know how other databases do it? Like MySQL or Oracle (which both run on RedHat, which should mean RPMs, right?) I know MSSQL refuses to install with blank passwords these days (didn't use to be that way, no, which is why there are still a lot of installations out there with blank sa passwords). //Magnus ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Assisting developers
Christopher Kings-Lynne wrote: One failing that has appeared during the 7.5 development cycle is that we as a community haven't been able to provide timely feedback to developers working on large feature additions. I am particularly thinking of Alvaro (nested transactions) and Simon (PITR), where we haven't been able to give them sufficient feedback to make them fully productive. I am not sure what can be done to solve this in the future. There are only a limited number of us who have the experience and time to review and comment on very complex patches. Hopefully this is just growing pains and the community will grow to the point where we can have more people focused on assisting developers adding complex features. Well, I note (and I'm not being unkind or anything here) that a lot of the high level committers we have haven't been so active this release. Peter and Joe haven't been around much and Jan has been busy with Slony. We also lost Thomas Lockhart. Neil's also away on holidays. You and Tom have basically been doing all the reviewing - a great job - but I can't believe Tom hasn't cracked yet :P Excellent analysis. I've been around for years, but I've never really gotten into the depths of things, so I'm not much use for checking complex patches, but I'm willing to review simpler stuff :) Maybe you should promote a new committer? (Not me!) The committing isn't really the issue. It is reviewing and giving feedback to developers of complex features. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Is trust really a good default?
On Tue, 2004-07-13 at 11:03, Magnus Hagander wrote:
This isn't happening for a number of reasons, the most
obvious being
that we cannot require initdb to be run interactively.
(That stern
warning will not impress /dev/null.)
This is the very reason --pwfile was added.
No, that does not address the objection in the least. That
simply allows a level of indirection. There still has to be
user interaction going on somewhere to supply the password.
Right. I realised that after posting.
I still think it would be a good move to add a switch you have to
explicitly put in there to make it use trust auth by default. This can
spit out some warnings, which will of course be ignored by RPM and such
packagers. But for those installs the package creator made the
decisions, and we will still get most of the
install-from-source-but-don't-know-about-the-switches people.
It would, of course, be better if the RPMs could do this as well. Don'tk
now how they work, though, but I assume this is the part that has been
discussed before by people who do.
Or we could initdb with requiring password, and just refuse logins with
blank passwords. That way you get a system that can be installed, but
you cannot actually connect to it until you have set a password. We'd
need to supply a tool that could change the password without connecting
(since you can't connect with no password, catch-22) but that should be
fairly straightforward with a standalone backend, right?
For reference, does anybody know how other databases do it? Like MySQL
or Oracle (which both run on RedHat, which should mean RPMs, right?) I
know MSSQL refuses to install with blank passwords these days (didn't
use to be that way, no, which is why there are still a lot of
installations out there with blank sa passwords).
I am sure Chris would back me up on saying that the inability to
authenticate a database connection is the #1 support problem on the
phppgadmin mailing lists and you want to make this harder for
people??
I think we are obliged to provide security mechanisms that people can
use, and to make sure our software is a not a conduit of exploits for
the systems we're installed on, but forcing people to deploy the
software in a fasion that we deem acceptable for production use goes
above and beyond what we should be doing.
Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Release planning (was: Re: [HACKERS] Status report)
Jan Wieck wrote: On 7/10/2004 11:02 PM, Bruce Momjian wrote: If you get full control of PostgreSQL, you can dictate what will happen. Until then, I will follow the community consensus, which may or may not match your opinion. Marc isn't the only one who didn't like waiting for more features going into 7.5 two months ago. I agree that it is too late now to push things just out. But the mistake made wasn't ours. What this repeated discussion about release plans and schedules (and we have it every release) indicates to me is that we might miss something. As you pointed out elsewhere, a 9-12 month development cycle just isn't enough to get those big features done. But I think that stretching the release cycle to two years and holding back all the smaller features as well isn't a good idea either. I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. PITR could be done that way though. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Release planning (was: Re: [HACKERS] Status report)
On 7/10/2004 11:02 PM, Bruce Momjian wrote: If you get full control of PostgreSQL, you can dictate what will happen. Until then, I will follow the community consensus, which may or may not match your opinion. Marc isn't the only one who didn't like waiting for more features going into 7.5 two months ago. I agree that it is too late now to push things just out. But the mistake made wasn't ours. What this repeated discussion about release plans and schedules (and we have it every release) indicates to me is that we might miss something. As you pointed out elsewhere, a 9-12 month development cycle just isn't enough to get those big features done. But I think that stretching the release cycle to two years and holding back all the smaller features as well isn't a good idea either. I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Jan --- Marc G. Fournier wrote: On Sat, 10 Jul 2004, Bruce Momjian wrote: My point is that it isn't the patch submitters we are waiting on anymore. It is the backlog of patches that need review/adjustment. Of course, many of the patches I kept need some adjustment to get applied ... ... to me, that indicates that even after review, they will have to go back to the submitter to be adjusted, and sent back, and reviewed, and ... Get in what you feel comfortable adding over the next week, the rest can wait until 7.6 ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Nested Transactions, Abort All
On 7/10/2004 6:55 PM, Josh Berkus wrote: Bruce, It seems anonymous savepoints really don't buy us anything. They don't match the Oracle behavior, and don't do anything more than nested transactions. I agree we want them, but I don't see the value they add value right now. Anonymous Savepoints == Nested Transactions Almost This issue is whether we're going to use a PostgreSQL-specific, non-standard, syntax for NTs, or use a syntax that puts us on the road to implementing spec-compliant savepoints. Given that the functionality is exactly the same in either case, I don't see why you would want to implement syntax which is 100% Postgres-specific. I don't think they are 100% the same. The SQL3 spec defines in 7.15 and 13.4 that each sql procedure statement and each subquery on close implicitly destroy all savepoints that have been created during that statement or subquery. I am however certain that nested transactions do not offer any additional functionality that would not be available through savepoints. So what I am missing is the reason why we would want a non-standard syntax at all. Especially using the keyword BEGIN in the syntax would strike me as dumb, because it will create a parsing and reading nightmare for PL/pgSQL, since that language uses BEGIN ... END; for grouping statements like C uses curly braces. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Nested Transactions, Abort All
On 7/11/2004 12:22 AM, Alvaro Herrera wrote: There is not a lot of difference. This was allowed in nested transactions because we wanted the nesting be to OK when using it in a possibly aborted transaction block, so the user would not commit a transaction that could not have been created. In savepoints it's a nonissue because the command to end the outer xact is different. My opinion is that we should disallow both SAVEPOINT and RELEASE when in an aborted transaction block. Only ROLLBACK TO, ROLLBACK and COMMIT would be allowed. In this scenario, ROLLBACK TO would always return to a non-aborted transaction state, or the target savepoint would not exist and the state would remain the same. As I interpret the spec ROLLBACK TO foo will rollback all savepoints that have been created since savepoint foo was created including ones explicitly released. That means, that every subxid = foo is aborted, and a new foo subtransaction created. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Assisting developers
On Jul 13, 2004, at 17:02, Bruce Momjian wrote: One failing that has appeared during the 7.5 development cycle is that we as a community haven't been able to provide timely feedback to developers working on large feature additions. I am particularly thinking of Alvaro (nested transactions) and Simon (PITR), where we haven't been able to give them sufficient feedback to make them fully productive. I'm just a bystander here, but it seems to me that in-depth discussion of a feature only starts when someone realizes that he must speak now or the darn thing might get committed. In other words, the emphasis is placed in preventing something half-baked getting included. And that's perfectly natural because it is much easier and quicker than commenting thoughtfully on every idea that someone might come up with. But it of course means that the price of admission is a patch that poses a real risk of getting committed. From a pure resource utilization perspective, I don't see a way around this. There's not enough expertise of pgsql internals to go around. As long as that's the case, there will always be a barrier to entry. But a high-risk patch isn't the only thing that can get you over such a barrier; the only thing to control the distribution of this scarce resource. Cash comes to mind as an alternative. mk ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Assisting developers
Bruce Momjian wrote: I am not sure what can be done to solve this in the future. There are only a limited number of us who have the experience and time to review and comment on very complex patches. The issue as I see it is not reviewing patches, but defining features. Someone sets out to develop nested transactions, and three days after feature freeze we have the first large discussion about what nested transactions really are, what they are good for, and how they should work. Maybe next time think more about the old requirements, design, implementation, testing cycle. Of course people did post plans, status updates, etc., but maybe it wasn't enough, not clear enough, or something else. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Is trust really a good default?
Robert Treat [EMAIL PROTECTED] writes: I am sure Chris would back me up on saying that the inability to authenticate a database connection is the #1 support problem on the phppgadmin mailing lists and you want to make this harder for people?? The other thing that bothers me about this proposal is that password auth is certainly the least convenient-to-use auth method we have, and it encourages insecure practices like coding passwords right into access scripts. So I'm not pleased with the idea of making it the default. For local-access-only installations, either IDENT or socket-file-permissions-based access control is likely to be a much more usable choice, but I don't think we can usefully make either of those the default either. So it still comes down to the DBA having to make a conscious choice. If what you want to do is raise the visibility of the need to make that choice, we could do something like this: initdb --trust installs pg_hba.conf with TRUST local auth, same as now initdb with -W or --pwfile installs pg_hba.conf with MD5 local auth initdb with no relevant switch installs pg_hba.conf with REJECT local auth thus forcing the DBA to make some choice before he can do anything. We could also add initdb --ident to install with IDENT local auth, which would be a cleaner solution for the distros that are currently enforcing that policy via a patch to pg_hba.conf.sample. I suspect however that we'd wind up reverting the whole thing before we get out of beta, because one thing I guarantee you is there will be lots of complaints. The only part of this discussion that I'd really be prepared to buy into is the part about *if* you use -W or --pwfile, then set up pg_hba.conf with MD5 as the default auth (because that's probably what the user wants anyway). But otherwise I think we should leave initdb's behavior alone. I do not agree with trying to force people to use passwords. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Is trust really a good default?
At this stage, I would be happy adding --ident to enable only ident, and -W/--pwfile to enable only MD5, and allow initdb to default to full local access (with a warning printed that package builders would at least see). --- Tom Lane wrote: Robert Treat [EMAIL PROTECTED] writes: I am sure Chris would back me up on saying that the inability to authenticate a database connection is the #1 support problem on the phppgadmin mailing lists and you want to make this harder for people?? The other thing that bothers me about this proposal is that password auth is certainly the least convenient-to-use auth method we have, and it encourages insecure practices like coding passwords right into access scripts. So I'm not pleased with the idea of making it the default. For local-access-only installations, either IDENT or socket-file-permissions-based access control is likely to be a much more usable choice, but I don't think we can usefully make either of those the default either. So it still comes down to the DBA having to make a conscious choice. If what you want to do is raise the visibility of the need to make that choice, we could do something like this: initdb --trust installs pg_hba.conf with TRUST local auth, same as now initdb with -W or --pwfile installs pg_hba.conf with MD5 local auth initdb with no relevant switch installs pg_hba.conf with REJECT local auth thus forcing the DBA to make some choice before he can do anything. We could also add initdb --ident to install with IDENT local auth, which would be a cleaner solution for the distros that are currently enforcing that policy via a patch to pg_hba.conf.sample. I suspect however that we'd wind up reverting the whole thing before we get out of beta, because one thing I guarantee you is there will be lots of complaints. The only part of this discussion that I'd really be prepared to buy into is the part about *if* you use -W or --pwfile, then set up pg_hba.conf with MD5 as the default auth (because that's probably what the user wants anyway). But otherwise I think we should leave initdb's behavior alone. I do not agree with trying to force people to use passwords. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Release planning (was: Re: [HACKERS] Status report)
Bruce Momjian [EMAIL PROTECTED] writes: Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. I think the same would be true of almost any really large feature --- ifdefs all over the code base are just too much of a mess. To be honest I think that releasing more often isn't necessarily an appropriate goal for the project anymore. Five or six years ago we were doing a major (initdb-forcing) release every three or four months, and that was appropriate at the time, but the project has matured and our user population has changed. Look at how many people are still using 7.2 or 7.3. One major release a year may be about right now, because you can't get people to adopt new major revs faster than that anyway. Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Assisting developers
Chris, all: Well, I note (and I'm not being unkind or anything here) that a lot of the high level committers we have haven't been so active this release. Peter and Joe haven't been around much and Jan has been busy with Slony. We also lost Thomas Lockhart. Neil's also away on holidays. You and Tom have basically been doing all the reviewing - a great job - but I can't believe Tom hasn't cracked yet :P Actually, this is a *big* part of the problem. In 7.3, by unplanned circumstance, we got into a cycle of doing feature freeze and starting beta during the summer. This is a bad cycle -- the Europeans take their 3-week vacations, the Americans go to (and spend many hours preparing for) a bunch of conventions, Students go to internships, people take long weekends, get married, etc. ( This is probably why many American corporations end their fiscal year in midsummer; nothing is going to get done anyway, might as well clean up. ) The result is that we chronically have less manpower when just when we need everybody. And some of us end up spending 11 hours a day in front of the screen when we should be outside soaking up Vitamin D. Therefore, I propose that the next version either feature freeze in March or in October, but NOT in May-August. Which we do -- March or October -- can be based on an evaluation of the outstanding post-7.5 patches in January. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Release planning (was: Re: [HACKERS] Status report)
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. I think the same would be true of almost any really large feature --- ifdefs all over the code base are just too much of a mess. To be honest I think that releasing more often isn't necessarily an appropriate goal for the project anymore. Five or six years ago we were doing a major (initdb-forcing) release every three or four months, and that was appropriate at the time, but the project has matured and our user population has changed. Look at how many people are still using 7.2 or 7.3. One major release a year may be about right now, because you can't get people to adopt new major revs faster than that anyway. Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. Totally agree. I think the only downside to a longer release cycle is that features developed would take longer to get out to the public. Perhaps we need to start thinking of add-ons to existing releases such as an ARC or vacuum-delay add-on to the 7.4.X release. The patch would potentially have to be recreated for every minor release. I would also like to see some psql message that shows the add-ons added to an official release. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Assisting developers
Josh Berkus wrote: Chris, all: Well, I note (and I'm not being unkind or anything here) that a lot of the high level committers we have haven't been so active this release. Peter and Joe haven't been around much and Jan has been busy with Slony. We also lost Thomas Lockhart. Neil's also away on holidays. You and Tom have basically been doing all the reviewing - a great job - but I can't believe Tom hasn't cracked yet :P Actually, this is a *big* part of the problem. In 7.3, by unplanned circumstance, we got into a cycle of doing feature freeze and starting beta during the summer. This is a bad cycle -- the Europeans take their 3-week vacations, the Americans go to (and spend many hours preparing for) a bunch of conventions, Students go to internships, people take long weekends, get married, etc. ( This is probably why many American corporations end their fiscal year in midsummer; nothing is going to get done anyway, might as well clean up. ) The result is that we chronically have less manpower when just when we need everybody. And some of us end up spending 11 hours a day in front of the screen when we should be outside soaking up Vitamin D. Therefore, I propose that the next version either feature freeze in March or in October, but NOT in May-August. Which we do -- March or October -- can be based on an evaluation of the outstanding post-7.5 patches in January. True, but northern-hemisphere summer is only 6 weeks old, and we have had these issues for many months --- it isn't a new problem. Alvaro didn't get the feedback he needed in March either. :-( -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Assisting developers
Peter Eisentraut [EMAIL PROTECTED] writes: The issue as I see it is not reviewing patches, but defining features. Someone sets out to develop nested transactions, and three days after feature freeze we have the first large discussion about what nested transactions really are, what they are good for, and how they should work. Bear in mind though that what we have here is a huge discussion about something that represents much less than 1% of the work involved in the feature. The hard part of nested transactions (or savepoints or whatever you care to call 'em) is the implementation support for reverting the backend's state to an earlier point without going all the way back to ground-zero-idle state. Alvaro's naturally spent most of his time on the implementation, because without that there is no point in debating syntax. And it was the state of the implementation, not the API which was understood to be unfinished, that drove the decision about whether this was ready to be included in 7.5. If we end up backing this out of 7.5, it will be because the remaining implementation work doesn't get done, not because we are unable to agree on a syntax. (In which connection I'm a bit disturbed that Alvaro seems to be spending time arguing with people rather than continuing to work on internals...) regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Assisting developers
Bruce Momjian wrote: The result is that we chronically have less manpower when just when we need everybody. And some of us end up spending 11 hours a day in front of the screen when we should be outside soaking up Vitamin D. Therefore, I propose that the next version either feature freeze in March or in October, but NOT in May-August. Which we do -- March or October -- can be based on an evaluation of the outstanding post-7.5 patches in January. True, but northern-hemisphere summer is only 6 weeks old, and we have had these issues for many months --- it isn't a new problem. Alvaro didn't get the feedback he needed in March either. :-( Oh, and neither did Simon. :-( In fact Simon coded a whole client-side approach to PITR before I studied his ideas and realized it needs to be done as a server subprocess. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. I think the only downside to a longer release cycle is that features developed would take longer to get out to the public. Perhaps we need to start thinking of add-ons to existing releases such as an ARC or vacuum-delay add-on to the 7.4.X release. Mmm ... for people whose pain-point has to do with compatibility testing, adding features in minor releases would turn them into major releases, because they'd still have to wonder whether the new version would break anything. I think our policy of only bug fixes in stable releases is important to maintain, because it makes it easy for people to upgrade within a stable release series. Also, we do not really have the manpower to test multiple concurrently developed versions ... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. I think the only downside to a longer release cycle is that features developed would take longer to get out to the public. Perhaps we need to start thinking of add-ons to existing releases such as an ARC or vacuum-delay add-on to the 7.4.X release. Mmm ... for people whose pain-point has to do with compatibility testing, adding features in minor releases would turn them into major releases, because they'd still have to wonder whether the new version would break anything. I think our policy of only bug fixes in stable releases is important to maintain, because it makes it easy for people to upgrade within a stable release series. Also, we do not really have the manpower to test multiple concurrently developed versions ... When I say add-ons, I am thinking of source code patches that are avaiable as options to the standard subreleases. I agree we don't want to change our current subrelease criteria. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Assisting developers
On Tue, 13 Jul 2004, Bruce Momjian wrote: True, but northern-hemisphere summer is only 6 weeks old, and we have had these issues for many months --- it isn't a new problem. Alvaro didn't get the feedback he needed in March either. :-( This is true, but Josh does have a point ... you took a much needed 12 days off end of June, Tom took a few days in July ... both in middle of the feature freeze ... if we were in 'dev mode' through the summer months, those wouldn't have been as critical, but even once you got back, you had, what, 2k messages to weed through? At least up in Canada, our summers are so short, we try and squeeze as much out of it as possible, so our time is more split then the rest of the year when we tend to stay indoors alot more ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Assisting developers
Marc G. Fournier wrote: On Tue, 13 Jul 2004, Bruce Momjian wrote: True, but northern-hemisphere summer is only 6 weeks old, and we have had these issues for many months --- it isn't a new problem. Alvaro didn't get the feedback he needed in March either. :-( This is true, but Josh does have a point ... you took a much needed 12 days off end of June, Tom took a few days in July ... both in middle of the feature freeze ... if we were in 'dev mode' through the summer months, those wouldn't have been as critical, but even once you got back, you had, what, 2k messages to weed through? At least up in Canada, our summers are so short, we try and squeeze as much out of it as possible, so our time is more split then the rest of the year when we tend to stay indoors alot more ... True. Things for the past few weeks have been worse, but we are geting though that quickly. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Assisting developers
On Tue, 13 Jul 2004, Bruce Momjian wrote: Marc G. Fournier wrote: On Tue, 13 Jul 2004, Bruce Momjian wrote: True, but northern-hemisphere summer is only 6 weeks old, and we have had these issues for many months --- it isn't a new problem. Alvaro didn't get the feedback he needed in March either. :-( This is true, but Josh does have a point ... you took a much needed 12 days off end of June, Tom took a few days in July ... both in middle of the feature freeze ... if we were in 'dev mode' through the summer months, those wouldn't have been as critical, but even once you got back, you had, what, 2k messages to weed through? At least up in Canada, our summers are so short, we try and squeeze as much out of it as possible, so our time is more split then the rest of the year when we tend to stay indoors alot more ... True. Things for the past few weeks have been worse, but we are geting though that quickly. Granted, but that wasn't Josh's point :) The point is, a March/October feature freeze would (should) have alot less issues as far as ppls time is concerned then a summer one (either Southern or Northern hemisphere summer) ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [pgsql-www] Problems logging into CVS server
Marc G. Fournier wrote: Damn ... I'll have to look at it ... we had a hacker get in through the way anoncvs was setup, so I set a passwd on in /etc/passwd (but didn't touch the anoncvs setup itself) ... will play with it tonight and see if I can figure out how to do a more secure anon-cvs ;( I have to be missing something in the config *sigh* Um, that sounds worrying. Was the activity of the hacker anything that would affect PG code, or access to anything sensitive (account passwords, etc)? Regards and best wishes, Justin Clift ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] plperl (7.5)
posted mailed
Tom Lane wrote:
Alvaro Herrera [EMAIL PROTECTED] writes:
On Sat, Jul 10, 2004 at 09:18:28PM -0700, elein wrote:
The new plperl returns sets by having
the function return an array.
I think RETURN NEXT does the same thing anyway ... they just store
tuples in a Tuplestore and then the whole thing is returned. So the
function actually doesn't return until the whole function is done.
However, it's likely that the tuplestore infrastructure can deal
comfortably with sets far larger than a Perl array can. (For one thing,
it will swap tuples out to a temp file on disk once the set size exceeds
work_mem.) I think elein's concern is justified, unless someone can
produce a test case showing that plperl actually performs OK with a
large result set.
As a simple test for plpgsql's speed with such things, I tried
create function seq(int) returns setof int as '
begin
for i in 1..$1 loop
return next i;
end loop;
return;
end' language plpgsql;
regression=# \timing
Timing is on.
regression=# select count(*) from seq(10);
count
10
(1 row)
Time: 396.524 ms
regression=# select count(*) from seq(100);
count
-
100
(1 row)
Time: 3615.115 ms
regression=# select count(*) from seq(1000);
count
--
1000
(1 row)
Time: 40356.972 ms
My Perl is too rusty to immediately whip out the equivalent incantation
in plperl; would someone like to compare the timings on their own machine?
I don't have access to a machine with plperl installed, but it would be very
close to this:
create function seq(int) returns setof int as $$
my $count = shift;
my $ret = [];
for my $i ( 1 .. $count ) {
push @$ret, $i;
}
return $ret;
$$ language 'plperl';
... hmmm... the push line may need to be:
push @$ret, { val = $i };
Hope it helps!
regards, tom lane
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
--
--miker
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [HACKERS] Nested Transactions, Abort All
posted mailed
Dennis Bjorklund wrote:
On Sat, 10 Jul 2004, Mike Rylander wrote:
They do, if only to make particular constructs easier to write. This is
an opinion, but for example an EXCEPTION framework for plpgsql would be
easier to implement and use if it used the nested transactions rather
than savepoint syntax:
CREATE FUNCTION blah () RETURNS INT LANGUAGE PLPGSQL AS '
BEGIN
BEGIN NESTED;
do some work...
BEGIN NESTED;
do other work...
EXCEPTION WHEN SQLSTATE = already_exists THEN
do alternate work with its own error checking...
END NESTED;
EXCEPTION WHEN SQLSTATE = fkey_violation THEN
ROLLBACK NESTED;
END NESTED;
END;';
I realize this can be done with nested savepoints and that is what the
spec requires,
Lets look at what it can look like:
BEGIN
SAVEPOINT nested;
do some work...
SAVEPOINT nested2;
do other work...
EXCEPTION WHEN SQLSTATE = already_exists THEN
ROLLBACK TO SAVEPOINT nested2;
do alternate work with its own error checking...
RELEASE nested2;
EXCEPTION WHEN SQLSTATE = fkey_violation THEN
ROLLBACK TO SAVEPOINT nested;
RELEASE nested;
END;
Now, in what way is this more complicated?
Only in that you need to define a name for each savepoint in order to create
the hierarchy. And that is my point, savepoints impose more work on the
user to create a logical hierarchy, not that they cannot be used for
hierarchical structures.
I'm not 100% sure how the exceptions that you used above work. Do that
always rollback the transaction thay are in? In one of the exceptions you
did a rollback but not in the other. In my example I added a rollback in
the first exception handler. Maybe you forgot it there?
That was just pseudo-code and wholly invented in my head, but based on an
earlier expample of possible EXCEPTION syntax. The idea is that when a
subtransaction is in an aborted state due to an error the EXCEPTION clause
would implicitly roll back that subtransaction and open a new transaction
from its own block. This EXCEPTION subtrans is only used in the case of an
error in the matching BEGIN NESTED block, and the two share the COMMIT
statement, syntacticly speaking. Think of it as a try { ... } catch
[type] { ... } finally { commit } type structure.
In any case. I don't see this as any harder then your example.
It's not harder, per se, but it does impose a more difficult to maintain
syntax, IMHO.
Savepoints have more possibilities, you can invalidate older savepoints
then the last (with subtransactions you can only commit/rollback the
last).
This implies that savepoints are flat. It won't be that way under the
covers, but it does give that impression, and flat savepoint space is
definitely suited to a different class of problems than nested
transactions.
First, my claim above was wrong. As Gavin pointed out in another mail, if
one have savepoints p1 and p2 and release p1 then also p2 is released.
It's possible to implement both kinds of behaviour using Alvaros work, but
the standard demands the simpler one where p2 is also released.
Now, about the flatness. Savepoints are not flat. They are sort of flat in
a savepoint level. But, for example when you call a function you get a new
savepoint level. I actually don't want to call it flat at all. The example
above does not overwrite the savepoints nested and nested2 that might
exist before the call, since this is a higher savepoint level.
OK, savepoints are not REALLY flat, but they are not hierarchically nested
either. They are cumulative. They can be used, as you showed above, in a
hierarchy, but as I said, they are not by their nature nested.
I'm not sure exactly what it is that defines a new savepoint level, but a
function call does and maybe some other things.
As for savepoint levels in functions, that is a scoping issue imposed by the
functions themselves, not by the savepoint syntax. It would be nonsensical
to rollback to a savepoint outside a function, just as it would be
nonsensical to rollback the outer transaction from within the function.
Allowing either would cause undesired action at a distance and possibly
violate the A in ACID. The way I see it, savepoint levels should be
specified by function calls, as you said, and by the transaction nesting
level.
BTW, I would imagine that savepoints will be implemented as nested
transactions with detachable labels... the label can move from a
transaction to one of its descendants, and that outer (sub)transaction
will be implicitly COMMITed with its parent.
Yes. That's my view as well.
Well, at least we agree on that ;)
Alvaro found it
Re: [HACKERS] Nested Transactions, Abort All
posted mailed Dennsnippetssklund wrote: On Fri, 9 Jul 2004, Mike Rylander wrote: Nested transactions and savepoints serve two different purposes. They have some overlap, but for the most part solve two distinct problems. Then show some examples that illustrait the difference. So far all examples shown that uses subtransactions could just as well have been written using savepoints. After seeing some more snippets of the SQL2003 spec it seems that this is true, and that there is more of a syntactic difference than functional. This does not seem to be the case for Oracle (the other major implementation that has been cited for SAVEPOINT syntax), as savepoints in Oracle are not logically nested. Note that I am going on the statements from others on this list for this point... I don't agree that they have two different purposes. They do, if only to make particular constructs easier to write. This is an opinion, but for example an EXCEPTION framework for plpgsql would be easier to implement and use if it used the nested transactions rather than savepoint syntax: CREATE FUNCTION blah () RETURNS INT LANGUAGE PLPGSQL AS ' BEGIN BEGIN NESTED; do some work... BEGIN NESTED; do other work... EXCEPTION WHEN SQLSTATE = already_exists THEN do alternate work with its own error checking... END NESTED; EXCEPTION WHEN SQLSTATE = fkey_violation THEN ROLLBACK NESTED; END NESTED; END;'; I realize this can be done with nested savepoints and that is what the spec requires, but in other major implementations of savepoints this nested exception handling would be more difficult to write. Again, simply my opinion. I don't think so, especially as there has been some talk of implementing savepoints as a subset of nested transactions. It is not a subset. It's the other way around. Nested transactions are a subset of savepoints Perhaps I got my wires crossed a bit there. And again, after looking at some more of the SQL2003 spec this does seem to be the case. I cry your pardon! :) Savepoints have more possibilities, you can invalidate older savepoints then the last (with subtransactions you can only commit/rollback the last). This implies that savepoints are flat. It won't be that way under the covers, but it does give that impression, and flat savepoint space is definitely suited to a different class of problems than nested transactions. If you don't use that then it's exactly the same as subtransactions. I don't see this. Nested transactions present a hierarchal interface to the user, savepoints don't, especially considering that those familiar with PL/SQL know that savepoints are not nested. Now, savepoints can be used IN a hierarchy, but they do not DEFINE one as nested transactions do. I look at it this way: Let's have both, and where a user wants a flat transaction space, as that may suit the needs of the problem, they will use SAVEPOINT syntax; if the user would perfer an explicit hierarchy they can use nested transactions. Everyone wins! The only feature subtransactions have that savepoints doesn't is the lack of names. Every savepoint have a name. If we want an extension it could be to get the database to generate a fresh savepoint name. The client can of course also generate unique savepoint names if it want. I don't think they can be compared like that, feature for feature. Although I agree with you that they provide extremely similar feature sets, the present different interfaces to the user. They may end up being backed by the exact same code but the syntax and logical structure will surely differ, and when a user wants labeled rollback point they will use savepoints. When s/he wants hierarchical rollback points they will use the nested transactions syntax. BTW, I would imagine that savepoints will be implemented as nested transactions with detachable labels... the label can move from a transaction to one of its descendants, and that outer (sub)transaction will be implicitly COMMITed with its parent. That subtransactions do more than savepoints is just smoke an mirrors. So far there have been no example to validate that point of view, and I don't think there will be any. If anyone know of something that you can do with subtransactions and not with savepoints, please speak up. You have opened my eyes to the fact that savepoints and nested transactions can be used for most of the same problems, however I don't see this as a one-or-the-other proposition. Alvaro found it easier to implement nested transactions, he forged ahead and did it. Now, because of good design or simple luck, we should be able to implement savepoints fairly easily. To me this is the best we could have hoped for, as it means that not only will be support the entire SQL2003 spec WRT savepoints, we actually get to present a richer
Re: [HACKERS] Assisting developers
Josh Berkus wrote: ( This is probably why many American corporations end their fiscal year in midsummer; nothing is going to get done anyway, might as well clean up. ) And that is also the time when volunteers will have the most time to spare. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Proposal for detecting encoding mismatch in initdb
I've worked out a scheme that should adequately detect encoding mismatches in initdb. Please comment on the following behavior. The locale is still taken from the environment or the command line; no change. If the locale is C or POSIX, then we set the encoding to SQL_ASCII or whatever was specified on the command line, and do nothing further. (No useful matching can be done in this case.) If the locale is not C or POSIX: If the encoding is specified, check for compatibility. If not compatible, print a warning. Continue in any case. If the encoding was not specified, pick a matching one, print it out, continue. (This is probably the most usual case.) If no matching encoding could be found, print an error message asking the user to set one explicitly. Here are some screenshots: $ initdb -D pg-install/var/data [EMAIL PROTECTED] [...] The database cluster will be initialized with locale [EMAIL PROTECTED] The default database encoding has accordingly been set to LATIN9. $ initdb -D pg-install/var/data [EMAIL PROTECTED] --encoding=UNICODE [...] The database cluster will be initialized with locale [EMAIL PROTECTED] initdb: warning: encoding mismatch The encoding you selected (UNICODE) and the encoding that the selected locale uses (ISO-8859-15) are not known to match. This may lead to misbehavior in various character string processing functions. To fix this situation, rerun initdb and either do not specify an encoding explicitly, or choose a matching combination. [continues...] $ initdb -D pg-install/var/data --locale=japanese.sjis [...] The database cluster will be initialized with locale japanese.sjis. initdb: could not find suitable encoding for locale japanese.sjis Rerun initdb with the -E option. Try initdb --help for more information. [exit 1] -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Is trust really a good default?
The only part of this discussion that I'd really be prepared to buy into is the part about *if* you use -W or --pwfile, then set up pg_hba.conf with MD5 as the default auth (because that's probably what the user wants anyway). But otherwise I think we should leave initdb's behavior alone. I do not agree with trying to force people to use passwords. Ok. Here is a patch that does this. I still think there should be a warning when trust is set, but I'm clearly not convincing enough about this. Might still be worth adding --ident as a parameter anyway, but in that case only to help the distros that need it. Or not, because they already have a way to deal with it. //Magnus initdb_pwd.patch Description: initdb_pwd.patch ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[HACKERS] Another locale test program
Regarding the encoding/locale matching issue, here's another test
routine I would like you to run if you want to see your operating
system supported. It reflects more accurately the actual
implementation I'm working on.
Compile the attached test program and then run
for x in `locale -a`; do LC_ALL=$x ./test; done | sort -u
If you don't have a locale command, maybe something like this will work:
for x in `ls /usr/share/locale`; do LC_ALL=`basename $x` ./test; done |
sort -u
I already have Linux and FreeBSD covered. Thanks.
(If the program doesn't compile or misbehaves, that would be useful
information as well.)
--
Peter Eisentraut
http://developer.postgresql.org/~petere/
#include stdio.h
#include locale.h
#include nl_types.h
#include langinfo.h
int
main(int argc, char *argv[])
{
char *foo;
setlocale(LC_ALL, );
foo = nl_langinfo(CODESET);
printf(%s\n, foo);
return 0;
}
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] Point in Time Recovery
On Tue, 2004-07-13 at 15:29, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Please tell me that we can ignore the state of the clog, We can. In general, you are of course correct. The reason that keeping track of timelines is interesting for xlog is simply to take pity on the poor DBA who needs to distinguish the various archived xlog files he's got laying about, and so that we can detect errors like supplying inconsistent sets of xlog segments during restore. This does not apply to clog because it's not archived. It's no more than a data file. If you think you have trouble recreating clog then you have the same issues recreating data files. I'm getting carried away with the improbablebut this is the rather strange, but possible scenario I foresee: A sequence of times... 1. We start archiving xlogs 2. We take a checkpoint 3. we commit an important transaction 4. We take a backup 5. We take a checkpoint As stands currently, when we restore the backup, controlfile says that last checkpoint was at 2, so we rollforward from 2 THRU 4 and continue on past 5 until end of logs. Normally, end of logs isn't until after 4... When we specify a recovery target, it is possible to specify the rollforward to complete just before point 3. So we use the backup taken at 4 to rollforward to a point in the past (from the backups perspective). The backup taken at 4 may now have data and clog records written by bgwriter. Given that time between checkpoints is likely to be longer than previously was the case...this becomes a non-zero situation. I was trying to solve this problem head on, but the best way is to make sure we never allow ourselves such a muddled situation: ISTM the way to avoid this is to insist that we always rollforward through at least one checkpoint to guarantee that this will not occur. ...then I can forget I ever mentioned the ** clog again. I'm ignoring this issue for nowwhether it exists or not! Best Regards, Simon Riggs ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Is trust really a good default?
Magnus Hagander wrote: The only part of this discussion that I'd really be prepared to buy into is the part about *if* you use -W or --pwfile, then set up pg_hba.conf with MD5 as the default auth (because that's probably what the user wants anyway). But otherwise I think we should leave initdb's behavior alone. I do not agree with trying to force people to use passwords. Ok. Here is a patch that does this. I still think there should be a warning when trust is set, but I'm clearly not convincing enough about this. I think there should be a warning. The warning will not be 100% effective, but I see no reason _not_ to give a warning. This is an ease-of-user issues which are usuaully not 100% but can be very helpful. Might still be worth adding --ident as a parameter anyway, but in that case only to help the distros that need it. Or not, because they already have a way to deal with it. I think --ident would be very helpful, and we know with OS's support ident too. Actually looking at the code, we need some way to define this so initdb would know if ident was a reasonable value for this OS: errmsg(Ident authentication is not supported on local connections on this platform))); Right now it is burried down inside a bunch of define tests. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Portals and nested transactions
I've been thinking about what to do with cursors in subtransactions. The problem really includes both cursors (created with DECLARE CURSOR) and portals (created with the V3-protocol Bind message) since they are the same kind of animal internally, namely a Portal. In previous discussion I think everyone agreed that we would like the following properties: 1. A Portal created within a successful (committed) subtransaction remains open and usable by the parent transaction, as well as by subsequent child subtransactions. 2. If a subtransaction uses (fetches from) a pre-existing Portal, the Portal state change persists after subxact commit. What was not totally settled was what to do on subtransaction abort: Q1: Should Portals successfully created within the failed subxact be closed? Or should they remain open? Q2: If the subxact changed the state of a pre-existing Portal, should that state change roll back? In particular, can a Close Portal operation roll back? Taking a transactional view means answering yes to both questions (so that all portal state returns to what it was at subxact entry). But there was also support for a nontransactional view in which both questions are answered no. The discussion sort of trailed off there because we had no ideas how to implement either. I will now sketch some implementation ideas about how to do the nontransactional way. We could support the transactional behavior as well, but not very efficiently (at least not in the first cut). An important limitation that I think we must make is that any error occurring while a specific Portal is executing kills that Portal; you cannot do anything further with it except close it, even if the Portal would otherwise have survived the subtransaction abort caused by the error. The reason for this is that we can't be sure we have consistent internal state for the Portal when an error occurred at a random point. (Example: a btree index scan could have released lock on one buffer and gotten an error while trying to read the next page of the index; it's not certain that the scan data structures accurately reflect this intermediate state.) Later on we might be able to relax this restriction, but it will take a lot of care to decide which errors are safe. So for the moment, an error during a FETCH (or protocol-level Execute) leaves that Portal in a state where any subsequent fetch or execute draws ERROR: portal execution cannot be continued. How to do it non-transactionally The key insight I had while thinking about this is that subtransactions are the wrong units for managing ownership of resources used by queries (buffers, locks, etc). When portals can outlive subtransactions, those resources really need to be thought of as belonging to the portals not the subtransactions. However I think we *also* need to allow subtransaction to own resources --- at least locks. We usually want to hold table locks until main transaction end, and it would be bad to have to keep Portals around just to remember some locks. It would be better to reassign ownership of the locks to the current transaction when a Portal is closed. What I think we ought to do to support this is to invent the concept of ResourceOwner objects, which will be very much like MemoryContexts except that they represent held buffer pins, table locks, and anything else that we decide needs to be managed in this fashion (rtree index scans are one example). In particular we'll let ResourceOwners have child ResourceOwner objects so that there can be forests of the things, just as with MemoryContexts. There would be a CurrentResourceOwner global variable analogous to CurrentMemoryContext, which would for instance tell PinBuffer which ResourceOwner to affix ownership of the pin to. (I am half tempted to unify ResourceOwners and MemoryContexts completely, but that's probably overkill, since we have many short-lived MemoryContexts that would never be appropriate owners of query-level resources. In particular I think CurrentResourceOwner would usually be different from CurrentMemoryContext.) Depending on the resource in question, we could let ResourceOwners point to owned objects (for instance, I'm thinking of storing an array of Buffer numbers in a ResourceOwner to represent buffer pins) or vice versa (for instance, the best way to keep track of rtree indexscan ownership is probably to store a pointer to the ResourceOwner object in the rtree indexscan struct). This infrastructure shouldn't be much work to create, since we have the MemoryContext stuff available to serve as a model. Once we have it, we'll create a ResourceOwner for each transaction or subtransaction as well as one for each Portal. (We need one for a transaction because, for example, query parsing requires buffer and lock access, and that happens before we create a Portal to execute the query.) The reason this solves our problems is that while executing a portal's query,
Re: Release planning (was: Re: [HACKERS] Status report)
Note that I'm all for a long (2 year? or even 'indefinite') development cycle for a major release if we continue on with what has been happening more often lately, where stuff is back patched to the last stable release ... there is alot of stuff that doesn't require a reload of the database (or initdb) that could very easily be backpatched ... As Jan points out, its the 'small features that are done' that we've been griping about having to wait for, not the big ones which we know aren't done ... The nice thing about doing something lke that is those small features would get a degree of testing happening in a live environment ... On Tue, 13 Jul 2004, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. I think the same would be true of almost any really large feature --- ifdefs all over the code base are just too much of a mess. To be honest I think that releasing more often isn't necessarily an appropriate goal for the project anymore. Five or six years ago we were doing a major (initdb-forcing) release every three or four months, and that was appropriate at the time, but the project has matured and our user population has changed. Look at how many people are still using 7.2 or 7.3. One major release a year may be about right now, because you can't get people to adopt new major revs faster than that anyway. Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. regards, tom lane Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: Note that I'm all for a long (2 year? or even 'indefinite') development cycle for a major release if we continue on with what has been happening more often lately, where stuff is back patched to the last stable release ... there is alot of stuff that doesn't require a reload of the database (or initdb) that could very easily be backpatched ... As Jan points out, its the 'small features that are done' that we've been griping about having to wait for, not the big ones which we know aren't done ... The nice thing about doing something lke that is those small features would get a degree of testing happening in a live environment ... Of course that last sentence is the downside too --- people don't want to have to retest their setups after a minor release upgrade. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Is trust really a good default?
Magnus Hagander [EMAIL PROTECTED] writes: The only part of this discussion that I'd really be prepared=20 to buy into is the part about *if* you use -W or --pwfile, then set up pg_hba.conf with MD5 as the default auth (because that's probably what the user wants anyway). Ok. Here is a patch that does this. ... and rather severely mangles the comments, too; not to mention the more basic problem that the comments will now be wrong. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Anoncvs down?
should be fixed now ... please let me know if it isn't working (or if you notice any other problems) ... upgrade to 1.11.17 of CVS is complete ... On Tue, 13 Jul 2004, Simon Riggs wrote: On Tue, 2004-07-13 at 02:44, Marc G. Fournier wrote: temporarily while I figure out what I screwed up that allowed a hacker to make use of he anoncvs account :( and, no, anoncvs doesn't have access to the core cvsroot ... On Tue, 13 Jul 2004, Christopher Kings-Lynne wrote: -bash-2.05b$ cvs up cvs update: authorization failed: server anoncvs.postgresql.org rejected access to /projects/cvsroot for user anoncvs Still down Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Is trust really a good default?
The only part of this discussion that I'd really be prepared=20 to buy into is the part about *if* you use -W or --pwfile, then set up pg_hba.conf with MD5 as the default auth (because that's probably what the user wants anyway). Ok. Here is a patch that does this. ... and rather severely mangles the comments, too; Um, no, it doesn't. At least not on my installation. not to mention the more basic problem that the comments will now be wrong. That, however, it is correct :-( Sloppy. How about a text along the line of: CAUTION: Configuring the system for trust authentication allows any local user to connect using any PostgreSQL user name, including the superuser, over either Unix domain sockets or TCP/IP. If you are on a multiple-user machine, this is probably not good. Change it to use something other than trust authentication. Or something along that line? Since it would no longer actually be default. Or do we want something like On some installations, the default is...? //Magnus ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Point in Time Recovery
Simon Riggs [EMAIL PROTECTED] writes: I'm getting carried away with the improbablebut this is the rather strange, but possible scenario I foresee: A sequence of times... 1. We start archiving xlogs 2. We take a checkpoint 3. we commit an important transaction 4. We take a backup 5. We take a checkpoint When we specify a recovery target, it is possible to specify the rollforward to complete just before point 3. No, it isn't possible. The recovery *must* proceed at least as far as wherever the end of the log was at the time the backup was completed. Otherwise everything is broken, not only clog, because you may have disk blocks in your backup that postdate where you stopped log replay. To have a consistent recovery at all, you must replay the log starting from a checkpoint before the backup began and extending to the time that the backup finished. You only get to decide where to stop after that point. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Is trust really a good default?
Bruce Momjian [EMAIL PROTECTED] writes: Magnus Hagander wrote: Might still be worth adding --ident as a parameter anyway, but in that case only to help the distros that need it. Or not, because they already have a way to deal with it. I think --ident would be very helpful, and we know with OS's support ident too. If we're going to be doing sed-like substitutions on pg_hba.conf.sample, then we really really wanna discourage distros from hacking the sample file directly, because that could break the sed results. So I think it's important to provide the switch. I was toying with the notion of a different editing mechanism though, so that initdb could emit a pg_hba.conf containing comments that are actually pertinent to the selected behavior. One simple way would be to prefix each line with a keyword to select when to emit it: ALWAYS this text is always emitted NEVER this text is never emitted (a meta-comment) TRUST this text is emitted if we're selecting TRUST mode IDENT this text is emitted if we're selecting IDENT mode etc. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Release planning (was: Re: [HACKERS] Status report)
On Tuesday 13 July 2004 17:12, Marc G. Fournier wrote: ... there is alot of stuff that doesn't require a reload of the database (or initdb) that could very easily be backpatched ... As Jan points out, its the 'small features that are done' that we've been griping about having to wait for, not the big ones which we know aren't done ... Split the feature out into either a patch or a module and put it in contrib until the next major version. Let contrib hold the smaller, non-initdb-forcing patches and such until the next major version rolls them into the mainline. Or even a patches tree parallel to contrib. Either way, the patch or module or whatever wouldn't be in the mainline unless the user needed it. Or maybe we need to rethink what a major version is. But even minor things can force an initdb, although we're better than we have been about this. But Tom's assertion is true. We have enough trouble getting patches rolled out; adding parallel branches is just begging for trouble, due to our relatively small resource size. Although, we probably have enough developers at this point to make it happen. Bruce I would want to be the patchmeister for the stable branch. Someone else (with Bruce's oversight, or Tom's, or whoever) could do the patchmunging and review for the development tree. But I want a stable hand on patches that go into the stable tree. The BSD's release something like that, with CURRENT, TESTING, and STABLE, right? (I'm not a big BSD user...) The Linux kernel has parallel development, and a gatekeeper for each stable release (2.0.x, 2.2.x, 2.4.x, and now 2.6.x). A 2.0.x kernel release happened not too long ago, in fact. We probably could have enough volunteers to do something like that. Gatekeeping a stable release would not be as complex as developing the new release, but, again, I would want an experienced hand on the last stable release where the temptation of backporting 'features' is great. I think a gatekeeper for 7.2.x, for example, would have very little to do except once in a great while if and when a serious bug is found. At that point, that gatekeeper can make a release (the current process is too tied up in people, IMO, and that includes the RPM mechanism (which I have every right to criticize!)). -- Lamar Owen Director of Information Technology Pisgah Astronomical Research Institute 1 PARI Drive Rosman, NC 28772 (828)862-5554 www.pari.edu ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Is trust really a good default?
Magnus Hagander wrote: The only part of this discussion that I'd really be prepared=20 to buy into is the part about *if* you use -W or --pwfile, then set up pg_hba.conf with MD5 as the default auth (because that's probably what the user wants anyway). Ok. Here is a patch that does this. ... and rather severely mangles the comments, too; Um, no, it doesn't. At least not on my installation. not to mention the more basic problem that the comments will now be wrong. That, however, it is correct :-( Sloppy. How about a text along the line of: CAUTION: Configuring the system for trust authentication allows any local user to connect using any PostgreSQL user name, including the superuser, over either Unix domain sockets or TCP/IP. If you are on a multiple-user machine, this is probably not good. Change it to use something other than trust authentication. New wording: CAUTION: Configuring the system for local trust authentication allows any local user to connect as any PostgreSQL user, including the database superuser. If you do not trust all your local users, use another authenication method. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Is trust really a good default?
Magnus Hagander wrote: not to mention the more basic problem that the comments will now be wrong. That, however, it is correct :-( Sloppy. How about a text along the line of: CAUTION: Configuring the system for trust authentication allows any local user to connect using any PostgreSQL user name, including the superuser, over either Unix domain sockets or TCP/IP. If you are on a multiple-user machine, this is probably not good. Change it to use something other than trust authentication. Or something along that line? Since it would no longer actually be default. Or do we want something like On some installations, the default is...? Woh, I didn't think we agreed that the default would change from 'trust', only that we would now emit a warning and allow other authentication methods to be specified at initdb time. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Release planning (was: Re: [HACKERS] Status report)
On Tuesday 13 July 2004 08:52 am, Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Take my opinion with a grain of salt, as I'm young and don't have much experience with large, super-stable C projects. However, looking at how the Linux community handles it, I think we can borrow a lot of concepts from them. Let's seperate out into two different communities: Stable and Dev. The stable folks only want patches when necessary. They don't want new features - they don't use them. They don't want anything but security or stability patches. They don't want to be forced to upgrade. New major versions are moved into the stable community, but they don't replace old versions. No one ever says upgrade or else! Each major version is kept by interested parties and patched as needed. They never die, but are abandoned. The dev community wants to try out all kinds of things. They aren't running production code, so they can risk losing data. They don't have to be so sensitive about backwards compatibility and reliability and security. In the dev community, there is the main branch with all the accepted features. When people feel it is time for a new major version, they spend all their effort stabilizing that main branch. When it is finally stable, they create a new major version and hand it off to the stable community. People keep their own versions of postgresql in the dev community. These compete with each other. For instance, if Tom and others don't feel like a feature should go into the main dev branch, that doesn't mean that Bruce won't put it in his own version and encourage people to develop against that. When Bruce can prove that it is a useful feature and it works and is stable enough, he will work on getting it in the main dev branch. I know this isn't the way things have been done. I know that one of the arguments against proposals like this is that it seperates out our pool of developers. Yes, it will do that. But it will also open up development to more people and more experimentation. Maybe in the end, we will have a stable and dev community that is larger than the entire community we have now. -- Jonathan Gardner [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Point in Time Recovery
On Tue, 2004-07-13 at 22:19, Tom Lane wrote: To have a consistent recovery at all, you must replay the log starting from a checkpoint before the backup began and extending to the time that the backup finished. You only get to decide where to stop after that point. So the situation is: - You must only stop recovery at a point in time (in the logs) after the backup had completed. No way to enforce that currently, apart from procedurally. Not exactly frequent, so I think I just document that and move on, eh? Thanks for your help, Best regards, Simon Riggs ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: Nobody would be required to upgrade to a new minor release either ... nobody is *require* to upgrade to any release, for that matter ... Most people don't have the time to investigate release notes, release policy details, etc. When there are bug fix updates, you use them. When there are feature updates, you use them for the next installation. Anything in between, or more variations added to that, just cause confusion. And frankly, for the examples thrown around that use this kind of policy, I can't see those as being very successful. I don't want to use a stable branch of a database system that still changes for random reasons. And I don't want a current branch that takes years to finalize. More frequent major releases, to the point that we can stem it, lead to more people getting more features earlier, which is good. Any of the other proposal just make things worse in my mind. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Is trust really a good default?
On Tue, 2004-07-13 at 22:27, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I think --ident would be very helpful, and we know with OS's support ident too. If we're going to be doing sed-like substitutions on pg_hba.conf.sample, then we really really wanna discourage distros from hacking the sample file directly, because that could break the sed results. So I think it's important to provide the switch. Speaking for Debian, I should like to explain how pg_hba.conf is managed (at least at present and probably in the next stable release). The basic assumption is that a system-installed package is of universal applicability, so there is only one (official) database cluster. The configuration files in that cluster are actually symlinks to /etc/postgresql/*. The Debian packaged version of initdb is hacked to write those symlinks rather than copy the sample files. (An extra command option --debian-conffile does this, and is used by the installation script.) (A local user running initdb in his own space would get the upstream behaviour, but this is not the normal case for package installations.) The reasons for the changes are found in Debian policy: 1. All configuration files [conffiles] must be in /etc . [motivation: administrators should be able to find configuration files quickly, without having to research each package separately.] 2. No conffile may be changed by a package upgrade without the administrator's consent. A package (such as postgresql) cannot simply overwrite a conffile such as pg_hba.conf with a new version. Its new version is written in parallel (/etc/postgresql/pg_hba.conf.dpkg-new) and only overwrites the old one if the administrator consents. [motivation: system administrators should not be surprised by having their systems redefined without their consent.] The default pg_hba.conf installed by a new package installation is configured thus: local all postgres ident sameuser local all all ident sameuser hostall all 127.0.0.1 255.255.255.255 ident sameuser hostall all ::1::::::: ident sameuser hostall all :::127.0.0.1/128 ident sameuser hostall all 0.0.0.00.0.0.0 reject that is, to accept local connections authenticated by ident and reject the rest. The adminstrator is advised not to change the first line, so as to allow cron jobs to run. [motivation: to install the package with a sufficient level of security that it will not open the machine to remote exploits and to ensure that local users cannot spoof their identity to the database or change other people's data without permission. We trust the local ident server, since it is installed by the same administrator that is installing postgresql.] The point of this explanation is that as Debian maintainer I would have to disable any procedures that attempt to edit these conffiles, or at least ensure that their operation is under package control and produce only the effects that I desire. When initdb is rerun during major upgrades, it must then leave the previous configuration unchanged. Ensuring this is part of ensuring a smooth upgrade path, which is a major part of the package maintainer's job. -- Oliver Elphick [EMAIL PROTECTED] Isle of Wight http://www.lfix.co.uk/oliver GPG: 1024D/A54310EA 92C8 39E7 280E 3631 3F0E 1EC0 5664 7A2F A543 10EA Let your character be free from the love of money, being content with what you have; for He Himself has said, I will never desert you, nor will I ever forsake you. Hebrews 13:5 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier [EMAIL PROTECTED] writes: On Tue, 13 Jul 2004, Lamar Owen wrote: But Tom's assertion is true. We have enough trouble getting patches rolled out; adding parallel branches is just begging for trouble, due to our relatively small resource size. Although, we probably have enough developers at this point to make it happen. Except, we already have parallel branches, else we'd never have made a 7.4.x release ... The point though is that we expend only very minimal effort on maintaining the stable branches. We only back-patch bug fixes, and 99% of the time the patch is nearly verbatim the same change as we developed to fix the same problem in HEAD. If the code involved has changed enough that a significantly different fix would be required, most of the time we simply don't fix the stable branch. And we spend very nearly zero effort on QA for the stable branch --- there's certainly no significant push to get people to beta-test minor releases. If we did anything much in the way of back-porting features then the level of investment in this would have to rise greatly. In fact, if the proposal is to let people pick and choose which back-ported things they install, then we are not talking about just one stable version but 2^N stable versions for N options. We couldn't possibly test every combination. The BSD's release something like that, with CURRENT, TESTING, and STABLE, right? (I'm not a big BSD user...) Yeah, but they don't support mix-and-match feature sets. A back release has only one current version. We could certainly do something along that line if we had a few people willing to be gatekeepers. We'd have to work harder at making the release generation process open and documented though. Right now there are plenty of steps that only you, Bruce, or Lamar (respectively) know how to do... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Proposal for detecting encoding mismatch in initdb
Tom Lane wrote: The behavioral description sounds fine, but I was eagerly awaiting your description of exactly how you'd test for compatibility or search for a compatible encoding ... without that algorithm the whole thing's moot. It's just an explicit list of things that spell similarly. There's not much more we can do, but I don't see any obvious candidates were this could lead to trouble. BTW, what happens if there is more than one apparently-matching encoding? (It might be best to error out in this case, on the theory that we evidently don't have a correct matching.) I just won't put something like that into the list. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: On Tue, 13 Jul 2004, Bruce Momjian wrote: We have always recommended upgrading to the most recent minor release, at least as a project policy for support. Also, the upgrade is only stop/install/restart so it is quite easy to do, and folks like the fact they don't have to test for new functionality. Right, and this made sense when we were dealing with a release every 4 months or so ... now we're talking about one every year, or two ... I meant if they are on 7.4.X they should be on 7.4.3. I don't suggest they have to upgrade 7.2.X to 7.4.X. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Point in Time Recovery
Simon Riggs [EMAIL PROTECTED] writes: So the situation is: - You must only stop recovery at a point in time (in the logs) after the backup had completed. Right. No way to enforce that currently, apart from procedurally. Not exactly frequent, so I think I just document that and move on, eh? The procedure that generates a backup has got to be responsible for recording both the start and stop times. If it does not do so then it's fatally flawed. (Note also that you had better be careful to get the time as seen on the server machine's clock ... this could be a nasty gotcha if the backup is run on a different machine, such as an NFS server.) regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] Is trust really a good default?
On Tue, 2004-07-13 at 17:44, Bruce Momjian wrote:
Magnus Hagander wrote:
not to mention the
more basic problem that the comments will now be wrong.
That, however, it is correct :-( Sloppy.
How about a text along the line of:
CAUTION: Configuring the system for trust authentication allows any
local user to connect using any PostgreSQL user name, including the
superuser, over either Unix domain sockets or TCP/IP. If you are on
a multiple-user machine, this is probably not good. Change it to use
something other than trust authentication.
Or something along that line? Since it would no longer actually be
default. Or do we want something like On some installations, the
default is...?
Woh, I didn't think we agreed that the default would change from
'trust', only that we would now emit a warning and allow other
authentication methods to be specified at initdb time.
I sure hope not (and that was my understanding as well)
Incidentally that warning is a little misleading since it isn't just
trust authentication that allows the wide open connections, but the
combo of all users / all dbs / trust that does it. For example on one
of my development machine I have a guest user who only has read access
to a specific database from a limited subnet, but with trust
authentication since random people inside the company will sometimes
want to take a look at what I am cooking up. For my needs I use the
superuser account who can access all databases but must come through
ident on a unix socket. Different strokes for different folks eh?
Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
Re: [HACKERS] Point in Time Recovery
On Tue, 2004-07-13 at 23:42, Bruce Momjian wrote: Simon Riggs wrote: On Tue, 2004-07-13 at 22:19, Tom Lane wrote: To have a consistent recovery at all, you must replay the log starting from a checkpoint before the backup began and extending to the time that the backup finished. You only get to decide where to stop after that point. So the situation is: - You must only stop recovery at a point in time (in the logs) after the backup had completed. No way to enforce that currently, apart from procedurally. Not exactly frequent, so I think I just document that and move on, eh? If it happens, could you use your previous full backup and the PITR logs from before stop stopped logging, and then after? Yes. Is there a period where they could not restore reliably? Good question. No is the answer. The situation is that the backup isn't timestamped with respect to the logs, so its possible to attempt to use the wrong backup for recovery. The solution is procedural - make sure you timestamp your backup files, so you know which ones to recover with... Best Regards, Simon Riggs ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] Is trust really a good default?
Robert Treat wrote: Woh, I didn't think we agreed that the default would change from 'trust', only that we would now emit a warning and allow other authentication methods to be specified at initdb time. I sure hope not (and that was my understanding as well) Incidentally that warning is a little misleading since it isn't just trust authentication that allows the wide open connections, but the combo of all users / all dbs / trust that does it. For example on one of my development machine I have a guest user who only has read access to a specific database from a limited subnet, but with trust authentication since random people inside the company will sometimes want to take a look at what I am cooking up. For my needs I use the superuser account who can access all databases but must come through ident on a unix socket. Different strokes for different folks eh? Sure, but the point is that the 'trust' line added by initdb is wide-open. Folks who do that fine-grained control will not get confused by the warning, hopefully. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Point in Time Recovery
Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: So the situation is: - You must only stop recovery at a point in time (in the logs) after the backup had completed. Right. No way to enforce that currently, apart from procedurally. Not exactly frequent, so I think I just document that and move on, eh? The procedure that generates a backup has got to be responsible for recording both the start and stop times. If it does not do so then it's fatally flawed. (Note also that you had better be careful to get the time as seen on the server machine's clock ... this could be a nasty gotcha if the backup is run on a different machine, such as an NFS server.) OK, but procedurally, how do you correlate the start/stop time of the tar backup with the WAL numeric file names? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Point in Time Recovery
Bruce Momjian [EMAIL PROTECTED] writes: OK, but procedurally, how do you correlate the start/stop time of the tar backup with the WAL numeric file names? Ideally the procedure for making a backup would go something like: 1. Inquire of the server its current time and the WAL position of the most recent checkpoint record (which is what you really need). 2. Make the backup. 3. Inquire of the server its current time and the current end-of-WAL position. 4. Record items 1 and 3 along with the backup itself. I think the current theory was you could fake #1 by copying pg_control before everything else, but this doesn't really help for step #3, so it would probably be better to add some server functions to get this info. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Point in Time Recovery
On Wed, 2004-07-14 at 00:28, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: OK, but procedurally, how do you correlate the start/stop time of the tar backup with the WAL numeric file names? Ideally the procedure for making a backup would go something like: 1. Inquire of the server its current time and the WAL position of the most recent checkpoint record (which is what you really need). 2. Make the backup. 3. Inquire of the server its current time and the current end-of-WAL position. 4. Record items 1 and 3 along with the backup itself. I think the current theory was you could fake #1 by copying pg_control before everything else, but this doesn't really help for step #3, so it would probably be better to add some server functions to get this info. err...I think at this point we should review the PITR patch The recovery mechanism doesn't rely upon you knowing 1 or 3. The recovery reads pg_control (from the backup) and then attempts to de-archive the appropriate xlog segment file and then starts rollforward from there. Effectively, restore assumes it has access to an infinite timeline of logswhich clearly isn't the case, but its up to *you* to check that you have the logs that go with the backups. (Or put another way, if this sounds hard, buy some software that administers the procedure for you). That's the mechanism that allows infinite recovery. In brief, the code path is as identical as possible to the current crash recovery situation...archive recovery restores the files from archive when they are needed, just as if they had always been in pg_xlog, in a way that ensures pg_xlog never runs out of space. Recovery ends when: it reaches the recovery target you specified, or it runs out of xlogs (first it runs out of archived xlogs, then tries to find a more recent local copy if there is one). I think the current theory was you could fake #1 by copying pg_control before everything else, but this doesn't really help for step #3, so it would probably be better to add some server functions to get this info. Not sure what you mean by fake Best Regards, Simon Riggs ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Tom Lane wrote: We could certainly do something along that line if we had a few people willing to be gatekeepers. We'd have to work harder at making the release generation process open and documented though. Right now there are plenty of steps that only you, Bruce, or Lamar (respectively) know how to do... I think we could do it if we restricted it to *just* one release back ... but I do agree with Peter about ppls motivations for upgrading (bug fixes vs new features) ... we'd have to have a 'two branch stable', one would be only bug fixes, the other would be bug fixes+feature backpatch ... would could get interesting trying to figure out release naming conventions :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
On Wed, 14 Jul 2004, Peter Eisentraut wrote: Marc G. Fournier wrote: Nobody would be required to upgrade to a new minor release either ... nobody is *require* to upgrade to any release, for that matter ... Most people don't have the time to investigate release notes, release policy details, etc. When there are bug fix updates, you use them. When there are feature updates, you use them for the next installation. Anything in between, or more variations added to that, just cause confusion. And frankly, for the examples thrown around that use this kind of policy, I can't see those as being very successful. I don't want to use a stable branch of a database system that still changes for random reasons. And I don't want a current branch that takes years to finalize. More frequent major releases, to the point that we can stem it, lead to more people getting more features earlier, which is good. Any of the other proposal just make things worse in my mind. 'k, note that I'm not arguing for more (or less) frequent releases ... but, do you have any thoughts on how we can effectively continue with the 'frequent releases' while bringing in the large features like Nested Xacts? We could start looking at 'development branches' for stuff like this, that could be merged up when ready for prime time, but would at least be available in CVS for those wishing to play with them ... ? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Bruce Momjian wrote: I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. I think this would be a bad direction to take ... I can just see us flooded with a bunch of 'the patch didn't apply to my source tree' questions :( It might work, I just have my doubts ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Lamar Owen wrote: But Tom's assertion is true. We have enough trouble getting patches rolled out; adding parallel branches is just begging for trouble, due to our relatively small resource size. Although, we probably have enough developers at this point to make it happen. Except, we already have parallel branches, else we'd never have made a 7.4.x release ... Bruce I would want to be the patchmeister for the stable branch. Someone else (with Bruce's oversight, or Tom's, or whoever) could do the patchmunging and review for the development tree. But I want a stable hand on patches that go into the stable tree. The BSD's release something like that, with CURRENT, TESTING, and STABLE, right? (I'm not a big BSD user...) We have a CURRENT branch where all the 'innovations' are done (SMP re-writes, etc) ... and a STABLE which is a release with stuff patched down from CURRENT *if* applicable ... periodically, a RELEASE is made along either branch, and, some day, what is currently CURRENT will be re-tag'd as -STABLE, and then CURRENT will become a new branch ... So, for instance, the way we number: 7.4.x would be -STABLE HEAD would be -CURRENT once 7.5 is released, it would surplant 7.4 as -STABLE, previous versions would only ever see 'security related patches' and work towards 7.6 would be -CURRENT ... Now, if we went with a 'long term dev cycle', then we might look at 7.x as being -STABLE, while work towards 8.x would be considered -CURRENT ... As a community, I don't think we should be 'supporting' anything older then the last STABLE ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Proposal for detecting encoding mismatch in initdb
Peter Eisentraut [EMAIL PROTECTED] writes: I've worked out a scheme that should adequately detect encoding mismatches in initdb. Please comment on the following behavior. The behavioral description sounds fine, but I was eagerly awaiting your description of exactly how you'd test for compatibility or search for a compatible encoding ... without that algorithm the whole thing's moot. BTW, what happens if there is more than one apparently-matching encoding? (It might be best to error out in this case, on the theory that we evidently don't have a correct matching.) regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Point in Time Recovery
PITR Patch v5_1 just posted has Point in Time Recovery working Still some rough edgesbut we really need some testers now to give this a try and let me know what you think. Klaus Naumann and Mark Wong are the only [non-committers] to have tried to run the code (and let me know about it), so please have a look at [PATCHES] and try it out. Many thanks, Simon Riggs ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Point in Time Recovery
Simon Riggs wrote: On Tue, 2004-07-13 at 22:19, Tom Lane wrote: To have a consistent recovery at all, you must replay the log starting from a checkpoint before the backup began and extending to the time that the backup finished. You only get to decide where to stop after that point. So the situation is: - You must only stop recovery at a point in time (in the logs) after the backup had completed. No way to enforce that currently, apart from procedurally. Not exactly frequent, so I think I just document that and move on, eh? If it happens, could you use your previous full backup and the PITR logs from before stop stopped logging, and then after? Is there a period where they could not restore reliably? -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: On Tue, 13 Jul 2004, Bruce Momjian wrote: I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. I think this would be a bad direction to take ... I can just see us flooded with a bunch of 'the patch didn't apply to my source tree' questions :( It might work, I just have my doubts ... I have my doubts too, of course. :-) -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Point in Time Recovery
On Wed, 2004-07-14 at 00:01, Bruce Momjian wrote: Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: So the situation is: - You must only stop recovery at a point in time (in the logs) after the backup had completed. Right. No way to enforce that currently, apart from procedurally. Not exactly frequent, so I think I just document that and move on, eh? The procedure that generates a backup has got to be responsible for recording both the start and stop times. If it does not do so then it's fatally flawed. (Note also that you had better be careful to get the time as seen on the server machine's clock ... this could be a nasty gotcha if the backup is run on a different machine, such as an NFS server.) OK, but procedurally, how do you correlate the start/stop time of the tar backup with the WAL numeric file names? No need. You just correlate the recovery target with the backup file times. Mostly, you'll only ever use your last backup and won't need to fuss with the times. Backup should begin with a CHECKPOINT...then wait for that to complete, just to make the backup as current as possible. If you want to start purging your archives of old archived xlogs, you can use the filedate (assuming you preserved that on your copy to archive - but even if not, they'll be fairly close). Best regards, Simon Riggs ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Bruce Momjian wrote: We have always recommended upgrading to the most recent minor release, at least as a project policy for support. Also, the upgrade is only stop/install/restart so it is quite easy to do, and folks like the fact they don't have to test for new functionality. Right, and this made sense when we were dealing with a release every 4 months or so ... now we're talking about one every year, or two ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 2004-07-13 at 13:03 -0400, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Jan Wieck wrote: I think in the future we have to force all large features, those that probably need more than 6 months of development time, to be properly #ifdef'd. Then it wouldn't be that big of a deal to release more often. Alvaro started out with ifdef's but it got too confusing and we all agreed to just go with a normal patch. He just hits too much code. I think the same would be true of almost any really large feature --- ifdefs all over the code base are just too much of a mess. To be honest I think that releasing more often isn't necessarily an appropriate goal for the project anymore. Five or six years ago we were doing a major (initdb-forcing) release every three or four months, and that was appropriate at the time, but the project has matured and our user population has changed. Look at how many people are still using 7.2 or 7.3. One major release a year may be about right now, because you can't get people to adopt new major revs faster than that anyway. Of course this all ties into the pain of having to dump/reload large databases, and maybe if we had working pg_upgrade the adoption rate would be faster, but I'm not at all sure of that. We're playing in a different league now. Big installations tend to want to do significant amounts of compatibility testing before they roll out a new database version. regards, tom lane It sounds like your only taking the point of view of people upgrading previous installations. What about new installations? I'm sure there are hundreds, if not thousands of new installations happening every week. These people are going to grab the latest stable version without a doubt. I think releasing more often would result in features getting tested much more. Then the big installations can see that a major feature has been in two stable releases (even if the time period was only a year) and feel much more comfortable in upgrading. Why would they have to upgrade more often then necessary anyways? Assuming security exploits are back ported of course. -- Mike Benoit [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: On Tue, 13 Jul 2004, Bruce Momjian wrote: The nice thing about doing something lke that is those small features would get a degree of testing happening in a live environment ... Of course that last sentence is the downside too --- people don't want to have to retest their setups after a minor release upgrade. Nobody would be required to upgrade to a new minor release either ... nobody is *require* to upgrade to any release, for that matter ... Hell, when we have a client that comes to us with a problem, we don't recommend upgrading unless we find something in the RELEASE NOTES to indicate something has been fixed related to their problem ... it isn't a automatic well, upgrade to the latest stable first, and then we can help you ... God, we still have clients using 7.2 servers, cause they've had no reason yet to upgrade to the latest ... it works, why upgrade? is generally the opinion ... We have always recommended upgrading to the most recent minor release, at least as a project policy for support. Also, the upgrade is only stop/install/restart so it is quite easy to do, and folks like the fact they don't have to test for new functionality. One idea would be for us to release 7.5 and 7.5.0.1, and allow 7.5.1 to have minor new features. That way, 7.5.0.[1-9] are no new features, and 7.5.[1-9] are minor improvements against 7.5. Of course this does require more project management, and we already don't have enough manpower. I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 13 Jul 2004, Bruce Momjian wrote: The nice thing about doing something lke that is those small features would get a degree of testing happening in a live environment ... Of course that last sentence is the downside too --- people don't want to have to retest their setups after a minor release upgrade. Nobody would be required to upgrade to a new minor release either ... nobody is *require* to upgrade to any release, for that matter ... Hell, when we have a client that comes to us with a problem, we don't recommend upgrading unless we find something in the RELEASE NOTES to indicate something has been fixed related to their problem ... it isn't a automatic well, upgrade to the latest stable first, and then we can help you ... God, we still have clients using 7.2 servers, cause they've had no reason yet to upgrade to the latest ... it works, why upgrade? is generally the opinion ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Is trust really a good default?
On Monday 12 July 2004 17:10, Merlin Moncure wrote: IMO, forcing su password at initdb time (allowing blank password with a very stern warning) and bumping localhost to auth is the right way to go. As far as RPM's, etc. I don't think RPM considerations should be driving security concerns. FWIW, the RPMs default to ident authentication, and trust is off. This is however done as a patch to the sample pg_hba.conf. A command line switch to initdb to mung up an ident default would be fine with me, though. -- Lamar Owen Director of Information Technology Pisgah Astronomical Research Institute 1 PARI Drive Rosman, NC 28772 (828)862-5554 www.pari.edu ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Another locale test program
Peter Eisentraut [EMAIL PROTECTED] writes: Regarding the encoding/locale matching issue, here's another test routine I would like you to run if you want to see your operating system supported. Oh, disregard previous complaint then... On HPUX 10.20 I get --snip-- SJIS arabic8 big5 ccdc eucJP eucKR eucTW greek8 hebrew8 hp15CN iso88591 iso885915 iso88592 iso88595 iso88596 iso88597 iso88598 iso88599 kana8 roman8 tis620 turkish8 utf8 --snip-- On Mac OS X 10.3.4, there doesn't seem to be a locale program. Using the ls /usr/share/locale technique, I get --snip-- Big5 CP1251 CP866 ISCII-DEV ISO8859-1 ISO8859-13 ISO8859-15 ISO8859-2 ISO8859-4 ISO8859-5 ISO8859-7 ISO8859-9 KOI8-R KOI8-U SJIS US-ASCII UTF-8 eucCN eucJP eucKR --snip-- Note the blank line in there! It appears that the CODESET routine is simply emitting the last part of the locale name, as the output works out like this: locale CODESET ... de_AT de_AT.ISO8859-1 ISO8859-1 de_AT.ISO8859-15ISO8859-15 de_AT.UTF-8 UTF-8 de_CH ... In short it looks like this is going to be a bit limited on OSX :-( regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Release planning (was: Re: [HACKERS] Status report)
However, looking at how the Linux community handles it, I think we can borrow a lot of concepts from them. I was thinking quite the opposite. The Linux community doesn't exactly have a great track-record for their stable releases. The thoughts behind the process might be good, but do we have examples where it has worked out well? The 2.4 series seems to have been particularly bad for new major issues in their stable releases. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Release planning (was: Re: [HACKERS] Status report)
On Tue, 2004-07-13 at 20:49, Marc G. Fournier wrote: On Tue, 13 Jul 2004, Tom Lane wrote: We could certainly do something along that line if we had a few people willing to be gatekeepers. We'd have to work harder at making the release generation process open and documented though. Right now there are plenty of steps that only you, Bruce, or Lamar (respectively) know how to do... I think we could do it if we restricted it to *just* one release back ... but I do agree with Peter about ppls motivations for upgrading (bug fixes vs new features) ... we'd have to have a 'two branch stable', one would be only bug fixes, the other would be bug fixes+feature backpatch ... would could get interesting trying to figure out release naming conventions :) FreeBSD does do this. They tag the bug-fix only branches as being SECURITY releases, since typically they only contain fixes for security or reliability related issues. The 5.2.1 release which followed 5.2.0 was an example of one of the few releases the Security team has done. They usually just allow users to use CVSUP to follow the branch. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Release planning (was: Re: [HACKERS] Status report)
God, we still have clients using 7.2 servers, cause they've had no reason yet to upgrade to the latest ... it works, why upgrade? is generally the opinion ... Because there's shocking failure-to-restart bugs in 7.2...and if you upgrade to 7.4 you're forced to get new features as well. Chris ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Release planning (was: Re: [HACKERS] Status report)
I was thinking of something much simpler where Jan would create an ARC patch against 7.4.X and have it either in /contrib for 7.4.X or on our ftp servers, or on a web site. I could create a mechanism so SELECT version() would display Jan's add-on. I think you guys just need to learn to become comfortable with being hard-nosed about this. Just Do Not Care about people who want ARC right this second. Do you see them calling up Oracle and saying please backport the new stuff from 11i-devel so we can use it now please? You learn after a long while in the software industry that if left to themselves the users will make _endless_ demands and they think that they NEED everything NOW. Of course in reality, they don't NEED it NOW - they can wait. And even if they did get it now, then it's buggy and then they complain even louder. You cannot win. The arguments about things getting more testing in production is nonsense - we are a major database server - you cannot have users doing the testing for us They have signed on to the PostgreSQL project on our terms of development - if they want to sponsor someone to backport something for their own situation, they can do so. Otherwise, they perhaps should have not been using PostgreSQL for their particular app in the first place! It's not worth thinking along the lines of oh if only we could get this feature out RIGHT NOW, we'd get more users and more people uprading and convertign and better reviews and stuff - but that's a distraction. There will ALWAYS be Just One More Feature that we need to be really great. Just don't worry about it. Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Release planning
The thoughts behind the process might be good, but do we have examples where it has worked out well? The 2.4 series seems to have been particularly bad for new major issues in their stable releases. PHP's the same. Absolutely dreadful. They put all sorts of new features mixed in with security and bug fixes in their minor releases. The NUMBER OF TIMES I've upgraded PHP to fix a bug and they've introduced a new global function that conflicts with one of my user one and worse... Chris ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Release planning (was: Re: [HACKERS] Status report)
Marc G. Fournier wrote: snip As Jan points out, its the 'small features that are done' that we've been griping about having to wait for, not the big ones which we know aren't done ... snip Hmmm... so we do things slightly differently than previously... This upcoming version could be PG version 8.0, We continue with bugfixes on 7.4.x, That still leaves 7.5.x, 7.6.x (etc if needed), for releasing new versions of PG without the big features. Kind of like an in-between thing, whilst waiting for the major features in the major releases? That would mean we'd have: Version 8.0 as our next main release, Version 9.0 being the version after that with the next big features in it. Version 8.x being version 8 plus smaller features, prior to 9.0 Version 8.x.x being version 8.x plus bug fixes. Sounds like it could get hairy if we're not careful, but I reckon the PG Community is mature enough to make the right calls where needed. :) Regards and best wishes, Justin Clift ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Point in Time Recovery
Can you give us some suggestions of what kind of stuff to test? Is there a way we can artificially kill the backend in all sorts of nasty spots to see if recovery works? Does kill -9 simulate a 'power off'? Chris Simon Riggs wrote: PITR Patch v5_1 just posted has Point in Time Recovery working Still some rough edgesbut we really need some testers now to give this a try and let me know what you think. Klaus Naumann and Mark Wong are the only [non-committers] to have tried to run the code (and let me know about it), so please have a look at [PATCHES] and try it out. Many thanks, Simon Riggs ---(end of broadcast)--- TIP 8: explain analyze is your friend ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Release planning
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tuesday 13 July 2004 7:33 pm, Christopher Kings-Lynne wrote: PHP's the same. Absolutely dreadful. They put all sorts of new features mixed in with security and bug fixes in their minor releases. The NUMBER OF TIMES I've upgraded PHP to fix a bug and they've introduced a new global function that conflicts with one of my user one and worse... I was arguing quite the opposite. New features wouldn't be introduced into code that is in the stable community. Only bug fixes and security patches would be introduced to the stable community, and very carefully. - -- Jonathan Gardner [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQFA9J89qp6r/MVGlwwRAh6CAKDCJhuomf8hWbHMHGqrYfCGi5QzxQCfT4Hs feHVoYbFW0tq6BwJtFxy+EE= =BmHk -END PGP SIGNATURE- ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] [HACKERS] Is trust really a good default?
Oliver Elphick [EMAIL PROTECTED] writes: ... The point of this explanation is that as Debian maintainer I would have to disable any procedures that attempt to edit these conffiles, or at least ensure that their operation is under package control and produce only the effects that I desire. Uh, is this relevant at all? There has been no suggestion that initdb should try any harder or less hard than it does now to write $PGDATA/pg_hba.conf. All that's been discussed is what it should write there. If you are going to hack on it to enforce your opinion of what it should do, then you'll be making the same hack either way. regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Assisting developers
Bruce Momjian wrote: Christopher Kings-Lynne wrote: Well, I note (and I'm not being unkind or anything here) that a lot of the high level committers we have haven't been so active this release. Peter and Joe haven't been around much and Jan has been busy with Slony. I think this is (in part at least) a reflection of the economy finally picking up. Last year this time I had one major project going on -- right now I'm juggling about a half dozen or so, any one of which could keep me busy full time. Add a couple of OSCON presentations to prepare, and several serial disasters/distractions at home (nothing life threatening, but all time consuming), and that leaves little-to-no time for recreational Postgres coding :( The committing isn't really the issue. It is reviewing and giving feedback to developers of complex features. This is very true. But the fact that I have been able to commit what little I have done this release has at least off-loaded some work from you Bruce, hasn't it? I'm still green enough though that I'm not in a position to review anything too complex -- someone like Tom or Bruce would still need to follow up behind me, so it wouldn't really save them any time. Joe ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Assisting developers
Joe Conway wrote: Bruce Momjian wrote: Christopher Kings-Lynne wrote: Well, I note (and I'm not being unkind or anything here) that a lot of the high level committers we have haven't been so active this release. Peter and Joe haven't been around much and Jan has been busy with Slony. I think this is (in part at least) a reflection of the economy finally picking up. Last year this time I had one major project going on -- right now I'm juggling about a half dozen or so, any one of which could keep me busy full time. Add a couple of OSCON presentations to prepare, and several serial disasters/distractions at home (nothing life threatening, but all time consuming), and that leaves little-to-no time for recreational Postgres coding :( The committing isn't really the issue. It is reviewing and giving feedback to developers of complex features. This is very true. But the fact that I have been able to commit what little I have done this release has at least off-loaded some work from you Bruce, hasn't it? I'm still green enough though that I'm not in a position to review anything too complex -- someone like Tom or Bruce would still need to follow up behind me, so it wouldn't really save them any time. Yes, others have stepped up to commit things while I was busy --- Neil and Joe come to mind. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
