Re: [HACKERS] recovering prepared transaction after serverrestart message
On Fri, 2006-11-03 at 01:48 -0500, Tom Lane wrote: I agree that there's a usability issue here though; I've been burnt by forgotten prepared xacts myself (eg by control-C'ing pg_regress at just the wrong time). Would it help if we included prepared xacts in the pg_stat_activity view? Is there any other place we could make them more visible during normal server operation? We only care when they break, otherwise its just situation normal, yes? Is there a way to see prepared transactions where the original session that prepared then has died? Perhaps the message at startup should be you have at least one prepared transaction that needs resolution. We need something at that point, otherwise a PITR recovery is fairly likely to contain them and we need to know that. Otherwise on a system using prepared transactions heavily you may not spot the odd or two that have crashed and need resolution. Presumably they will effect oldestxmin, so its fairly important to be able to resolve them in a timely manner or at least know they are there. Not that I am advising their use though -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] WAL logging freezing
Seems like either we go back to ignoring non-connectable databases (with the risks that entails), or adopt some more-aggressive policy for launching autovacuums on them, or Um, I think we would want to still have the ability to not need to frequently vacuum known frozen databases. Could we set datvacuumxid to frozenxid after freezing template0 and making it non-connectable ? There was discussion about making a non-connectable db connectable, one step would be to set datvacuumxid to currentxid. Then the rule would be: ignore db's that have datvacuumxid == frozenxid. (only readonly db's would be allowed to have datvacuumxid == frozenxid) Andreas PS: I agree that the rename from datfrozenxid to datvacuumxid should be undone ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] WAL logging freezing
Zeugswetter Andreas ADI SD wrote: Seems like either we go back to ignoring non-connectable databases (with the risks that entails), or adopt some more-aggressive policy for launching autovacuums on them, or Um, I think we would want to still have the ability to not need to frequently vacuum known frozen databases. Could we set datvacuumxid to frozenxid after freezing template0 and making it non-connectable ? There was discussion about making a non-connectable db connectable, one step would be to set datvacuumxid to currentxid. I tried to do this at some point, but it doesn't work. I think the problem appears when you create a new database after the template database has been frozen for a while. IIRC one of my proposed solutions was to force a vacuum of the new database right away, but that was shot down (for good reasons, I think). Another one was to mark the tables as frozen, which led to the suggestion of pg_class_nt, which eventually we shot down for other reasons. The last suggestion was to force the vacuuming of frozen databases, which is where we are now. I think the best solution for now is to force the vacuum of template databases. They are generally small, so this _shouldn't_ be an issue. (There _would_ be an issue if someone made big databases non-connectable for the purposes of avoiding vacuum; this was mentioned back then, but no one said that this would be an issue.) We can change that in 8.3 by reintroducing the concept of pg_class_nt and possibly pg_database_nt, which would allow us to really mark tables as completely frozen, and databases as not needing vacuum at all. PS: I agree that the rename from datfrozenxid to datvacuumxid should be undone Actually, the one that was renamed from datfrozenxid was datminxid, which is now being proposed to be removed. We are now changing the semantics of datvacuumxid to the old datfrozenxid. I also agree with the change (but keep in mind that I'm not affected as a beta tester, so my opinion may not count). PS -- yeah, I neglected the maintenance.sgml docs at the time :-( I noticed some days ago and I was going to submit appropriate changes. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] recovering prepared transaction after serverrestart message
Simon Riggs [EMAIL PROTECTED] writes: We only care when they break, otherwise its just situation normal, yes? No, the trouble case is where the XA manager that owns the transaction has forgotten about it. Is there a way to see prepared transactions where the original session that prepared then has died? Perhaps the message at startup should be you have at least one prepared transaction that needs resolution. I am completely baffled by this focus on database startup time. That's not where the problem is. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] recovering prepared transaction after serverrestart
On Fri, 3 Nov 2006, Tom Lane wrote: Is there a way to see prepared transactions where the original session that prepared then has died? Perhaps the message at startup should be you have at least one prepared transaction that needs resolution. I am completely baffled by this focus on database startup time. That's not where the problem is. regards, tom lane I'm not alluding to anyone in particular, just responding to the focus on startup time; When I joined Ingres as a Consultant (back when that was a revered job), we saw this a lot, too, bubbling through the ranks from technical support. Engineering was having a cow over it. We Consultants were expected to backline such problems and be the interface between engineering and the rest of the world. What we found was that in what we'd call the ligitimate cases, the cause for concern over startup time had to do with bugs that forced, one way or another, a server restart. Illigitimate cases - the VAST majority - were the result of, well, let's call them less-than-successful DBAs, thrashing their installations with their management breathing down their necks, often with flailing arms and fire coming out of their mouths saying things like, I bet my business on this!... The usual causes there were inappropriate configurations, and a critical cause of _that_ was an instalation toolset that didn't help people size/position things properly. Often a sales guy or trainee would configure a test system and then the customer would put that into production without ever reexamining the settings. I realized there was an opportunity here; I put together a training program and we sold it as a service along with installation to new customers to help them get off on the right foot. Once we did that, new customers were essentially put on notice that they could either pay us to help set them up, or they could do it, but that continuing along with what the salesman or junior techie had done wasn't sufficient for a production environment that you could bet your business on. ...The complaint and concern about startup time dropped out of sight nearly immediately... Opportunity here, for PostgreSql: A Technical Document of some kind entitled something like: How to move your testing environment into production. No, unfortunately, I can't volunteer to be the point person on this one. And to the underlying question: is this the case with PostgreSql? I can't say... Regards, Richard -- Richard Troy, Chief Scientist Science Tools Corporation 510-924-1363 or 202-747-1263 [EMAIL PROTECTED], http://ScienceTools.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] recovering prepared transaction after serverrestart message
Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: We only care when they break, otherwise its just situation normal, yes? No, the trouble case is where the XA manager that owns the transaction has forgotten about it. Yeah, and there's no way the DBMS can detect that. Is there a way to see prepared transactions where the original session that prepared then has died? Perhaps the message at startup should be you have at least one prepared transaction that needs resolution. I am completely baffled by this focus on database startup time. That's not where the problem is. Agreed. Though one way to have orphaned prepared transactions is to recover from a PITR backup or bring a warm stand-by live. The transaction manager might have committed a transaction after the backup was taken. Recovering from the backup resurrects the transaction again and the TM won't know about it. The problem of orphaned transactions is most likely to occur on a dev/test environment, where the TM is run on a developer's laptop and might be killed and reinstalled or reconfigured at any time. And unfortunately there's also a lot of broken TMs out there that don't recover from crashes properly. I think it's a good idea to at least LOG about prepared transactions at startup. But it would be nice to also have a timeout, after which a big fat WARNING would be printed. I don't believe in killing transactions automatically though, that's a job for the administrator. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] Bug in WAL backup documentation
Bernd Helmle [EMAIL PROTECTED] writes: Our WAL backup documentation says in some parts of it: ...%p is replaced by the absolute path of the file to archive... [1] I think this is (at least for 8.1 and upcoming 8.2 releases) wrong, since the archiver replaces this with pg_xlog/LOGFILENAME only, Good point. Do we want to consider that this is a code bug rather than a doc bug? The relative path is more efficient as long as the archiver script doesn't do a cd, but if it does then there'd be a problem. You could argue that the code should be tweaked to continue supplying an absolute path. Since 8.1 has done this all along and no one's actually complained about it, I guess no one is using scripts that do cd. I'm inclined to go with Bernd's suggestion to change the docs to match the code, but does anyone have a contrary opinion? regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Design Considerations for New Authentication Methods
To be honest, I have often wondered *why* we support kerberos outside of the uber l33t geek factor. I have not once in a commercial deployment had a business requirement for the beast. LDAP? Now that is a whole other issue :) Single sign-on in a Windows/AD environment (I'm talking clients on windows, servers on linux here - at least in my case). I know several people who use it, most just don't post here ;-) Wouldn't the LDAP auth in 8.2 resolve that? Now, it would likely be a lot *easier* to do this with GSSAPI than the pure kerberos stuff we have now, given that the Windows native APIs support GSSAPI compatible stuff, but not the stuff we have now. Nod. Sincerely, Joshua D. Drake //Magnus -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] Bug in WAL backup documentation
On Fri, Nov 03, 2006 at 11:25:09AM -0500, Tom Lane wrote: Since 8.1 has done this all along and no one's actually complained about it, I guess no one is using scripts that do cd. I'm inclined to go with Bernd's suggestion to change the docs to match the code, but does anyone have a contrary opinion? Arguably you could give people a choice, say %P for the absolute path and %p for the relative one. In Unix you can easily prepend $PWD to the string, but I don't know how easy that is in Windows. Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] Coding style question
I think Tom stated it pretty well: When the variable is going to be set anyway in straight-line code at the top of the function, then it's mostly a matter of taste whether you set it with an initializer or an assignment. the key phrase is: "set anyway in straigh-tline code at the top of the function" (I don't go so far as to introduce artificial scopes just for the sake of nesting variable declarations). I don't introduce artificial scopes either. However, I do try to declare variables in the most-tightly-enclosing scope. For example, if a variable is only used in one branch of an if statement, declare the variable inside that block, not in the enclosing scope. good... This may not inform the current conversation at all, but a while back I went on a cross-compiler compatibility binge for all of my active projects, and I found that some compilers (*cough* Borland *cough) had some very strange compiler/run time errors unless all variables were declared at the top of the function, before any other code gets executed. For better or for worse, I started strictly declaring all variables in this manner, with initialization happening afterward, and the behavior has stuck with me. I don't know whether any compilers used for postgres builds still have this issue - it's been a few years. I also find that if you're declaring a lot of variables in a single block, that's usually a sign that the block is too large and should be refactored (e.g. by moving some code into separate functions). If you keep your functions manageably small (which is not always the case in the Postgres code, unfortunately), the declarations are usually pretty clearly visible. I couldn't agree more. Insert emphatic agreement here. Refactoring into smaller functions or doing a bit of object orientation almost always solves that readability problem for me. -- Nolan Cafferky Software Developer IT Department RBS Interactive [EMAIL PROTECTED]
Re: [HACKERS] Coding style question
Nolan Cafferky wrote: This may not inform the current conversation at all, but a while back I went on a cross-compiler compatibility binge for all of my active projects, and I found that some compilers (*cough* Borland *cough) had some very strange compiler/run time errors unless all variables were declared at the top of the function, before any other code gets executed. For better or for worse, I started strictly declaring all variables in this manner, with initialization happening afterward, and the behavior has stuck with me. I don't know whether any compilers used for postgres builds still have this issue - it's been a few years. We expect the compiler to be C89 compliant at a minimum. If it rejects simple initialisation in the declarations or variables declared in an inner scope then it's hopeless for our purposes, surely. We have lots of such code. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [PATCHES] Bug in WAL backup documentation
Tom Lane wrote: Bernd Helmle [EMAIL PROTECTED] writes: Since 8.1 has done this all along and no one's actually complained about it, I guess no one is using scripts that do cd. I'm inclined to go with Bernd's suggestion to change the docs to match the code, but does anyone have a contrary opinion? I think supplying the absolute path makes archiving scripts less error-prone, which is a good time. So I'd vote for absolute paths. greetings, Florian Pflug ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Design Considerations for New Authentication Methods
To be honest, I have often wondered *why* we support kerberos outside of the uber l33t geek factor. I have not once in a commercial deployment had a business requirement for the beast. LDAP? Now that is a whole other issue :) Single sign-on in a Windows/AD environment (I'm talking clients on windows, servers on linux here - at least in my case). I know several people who use it, most just don't post here ;-) Wouldn't the LDAP auth in 8.2 resolve that? No. LDAP gives me single credentials, but not single sign-on. I still have to enter my password every time I connect. //Magnus ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] Bug in WAL backup documentation
On Fri, 2006-11-03 at 17:34 +0100, Martijn van Oosterhout wrote: On Fri, Nov 03, 2006 at 11:25:09AM -0500, Tom Lane wrote: Since 8.1 has done this all along and no one's actually complained about it, I guess no one is using scripts that do cd. I'm inclined to go with Bernd's suggestion to change the docs to match the code, but does anyone have a contrary opinion? In Unix you can easily prepend $PWD to the string, but I don't know how easy that is in Windows. Windows input anyone? Given the lack of a comprehensive test suite at this stage, I'd vote on the side of least change right now. We know the existing mechanism works, and as Martijn point out there is a workaround, plus as Tom discusses this would only happen if people cd which in my book would be bad programming form anyway. +1 Doc bug for 8.2, feature request for 8.3, unless Windows bites. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] Bug in WAL backup documentation
Simon Riggs wrote: On Fri, 2006-11-03 at 17:34 +0100, Martijn van Oosterhout wrote: On Fri, Nov 03, 2006 at 11:25:09AM -0500, Tom Lane wrote: Since 8.1 has done this all along and no one's actually complained about it, I guess no one is using scripts that do cd. I'm inclined to go with Bernd's suggestion to change the docs to match the code, but does anyone have a contrary opinion? In Unix you can easily prepend $PWD to the string, but I don't know how easy that is in Windows. Windows input anyone? Of course you can get the current directory on Windows, if that's what the question is. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] Bug in WAL backup documentation
Since 8.1 has done this all along and no one's actually complained about it, I guess no one is using scripts that do cd. I'm inclined to go with Bernd's suggestion to change the docs to match the code, but does anyone have a contrary opinion? In Unix you can easily prepend $PWD to the string, but I don't know how easy that is in Windows. Windows input anyone? %CD% gives the same as $PWD in a command shell: C:\Program Files\Microsoft Visual Studio 8\VCecho %CD% C:\Program Files\Microsoft Visual Studio 8\VC //Magnus ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
