Re: [HACKERS] Skytools committed without hackers discussion/review
We allow /contrib to be more lax about beta changes. the postgresql ecosystem is growing and there is a lot of people like packagers that will be a quite irritated if we keep randomly adding completely new code and modules during BETA. Should packagers be concerned with /contrib at all ? Our users want it. Because we have important features that live there. As noted before /contrib is a technical way of ensuring that something gets updated together with core, not a recommendation to include it in a package. Then why did it get added there with the motivation that a lot of users will want it? /Magnus ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Skytools committed without hackers discussion/review
Ühel kenal päeval, K, 2007-10-10 kell 07:44, kirjutas Magnus Hagander: We allow /contrib to be more lax about beta changes. the postgresql ecosystem is growing and there is a lot of people like packagers that will be a quite irritated if we keep randomly adding completely new code and modules during BETA. Should packagers be concerned with /contrib at all ? Our users want it. Because we have important features that live there. Sure, but some features are also moved from /contrib to pgfoundry, and users still may want these too. And sure there are features in contrib that most users don't want. As noted before /contrib is a technical way of ensuring that something gets updated together with core, not a recommendation to include it in a package. Then why did it get added there with the motivation that a lot of users will want it? I think that the main motivation was to ensure that a feature that a lot of users will want will be stable, that is, maintained together with core postgres. exposing stable xid and snapshot to userspace is something needed by more than one postgres add-on (Slony1 replication, Skytools universal queueing and replication) makes it much easier to develop these packages without need to have an extra package to maintain separately from these, yet in sync with core postgres. Hannu ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Skytools committed without hackers discussion/review
On Tue, 2007-10-09 at 18:35 -0400, Andrew Dunstan wrote: I think this project has got too big for us to make things up as we go along. We need to follow processes that are well understood and transparent. Well said, I very much agree. Mostly we do, but since we've just spent more than 6 months between Feature Freeze and Beta. There were no well understood or transparent processes during that period, so nobody is on solid ground trying to enforce a small number of rules with a single individual. Especially when discussing what might be argued was a major item in 8.3 -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql: Added the Skytools extended transaction ID module to contrib as
On Wed, 2007-10-10 at 01:14 -0300, Euler Taveira de Oliveira wrote: Simon Riggs wrote: I would prefer that we backported pg_standby into 8.2 contrib, so the solution is where people need it to be. If not... Don't know about the policy to put things in already-released-version but if it's not the case, we could at least put the code somewhere in the ftp.postgresql.org. IMHO pgfoundry project will confuse people. Both: ftp and pgfoundry. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Skytools committed without hackers discussion/review
Hi, On Tue, 2007-10-09 at 17:10 -0700, Joshua D. Drake wrote: (Will all respect to pginstaller team, I'm *think* it won't take much time to add txid to installer, at least compared to the time that we spent discussing this issue.) With respect, you don't know. My understanding of the pginstaller project is that it is a fairly heft undertaking. It isn't as simple on Linux as just calling to the OS level dependencies because library versions etc.. To avoid any problem/misunderstanding: I do never ever underestimate any packaging work. I fully respect what Dave, Magnus, Hiroshi and others do for pginstaller -- and it is the work that I personally cannot do. I just tried to compare the amount of work from RPM perspective. Regards, -- Devrim GÜNDÜZ PostgreSQL Replication, Consulting, Custom Development, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/ signature.asc Description: This is a digitally signed message part
Re: [HACKERS] [COMMITTERS] pgsql: Added the Skytoolsextended transa ction ID module to contrib as
Simon Riggs wrote: I would prefer that we backported pg_standby into 8.2 contrib, so the solution is where people need it to be. If not... Don't know about the policy to put things in already-released-version but if it's not the case, we could at least put the code somewhere in the ftp.postgresql.org. IMHO pgfoundry project will confuse people. Both: ftp and pgfoundry. Everything released through the pgfoundry release system *is* on the ftp site automatically already... /Magnus ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Skytools committed without hackers discussion/review
We allow /contrib to be more lax about beta changes. the postgresql ecosystem is growing and there is a lot of people like packagers that will be a quite irritated if we keep randomly adding completely new code and modules during BETA. Should packagers be concerned with /contrib at all ? Our users want it. Because we have important features that live there. Sure, but some features are also moved from /contrib to pgfoundry, and users still may want these too. Sure. This is why Dave created stackbuilder. The point is users expect us to ship whatever isincluded in core postgresql, which includes contrib. And sure there are features in contrib that most users don't want. Sure. There are features in the backend that most users don't even know about, much less use/want. As noted before /contrib is a technical way of ensuring that something gets updated together with core, not a recommendation to include it in a package. Then why did it get added there with the motivation that a lot of users will want it? I think that the main motivation was to ensure that a feature that a lot of users will want will be stable, that is, maintained together with core postgres. IMSHO, this is far from enough motivation to put something in post beta. I could accept it *with* proper discussion, during feature freze. Where the argument of not putting it in contrib, but backend-actual instead, could've been properly made. exposing stable xid and snapshot to userspace is something needed by more than one postgres add-on (Slony1 replication, Skytools universal queueing and replication) makes it much easier to develop these packages without need to have an extra package to maintain separately from these, yet in sync with core postgres. So, it should be in the backend, not contrib. And since we're past beta, that means 8.4. If discussion had happened just a day or two before the comit, we could've delayed beta a day to get it in /Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Skytools committed without hackers discussion/review
Simon Riggs wrote: Personally, I want to see Jan contribute more, not less. The link with Slony and related replication technology is critically important to Postgres, which is why Jan has spent so long on it. Generally we should be encouraging everybody to contribute; the project must have an orientation towards action and growth if we are to survive. If Jan had not done this, would there have been a storm of protest? +1 Having said that, the fact that the discussion happened in a forum other than -hackers is clearly something to be avoided in future (ISTR something similar happening with some changes coming from Bizgres a while ago). I don't know how we can avoid this sort of thing happening every now and again, except to encourage folk to include -hackers in their discussions before sending the final patch in (Bruce and others have stated this previously - maybe we need to re-iterate it few weeks *before* every feature freeze...) Cheers Mark ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Possible bugreport 8.3 beta1 on Win32: Looking like a deadlock with AutoVacuum
Alvaro Herrera schreef: Deblauwe Gino wrote: OS: Windows XP Pro SP2 CPU: AMD Athlon 64 3500+ RAM: 2GB DB: PostgreSQL 8.3beta1, compiled by Visual C++ build 1400 I've come to the conclusion that it seems like a deadlock occurs when dropping a column in a table the same moment that table is autovacuumed. Example: ALTER TABLE bondetail DROP COLUMN btw; (user=gino, 16252 records) deadlocks with VACUUM ANALYZE public.bondetail; (user=postgres) Does it really deadlock, or is it just locked waiting for the vacuum to finish? If it deadlocks you should get a message about it and a transaction rollback. Otherwise you should be able to see the ungranted lock in pg_locks. Also it's not clear if autovacuum is involved, or you invoked the VACUUM ANALYZE manually. Can you clarify? No it just looks like a deadlock on first sight. It just takes a very long time. In this case, it takes 10 minutes instead of 5 seconds to execute the query. I was only able to reproduce this on 'ALTER TABLE x DROP COLUMN y;' queries. Those things happen while upgrading our software to a newer version. The more common instructions (SELECT/INSERT/UPDATE/DELETE) work fine the same as adding/changing columns/tables. Greetings Deblauwe Gino
Re: [HACKERS] Skytools committed without hackers discussion/review
Devrim GÜNDÜZ wrote: (Will all respect to pginstaller team, I'm *think* it won't take much time to add txid to installer, at least compared to the time that we spent discussing this issue.) Time is not the issue. /D ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, 2007-10-10 at 07:08 +0100, Simon Riggs wrote: On Tue, 2007-10-09 at 18:35 -0400, Andrew Dunstan wrote: I think this project has got too big for us to make things up as we go along. We need to follow processes that are well understood and transparent. Well said, I very much agree. Mostly we do, but since we've just spent more than 6 months between Feature Freeze and Beta. There were no well understood or transparent processes during that period, so nobody is on solid ground trying to enforce a small number of rules with a single individual. Especially when discussing what might be argued was a major item in 8.3 I should add that I'm not unhappy about how things have happened and I have no complaints to lodge anywhere with anybody. Just wanted to give Jan a bit of moral support, and I've done that now, so time for me to stop. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Locale + encoding combinations
Peter Eisentraut wrote: Dave Page wrote: setlocale(LC_CTYPE, English_United Kingdom.65001) will return null (and not change anything) because it doesn't like the combination of the locale and that encoding (UTF-8). The reason that that call fails is probably that the operating system does not provide such a locale. It doesn't - UTF-8/65001 is a pseudo codepage on Windows with no NLS file defining collation rules etc. as we already discussed. But that's not what we are interested in. We are interested in compatibility between *existing* operating system locales and *PostgreSQL* encoding names. Yes. Let me put my question another way. Latin1 is a perfectly valid encoding for my locale English_United Kingdom. It is accepted by setlocale for LC_ALL. Why does initdb reject it? Why does it insist the encoding is not valid for the locale? /D ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Locale + encoding combinations
Am Mittwoch, 10. Oktober 2007 schrieb Dave Page: Latin1 is a perfectly valid encoding for my locale English_United Kingdom. It is accepted by setlocale for LC_ALL. Why does initdb reject it? Why does it insist the encoding is not valid for the locale? Because initdb works with a finite list of known matches, and your particular combination might not be in that list -- yet. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Skytools committed without hackers discussion/review
On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote: On Tue, 9 Oct 2007 18:35:52 -0500 Michael Glaesemann [EMAIL PROTECTED] wrote: On Oct 9, 2007, at 0:06 , Bruce Momjian wrote: I am surprised we are not backing out the patch and requiring that the patch go through the formal review process. I have no opinion as to the patch itself (other than the fact that it's a not bug fix), but I think this patch should be reverted because it's (a) after feature freeze, (b) had no discussion on hackers (or patches), (c) is not a bug fix. IMO rules can be bent but there should always at least be discussion before a new feature is committed after feature freeze and definitely after beta. Otherwise, the rule appears to be if you can get it in somehow, it's in. I think this almost says it all. My particular gripe about this whole thing is that there are other features that are not too intrusive (or appear so anyway) that are easily more useful that are not being considered at all. Namely, http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It makes the whole process seem tilted and subjective. IMO, the patch is reverted, and submitted for 8.4 or pgfoundry. Yes, reverting is an option, but please, do that at least with an understanding what actually happened. Current discussion seems to give picture that Jan committed some private piece of code without consulting anybody which was not the case. It was actually my patch that was reviewed by 2 senior PostgreSQL developers: Jan and Tom, then later committed by Jan. I don't think the fact that Jan was an interested party by being Slony developer invalidates his status as PostgreSQL developer. Obviously that does not make skipping -hackers less mistake, but there was no evil from anybody and the process for such exceptional case was _mostly_ followed. Now the skipping -hackers part - that was also my mistake, I should have Cc-d the design and code review discussion here also. I just saw the contrib-acceptance as minor question, the main issue was whether Slony was prepared to such a major rewrite of its core parts on such short notice, so I wanted to sync with them first. Also I think several people are annoyed by the Jan asked permission from -core part of the process. But I think if you replace the -core with release manager it will become more understandable. The fact is there are only few people responsible for releases and non-technical decisions need to be made by them. And yes, it should have been accompanied by technical review in -hackers. -- marko ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Locale + encoding combinations
Peter Eisentraut wrote: Am Mittwoch, 10. Oktober 2007 schrieb Dave Page: Latin1 is a perfectly valid encoding for my locale English_United Kingdom. It is accepted by setlocale for LC_ALL. Why does initdb reject it? Why does it insist the encoding is not valid for the locale? Because initdb works with a finite list of known matches, and your particular combination might not be in that list -- yet. So is it just a case of us generating a list of matches that may be Windows specific, or is there more to it than that? /D ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] quote_literal with NULL
On Wed, 2007-10-10 at 14:57 +1000, Brendan Jurd wrote: Wouldn't it be more useful if quote_literal(NULL) yielded the text value 'NULL'? I don't think you can change that now. There could be code out there that relies on that behaviour. It isn't very helpful to return the word NULL in many cases, since the WHERE clause col = NULL does not do the same thing as col is NULL. So you need to know about NULL values and how to handle them in many cases. It might be useful to define a new text concatenation operator ||| that treats NULL values as zero-length strings, so that 'help ' ||| NULL ||| 'me' returns 'help me' -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Skytools committed without hackers discussion/review
On 10/10/2007 12:05 AM, Shane Ambler wrote: Devrim GÜNDÜZ wrote: Hi, On Tue, 2007-10-09 at 16:50 -0700, Joshua D. Drake wrote: IMO, the patch is reverted, and submitted for 8.4 or pgfoundry. You know, txid was discussed in Slony-I + Skytools lists for a reasonably long time, and Tom also commented in that thread. I agree that we broke the policy this time, but this does not mean the end of the world. If it has been discussed and planned for so long then it should have been considered for inclusion earlier, not just slipped under the radar. Even if at feature freeze it wasn't ready it could have been discussed whether it could be added after feature freeze if it reached an acceptable standard. If Slony or Skytools need this for a new feature in their x.y release then it can be a patch that is included with their release or be a prerequisite for their version x.y and detailed in their install steps. There was no intend to slip it in under the radar. Discussion had happened and I failed to realize at the time the code actually looked good that the discussion had happened somewhere other than -hackers. I can certainly take a good amount of flak, especially considering that there was a fault on my side. But what is going on right now here is getting annoying. Also Slony doesn't need this module. We can certainly wait until we are forced by other reasons to bump Slony to version 3.0, declare 3.0 incompatible with 8.3 and switch to using txid then. Slony can continue using its own xxid data type until then. No problem here. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Locale + encoding combinations
Am Mittwoch, 10. Oktober 2007 schrieb Dave Page: So is it just a case of us generating a list of matches that may be Windows specific, or is there more to it than that? You want to peruse src/port/chklocale.c. There is already explicit Windows support in there, so maybe you just need to add on your particular cases. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] quote_literal with NULL
On 10/10/07, Simon Riggs [EMAIL PROTECTED] wrote: On Wed, 2007-10-10 at 14:57 +1000, Brendan Jurd wrote: Wouldn't it be more useful if quote_literal(NULL) yielded the text value 'NULL'? I don't think you can change that now. There could be code out there that relies on that behaviour. Bummer. But I take your point. If there's a good chance someone is going to have their application murdered by a change here, best to leave it alone. I've already gotten around this in my own apps by adding a UDF alternative to quote_literal that plays nicely with NULLs, but thought I'd mention it here in case others were of the same mind. It isn't very helpful to return the word NULL in many cases, since the WHERE clause col = NULL does not do the same thing as col is NULL. So you need to know about NULL values and how to handle them in many cases. Well if you're expecting a possibly-NULL value in your dynamic query you're going to be using something like 'WHERE foo IS NOT DISTINCT FROM ' || quote_literal(bar) anyway. Either way possibly-NULL values need to be anticipated and treated specially. With the string 'NULL' you need DISTINCT FROM. With an actual NULL you need COALESCE. It just seemed to me that the string 'NULL' result was more in line with what quote_literal was supposed to do; and leads to less cluttered code. It might be useful to define a new text concatenation operator ||| that treats NULL values as zero-length strings, so that 'help ' ||| NULL ||| 'me' returns 'help me' That could be cool. Not immediately practical for the dynamic query scenario though: If I do 'WHERE foo IS NOT DISTINCT FROM ' ||| quote_literal(bar) it'll still give me an invalid query string if bar is NULL. Cheers, BJ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] spam on pgfoundry
hello there are lot of spam. Can I remove it from my project? http://pgfoundry.org/tracker/index.php?group_id=1000113atid=497 Pavel ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] pgstattuple module
Hi, I have tested pgstattuple with lot of scenarios want to clarify some of the things. In the Readme file you have explained clearly about pgstattuple function. I mean explanation for each column. But for some of the functions like btree index metapage,single btree pages,bt page items are not clear to me. I have gone through the portal also but unable to find correct information. So if you have any links or document for the above functions please share with me. Actually I want to understand what these columns are the sizes are in (KB, bytes) Thanks Regards M.AHRAM KHAN
Re: [HACKERS] First steps with 8.3 and autovacuum launcher
Simon Riggs wrote: My thoughts are that it doesn't need to. Typically we create objects and then fill them. It isn't that frequent that we would load data, then delete or update more than 20% of it, then attempt other DDL. One scenario that comes to mind is a table that's used in OLTP fashion during day, but it's taken offline for data loading during night. To speed up the data loading, indexes are dropped before the load and recreated afterwards. Even if there's no dead rows in a table, autovacuum will still kick in to freeze it at some point. If a COPY fails it will create dead rows, which should be cleared up by an autoVACUUM. If a COPY fails, the user knows to run a VACUUM or a re-TRUNCATE before re-attempting a modified COPY. So there is potential for more than one VACUUM to be attempted in that case. I wish the user didn't have to know to do that. So there could be an argument for TRUNCATE causing a cancellation of a VACUUM, but I don't see the use case for other DDL. Maybe it would be easier to make all conflicting lock requestors cancel VACUUM. Any VACUUM, or just autovacuum? The only danger I can see is that the autovacuum is always killed and never gets to finish, leading to degrading performance at first and shutdown to prevent xid wraparound at the extreme. Doesn't seem likely under normal circumstances, though. A scenario that comes to mind is having very lazy autovacuum settings, so that vacuum of the table takes longer than 24h, and a daily cron job to run REINDEX. The priority inheritance scheme I proposed earlier would work well with that: instead of killing the autovacuum, set cost delay to zero to let it finish out of the way ASAP. It has it's own set of problems, though. An innocent-looking DROP INDEX would cause the autovacuum to go full steam ahead, hurting performance for others. I think it would be helpful if user-initiated VACUUMs waited behind another VACUUM that was already in progress on the table and then returned immediately as successful when the first VACUUM finishes. That would seem better than queuing up behind the first VACUUM and then repeating the process. I don't think that's a good idea. The second VACUUM wouldn't be a no-op, it would clean up any dead rows accumulated during the first VACUUM. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, 2007-10-10 at 11:50 +0300, Marko Kreen wrote: On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote: IMO, the patch is reverted, and submitted for 8.4 or pgfoundry. Yes, reverting is an option Reverting is only an option if we need to solve a technical problem. If there is no problem then why would we revert it? The only argument I've seen for reverting the patch is that it broke a process. It's hard enough for Developers to get code in without a team of Anti-Developers trying to take it back out again. Perhaps we should have an anti-credits section in the release notes to allow all those who've managed to get work removed get full credit for their anti-efforts. :-) -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] First steps with 8.3 and autovacuum launcher
Heikki, Thanks for your comments, we do need some review on the expected behaviour. On Wed, 2007-10-10 at 11:17 +0100, Heikki Linnakangas wrote: Simon Riggs wrote: My thoughts are that it doesn't need to. Typically we create objects and then fill them. It isn't that frequent that we would load data, then delete or update more than 20% of it, then attempt other DDL. One scenario that comes to mind is a table that's used in OLTP fashion during day, but it's taken offline for data loading during night. To speed up the data loading, indexes are dropped before the load and recreated afterwards. Yes, delaying the index re-creation could cause an effective outage, even if it is more efficient to let the VACUUM happen then re-add the indexes. Even if there's no dead rows in a table, autovacuum will still kick in to freeze it at some point. Yeh, but that can wait a while. If a COPY fails it will create dead rows, which should be cleared up by an autoVACUUM. If a COPY fails, the user knows to run a VACUUM or a re-TRUNCATE before re-attempting a modified COPY. So there is potential for more than one VACUUM to be attempted in that case. I wish the user didn't have to know to do that. Yeh, but thats an 8.4 feature now. So there could be an argument for TRUNCATE causing a cancellation of a VACUUM, but I don't see the use case for other DDL. Maybe it would be easier to make all conflicting lock requestors cancel VACUUM. Any VACUUM, or just autovacuum? After some thought, you and Michael have persuaded me that there is cause to do this for VACUUM as well, but just autovacuum, I think. That also makes the patch simpler, since we don't need to delve inside the av worker to see what it is doing. Alvaro: That means we can just skip your patch altogether, or at least we can discuss them separately now. The only danger I can see is that the autovacuum is always killed and never gets to finish, leading to degrading performance at first and shutdown to prevent xid wraparound at the extreme. Doesn't seem likely under normal circumstances, though. Yeh agreed. Table locks aren't that common, so I think we are safe for 100s of millions of transactions. The user has log messages to warn of that, so I think we're good. A scenario that comes to mind is having very lazy autovacuum settings, so that vacuum of the table takes longer than 24h, and a daily cron job to run REINDEX. A table that big would have a REINDEX run for a very long time too, so I hope the user would notice before too long. The priority inheritance scheme I proposed earlier would work well with that: instead of killing the autovacuum, set cost delay to zero to let it finish out of the way ASAP. It has it's own set of problems, though. An innocent-looking DROP INDEX would cause the autovacuum to go full steam ahead, hurting performance for others. Not very nice performance behaviour. I think it would be helpful if user-initiated VACUUMs waited behind another VACUUM that was already in progress on the table and then returned immediately as successful when the first VACUUM finishes. That would seem better than queuing up behind the first VACUUM and then repeating the process. I don't think that's a good idea. The second VACUUM wouldn't be a no-op, it would clean up any dead rows accumulated during the first VACUUM. That was my first reaction to that thought too! In practice, whoever submitted the first VACUUM can re-run it. So that might be a custom program emitting a stream of VACUUMs or autovacuum doing the same thing. If we need to VACUUM almost continuously then autovacuum will realise this and re-submit. Sitting in the locking queue won't make anything more efficient; like waiting for Rolling Stones tickets at 4am doesn't make them play any better at the gig. I'm not proposing to do this latter idea for now though. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] pgstattuple module
Khan, Mahmood Ahram wrote: But for some of the functions like btree index metapage,single btree pages,bt page items are not clear to me. Those functions display the contents of b-tree pages. They're really meant for debugging purposes. You'll need to understand the internal structure of a b-tree index to make sense of the output. The b-tree internals are explained in the source tree, in src/backend/access/nbtree/README, and the source files in the same directory. In fact, in 8.3, those functions have been moved to a new, separate contrib module called pageinspect, along with similar functions for debugging heap contents. Hopefully that makes the distinction between functions useful for DBAs and functions useful for developers more clear. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, Oct 10, 2007 at 11:50:12AM +0300, Marko Kreen wrote: On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote: On Tue, 9 Oct 2007 18:35:52 -0500 Michael Glaesemann [EMAIL PROTECTED] wrote: On Oct 9, 2007, at 0:06 , Bruce Momjian wrote: I am surprised we are not backing out the patch and requiring that the patch go through the formal review process. I have no opinion as to the patch itself (other than the fact that it's a not bug fix), but I think this patch should be reverted because it's (a) after feature freeze, (b) had no discussion on hackers (or patches), (c) is not a bug fix. IMO rules can be bent but there should always at least be discussion before a new feature is committed after feature freeze and definitely after beta. Otherwise, the rule appears to be if you can get it in somehow, it's in. I think this almost says it all. My particular gripe about this whole thing is that there are other features that are not too intrusive (or appear so anyway) that are easily more useful that are not being considered at all. Namely, http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It makes the whole process seem tilted and subjective. IMO, the patch is reverted, and submitted for 8.4 or pgfoundry. Yes, reverting is an option, but please, do that at least with an understanding what actually happened. Current discussion seems to give picture that Jan committed some private piece of code without consulting anybody which was not the case. At least I am fully aware that it's not a private piece of code. And in general, I trust Jan (and of course Tom as well) to take a patch from elsewhere and put it in. My objections are twofold: 1) We don't add things after beta. I can live with adding it during feature freeze since it's contrib, and reviewed by these people, but I think it's horrible to do it after we've shipped beta1. 2) I get the strong feeling that it's going into contrib only because it missed feature freeze. If it hadn't missed feature freeze, it wuold be in the backend and not contrib. If the plan is that it lives in contrib forever, that argument falls. But if the plan is to migrate it into the backend for 8.4, then I strongly object to using contrib just as a way to get it in even though we're feature-frozen. It was actually my patch that was reviewed by 2 senior PostgreSQL developers: Jan and Tom, then later committed by Jan. I don't think the fact that Jan was an interested party by being Slony developer invalidates his status as PostgreSQL developer. Certainly not. Obviously that does not make skipping -hackers less mistake, but there was no evil from anybody and the process for such exceptional case was _mostly_ followed. I don't think anybody thinks there were evil intentions behind it. I certainly don't think Jan would've committed it if he expected people to dislike it technically. All objections have been procedural, AFICS. Now the skipping -hackers part - that was also my mistake, I should have Cc-d the design and code review discussion here also. I just saw the contrib-acceptance as minor question, the main issue was whether Slony was prepared to such a major rewrite of its core parts on such short notice, so I wanted to sync with them first. Also I think several people are annoyed by the Jan asked permission from -core part of the process. But I think if you replace the -core with release manager it will become more understandable. The fact is there are only few people responsible for releases and non-technical decisions need to be made by them. And yes, it should have been accompanied by technical review in -hackers. AFAIK, our release manager is -hackers concensus. We don't *have* a release manager as such. The closest thing you'd get in that case is the -packagers list which is for those building the binary packages, and they were not consulted... But again, I don't see any issues with the technical side of things. It's procedural, and placement (is contrib really the right place for it). //Magnus ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Locale + encoding combinations
Peter Eisentraut wrote: Am Mittwoch, 10. Oktober 2007 schrieb Dave Page: So is it just a case of us generating a list of matches that may be Windows specific, or is there more to it than that? You want to peruse src/port/chklocale.c. There is already explicit Windows support in there, so maybe you just need to add on your particular cases. Yup, found that - thanks. I'll look at updating that list. /D ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Skytools committed without hackers discussion/review
Magnus Hagander wrote: On Wed, Oct 10, 2007 at 11:50:12AM +0300, Marko Kreen wrote: On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote: On Tue, 9 Oct 2007 18:35:52 -0500 Michael Glaesemann [EMAIL PROTECTED] wrote: On Oct 9, 2007, at 0:06 , Bruce Momjian wrote: I am surprised we are not backing out the patch and requiring that the patch go through the formal review process. I have no opinion as to the patch itself (other than the fact that it's a not bug fix), but I think this patch should be reverted because it's (a) after feature freeze, (b) had no discussion on hackers (or patches), (c) is not a bug fix. IMO rules can be bent but there should always at least be discussion before a new feature is committed after feature freeze and definitely after beta. Otherwise, the rule appears to be if you can get it in somehow, it's in. I think this almost says it all. My particular gripe about this whole thing is that there are other features that are not too intrusive (or appear so anyway) that are easily more useful that are not being considered at all. Namely, http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It makes the whole process seem tilted and subjective. IMO, the patch is reverted, and submitted for 8.4 or pgfoundry. Yes, reverting is an option, but please, do that at least with an understanding what actually happened. Current discussion seems to give picture that Jan committed some private piece of code without consulting anybody which was not the case. At least I am fully aware that it's not a private piece of code. And in general, I trust Jan (and of course Tom as well) to take a patch from elsewhere and put it in. My objections are twofold: 1) We don't add things after beta. I can live with adding it during feature freeze since it's contrib, and reviewed by these people, but I think it's horrible to do it after we've shipped beta1. yeah that is exactly the point - if we do have a feature freeze we should hold to it. if we are in BETA we should not add any new code. 2) I get the strong feeling that it's going into contrib only because it missed feature freeze. If it hadn't missed feature freeze, it wuold be in the backend and not contrib. If the plan is that it lives in contrib forever, that argument falls. But if the plan is to migrate it into the backend for 8.4, then I strongly object to using contrib just as a way to get it in even though we're feature-frozen. yeah I agree that code like this should be either in core or somewhere else (either pgfoundry or even shipped as part of the replication solutions mentioned which is basically something slony did for ages with the xxid stuff). Just pushing it now into contrib results in people wanting to use one of those solution having to deal with 3 kinds of packages: 1. postgresql 2. postgresql-contrib 3. skytools/slony/... instead of just two which does not strike me as much of an improvement. Stefan ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] First steps with 8.3 and autovacuum launcher
Simon Riggs wrote: OK, I've got this working now. It successfully handles this test case, which trips up on an auto ANALYZE every time I run it. ... I notice when we cancel an AV worker it always says cancelling autovacuum of table, even when its just an ANALYZE. Wasn't important before but now looks a little strange. ... Any other input anyone? What about VACUUM (not just ANALYZE)? The starter of the thread Possible bugreport 8.3 beta1 on Win32: Looking like a deadlock with AutoVacuum complained about vacuum, not analyze. It is just as Tom said earlier: it will be before end of beta that people will complain about more than just restoring dumps. ;-) So does this approach work for both analyze as well as vacuum? Best Regards Michael Paesold ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] First steps with 8.3 and autovacuum launcher
On Wed, 2007-10-10 at 11:04 +0200, Michael Paesold wrote: Simon Riggs wrote: OK, I've got this working now. It successfully handles this test case, which trips up on an auto ANALYZE every time I run it. ... I notice when we cancel an AV worker it always says cancelling autovacuum of table, even when its just an ANALYZE. Wasn't important before but now looks a little strange. ... Any other input anyone? What about VACUUM (not just ANALYZE)? The starter of the thread Possible bugreport 8.3 beta1 on Win32: Looking like a deadlock with AutoVacuum complained about vacuum, not analyze. It is just as Tom said earlier: it will be before end of beta that people will complain about more than just restoring dumps. ;-) I'm not looking at this from the perspective of how to make restores work better, I'm looking at the common use cases. Re-adding FKs after a major data load is part of the Performance Tips section. So does this approach work for both analyze as well as vacuum? It doesn't work with VACUUM. My thoughts are that it doesn't need to. Typically we create objects and then fill them. It isn't that frequent that we would load data, then delete or update more than 20% of it, then attempt other DDL. If a COPY fails it will create dead rows, which should be cleared up by an autoVACUUM. If a COPY fails, the user knows to run a VACUUM or a re-TRUNCATE before re-attempting a modified COPY. So there is potential for more than one VACUUM to be attempted in that case. So there could be an argument for TRUNCATE causing a cancellation of a VACUUM, but I don't see the use case for other DDL. Maybe it would be easier to make all conflicting lock requestors cancel VACUUM. I think it would be helpful if user-initiated VACUUMs waited behind another VACUUM that was already in progress on the table and then returned immediately as successful when the first VACUUM finishes. That would seem better than queuing up behind the first VACUUM and then repeating the process. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Locale + encoding combinations
Dave Page wrote: Peter Eisentraut wrote: Am Mittwoch, 10. Oktober 2007 schrieb Dave Page: So is it just a case of us generating a list of matches that may be Windows specific, or is there more to it than that? You want to peruse src/port/chklocale.c. There is already explicit Windows support in there, so maybe you just need to add on your particular cases. Yup, found that - thanks. I'll look at updating that list. OK so I added the appropriate entries (and posted the patch to -patches), but my original question remains: why can I only select the *default* encoding for the chosen locale, but not other ones that are also be valid according to setlocale? Is this a bug, or is there some technical reason? /D ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [COMMITTERS] pgsql: Added the Skytools extended transaction ID module to contrib as
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
- --On Wednesday, October 10, 2007 07:09:20 +0100 Simon Riggs
[EMAIL PROTECTED] wrote:
On Wed, 2007-10-10 at 01:14 -0300, Euler Taveira de Oliveira wrote:
Simon Riggs wrote:
I would prefer that we backported pg_standby into 8.2 contrib, so the
solution is where people need it to be. If not...
Don't know about the policy to put things in already-released-version
but if it's not the case, we could at least put the code somewhere in
the ftp.postgresql.org. IMHO pgfoundry project will confuse people.
Both: ftp and pgfoundry.
ftp://ftp.postgresql.org/pub/projects/{gborg,pgFoundry}
-
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED]
Yahoo . yscrappy Skype: hub.orgICQ . 7615664
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.4 (FreeBSD)
iD8DBQFHDLxU4QvfyHIvDvMRAjkCAJ9nN3JxEV/drYMO7uMN2uH1phhJbwCgiCKN
etp2sBtJoBTtqoAVUgLSkzw=
=yAQh
-END PGP SIGNATURE-
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] Locale + encoding combinations
Am Mittwoch, 10. Oktober 2007 schrieb Dave Page: my original question remains: why can I only select the *default* encoding for the chosen locale, but not other ones that are also be valid according to setlocale? Is this a bug, or is there some technical reason? One locale works only with one encoding. There are no default or perhaps alternative encodings for one locale; there is only one. The whole point of the exercise is to determine what the spelling of that one encoding is in PostgreSQL. Perhaps you are confused about the naming. These are all entirely separate locales: en_GB.iso88591 en_GB.iso885915 en_GB.utf8 Someone was friendly enough to include the name of the encoding used by the locale into its name, but that doesn't mean that en_GB has three alternative encodings or something. At least that's the model we have on POSIX platforms. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Skytools committed without hackers discussion/review
On 10/10/07, Magnus Hagander [EMAIL PROTECTED] wrote: All objections have been procedural, AFICS. Lets not talk about mistakes we made for a moment. And I agree with the rest of the objections in general. But I'd like to summarise why I still hope the exception can be made even this late. This is directly related to attitude to the first submission to 8.2: unless Slony uses it we are not interested. Now is the only moment which won't come again in several years that it's possible to unify txid handling in Slony and Skytools and also make the functionality available to broader public. This due to the fact that Slony 2.0 which will be released with 8.3 will not support PostgreSQL version lower then 8.3. Yes, we realized the opportunity too late and now it's question if PostgreSQL is flexible enough to react to this. Note that rejection now does not cause any big problems to either Slony and Skytools, we will keep our internal versions of the module, invisible to anybody else. But the potential use of the module is huge - it's killer feature is that it allows to implement high-performance multi-reader / multi-writer queues inside database. Well, I know this sounds unimpressive, queues do not belong to standard toolbox when doing database developement. And those who have tried to implement them, carry a avoid at any cost tag, because thus far there has not been a way to implement robust and well-perfoming queue inside general-purpose database. Now txid can change that. E.g. in Skype, it has become irreplaceable tool for coordinating work between several databases. Here we are probably going overboard with usage of queues... -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Locale + encoding combinations
Dave Page [EMAIL PROTECTED] writes: However, setlocale() will also accept other valid combinations on Windows, which initdb will not, for example: English_United Kingdom.28591 (Latin1) Is there any reason not to accept other combinations that setlocale() is happy with? Are you certain that that acceptance actually represents support? Have you checked that it rejects combinations involving real code pages (ie, NOT 65001) that don't really work with the locale? regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Locale + encoding combinations
Peter Eisentraut wrote: Am Mittwoch, 10. Oktober 2007 schrieb Dave Page: my original question remains: why can I only select the *default* encoding for the chosen locale, but not other ones that are also be valid according to setlocale? Is this a bug, or is there some technical reason? One locale works only with one encoding. There are no default or perhaps alternative encodings for one locale; there is only one. The whole point of the exercise is to determine what the spelling of that one encoding is in PostgreSQL. Perhaps you are confused about the naming. These are all entirely separate locales: en_GB.iso88591 en_GB.iso885915 en_GB.utf8 Someone was friendly enough to include the name of the encoding used by the locale into its name, but that doesn't mean that en_GB has three alternative encodings or something. At least that's the model we have on POSIX platforms. OK, sorting out my terminology deficiencies has helped - thanks. The problem seems to be: initdb --locale English_United Kingdom.28591 works, but initdb -E LATIN1 --locale English_United Kingdom does not. That's good (albeit inconsistent), I know how to fix pgInstaller now. What isn't so good is: C:\pgbin\initdb --locale English_United Kingdom.9 -D data initdb: invalid locale name English_United Kingdom.9 initdb: invalid locale name English_United Kingdom.9 initdb: invalid locale name English_United Kingdom.9 initdb: invalid locale name English_United Kingdom.9 initdb: invalid locale name English_United Kingdom.9 initdb: invalid locale name English_United Kingdom.9 The files belonging to this database system will be owned by user Dave. This user must also own the server process. The database cluster will be initialized with locale English_United Kingdom.1252 . The default database encoding has accordingly been set to WIN1252. === Shouldn't that have failed? Regards, Dave ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, Oct 10, 2007 at 03:27:17PM +0300, Marko Kreen wrote: On 10/10/07, Magnus Hagander [EMAIL PROTECTED] wrote: All objections have been procedural, AFICS. Lets not talk about mistakes we made for a moment. And I agree with the rest of the objections in general. But I'd like to summarise why I still hope the exception can be made even this late. This is directly related to attitude to the first submission to 8.2: unless Slony uses it we are not interested. Now is the only moment which won't come again in several years that it's possible to unify txid handling in Slony and Skytools and also make the functionality available to broader public. This due to the fact that Slony 2.0 which will be released with 8.3 will not support PostgreSQL version lower then 8.3. Yes, we realized the opportunity too late and now it's question if PostgreSQL is flexible enough to react to this. Note that rejection now does not cause any big problems to either Slony and Skytools, we will keep our internal versions of the module, invisible to anybody else. But the potential use of the module is huge - it's killer feature is that it allows to implement high-performance multi-reader / multi-writer queues inside database. Well, I know this sounds unimpressive, queues do not belong to standard toolbox when doing database developement. And those who have tried to implement them, carry a avoid at any cost tag, because thus far there has not been a way to implement robust and well-perfoming queue inside general-purpose database. Now txid can change that. E.g. in Skype, it has become irreplaceable tool for coordinating work between several databases. Here we are probably going overboard with usage of queues... If it is this irreplacable killer feature, it should *not* be in contrib. It should be in the core backend, and we should be discussing if we can bend the rules for that. This is the proper forum for discussing that, so let's bring that question to the table. Our beta-1 is already fairly broken (the locale stuff on our most downloaded platform), so perhaps we should pull that one back, put this stuff in the backend, and try to get a beta2 out ASAP? Putting it in contrib just because we were too late to put it in the backend, but it is reallyi really important for our users just doesn't make sense. //Magnus ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Locale + encoding combinations
Tom Lane wrote: Dave Page [EMAIL PROTECTED] writes: However, setlocale() will also accept other valid combinations on Windows, which initdb will not, for example: English_United Kingdom.28591 (Latin1) Is there any reason not to accept other combinations that setlocale() is happy with? Are you certain that that acceptance actually represents support? Have you checked that it rejects combinations involving real code pages (ie, NOT 65001) that don't really work with the locale? It fails with ones that Microsoft have decided don't belong in my language group and therefore aren't installed. It accepts all the others I've tried, but then from the sample I've looked, they all have 0-9a-zA-Z in them so I guess they're all capable of handling English. Regards, Dave. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Timezone database changes
Peter Eisentraut [EMAIL PROTECTED] writes: We are not considering an interval scheduling system, we are considering a database system. Such a system should have the basic property that if you store A, it will read out as A. I'm not sure that I think this sort of rigid thinking works very well in the wonderland that is date/time behavior. When the rules of the game (ie, DST laws) are changing underneath you, who is to say exactly what reading out as A means? Arguably, TIMESTAMP WITH TIME ZONE does the right thing now, and would cease to do the right thing if we changed it as I think you intend. Given that all involved agree that the SQL spec is hopelessly broken in this area, becoming more compliant with it is not a goal that I think we should strive for blindly. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] permission denied for tablespace pg_global?
Hiroshi Saito [EMAIL PROTECTED] writes: postgres=# SELECT pg_size_pretty(pg_tablespace_size(1664)); ERROR: permission denied for tablespace pg_global This is an intentional change, documented in the release notes: * Put some security restrictions on the dbsize functions (Tom) Restrict pg_database_size() to users who can connect to the target database (note that CONNECT privilege is granted by default, so this does not change the default behavior). Restrict pg_tablespace_size() to users who have CREATE privilege on the tablespace (which is not granted by default), except when the tablespace is the default tablespace for the current database (since we treat that as implicitly allowing use of the tablespace). There was a long thread about this in -hackers two months ago: http://archives.postgresql.org/pgsql-hackers/2007-08/msg00948.php regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Locale + encoding combinations
Dave Page [EMAIL PROTECTED] writes: Tom Lane wrote: Are you certain that that acceptance actually represents support? Have you checked that it rejects combinations involving real code pages (ie, NOT 65001) that don't really work with the locale? It fails with ones that Microsoft have decided don't belong in my language group and therefore aren't installed. It accepts all the others I've tried, but then from the sample I've looked, they all have 0-9a-zA-Z in them so I guess they're all capable of handling English. That doesn't exactly fill me with confidence. Maybe you need to make some tests involving a non-English base locale? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] type money causes unrestorable dump
Gregory Stark [EMAIL PROTECTED] writes: Long term I liked the idea from a few years ago of having a default format which would be attached to a column just like a default collation can be attached. Then you can declare your currency columns as regular integers but mark them as being formatted as currency by default. pg_dump would presumably explicitly override the default and format the integers as plain integers and restore the default format string as part of its DDL. At least for the case at hand, this seems a pretty horrid solution. It could easily lead to a value that had been $1.01 being reloaded as 101 Yen, or vice versa, neither of which would make anyone happy. If anything I would gripe that type money is not locale-specific enough; it doesn't have a way to prevent similar confusions between say US$ and AU$, or any two currencies using the same symbol. The better long-term solution would be to go over to a tagged-type arrangement, in which each value is *explicitly* marked with its currency. This needn't be a whole lot slower than the current arrangement --- I think D'Arcy already took the main speed hit when he went from int4 (pass by value) to int8 (pass by reference). regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Timezone database changes
* Peter Eisentraut [EMAIL PROTECTED] [071010 09:58]: If we make an appointment at 12-November-2007 at 10:00 CET (winter time) and next week those in charge decide to postpone the change to winter time from 28-October-2007 to 25-November-2007, what becomes of the appointment? Do we still meet when the hands point to 10, or when? And to make matters worse, what if your appointment includes a audio(or video) conference with colleagues sitting in London, who you've told to meet at 16:00 (OK, my timezones/daylight savings, etc may be off) So, do you meet when the hands point at 10, or do they meet when the hands point at 4? -- Aidan Van Dyk Create like a god, [EMAIL PROTECTED] command like a king, http://www.highrise.ca/ work like a slave. signature.asc Description: Digital signature
Re: [HACKERS] Locale + encoding combinations
Tom Lane wrote: Dave Page [EMAIL PROTECTED] writes: Tom Lane wrote: Are you certain that that acceptance actually represents support? Have you checked that it rejects combinations involving real code pages (ie, NOT 65001) that don't really work with the locale? It fails with ones that Microsoft have decided don't belong in my language group and therefore aren't installed. It accepts all the others I've tried, but then from the sample I've looked, they all have 0-9a-zA-Z in them so I guess they're all capable of handling English. That doesn't exactly fill me with confidence. Maybe you need to make some tests involving a non-English base locale? Hmm, I'm guessing these probably shouldn't work: [EMAIL PROTECTED]:~$ setlc Japanese_Japan.28605 Japanese_Japan.28605 [EMAIL PROTECTED]:~$ setlc Japanese_Japan.28595 Japanese_Japan.28595 [EMAIL PROTECTED]:~$ setlc Russian_Russia.1252 Russian_Russia.1252 [EMAIL PROTECTED]:~$ setlc Russian_Russia.28591 Russian_Russia.28591 1252 == WIN1252 28591 == LATIN1 28605 == LATIN9 28595 == ISO8859-5 (Cyrillic) 28597 == ISO8859-7 (Greek) In fact, it looks like it'll allow me to use anything thats installed, regardless of whether they're liekly to be compatible. So much for trusting setlocale() :-( /D ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Timezone database changes
Am Mittwoch, 10. Oktober 2007 schrieb Tom Lane: I'm not sure that I think this sort of rigid thinking works very well in the wonderland that is date/time behavior. When the rules of the game (ie, DST laws) are changing underneath you, who is to say exactly what reading out as A means? Arguably, TIMESTAMP WITH TIME ZONE does the right thing now, and would cease to do the right thing if we changed it as I think you intend. If we make an appointment at 12-November-2007 at 10:00 CET (winter time) and next week those in charge decide to postpone the change to winter time from 28-October-2007 to 25-November-2007, what becomes of the appointment? Do we still meet when the hands point to 10, or when? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Locale + encoding combinations
Dave Page [EMAIL PROTECTED] writes: OK so I added the appropriate entries (and posted the patch to -patches), but my original question remains: why can I only select the *default* encoding for the chosen locale, but not other ones that are also be valid according to setlocale? Is this a bug, or is there some technical reason? Well, the chklocale code is designed around the assumption that there *is* only one encoding for which a locale setting will work, with C/POSIX being a special case. I think we are talking a bit at cross-purposes here, because the Windows equivalent to this notion seems to be English_United Kingdom.1252 whereas you seem to be defining locale as just English_United Kingdom. Does it not work the way you want if you make the installer pass locale strings of the first form to initdb? regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Locale + encoding combinations
Tom Lane wrote: Dave Page [EMAIL PROTECTED] writes: OK so I added the appropriate entries (and posted the patch to -patches), but my original question remains: why can I only select the *default* encoding for the chosen locale, but not other ones that are also be valid according to setlocale? Is this a bug, or is there some technical reason? Well, the chklocale code is designed around the assumption that there *is* only one encoding for which a locale setting will work, with C/POSIX being a special case. I think we are talking a bit at cross-purposes here, because the Windows equivalent to this notion seems to be English_United Kingdom.1252 whereas you seem to be defining locale as just English_United Kingdom. Does it not work the way you want if you make the installer pass locale strings of the first form to initdb? Yes, it seems it does (see my previous email to Peter): http://archives.postgresql.org/pgsql-hackers/2007-10/msg00447.php So I guess that's how I'll fix the installer. There is another issue though as I mentioned in the post above - that it complains about an invalid encoding specifier on the encoding name, then ignores it and uses the default which seems wrong to me. /D ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Skytools committed without hackers discussion/review
Marko Kreen wrote: On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote: On Tue, 9 Oct 2007 18:35:52 -0500 Michael Glaesemann [EMAIL PROTECTED] wrote: On Oct 9, 2007, at 0:06 , Bruce Momjian wrote: I am surprised we are not backing out the patch and requiring that the patch go through the formal review process. I have no opinion as to the patch itself (other than the fact that it's a not bug fix), but I think this patch should be reverted because it's (a) after feature freeze, (b) had no discussion on hackers (or patches), (c) is not a bug fix. IMO rules can be bent but there should always at least be discussion before a new feature is committed after feature freeze and definitely after beta. Otherwise, the rule appears to be if you can get it in somehow, it's in. I think this almost says it all. My particular gripe about this whole thing is that there are other features that are not too intrusive (or appear so anyway) that are easily more useful that are not being considered at all. Namely, http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It makes the whole process seem tilted and subjective. IMO, the patch is reverted, and submitted for 8.4 or pgfoundry. Yes, reverting is an option, but please, do that at least with an understanding what actually happened. Current discussion seems to give picture that Jan committed some private piece of code without consulting anybody which was not the case. It was actually my patch that was reviewed by 2 senior PostgreSQL developers: Jan and Tom, then later committed by Jan. I don't think the fact that Jan was an interested party by being Slony developer invalidates his status as PostgreSQL developer. Obviously that does not make skipping -hackers less mistake, but there was no evil from anybody and the process for such exceptional case was _mostly_ followed. Now the skipping -hackers part - that was also my mistake, I should have Cc-d the design and code review discussion here also. I just saw the contrib-acceptance as minor question, the main issue was whether Slony was prepared to such a major rewrite of its core parts on such short notice, so I wanted to sync with them first. Also I think several people are annoyed by the Jan asked permission from -core part of the process. But I think if you replace the -core with release manager it will become more understandable. The fact is there are only few people responsible for releases and non-technical decisions need to be made by them. And yes, it should have been accompanied by technical review in -hackers. I don't think this is accurate. Jan talked to Tom, not all of core, and Tom just gave general approval. Tom still expected this to go through the hackers review process. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [COMMITTERS] pgsql: Added the Skytools extended transaction ID module to contrib as
On Wednesday 10 October 2007 02:09, Simon Riggs wrote:
On Wed, 2007-10-10 at 01:14 -0300, Euler Taveira de Oliveira wrote:
Simon Riggs wrote:
I would prefer that we backported pg_standby into 8.2 contrib, so the
solution is where people need it to be. If not...
Don't know about the policy to put things in already-released-version
but if it's not the case, we could at least put the code somewhere in
the ftp.postgresql.org. IMHO pgfoundry project will confuse people.
Both: ftp and pgfoundry.
Putting it on pgfoundry would automatically put it in the ftp tree
(ftp://ftp.postgresql.org/pub/projects/pgFoundry). If it was to go on
pgfoundry (which I'd recommend) I'd suggest removing it from 8.3 contrib
before we release (cause having it in both places is really going to cause
confusion)
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Re: [HACKERS] Skytools committed without hackers discussion/review
Agreed. I think if we had followed procedure the code would have been accepted post-beta1. --- Marko Kreen wrote: On 10/10/07, Magnus Hagander [EMAIL PROTECTED] wrote: All objections have been procedural, AFICS. Lets not talk about mistakes we made for a moment. And I agree with the rest of the objections in general. But I'd like to summarise why I still hope the exception can be made even this late. This is directly related to attitude to the first submission to 8.2: unless Slony uses it we are not interested. Now is the only moment which won't come again in several years that it's possible to unify txid handling in Slony and Skytools and also make the functionality available to broader public. This due to the fact that Slony 2.0 which will be released with 8.3 will not support PostgreSQL version lower then 8.3. Yes, we realized the opportunity too late and now it's question if PostgreSQL is flexible enough to react to this. Note that rejection now does not cause any big problems to either Slony and Skytools, we will keep our internal versions of the module, invisible to anybody else. But the potential use of the module is huge - it's killer feature is that it allows to implement high-performance multi-reader / multi-writer queues inside database. Well, I know this sounds unimpressive, queues do not belong to standard toolbox when doing database developement. And those who have tried to implement them, carry a avoid at any cost tag, because thus far there has not been a way to implement robust and well-perfoming queue inside general-purpose database. Now txid can change that. E.g. in Skype, it has become irreplaceable tool for coordinating work between several databases. Here we are probably going overboard with usage of queues... -- marko -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Skytools committed without hackers discussion/review
Magnus Hagander wrote: Now txid can change that. E.g. in Skype, it has become irreplaceable tool for coordinating work between several databases. Here we are probably going overboard with usage of queues... If it is this irreplacable killer feature, it should *not* be in contrib. It should be in the core backend, and we should be discussing if we can bend the rules for that. This is the proper forum for discussing that, so let's bring that question to the table. Our beta-1 is already fairly broken (the locale stuff on our most downloaded platform), so perhaps we should pull that one back, put this stuff in the backend, and try to get a beta2 out ASAP? Putting it in contrib just because we were too late to put it in the backend, but it is reallyi really important for our users just doesn't make sense. I also agree with this. We have to pretend it isn't in /contrib now, figure out where want it, then put it there (contrib, pgfoundry, core). -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] some points for FAQ
Pavel Stehule wrote: OK, how do we even explain this idea in the FAQ. It pulls 20 random values from 1 to 1? That seems pretty hard to code to me. Where do you get the 1 number from? How do you know you will hit a match in 20 tries? Number 1 you have to store in application .. it's magic constant. It similar our statistics. And sometimes you have to actualise it. This is stochastic methods, so it's possible so it doesn't return any value, and you have to repeat it. Using this method expect knowledge about generating random numbers. This method is far to ideal, but on databases with big traffic only this is usable. OK, but this is clearly something I can't just throw into the FAQ and expect people to figure it out, and going into major detail to explain it in the FAQ isn't logical either. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Skytools committed without hackers discussion/review
Magnus Hagander wrote: If it is this irreplacable killer feature, it should *not* be in contrib. It should be in the core backend, and we should be discussing if we can bend the rules for that. This is the proper forum for discussing that, so let's bring that question to the table. Our beta-1 is already fairly broken (the locale stuff on our most downloaded platform), so perhaps we should pull that one back, put this stuff in the backend, and try to get a beta2 out ASAP? Putting it in contrib just because we were too late to put it in the backend, but it is reallyi really important for our users just doesn't make sense. I agree with most of this. If it's really that important we should abandon beta1 in effect and open the tree for just this and as much of the extra tsearch samples as we care to take. But that's a decision that should be taken openly, after discussion on -hackers. I don't think I'm being arrogant when I say that if I was completely unaware of this proposal then a great many other people probably were too. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Skytools committed without hackers discussion/review
Magnus Hagander wrote: If it is this irreplacable killer feature, it should *not* be in contrib. It should be in the core backend, and we should be discussing if we can bend the rules for that. This is the proper forum for discussing that, so let's bring that question to the table. +1 there, I don't think it should go into contrib just cause it was a late entry. It really seems to be a matter of whether it gets into 8.3 or 8.4 Our beta-1 is already fairly broken (the locale stuff on our most downloaded platform), so perhaps we should pull that one back, put this stuff in the backend, and try to get a beta2 out ASAP? The question there is how long will it take to reach a decision of where the patch belongs? (8.3 8.4 or contrib) Putting it in contrib just because we were too late to put it in the backend, but it is reallyi really important for our users just doesn't make sense. +1 -- Shane Ambler [EMAIL PROTECTED] Get Sheeky @ http://Sheeky.Biz ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Skytools committed without hackers discussion/review
Simon Riggs wrote: On Tue, 2007-10-09 at 18:35 -0400, Andrew Dunstan wrote: I think this project has got too big for us to make things up as we go along. We need to follow processes that are well understood and transparent. Well said, I very much agree. Mostly we do, but since we've just spent more than 6 months between Feature Freeze and Beta. There were no well understood or transparent processes during that period, so nobody is on solid ground trying to enforce a small number of rules with a single individual. Especially when discussing what might be argued was a major item in 8.3 What? Even Jan agrees he didn't follow the procedures. I have no idea how you are coming to the conclusion there are no well understood processes. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Timezone database changes
Aidan Van Dyk [EMAIL PROTECTED] writes: * Peter Eisentraut [EMAIL PROTECTED] [071010 09:58]: If we make an appointment at 12-November-2007 at 10:00 CET (winter time) and next week those in charge decide to postpone the change to winter time from 28-October-2007 to 25-November-2007, what becomes of the appointment? Do we still meet when the hands point to 10, or when? And to make matters worse, what if your appointment includes a audio(or video) conference with colleagues sitting in London, who you've told to meet at 16:00 (OK, my timezones/daylight savings, etc may be off) Exactly ... there is more than one right answer here. The answer that PG's TIMESTAMP WITH TIME ZONE code deems to be right is that UTC is reality. That's a definition that is indeed useful for a wide variety of real-world problems. In a lot of cases where it's not so useful, TIMESTAMP WITHOUT TIME ZONE does the right thing. I'm not sure that there's a significant use-case for a third behavior, and I definitely don't think you can make an argument from first principles that the UTC-based definition is wrong. (FWIW, Red Hat has been struggling with this exact problem of cross-time-zone meeting times for some years now, and has pretty much arrived at the conclusion that company meeting times are to be defined in UTC...) The arguments that have been made for storing a zone along with the UTC value seem to mostly boil down to it should present the value the same way I entered it, but if you accept that argument then why do we have DateStyle? If it's OK to regurgitate 11-12-2007 as 2007-12-11, I'm not clear on why adjusting timezone isn't OK. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Locale + encoding combinations
Dave Page [EMAIL PROTECTED] writes: In fact, it looks like it'll allow me to use anything thats installed, regardless of whether they're liekly to be compatible. So much for trusting setlocale() :-( Yech :-(. Count on Microsloth to get this wrong. Anyone have any ideas on how to tell if a locale setting *really* works on Windows? regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [COMMITTERS] pgsql: Added the Skytools extended transaction ID module to contrib as
Robert Treat wrote: On Wednesday 10 October 2007 02:09, Simon Riggs wrote: On Wed, 2007-10-10 at 01:14 -0300, Euler Taveira de Oliveira wrote: Simon Riggs wrote: I would prefer that we backported pg_standby into 8.2 contrib, so the solution is where people need it to be. If not... Don't know about the policy to put things in already-released-version but if it's not the case, we could at least put the code somewhere in the ftp.postgresql.org. IMHO pgfoundry project will confuse people. Both: ftp and pgfoundry. Putting it on pgfoundry would automatically put it in the ftp tree (ftp://ftp.postgresql.org/pub/projects/pgFoundry). If it was to go on pgfoundry (which I'd recommend) I'd suggest removing it from 8.3 contrib before we release (cause having it in both places is really going to cause confusion) One of pgfoundry's explicit purposes is for backports of features. Given that we (rightly) don't backport new features in mainline releases, where else should they go? I don't buy the confusing argument. If necessary the author can plaster big red notices in a README on the pgfoundry release saying don't use this past postgres version x cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Skytools committed without hackers discussion/review
Magnus Hagander [EMAIL PROTECTED] writes: On Wed, Oct 10, 2007 at 03:27:17PM +0300, Marko Kreen wrote: Now txid can change that. E.g. in Skype, it has become irreplaceable tool for coordinating work between several databases. Here we are probably going overboard with usage of queues... If it is this irreplacable killer feature, it should *not* be in contrib. It should be in the core backend, and we should be discussing if we can bend the rules for that. It may be a killer feature for Skytools and Slony, but even so that's a small part of our userbase. I see nothing wrong with having it in contrib now with an eye to migrating to the core later, when and if we see there's enough demand for that. Another reason for that approach is that once it's in core it will be very much harder to make any tweaks to the API; and with the prospective uses being largely unwritten as yet, it hardly seems unlikely we might not want some changes. I think our two realistic options today are (1) leave the code where it is, or (2) remove it. While Jan clearly failed to follow the agreed procedures, I'm not convinced the transgression was severe enough to justify (2). regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] quote_literal with NULL
On Wed, Oct 10, 2007 at 4:57 AM, in message [EMAIL PROTECTED], Brendan Jurd [EMAIL PROTECTED] wrote: On 10/10/07, Simon Riggs [EMAIL PROTECTED] wrote: On Wed, 2007-10-10 at 14:57 +1000, Brendan Jurd wrote: Wouldn't it be more useful if quote_literal(NULL) yielded the text value 'NULL'? I don't think you can change that now. There could be code out there that relies on that behaviour. Bummer. But I take your point. If there's a good chance someone is going to have their application murdered by a change here, best to leave it alone. In particular, it seems like exactly what you would want for values you're going to use in an INSERT or the SET clause of an UPDATE. -Kevin ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Timezone database changes
Tom Lane wrote: Exactly ... there is more than one right answer here. The answer that PG's TIMESTAMP WITH TIME ZONE code deems to be right is that UTC is reality. That's a definition that is indeed useful for a wide variety of real-world problems. In a lot of cases where it's not so useful, TIMESTAMP WITHOUT TIME ZONE does the right thing. I'm not sure that there's a significant use-case for a third behavior, and I definitely don't think you can make an argument from first principles that the UTC-based definition is wrong. (FWIW, Red Hat has been struggling with this exact problem of cross-time-zone meeting times for some years now, and has pretty much arrived at the conclusion that company meeting times are to be defined in UTC...) The arguments that have been made for storing a zone along with the UTC value seem to mostly boil down to it should present the value the same way I entered it, but if you accept that argument then why do we have DateStyle? If it's OK to regurgitate 11-12-2007 as 2007-12-11, I'm not clear on why adjusting timezone isn't OK. I am thinking additional documention is the only good solution here. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Locale + encoding combinations
Dave Page [EMAIL PROTECTED] writes: ... There is another issue though as I mentioned in the post above - that it complains about an invalid encoding specifier on the encoding name, then ignores it and uses the default which seems wrong to me. Yeah, if you look at chklocale() in initdb.c this is clearly how it works, but there's a comment /* should we exit here? */ so whoever wrote it wasn't all that convinced it was the right behavior. Given that 8.3 is raising the stakes for having a correct locale specification at initdb time, it seems right to me to error out if a bogus locale switch is given, rather than whining and then substituting the environment default. Any objections? That still leaves us with the problem of how to tell whether a locale spec is bad on Windows. Judging by your example, Windows checks whether the code page is present but not whether it is sane for the base locale. What happens when there's a mismatch --- eg, what encoding do system messages come out in? regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Skytools committed without hackers discussion/review
Bruce Momjian [EMAIL PROTECTED] writes: Marko Kreen wrote: Also I think several people are annoyed by the Jan asked permission from -core part of the process. I don't think this is accurate. Jan talked to Tom, not all of core, and Tom just gave general approval. Tom still expected this to go through the hackers review process. Well, my view of the discussion was that Jan asked core if any of us would veto a late contrib addition. I trust you'll agree that if anyone on core were to vote against, it would not have gone in; and so it seemed reasonable to me for Jan to check this before expending any further effort on creating a patch. I asked a couple questions and then said it sounded okay to me; I don't recall now if anyone else commented, but this was definitely a discussion on -core not personal email. In any case, since that was in advance of seeing the code, I certainly was expecting a -patches submission before commit ... regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Skytools committed without hackers discussion/review
Bruce Momjian [EMAIL PROTECTED] writes: I also agree with this. We have to pretend it isn't in /contrib now, figure out where want it, then put it there (contrib, pgfoundry, core). Putting it in core now would mean forcing a post-beta1 initdb, which I don't think adequate cause has been shown for. Possibly we should sit on the decision for awhile and see if any initdb-forcing bugs are reported. But for the moment I think only the contrib or pgfoundry options are acceptable. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Skytools committed without hackers discussion/review
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote: Magnus Hagander [EMAIL PROTECTED] writes: On Wed, Oct 10, 2007 at 03:27:17PM +0300, Marko Kreen wrote: Now txid can change that. E.g. in Skype, it has become irreplaceable tool for coordinating work between several databases. Here we are probably going overboard with usage of queues... If it is this irreplacable killer feature, it should *not* be in contrib. It should be in the core backend, and we should be discussing if we can bend the rules for that. It may be a killer feature for Skytools and Slony, but even so that's a small part of our userbase. I see nothing wrong with having it in contrib now with an eye to migrating to the core later, when and if we see there's enough demand for that. Another reason for that approach is that once it's in core it will be very much harder to make any tweaks to the API; and with the prospective uses being largely unwritten as yet, it hardly seems unlikely we might not want some changes. Considering the core operations are now being in active use some 6-7 years, I really fail to see how there can be anything to tweak, unless you are speaking changing naming style. IMHO the core operations are already as stable as PostgreSQL use of MVCC, as the module just exports backend internal state... Current set of functions is the minimal necessary to implement queue operations, there is nothing more to remove. We might want to add some helper functions for query generation, but that does not affect core operation. Another thing can can be done is more compact representation for txid_snapshot type, but that also won't affect core operation. I am very happy for txid being in contrib, I'm not arguing against that, but the hint that txid module is something that just freshly popped out of thin air is irking me. I think our two realistic options today are (1) leave the code where it is, or (2) remove it. While Jan clearly failed to follow the agreed procedures, I'm not convinced the transgression was severe enough to justify (2). Thanks for being understanding. -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, Oct 10, 2007 at 11:30:47AM -0400, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I also agree with this. We have to pretend it isn't in /contrib now, figure out where want it, then put it there (contrib, pgfoundry, core). Putting it in core now would mean forcing a post-beta1 initdb, which I don't think adequate cause has been shown for. Ok. In that case, my vote is pgfoundry (heh, I'm sure that's clear by now). I don't think an adequate cause to break all our procedures to stick it in core has been shown either. Possibly we should sit on the decision for awhile and see if any initdb-forcing bugs are reported. But for the moment I think only the contrib or pgfoundry options are acceptable. This sounds like a good fallback - if the option opens up, I really think it should be put in the backend. (Assuming it's technically sound - I still haven't checked the actual code, but I'm assuming it's Ok since Jan approved it) //Magnus ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Skytools committed without hackers discussion/review
Hi, On Wed, 2007-10-10 at 09:19 +0100, Simon Riggs wrote: I should add that I'm not unhappy about how things have happened and I have no complaints to lodge anywhere with anybody. Just wanted to give Jan a bit of moral support I have the same feelings, so +1. Regards, -- Devrim GÜNDÜZ PostgreSQL Replication, Consulting, Custom Development, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/ signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Skytools committed without hackers discussion/review
Marko Kreen [EMAIL PROTECTED] writes: IMHO the core operations are already as stable as PostgreSQL use of MVCC, as the module just exports backend internal state... Well, it exports backend internal state that did not exist before 8.2 (ie, XID epoch). So it doesn't seem all that set in stone to me. Another thing can can be done is more compact representation for txid_snapshot type, but that also won't affect core operation. That's another thing that's likely to become very much harder to change once it's in core. People keep threatening to produce a working in-place-upgrade process, and once that's reality the on-disk representation of core types is going to be hard to change. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, 2007-10-10 at 10:29 -0400, Bruce Momjian wrote: Simon Riggs wrote: Mostly we do, but since we've just spent more than 6 months between Feature Freeze and Beta. There were no well understood or transparent processes during that period, so nobody is on solid ground trying to enforce a small number of rules with a single individual. Especially when discussing what might be argued was a major item in 8.3 What? Even Jan agrees he didn't follow the procedures. I have no idea how you are coming to the conclusion there are no well understood processes. Results look good to me, so I'm not lodging a complaint, just explaining my position. The main rule is discuss-it-first-on-Hackers and if Jan didn't follow that then he is wrong. But since he nearly got it right by discussing it with some people, nothing bad happened and the outcome is worth having I don't see why we would want to pick Jan up for anything, or back-out those changes. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [COMMITTERS] pgsql: Added the Skytools extended transaction ID module to contrib as
On Wednesday 10 October 2007 10:57, Andrew Dunstan wrote:
Robert Treat wrote:
On Wednesday 10 October 2007 02:09, Simon Riggs wrote:
On Wed, 2007-10-10 at 01:14 -0300, Euler Taveira de Oliveira wrote:
Simon Riggs wrote:
I would prefer that we backported pg_standby into 8.2 contrib, so the
solution is where people need it to be. If not...
Don't know about the policy to put things in already-released-version
but if it's not the case, we could at least put the code somewhere in
the ftp.postgresql.org. IMHO pgfoundry project will confuse people.
Both: ftp and pgfoundry.
Putting it on pgfoundry would automatically put it in the ftp tree
(ftp://ftp.postgresql.org/pub/projects/pgFoundry). If it was to go on
pgfoundry (which I'd recommend) I'd suggest removing it from 8.3 contrib
before we release (cause having it in both places is really going to
cause confusion)
One of pgfoundry's explicit purposes is for backports of features.
I can't think of any contrib modules we've added that also required backwards
comptible modules to be released on foundry at the same time. ISTM that such
a requirement would be an argument that such a thing doesn't belong in
contrib at all.
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] quote_literal with NULL
On Wed, 2007-10-10 at 10:12 -0500, Kevin Grittner wrote: On Wed, Oct 10, 2007 at 4:57 AM, in message [EMAIL PROTECTED], Brendan Jurd [EMAIL PROTECTED] wrote: On 10/10/07, Simon Riggs [EMAIL PROTECTED] wrote: On Wed, 2007-10-10 at 14:57 +1000, Brendan Jurd wrote: Wouldn't it be more useful if quote_literal(NULL) yielded the text value 'NULL'? I don't think you can change that now. There could be code out there that relies on that behaviour. Bummer. But I take your point. If there's a good chance someone is going to have their application murdered by a change here, best to leave it alone. In particular, it seems like exactly what you would want for values you're going to use in an INSERT or the SET clause of an UPDATE. Perhaps have quote_nullable() then as well? You then use quote_nullable() in INSERT and UPDATE SET clauses and quote_literal() in SELECT WHERE clauses. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Timezone database changes
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote: The arguments that have been made for storing a zone along with the UTC value seem to mostly boil down to it should present the value the same way I entered it, but if you accept that argument then why do we have DateStyle? If it's OK to regurgitate 11-12-2007 as 2007-12-11, I'm not clear on why adjusting timezone isn't OK. Actually, what I meant at least (not sure if others meant it), is storing the value in the timezone it was entered, along with what zone that was. That makes the value stable with respect to the zone it belongs to, instead of being stable with respect to UTC. When DST rules change, the value is in effect reinterpreted as if it were input using the new rules. To me that's also what the name of the type suggests it does. I imagine internally it would convert each value to UTC just before performing any calculations on it, and generally be irritating to work with. But the public interface would do the other right thing. Well, for political time zones anyway. I have no idea what that approach is supposed to do with numeric offsets, or the old PST8PDT type stuff. Anyway, getting back to documentation, I think it's just necessary to somehow point out the difference between these two behaviors in the section about the date and time types, and which type is more appropriate for which situation. I don't know if there's enough room to provide effective examples without getting too bogged down in details though. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Skytools committed without hackers discussion/review
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote: Marko Kreen [EMAIL PROTECTED] writes: IMHO the core operations are already as stable as PostgreSQL use of MVCC, as the module just exports backend internal state... Well, it exports backend internal state that did not exist before 8.2 (ie, XID epoch). So it doesn't seem all that set in stone to me. Ok. Lets say the API concepts are 6-7 years old. The epoch was a only thing missing in API ca. 2 years ago (the 8byte txid was written on 8.0), so now the API is complete. Another thing can can be done is more compact representation for txid_snapshot type, but that also won't affect core operation. That's another thing that's likely to become very much harder to change once it's in core. People keep threatening to produce a working in-place-upgrade process, and once that's reality the on-disk representation of core types is going to be hard to change. Good point. But txid_snapshot happens to have couple of free bits that can be used to create backwards compatibility, so I don't think that could be a big problem. Anyway, the in-place upgrade seems a month-two away couple of years now :) -- marko ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Skytools committed without hackers discussion/review
Simon Riggs wrote: On Wed, 2007-10-10 at 10:29 -0400, Bruce Momjian wrote: Simon Riggs wrote: Mostly we do, but since we've just spent more than 6 months between Feature Freeze and Beta. There were no well understood or transparent processes during that period, so nobody is on solid ground trying to enforce a small number of rules with a single individual. Especially when discussing what might be argued was a major item in 8.3 What? Even Jan agrees he didn't follow the procedures. I have no idea how you are coming to the conclusion there are no well understood processes. Results look good to me, so I'm not lodging a complaint, just explaining my position. The main rule is discuss-it-first-on-Hackers and if Jan didn't follow that then he is wrong. But since he nearly got it right by discussing it with some people, nothing bad happened and the outcome is worth having I don't see why we would want to pick Jan up for anything, or back-out those changes. The results have nothing to do with whether the process was followed. We do not ignore process violations just because the outcome was OK. And Jan did not come even close to following procedure. He just asked core if they would object to such an addition post-feature freeze. He did not show code to core so there is no sense the core approved the patch. And Jan did not show the patch to hackers or discuss the patch application. I do not want to keep bashing Jan but Simon's statements require me to set the record straight. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Skytools committed without hackers discussion/review
Magnus Hagander [EMAIL PROTECTED] writes: (Assuming it's technically sound - I still haven't checked the actual code, but I'm assuming it's Ok since Jan approved it) I hadn't looked at it either, but here are a few things that need review: * Why no binary I/O support for the new datatype? We tend to expect that for all core types. * Why is txid_current_snapshot() excluding subtransaction XIDs? That might be all right for the current uses in Slony/Skytools, but it seems darn close to a bug for any other use. * Why is txid_current_snapshot() reading SerializableSnapshot rather than an actually current snap as its name suggests? This isn't just misleading, this will fail completely when SerializableSnapshot goes away, as seems likely to happen in 8.4 (and no, we won't keep it just because txid might want it). regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, Oct 10, 2007 at 11:47:15AM -0400, Tom Lane wrote: Marko Kreen [EMAIL PROTECTED] writes: IMHO the core operations are already as stable as PostgreSQL use of MVCC, as the module just exports backend internal state... Well, it exports backend internal state that did not exist before 8.2 (ie, XID epoch). So it doesn't seem all that set in stone to me. Another thing can can be done is more compact representation for txid_snapshot type, but that also won't affect core operation. That's another thing that's likely to become very much harder to change once it's in core. People keep threatening to produce a working in-place-upgrade process, and once that's reality the on-disk representation of core types is going to be hard to change. Well, if that is a concern, than it's an equally big concern to have it in contrib. If people start using it, they're not going to care about us saying hey, it was in contrib, why did you use it, when we earlier said in order do use our whiz-bang stuff, you must install from contrib. We'll have complaints that it's too hard to install, but we won't manage to escape from the responsibility to keep it working. //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] quote_literal with NULL
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Perhaps have quote_nullable() then as well? You then use quote_nullable() in INSERT and UPDATE SET clauses and quote_literal() in SELECT WHERE clauses. I still don't see the use case. Wouldn't your app still need to check for nullability anyway, to avoid = NULL? (Aside: seems to me that SET foo = NULL; really should be SET foo TO NULL; to be consistent with WHERE foo IS NULL;) - -- Greg Sabino Mullane [EMAIL PROTECTED] PGP Key: 0x14964AC8 200710101221 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iD8DBQFHDPwyvJuQZxSWSsgRAwGaAJ92ICR+MyclkmWvJRkC4vazIw+b0ACghpZt WXbCxe0abFlp8jwr0ol/fac= =oWqD -END PGP SIGNATURE- ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [COMMITTERS] pgsql: Added the Skytools extended transaction ID module to contrib as
Robert Treat [EMAIL PROTECTED] writes: On Wednesday 10 October 2007 10:57, Andrew Dunstan wrote: One of pgfoundry's explicit purposes is for backports of features. I can't think of any contrib modules we've added that also required backwards comptible modules to be released on foundry at the same time. ISTM that such a requirement would be an argument that such a thing doesn't belong in contrib at all. AFAICT there isn't any market for a backport of txid. Slony won't depend on it before their next release, which will require PG = 8.3 for other reasons. Skytools already has an internal version in their existing releases. And the code won't work before PG 8.2 so any backport couldn't go very far anyway. So while Andrew's statement is true in general, I don't think it's very relevant to a consideration of what to do with txid. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Timezone database changes
Trevor Talbot [EMAIL PROTECTED] writes: Actually, what I meant at least (not sure if others meant it), is storing the value in the timezone it was entered, along with what zone that was. That makes the value stable with respect to the zone it belongs to, instead of being stable with respect to UTC. When DST rules change, the value is in effect reinterpreted as if it were input using the new rules. What happens if the rules change in a way that makes the value illegal or ambiguous (ie, it now falls into a DST gap)? But perhaps more to the point, please show use-cases demonstrating that this behavior is more useful than the pure-UTC behavior. For storage of actual time observations, I think pure-UTC is unquestionably the more useful. Peter's example of a future appointment time is a possible counterexample, but as observed upthread it's hardly clear which behavior is more desirable in such a case. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] permission denied for tablespace pg_global?
- Original Message - From: Tom Lane [EMAIL PROTECTED] To: Hiroshi Saito [EMAIL PROTECTED] Cc: pgsql-hackers@postgresql.org Sent: Wednesday, October 10, 2007 9:55 PM Subject: Re: [HACKERS] permission denied for tablespace pg_global? Hiroshi Saito [EMAIL PROTECTED] writes: postgres=# SELECT pg_size_pretty(pg_tablespace_size(1664)); ERROR: permission denied for tablespace pg_global This is an intentional change, documented in the release notes: * Put some security restrictions on the dbsize functions (Tom) Restrict pg_database_size() to users who can connect to the target database (note that CONNECT privilege is granted by default, so this does not change the default behavior). Restrict pg_tablespace_size() to users who have CREATE privilege on the tablespace (which is not granted by default), except when the tablespace is the default tablespace for the current database (since we treat that as implicitly allowing use of the tablespace). There was a long thread about this in -hackers two months ago: http://archives.postgresql.org/pgsql-hackers/2007-08/msg00948.php regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] quote_literal with NULL
On Oct 10, 2007, at 11:24 , Greg Sabino Mullane wrote: (Aside: seems to me that SET foo = NULL; really should be SET foo TO NULL; to be consistent with WHERE foo IS NULL;) The = character has different meanings in these two cases. UPDATE foos SET foo = NULL -- assignment WHERE bar IS NULL -- comparison AND foo = 'ignore me' -- comparison Or is that what the smiley was about? :) Michael Glaesemann grzm seespotcode net PGP.sig Description: This is a digitally signed message part
Re: [HACKERS] quote_literal with NULL
Greg Sabino Mullane [EMAIL PROTECTED] writes: Perhaps have quote_nullable() then as well? You then use quote_nullable() in INSERT and UPDATE SET clauses and quote_literal() in SELECT WHERE clauses. I still don't see the use case. Wouldn't your app still need to check for nullability anyway, to avoid = NULL? Well, it's clearly useful in INSERT and UPDATE. For WHERE cases, you might or might not be able to use it, but I note that quote_nullable() would work much more like what happens if you use a parameter symbol and then bind NULL as the actual parameter value ... In hindsight we should probably have done quote_literal the way the OP suggests, but I concur that it's too late to change it. An additional function seems a reasonable compromise. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] permission denied for tablespace pg_global?
Hi. Oops, I pursued the thread and was not competent. I will inquire thoroughly. Thanks!! Regards, Hiroshi Saito - Original Message - From: Tom Lane [EMAIL PROTECTED] Hiroshi Saito [EMAIL PROTECTED] writes: postgres=# SELECT pg_size_pretty(pg_tablespace_size(1664)); ERROR: permission denied for tablespace pg_global This is an intentional change, documented in the release notes: * Put some security restrictions on the dbsize functions (Tom) Restrict pg_database_size() to users who can connect to the target database (note that CONNECT privilege is granted by default, so this does not change the default behavior). Restrict pg_tablespace_size() to users who have CREATE privilege on the tablespace (which is not granted by default), except when the tablespace is the default tablespace for the current database (since we treat that as implicitly allowing use of the tablespace). There was a long thread about this in -hackers two months ago: http://archives.postgresql.org/pgsql-hackers/2007-08/msg00948.php regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Skytools committed without hackers discussion/review
Magnus Hagander [EMAIL PROTECTED] writes: On Wed, Oct 10, 2007 at 11:30:47AM -0400, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I also agree with this. We have to pretend it isn't in /contrib now, figure out where want it, then put it there (contrib, pgfoundry, core). Putting it in core now would mean forcing a post-beta1 initdb, which I don't think adequate cause has been shown for. Ok. In that case, my vote is pgfoundry (heh, I'm sure that's clear by now). I don't think an adequate cause to break all our procedures to stick it in core has been shown either. I just don't see the point in putting it in pgfoundry. It's already in pgfoundry as part of Skytools. The whole point of having such a datatype is to build common interface to abstract away the internals of the database. That makes the pgfoundry modules (Skytools and Slony) easier to maintain separately. Putting txid on pgfoundry just means splitting one pgfoundry package into two. Moving code around doesn't make it any easier to maintain, the portion that depends on the database internals will still depend on database internals. Putting it in core or contrib means that when we change the snapshot mechanics in 8.4 the same developer will be able to fix the module at the same time (and find out if his changes break it at the same time). Also different versions of Postgres would include appropriate txid modules without having to maintain a single version with a million ifdefs to cover multiple versions of Postgres. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, 10 Oct 2007 11:04:53 -0400 Tom Lane [EMAIL PROTECTED] wrote: Magnus Hagander [EMAIL PROTECTED] writes: On Wed, Oct 10, 2007 at 03:27:17PM +0300, Marko Kreen wrote: Now txid can change that. E.g. in Skype, it has become irreplaceable tool for coordinating work between several databases. Here we are probably going overboard with usage of queues... If it is this irreplacable killer feature, it should *not* be in contrib. It should be in the core backend, and we should be discussing if we can bend the rules for that. It may be a killer feature for Skytools and Slony, but even so that's a small part of our userbase. This is a valid point. Skytools is cool. Slony is cool, but their user bases compared to PostgreSQL proper are miniscule. We need to be considering the potential issues (if there are any) that this can cause for the larger community. I can think of a couple of immediate ones, I believe Tom has touched on some of these elsewhere in the thread. 1) Maintainability 2) The eventual (lord it is about 5 years late) upgrade in place feature 3) This possibly should be in core not contrib. *If* it should be in core, it needs to push to 8.4. It is that simple. Else we need to open the tree and start reviewing all those patches that got left behind before at feature freeze because of possible open issues. If it doesn't need to be in core, in certainly has zero need to be in contrib and can push to pgFoundry. I think our two realistic options today are (1) leave the code where it is, or (2) remove it. While Jan clearly failed to follow the agreed procedures, I'm not convinced the transgression was severe enough to justify (2). Well I would like to step back and remove the Jan element. Jan has really been taking a beating here. The reality is, A committer made a mistake. It doesn't matter who it was. The code needs to be removed. I just don't see a way around it, unless we are willing to go back and review other items. This patch has not been peer reviewed, it's benefit has not been discussed on -hackers, and in general it hasn't been vetted. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ signature.asc Description: PGP signature
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, 10 Oct 2007 18:33:03 +0300 Marko Kreen [EMAIL PROTECTED] wrote: Considering the core operations are now being in active use some 6-7 years, I really fail to see how there can be anything to tweak, unless you are speaking changing naming style. Well that is the problem right there isn't it? As someone who has financed, shipped, developed and tested an integrated Replication solution for *years*, this statement is obvious naivety. Your code, may be the most blessed, pristine and bug free code *in your environment* but your environment is hardly the only environment out there and things *always* come up. IMHO the core operations are already as stable as PostgreSQL use of MVCC, as the module just exports backend internal state... Current set of functions is the minimal necessary to implement queue operations, there is nothing more to remove. Having a hard time buying that. MVCC has the pleasure of being tested everyday by hundreds of thousands of installations. We might want to add some helper functions for query generation, but that does not affect core operation. But it does affect the inclusion argument. Another thing can can be done is more compact representation for txid_snapshot type, but that also won't affect core operation. You are starting to bring up things in your own post that may need to change before inclusion. This is *exactly* why the code should be removed. It wasn't vetted on -hackers, and if it had been we may have had a more complete piece of software. I am very happy for txid being in contrib, I'm not arguing against that, but the hint that txid module is something that just freshly popped out of thin air is irking me. Certainly, I can understand this as you have had a long time to work with, develop and mature the code. However it is just out of thin air. It doesn't exist except for a very small part of the PostgreSQL world. It may not be new to you, but it is certainly 100% new to many of the long time contributors of this project. I think our two realistic options today are (1) leave the code where it is, or (2) remove it. While Jan clearly failed to follow the agreed procedures, I'm not convinced the transgression was severe enough to justify (2). Thanks for being understanding. We all try to be :) but I do feel it needs to be removed, pgFoundry is the perfect place for this. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ signature.asc Description: PGP signature
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, 10 Oct 2007 13:01:54 +0200 Stefan Kaltenbrunner [EMAIL PROTECTED] wrote: yeah I agree that code like this should be either in core or somewhere else (either pgfoundry or even shipped as part of the replication solutions mentioned which is basically something slony did for ages with the xxid stuff). Just pushing it now into contrib results in people wanting to use one of those solution having to deal with 3 kinds of packages: 1. postgresql 2. postgresql-contrib 3. skytools/slony/... instead of just two which does not strike me as much of an improvement. I am not sure I am buying this argument. Skytools/slony is not PostgreSQL. Should we also include Greg's recent replication software? Or perhaps pgCluster? I can think of at least a dozen modules that would be cool to have in contrib... plpgsm or orafce? Yes this is just a small bit but Marko has already made suggestions in his own thread of items that should possibly change. This needs to be pulled out, put on pgFoundry or submitted for 8.4. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ signature.asc Description: PGP signature
Re: [HACKERS] Timezone database changes
Am Mittwoch, 10. Oktober 2007 schrieb Tom Lane: Peter's example of a future appointment time is a possible counterexample, but as observed upthread it's hardly clear which behavior is more desirable in such a case. Whereas the most realistic solution to my example might be, the parties involved reconfirm their appointment, I expect that public transportation companies such as railways and airlines have specific rules to deal with these situations. That might give us some insight what the industrial-strength resolution could be, even if we deem it inappropriate to implement it at the end. So if someone has knowledge in that area, I'd be interested. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Timezone database changes
Trevor Talbot wrote: , what I meant at least (not sure if others meant it), is storing the value in the timezone it was entered, along with what zone that was. That makes the value stable with respect to the zone it belongs to, instead of being stable with respect to UTC. When DST rules change, the value is in effect reinterpreted as if it were input using the new rules. To me that's also what the name of the type suggests it does. I would argue that this isn't necessarily more helpful than what we have. Many of us work in an in an international environment, and DST rules (and, I'm sure you all remember the Venezuela case, time zones) change, and at different instances in time. To reiterate what the SQL standard says, a WITH TIMEZONE element should have information on the original time zone it was stored as, but only in the form of an offset from UTC in hours and minutes. SQL has no notion of time zone labels, so if we decide to store these, we wouldn't be any closer to SQL compliancy. An interesting observation is that, as far as I can tell, the original time zone is only applied when casting the element to a string. Apart from that, it's not used. I would suggest that the WITH TIMEZONE elements are converted to UTC when inserted into the database. Since all operations on it are based on its UTC form, it's most efficient ( I believe) if the data is stored that way. To be compliant, an offset (hours and minutes) to the time zone that was used when storing the time should be stored as well. --Magne ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] pg_standby location (was added the Skytools extended transaction ID module)
On Wed, 10 Oct 2007, Robert Treat wrote: Simon Riggs wrote: I would prefer that we backported pg_standby into 8.2 contrib, so the solution is where people need it to be. If not... If it was to go on pgfoundry (which I'd recommend) I'd suggest removing it from 8.3 contrib before we release (cause having it in both places is really going to cause confusion) There is a perception that code in contrib, while not essential, has at least been reviewed by core to some degree and therefore is relatively safe to install. I've listened to people state that they're not installing code from some random pgfoundry package on the production system in a meeting, while contrib code was considered perfectly acceptable. pg_standby has such a wide potential audience that I'd hate to see it viewed as second-class code in this fashion were it moved to pgfoundry and pulled out of the 8.3 contrib. If the concern is to avoid confusion, I'd suggest clearly labeling the foundry project something like pg_standby 8.2 backport, so people know it's aimed at being stable when run against an 8.2 server; that would make it obvious it's not intended for 8.3 systems as well. Right now I've been suggesting people use the version that comes with 8.3, but I get the feeling not everyone is comfortable with that; having a release specifically labeled 8.2 compatible would be a help. Like Simon, I'd _like_ to see it show up in the 8.2 contrib, but as that goes against the project's stable version policies I don't expect that to happen. Having a backport (which may be the same code) as a pgfoundry project, with the understanding that it comes with the base contrib in 8.3, would help clear some of the concerns about the quality of the code I've run into. Pushing the entire thing onto pgfoundry will make it even harder to get certain type of corporate customers to use pg_standby, which is a clear step backwards as far as I'm concerned. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Skytools committed without hackers discussion/review
Joshua D. Drake [EMAIL PROTECTED] writes: If it doesn't need to be in core, in certainly has zero need to be in contrib and can push to pgFoundry. One advantage of having it in contrib is buildfarm testing, as indeed we already found out ... although it's true that *keeping* it there now that it passes probably won't teach us too much more. But I think the argument that was being made was mostly that the Slony and Skytools projects would find it easier to depend on a contrib module than on something that has to be fetched separately from pgfoundry. Now they could work around that by including copies of the pgfoundry project in their own distributions, but then they have a collision problem if someone tries to install both together. (I have no idea how likely that is, though; it might not be a big problem in practice?) regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, 2007-10-10 at 12:08 -0400, Bruce Momjian wrote: The results have nothing to do with whether the process was followed. We do not ignore process violations just because the outcome was OK. And Jan did not come even close to following procedure. He just asked core if they would object to such an addition post-feature freeze. He did not show code to core so there is no sense the core approved the patch. And Jan did not show the patch to hackers or discuss the patch application. I do not want to keep bashing Jan but Simon's statements require me to set the record straight. Record straight; please consider my comments withdrawn. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Skytools committed without hackers discussion/review
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote: Magnus Hagander [EMAIL PROTECTED] writes: (Assuming it's technically sound - I still haven't checked the actual code, but I'm assuming it's Ok since Jan approved it) I hadn't looked at it either, but here are a few things that need review: * Why no binary I/O support for the new datatype? We tend to expect that for all core types. As I said, the current module is minimal, my goal was to have code where there is nothing to remove. But for a data type that targets core, yes binary i/o support should be added. * Why is txid_current_snapshot() excluding subtransaction XIDs? That might be all right for the current uses in Slony/Skytools, but it seems darn close to a bug for any other use. In queue usage the transaction that stores snapshots does not do any other work on its own, so it does not matter, main requirement is that txid_current()/txid_current_snapshot() be symmetric - whatever the txid_current() outputs, the txid_current_snapshot() measures. But I agree, supporting subtransactions makes the API more universal. And it wouldn't break Slony/PgQ current usage. * Why is txid_current_snapshot() reading SerializableSnapshot rather than an actually current snap as its name suggests? This isn't just misleading, this will fail completely when SerializableSnapshot goes away, as seems likely to happen in 8.4 (and no, we won't keep it just because txid might want it). If you say so. This code is from original xxid module and has worked thus far. :) If the requirement mentioned above is not broken, the code needs to be brought in line with current backend coding standards. Will look into the problems and send patch tomorrow, today has been too tiring already... -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Skytools committed without hackers discussion/review
Ühel kenal päeval, K, 2007-10-10 kell 12:18, kirjutas Tom Lane: Magnus Hagander [EMAIL PROTECTED] writes: (Assuming it's technically sound - I still haven't checked the actual code, but I'm assuming it's Ok since Jan approved it) I hadn't looked at it either, but here are a few things that need review: * Why no binary I/O support for the new datatype? We tend to expect that for all core types. should be easy to add, likely a copy-past from any other varlena type. * Why is txid_current_snapshot() excluding subtransaction XIDs? That might be all right for the current uses in Slony/Skytools, but it seems darn close to a bug for any other use. Just thinking aloud here : I think that the reason has something to do with it being for stored snapshots used by different transactions for determining info about other committed transactions , and the stored snapshots and transaction ids become visible from SQL level only after both are committed. There may be cases where you want to use it from other places, say C code for user-defined function dealing with visibility of other transactions, but before adding in subtransactions and thus possibly bloating the storage, we should first identify such case. Most likely it is better to just use in-backend snapshots straight from backend internals if you dont need to store them. * Why is txid_current_snapshot() reading SerializableSnapshot rather than an actually current snap as its name suggests? This isn't just misleading, this will fail completely when SerializableSnapshot goes away, as seems likely to happen in 8.4 (and no, we won't keep it just because txid might want it). Why is SerializableSnapshot going away ? How will we do serialized isolation level in 8.4 then? Hannu ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Skytools committed without hackers discussion/review
Ühel kenal päeval, K, 2007-10-10 kell 11:06, kirjutas Joshua D. Drake: On Wed, 10 Oct 2007 18:01:34 +0100 Gregory Stark [EMAIL PROTECTED] wrote: Magnus Hagander [EMAIL PROTECTED] writes: On Wed, Oct 10, 2007 at 11:30:47AM -0400, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I also agree with this. We have to pretend it isn't in /contrib now, figure out where want it, then put it there (contrib, pgfoundry, core). I just don't see the point in putting it in pgfoundry. It's already in pgfoundry as part of Skytools. The whole point of having such a datatype is to build common interface to abstract away the internals of the database. That makes the pgfoundry modules (Skytools and Slony) easier to maintain separately. I missed the part that it is part of Skytools already but as counter point, what makes sense at that point is for Skytools to remove it and make it it's own module. Is'nt this just what happened when it was moved to contrib ? That way Slony (which is not a pgfoundry project) or anyone else that wants to make use of it can. Putting it in core or contrib means that when we change the snapshot mechanics in 8.4 the same developer will be able to fix the module at the same time (and find out if his changes break it at the same time). Which is very cool, for *8.4* :) Joshua D. Drake ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Timezone database changes
=?ISO-8859-1?Q?Magne_M=E6hre?= [EMAIL PROTECTED] writes: I would suggest that the WITH TIMEZONE elements are converted to UTC when inserted into the database. Since all operations on it are based on its UTC form, it's most efficient ( I believe) if the data is stored that way. To be compliant, an offset (hours and minutes) to the time zone that was used when storing the time should be stored as well. Well, the question is what would we *do* with the latter? If we have that override the TimeZone zone for output, we will break a lot of things. There's also the question of what to put into a computed timestamp value. Consider regression=# select timestamptz '2007/10/01 00:00 EDT'; timestamptz 2007-10-01 00:00:00-04 (1 row) regression=# select timestamptz '2007/10/01 00:00 EDT' + interval '3 months'; ?column? 2008-01-01 00:00:00-05 (1 row) I think the latter behavior (that you get midnight EST not EDT) is generally agreed to be desirable, but I don't see any very principled way to achieve it if UTC offsets (as opposed to timezones) are considered sticky. The proposal to store a zone identifier (*not* a raw UTC offset) is somewhat more defensible but it's still got issues. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Skytools committed without hackers discussion/review
On Wed, 10 Oct 2007 18:01:34 +0100 Gregory Stark [EMAIL PROTECTED] wrote: Magnus Hagander [EMAIL PROTECTED] writes: On Wed, Oct 10, 2007 at 11:30:47AM -0400, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I also agree with this. We have to pretend it isn't in /contrib now, figure out where want it, then put it there (contrib, pgfoundry, core). I just don't see the point in putting it in pgfoundry. It's already in pgfoundry as part of Skytools. The whole point of having such a datatype is to build common interface to abstract away the internals of the database. That makes the pgfoundry modules (Skytools and Slony) easier to maintain separately. I missed the part that it is part of Skytools already but as counter point, what makes sense at that point is for Skytools to remove it and make it it's own module. That way Slony (which is not a pgfoundry project) or anyone else that wants to make use of it can. Putting it in core or contrib means that when we change the snapshot mechanics in 8.4 the same developer will be able to fix the module at the same time (and find out if his changes break it at the same time). Which is very cool, for *8.4* :) Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ signature.asc Description: PGP signature
[HACKERS] full text search in 8.3
Hi All, You knew it was coming I have an 8.2 database that has full text searching. I tried to backup/restore it to 8.3 but got lots of errors: snip ERROR: could not access file $libdir/tsearch2: No such file or directory ERROR: function public.gtsq_in(cstring) does not exist ERROR: could not access file $libdir/tsearch2: No such file or directory ERROR: function public.gtsq_out(gtsq) does not exist ERROR: function gtsq_in(cstring) does not exist snip ERROR: type tsvector is only a shell ERROR: type public.tsdebug does not exist snip etc... I didn't really expect it to totally work, but I'm not sure how to move my db. Any pointers would be appreciated. I used the 8.3 pg_dump on my laptop to backup the 8.2 db from the server, and tried to restore on my laptop. I tried both pg_dump -Fc, and just a pg_dump. Am I going to need to backup the the data, and nothing else (pg_dump --data-only ). Will the tsvector column be ok? I tried doing a pg_dump --schema-only and restoring just that, but still got a bunch of errors (those above). If I clean that up of all the old text search stuff, and then run it, then do the data, will that work ok? One more dumb question: I don't have to enable or install anything, do I? Thanks, -Andy ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Skytools committed without hackers discussion/review
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote: Joshua D. Drake [EMAIL PROTECTED] writes: If it doesn't need to be in core, in certainly has zero need to be in contrib and can push to pgFoundry. One advantage of having it in contrib is buildfarm testing, as indeed we already found out ... although it's true that *keeping* it there now that it passes probably won't teach us too much more. But I think the argument that was being made was mostly that the Slony and Skytools projects would find it easier to depend on a contrib module than on something that has to be fetched separately from pgfoundry. Now they could work around that by including copies of the pgfoundry project in their own distributions, but then they have a collision problem if someone tries to install both together. (I have no idea how likely that is, though; it might not be a big problem in practice?) Well, if it is kicked from /contrib now, one way we could handle it is by shipping the same module inside both skytools/slony. That has obvious conflict problems. Unfortunately the problem has very easy fix - each one keeps using it's current module. Very easy, no work required. But that also scratches the common API possibility. -- marko ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Skytools committed without hackers discussion/review
Ühel kenal päeval, K, 2007-10-10 kell 18:23, kirjutas Magnus Hagander: On Wed, Oct 10, 2007 at 11:47:15AM -0400, Tom Lane wrote: Marko Kreen [EMAIL PROTECTED] writes: IMHO the core operations are already as stable as PostgreSQL use of MVCC, as the module just exports backend internal state... Well, it exports backend internal state that did not exist before 8.2 (ie, XID epoch). So it doesn't seem all that set in stone to me. Another thing can can be done is more compact representation for txid_snapshot type, but that also won't affect core operation. That's another thing that's likely to become very much harder to change once it's in core. People keep threatening to produce a working in-place-upgrade process, and once that's reality the on-disk representation of core types is going to be hard to change. Well, if that is a concern, than it's an equally big concern to have it in contrib. If people start using it, they're not going to care about us saying hey, it was in contrib, why did you use it, when we earlier said in order do use our whiz-bang stuff, you must install from contrib. We'll have complaints that it's too hard to install, but we won't manage to escape from the responsibility to keep it working. I guess Marko will be able to take that responsibility, as he has been doing for pg_crypto for years, an recently also pl/python to some degree. tsearch lived in contrib for quite long, evolving a lot and going through a big morphing when it moved to core. I can't see anything nearly as big happening to txid/snapshot types. Hannu ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
