Re: [HACKERS] Index bloat problem in 7.4
Hi, Alvaro, Alvaro Herrera wrote: I am aware that more recent versions 8.x have fixed this problem, I checked the 7.4 release notes but can't see if any of the fixes made it into 7.4. Usually, only critical data loss and security fixes are put into the minor updates (e. G. 7.4.0 to 7.4.13). So, if it's not in the release notes, it is unlikely that it dit make it into the 7.4 releases. Except it was solved in 7.4: Allow B-tree index compaction and empty page reuse (Tom) Yes, you're right, the change is listed on http://www.postgresql.org/docs/7.4/interactive/release-7-4.html So, as it _is_ in the release notes, this does not render my statement wrong, but pointless :-) Thanks for your correction, Markus -- Markus Schaber | Logical TrackingTracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in Europe! www.ffii.org www.nosoftwarepatents.org signature.asc Description: OpenPGP digital signature
Re: [HACKERS] [PATCHES] WIP: Hierarchical Queries - stage 1
Hi Tom, Thanks for your initial thoughts on this. On Wed, 2006-09-20 at 20:13 -0400, Tom Lane wrote: (cut) You really can't get away with having the identical representation for CTEs and ordinary sub-selects in the range table. For instance, it looks like your patch will think that select ... from (select ...) as x, x, ... is legal when it certainly is not. I think you need either a new RTEKind or an additional flag in the RTE to show that it's a CTE rather than a plain subselect. I'm not entirely sure that you even want the CTEs in the rangetable at all --- that still needs some thought. For semantic reasons, I can see why you are questioning whether or not the CTE should be contained within the rangetable - there is an implicit hint that RTEs reflect entries within the FROM clause, but then I also see that the rewriter adds RTEs when substituting view definitions into queries. The comments in parsenodes.h also suggest that an RTE is a namespace/data source reference for a named entity within the query. The main problem I can see with keeping the CTEs outside the rangetable is that according to the source, jointree nodes must currently have RANGETBLREF nodes as leaf nodes; as I understand it, your suggestion of maintaining the CTEs separately would involve something along the lines of keeping a separate CTETable and creating some form of CTETBLREF node that could be referenced within the jointree. While arguably it may be semantically neater, it appears to involve quite a bit of extra work... could you explain in more detail as to why you feel that CTEs should remain outside the rangetable? This comes back to the question of whether the CTE per se should be an RTE at all. Maybe only the reference to it should be an RTE. The behavior when seeing a plain RangeVar in FROM would be to first search the side list of valid CTEs, and only on failure go looking for a real table. This is effectively what the patch does, albeit not particularly elegantly. I'll spend some time on making those changes a bit more refined. Kind regards, Mark. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] Include file in regress.c
Strangely, if I try to do a cvs add gram.c, it fails with cvs add: `gram.c' added independently by second party I don't know what this means. (Why second party and not third party?). Even if I delete gram.c. Even if I remove it from .cvsignore. I think cvs add probably contacts the server, because I seem to recall that it gives different output depending on whether the file already exists on another branch, and there's no way to tell that from your local working directory contents. The CVS history for gram.c looks a bit confused: http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/parser /Attic/gram.c How did revisions 2.89 and 2.90 come into existence when the file was already cvs-removed? This may be confusing the server too. That definitely looks weird to me. Unfortunatly, it's way above me wrt CVS knowledge. I'm just going to have to live with it and remember to delete that part from my diffs... (It's not so hard, because it's several megabytes each time, and I don't normally produce patches that large..) //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] pg_upgrade: downgradebility
On 9/22/06, Tom Lane [EMAIL PROTECTED] wrote: I suggest again that you ought to be running your basic design past the list sooner rather than later. We will be posting it as soon as we can. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Index bloat problem in 7.4
On 22-Sep-06, at 3:58 AM, Markus Schaber wrote: Hi, Alvaro, Alvaro Herrera wrote: I am aware that more recent versions 8.x have fixed this problem, I checked the 7.4 release notes but can't see if any of the fixes made it into 7.4. Usually, only critical data loss and security fixes are put into the minor updates (e. G. 7.4.0 to 7.4.13). So, if it's not in the release notes, it is unlikely that it dit make it into the 7.4 releases. Except it was solved in 7.4: Allow B-tree index compaction and empty page reuse (Tom) Yes, you're right, the change is listed on http://www.postgresql.org/docs/7.4/interactive/release-7-4.html So, as it _is_ in the release notes, this does not render my statement wrong, but pointless :-) My understanding is that further work was done to address this in later versions as well. My client is experiencing index bloat in 7.4.x. Dave Thanks for your correction, Markus -- Markus Schaber | Logical TrackingTracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in Europe! www.ffii.org www.nosoftwarepatents.org ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Index bloat problem in 7.4
Dave Cramer wrote: On 22-Sep-06, at 3:58 AM, Markus Schaber wrote: Hi, Alvaro, Alvaro Herrera wrote: I am aware that more recent versions 8.x have fixed this problem, I checked the 7.4 release notes but can't see if any of the fixes made it into 7.4. Usually, only critical data loss and security fixes are put into the minor updates (e. G. 7.4.0 to 7.4.13). So, if it's not in the release notes, it is unlikely that it dit make it into the 7.4 releases. Except it was solved in 7.4: Allow B-tree index compaction and empty page reuse (Tom) Yes, you're right, the change is listed on http://www.postgresql.org/docs/7.4/interactive/release-7-4.html So, as it _is_ in the release notes, this does not render my statement wrong, but pointless :-) My understanding is that further work was done to address this in later versions as well. My client is experiencing index bloat in 7.4.x. My guess is that they will still experience it even with 8.2, because the situations on which the problem keeps happening have not, to my knowledge, been addressed in later versions. The main problem remaining is that partially filled paged are not merged, so if you delete multiple old tuples (say, indexed by a timestamp or a monotonously increasing key) and replace it with a single summary tuple keyed with a value in the same range, the index page where that tuple is stored will likely not contain a lot of other index entries. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] 8.3 Development Cycle
Following the recent discussion on this list and another on pgsql-core, we have decided that we would like to aim to meet the following schedule for the release of PostgreSQL 8.3: April 1st 2007 - Feature freeze May 1st 2007 - Beta 1 release June 1st 2007 - Release This will obviously be a short development cycle which will allow us to get some of the features that just missed 8.2 out of the door, as well as giving us the opportunity to try releasing before the summer (for those in the northern hemisphere) rather than after. We are also aware that this is a tight timetable, however given the shorter development cycle we feel it is an achievable goal. If anyone has any serious objections, please shout now! Regards, Dave -- Dave Page PostgreSQL Core Team ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [PATCHES] WIP: Hierarchical Queries - stage 1
Mark Cave-Ayland [EMAIL PROTECTED] writes: The main problem I can see with keeping the CTEs outside the rangetable is that according to the source, jointree nodes must currently have RANGETBLREF nodes as leaf nodes; as I understand it, your suggestion of maintaining the CTEs separately would involve something along the lines of keeping a separate CTETable and creating some form of CTETBLREF node that could be referenced within the jointree. No, what I'm thinking is that a *reference* to a CTE, from within the main query's FROM list, would create a CTERef RTE and then you'd have a normal RANGETBLREF node linking to that in the jointree. This solves the problem of where do you put the alias: on the RTE. What's not clear to me at this point is whether there can be multiple references in a query to the same CTE --- if there can, I suspect you must have a data structure like this. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] 8.3 Development Cycle
Dave Page wrote: Following the recent discussion on this list and another on pgsql-core, we have decided that we would like to aim to meet the following schedule for the release of PostgreSQL 8.3: April 1st 2007 - Feature freeze May 1st 2007 - Beta 1 release June 1st 2007 - Release This will obviously be a short development cycle which will allow us to get some of the features that just missed 8.2 out of the door, as well as giving us the opportunity to try releasing before the summer (for those in the northern hemisphere) rather than after. We are also aware that this is a tight timetable, however given the shorter development cycle we feel it is an achievable goal. If anyone has any serious objections, please shout now! Sounds fine, but announcing this now is almost certain to reduce the number of people migrating to 8.2. I am not saying we shouldn't announce it now, but it is something I wanted to mention. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 8.3 Development Cycle
-Original Message- From: Bruce Momjian [mailto:[EMAIL PROTECTED] Sent: 22 September 2006 15:26 To: Dave Page Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] 8.3 Development Cycle Sounds fine, but announcing this now is almost certain to reduce the number of people migrating to 8.2. I am not saying we shouldn't announce it now, but it is something I wanted to mention. Err right - if you had said this yesterday when we discussed the idea I could have mentioned it in my message. Right around the time you said Go for it would have been good :-) Regards, Dave. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 8.3 Development Cycle
Dave Page wrote: -Original Message- From: Bruce Momjian [mailto:[EMAIL PROTECTED] Sent: 22 September 2006 15:26 To: Dave Page Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] 8.3 Development Cycle Sounds fine, but announcing this now is almost certain to reduce the number of people migrating to 8.2. I am not saying we shouldn't announce it now, but it is something I wanted to mention. Err right - if you had said this yesterday when we discussed the idea I could have mentioned it in my message. Right around the time you said Go for it would have been good :-) I didn't think of it until today. Sorry. Personally, I don't like manipulating people by withholding information, so I still think we are doing the right thing. I am just saying it might have that effect. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] 8.3 Development Cycle
Bruce Momjian wrote: Dave Page wrote: -Original Message- From: Bruce Momjian [mailto:[EMAIL PROTECTED] Sent: 22 September 2006 15:26 To: Dave Page Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] 8.3 Development Cycle Sounds fine, but announcing this now is almost certain to reduce the number of people migrating to 8.2. I am not saying we shouldn't announce it now, but it is something I wanted to mention. Err right - if you had said this yesterday when we discussed the idea I could have mentioned it in my message. Right around the time you said Go for it would have been good :-) I didn't think of it until today. Sorry. Personally, I don't like manipulating people by withholding information, so I still think we are doing the right thing. I am just saying it might have that effect. They will upgrade if they want the new features, as usual. I had dinner with 3 significant users yesterday and mentioned to them the new facility to add or drop inheritance on tables, and their reaction was along the lines of When can I have it? I want it now! They won't be deterred by the fact that the next dev cycle will be a bit shorter. cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] 8.3 Development Cycle
-Original Message- From: Bruce Momjian [mailto:[EMAIL PROTECTED] Sent: 22 September 2006 15:35 To: Dave Page Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] 8.3 Development Cycle Err right - if you had said this yesterday when we discussed the idea I could have mentioned it in my message. Right around the time you said Go for it would have been good :-) I didn't think of it until today. Sorry. NP - I wrote that firmly tongue in cheek, hence the smiley. Personally, I don't like manipulating people by withholding information, so I still think we are doing the right thing. Yes, agreed. Hopefully it will help people to plan their time given as much advance notice of cycle dates as possible. I am just saying it might have that effect. For some people I'm sure it will. Others (myself included) will want to move to 8.2 as soon as possible. Regards, dave. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [PATCHES] Include file in regress.c
Magnus Hagander [EMAIL PROTECTED] writes: That definitely looks weird to me. Unfortunatly, it's way above me wrt CVS knowledge. I'm just going to have to live with it and remember to delete that part from my diffs... The weird thing is that it's not happening for other people. Have you tried blowing away the whole tree and doing a fresh checkout? What CVS version are you using? regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Fixed length data types issue
On Mon, Sep 11, 2006 at 19:05:12 -0400, Gregory Stark [EMAIL PROTECTED] wrote: I'm not sure how gmp and the others represent their data but my first guess is that there's no particular reason the base of the mantissa and exponent have to be the same as the base the exponent is interpreted as. That is, you can store a base 10 exponent but store it and the mantissa in two's complement integers. You can also store numbers as a relatively prime numerator and denominator, which will let store rational numbers exactly. Doing this isn't going to help with speed of operations though. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] advisory locks and permissions
Stephen Frost [EMAIL PROTECTED] writes: * Tom Lane ([EMAIL PROTECTED]) wrote: An admin who is concerned about this can revoke public access on the functions for himself ... but should that be the default out-of-the-box configuration? I feel more comfortable with saying you have to turn on this potentially-dangerous feature than with saying you have to turn it off. I agree with having it turned off by default, at least in 8.2. Do we have a consensus to do this for 8.2? Or are we going to leave it as is? Those are the only two realistic short-term options ... regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 8.3 Development Cycle
Dave Page wrote: -Original Message- From: Bruce Momjian [mailto:[EMAIL PROTECTED] Sent: 22 September 2006 15:26 To: Dave Page Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] 8.3 Development Cycle Sounds fine, but announcing this now is almost certain to reduce the number of people migrating to 8.2. I am not saying we shouldn't announce it now, but it is something I wanted to mention. Err right - if you had said this yesterday when we discussed the idea I could have mentioned it in my message. Right around the time you said Go for it would have been good :-) This will likely stop people from migrating to 8.2, but so what? It isn't going to stop new users and existing users in real production setting will likely wait for 8.3 anyway. Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] advisory locks and permissions
On 9/22/06, Tom Lane [EMAIL PROTECTED] wrote: Stephen Frost [EMAIL PROTECTED] writes: * Tom Lane ([EMAIL PROTECTED]) wrote: An admin who is concerned about this can revoke public access on the functions for himself ... but should that be the default out-of-the-box configuration? I feel more comfortable with saying you have to turn on this potentially-dangerous feature than with saying you have to turn it off. I agree with having it turned off by default, at least in 8.2. Do we have a consensus to do this for 8.2? Or are we going to leave it as is? Those are the only two realistic short-term options ... there are plenty of other potentially nasty things (like generate_series and the ! operator). why are advisory_locks handled specially? the way it stands right now is a user with command access can DoS a server after five minutes of research on the web. however, if we decide to lock them, it should be documented as such. advisory locks still show up as 'userlock' in the pg_locks view. does this matter? merlin ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Release Notes: Major Changes in 8.2
On Thu, 2006-09-21 at 21:45 -0400, Bruce Momjian wrote: Let me know how it looks. Very Good Very last, Minor change thoughts: * Continuous archiving enhancements change: Warm Standby enhancements The improvements to Continuous Archiving relate directly to the creation of Warm Standby servers, so it would be better to mention Warm Standby, not Continuous Archiving (and definitely not PITR) * Monitoring and logging additions add to end of line: improve performance tuning capability * COPY support for SELECT statements change: COPY TO support ... add to end of line: enhances data unload * Array and aggregate improvements add to end of line: , plus SQL:2003 statistical functions -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] advisory locks and permissions
Merlin Moncure [EMAIL PROTECTED] writes: advisory locks still show up as 'userlock' in the pg_locks view. does this matter? I'm disinclined to change that, because it would probably break existing client-side code for little gain. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] advisory locks and permissions
there are plenty of other potentially nasty things (like generate_series and the ! operator). why are advisory_locks handled specially? the way it stands right now is a user with command access can DoS a server after five minutes of research on the web. You don't even have to do any research, just fire off ab. Using a DOS to attack *any* database server via the web is a 3 second command. Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] advisory locks and permissions
On Sep 22, 2006, at 11:26 , Merlin Moncure wrote: On 9/22/06, Tom Lane [EMAIL PROTECTED] wrote: Stephen Frost [EMAIL PROTECTED] writes: * Tom Lane ([EMAIL PROTECTED]) wrote: An admin who is concerned about this can revoke public access on the functions for himself ... but should that be the default out-of- the-box configuration? I feel more comfortable with saying you have to turn on this potentially-dangerous feature than with saying you have to turn it off. I agree with having it turned off by default, at least in 8.2. Do we have a consensus to do this for 8.2? Or are we going to leave it as is? Those are the only two realistic short-term options ... there are plenty of other potentially nasty things (like generate_series and the ! operator). why are advisory_locks handled specially? the way it stands right now is a user with command access can DoS a server after five minutes of research on the web. however, if we decide to lock them, it should be documented as such. advisory locks still show up as 'userlock' in the pg_locks view. does this matter? I would be more worried about accidental collisions between applications. The lock ranges will now need to be in their respective application's configuration file in case of collision with another app once developers really start using locks for IPC. Ideally, the user-level lock functions would take strings instead of integers and hash them appropriately, no? Otherwise, someone will end up maintaining a registry of lock numbers in use. LISTEN doesn't use integers. -M ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] advisory locks and permissions
On Fri, Sep 22, 2006 at 12:03:46PM -0400, AgentM wrote: On Sep 22, 2006, at 11:26 , Merlin Moncure wrote: On 9/22/06, Tom Lane [EMAIL PROTECTED] wrote: Stephen Frost [EMAIL PROTECTED] writes: * Tom Lane ([EMAIL PROTECTED]) wrote: An admin who is concerned about this can revoke public access on the functions for himself ... but should that be the default out-of- the-box configuration? I feel more comfortable with saying you have to turn on this potentially-dangerous feature than with saying you have to turn it off. I agree with having it turned off by default, at least in 8.2. Do we have a consensus to do this for 8.2? Or are we going to leave it as is? Those are the only two realistic short-term options ... there are plenty of other potentially nasty things (like generate_series and the ! operator). why are advisory_locks handled specially? the way it stands right now is a user with command access can DoS a server after five minutes of research on the web. however, if we decide to lock them, it should be documented as such. advisory locks still show up as 'userlock' in the pg_locks view. does this matter? I would be more worried about accidental collisions between applications. The lock ranges will now need to be in their respective application's configuration file in case of collision with another app once developers really start using locks for IPC. Ideally, the user-level lock functions would take strings instead of integers and hash them appropriately, no? Otherwise, someone will end up maintaining a registry of lock numbers in use. LISTEN doesn't use integers. This is why I suggested we set aside some range of numbers that should not be used. Doing so would allow adding a better-managed numbering/naming scheme in the future. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 8.3 Development Cycle
Dave Page wrote: This will obviously be a short development cycle which will allow us to get some of the features that just missed 8.2 out of the door, as well as giving us the opportunity to try releasing before the summer (for those in the northern hemisphere) rather than after. Joshua's original mail suggested that only certain features would go in. Is that still on the cards, or will other features be considered if they're ready? I'm obviously thinking of enums which was ready (for review at least) a few weeks ago, but has probably bitrotted slightly since then given the number of patches that have landed in the tree. I intended to brush it up as soon as the 8.3 tree was open and resubmit it. Will that be a waste of time? Thanks Tom ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] 8.3 Development Cycle
Bruce, Dave, This will likely stop people from migrating to 8.2, but so what? It isn't going to stop new users and existing users in real production setting will likely wait for 8.3 anyway. And at this point most production users are only upgrading every 2-3 releases anyway (something which will get worse with time). Heck, I have former clients who are still running 7.2. Why would they upgrade? It's never been down, and it's not exposed to untrusted users. --Josh Berkus ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 8.3 Development Cycle
-Original Message- From: Tom Dunstan [EMAIL PROTECTED] To: Dave Page dpage@vale-housing.co.uk Cc: pgsql-hackers@postgresql.org pgsql-hackers@postgresql.org Sent: 22/09/06 17:21 Subject: Re: [HACKERS] 8.3 Development Cycle will other features be considered if they're ready? Yes, normal rules apply, just in a shorter timeframe. Regards, Dave ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 8.3 Development Cycle
Tom, I'm obviously thinking of enums which was ready (for review at least) a few weeks ago, but has probably bitrotted slightly since then given the number of patches that have landed in the tree. I intended to brush it up as soon as the 8.3 tree was open and resubmit it. Will that be a waste of time? Ooops. Are you sure these weren't committed? --Josh ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 8.3 Development Cycle
Tom Dunstan [EMAIL PROTECTED] writes: Joshua's original mail suggested that only certain features would go in. Is that still on the cards, or will other features be considered if they're ready? You'll note that Dave's mail said no such thing. There has been some talk of trying to agree on a roadmap for 8.3 and coordinate development efforts accordingly --- but that does not constitute an agreement to reject work not in the roadmap, just some coordination among those people who wish to coordinate. I'm obviously thinking of enums which was ready (for review at least) a few weeks ago, but has probably bitrotted slightly since then given the number of patches that have landed in the tree. I intended to brush it up as soon as the 8.3 tree was open and resubmit it. Please do. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] advisory locks and permissions
On 9/22/06, AgentM [EMAIL PROTECTED] wrote: I would be more worried about accidental collisions between applications. The lock ranges will now need to be in their respective i dont think this argument has merit because the lock is scoped to the current database. this would only be a problem if two applications used the same database...not likely. the old userlocks had 48 bits of lock space and now we have 64, im not complaining. application's configuration file in case of collision with another app once developers really start using locks for IPC. Ideally, the user-level lock functions would take strings instead of integers and hash them appropriately, no? Otherwise, someone will end up maintaining a registry of lock numbers in use. LISTEN doesn't use integers. application can translate the locks to strings via a very simple translation table. there is no downside to this besides a index lookup on a small table, which is more or less what the listen/notify does internally. advisory locks work off of the internal lock system which is an integer only system. the whole point is to get at these locks while bypassing the transaction system. you are suggesting something which does not fit into the current lock system. merlin ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Getting a move on for 8.2 beta
On Wed, Sep 13, 2006 at 22:22:12 -0700, Tom Dunstan [EMAIL PROTECTED] wrote: That's a worthwhile point. How many patches come from the general community vs out of the blue? Patches from regulars could probably get a free pass, which might cut down the review burden substantially. And how were you planning to tell if a patch cam from a regular? Hopefully you weren't planning on blindly trusting the from header. Misuse of the build farm in a way the effects other sites could get the project a big black eye, so you want to be very careful building and executing code from the patch queue. ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 8.3 Development Cycle
Tom Dunstan wrote: Dave Page wrote: This will obviously be a short development cycle which will allow us to get some of the features that just missed 8.2 out of the door, as well as giving us the opportunity to try releasing before the summer (for those in the northern hemisphere) rather than after. Joshua's original mail suggested that only certain features would go in. Is that still on the cards, or will other features be considered if they're ready? I'm obviously thinking of enums which was ready (for review at least) a few weeks ago, but has probably bitrotted slightly since then given the number of patches that have landed in the tree. I intended to brush it up as soon as the 8.3 tree was open and resubmit it. Will that be a waste of time? AFAIK, we aren't doing anything like that, and it would be quite unfair if we did. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] -HEAD planner issue wrt hash_joins on dbt3 ?
Tom Lane wrote: Matteo Beccati [EMAIL PROTECTED] writes: Tom Lane ha scritto: Matteo Beccati [EMAIL PROTECTED] writes: I cannot see anything bad by using something like that: if (histogram is large/representative enough) Well, the question is exactly what is large enough? I feel a bit uncomfortable about applying the idea to a histogram with only 10 entries (especially if we ignore two of 'em). With 100 or more, it sounds all right. What's the breakpoint? Yes, I think 100-200 could be a good breakpoint. I've committed this change with (for now) 100 as the minimum histogram size to use. Stefan, are you interested in retrying your benchmark? sure - but I'm having hardware (harddisk firmware) related issues on my testbox which will take a few further days to be resolved ... Stefan ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] advisory locks and permissions
On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: This is why I suggested we set aside some range of numbers that should not be used. Doing so would allow adding a better-managed numbering/naming scheme in the future. the whole point about advisory locks is that the provided lock space is unmanaged. for example, in the ISAM system I wrote which hooked into the acucobol virtual file system interface, I used a global sequence for row level pessimistic locking but reserved the 48th bit for table level locks. This system was extremely effective. on the current system I'm working on I use them to lock sequence oid's plus a high bit indicator for what i am doing. in short, advisory locks are application-defined in concept. merlin ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] 8.3 Development Cycle
Josh Berkus josh@agliodbs.com writes: Bruce, Dave, This will likely stop people from migrating to 8.2, but so what? It isn't going to stop new users and existing users in real production setting will likely wait for 8.3 anyway. And at this point most production users are only upgrading every 2-3 releases anyway (something which will get worse with time). The other side of that coin is that with a short devel cycle, 8.3 is not necessarily going to look like a must-have upgrade to many people either. If I were a DBA looking at the current plans, and I didn't have a desperate need for bitmap indexes (a feature with a still very unclear use-case footprint ...), I'd probably figure that updating to 8.2 soon is a more rewarding strategy than waiting for 8.3. Known benefits now versus unknown benefits later is a pretty easy call. In the end, any one user is going to find particular updates compelling or not based on specific features they need for their specific application. We can't any longer expect that everyone's going to adopt every release immediately ... indeed, that's why we're still supporting back release branches. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] silent install: silent error (even using the manual)
Good afternoon, dear PostgreSQL developers! First I want to thank all of you for the great DB Server! It's really useful, thank you! Second, here is my question(guys from #postgresql channel advised to write to dev list): I want to make a silent install of PostgreSQL version 8.1 using your recommendations on http://pginstaller.projects.postgresql.org/silent.html . But when I use this command: msiexec /i postgresql-8.1.msi /qb! /l*vx c:\pg_inst1.log INTERNALLAUNCH=1 ADDLOCAL=server,psql,pgadmin,jdbc,docs DOSERVICE=1 SERVICEDOMAIN=%COMPUTERNAME% SERVICEPASSWORD=suntek CREATESERVICEUSER=1 NOSHORTCUTS=1 I have nothing happened except appearing of window for 1 second with words: Preparing to install Then I can find msiexec.exe in processes with 0% CPU usage. I pasted full msi log below. What's the problem, can you tell me? Here is the log for it: ?=== Verbose logging started: 22.09.2006 19:20:59 Build type: SHIP UNICODE 3.01.4000.1823 Calling process: C:\WINDOWS\system32\msiexec.exe === MSI (c) (34:98) [19:20:59:062]: Resetting cached policy values MSI (c) (34:98) [19:20:59:062]: Machine policy value 'Debug' is 0 MSI (c) (34:98) [19:20:59:062]: *** RunEngine: *** Product: postgresql-8.1.msi *** Action: *** CommandLine: ** MSI (c) (34:98) [19:20:59:062]: Client-side and UI is none or basic: Running entire install on the server. MSI (c) (34:98) [19:20:59:062]: Grabbed execution mutex. MSI (c) (34:98) [19:20:59:078]: Cloaking enabled. MSI (c) (34:98) [19:20:59:078]: Attempting to enable all disabled priveleges before calling Install on Server MSI (c) (34:98) [19:20:59:078]: Incrementing counter to disable shutdown. Counter after increment: 0 MSI (s) (A4:30) [19:20:59:078]: Grabbed execution mutex. MSI (s) (A4:90) [19:20:59:078]: Resetting cached policy values MSI (s) (A4:90) [19:20:59:078]: Machine policy value 'Debug' is 0 MSI (s) (A4:90) [19:20:59:078]: *** RunEngine: *** Product: C:\Drive D\J-Pro's\TOMS\inst\postgresql-8.1.msi *** Action: *** CommandLine: ** MSI (s) (A4:90) [19:20:59:078]: Machine policy value 'DisableUserInstalls' is 0 MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: User non-assigned for product: 2367501907ACC3146B82D2C3BDBB09B6 MSI (s) (A4:90) [19:20:59:078]: Using cached product context: User non-assigned for product: 2367501907ACC3146B82D2C3BDBB09B6 MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: User non-assigned for product: 2516C73DFD9892D438FC6626005D3F89 MSI (s) (A4:90) [19:20:59:078]: Using cached product context: User non-assigned for product: 2516C73DFD9892D438FC6626005D3F89 MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: User non-assigned for product: AE36A588B2834DD47A554108B958756D MSI (s) (A4:90) [19:20:59:078]: Using cached product context: User non-assigned for product: AE36A588B2834DD47A554108B958756D MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: User non-assigned for product: BF0D824FD567BE04DB8D1A7E5F5C79AF MSI (s) (A4:90) [19:20:59:078]: Using cached product context: User non-assigned for product: BF0D824FD567BE04DB8D1A7E5F5C79AF MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: User non-assigned for product: D47ABDE8686099C4FBDD8F4976E8B593 MSI (s) (A4:90) [19:20:59:078]: Using cached product context: User non-assigned for product: D47ABDE8686099C4FBDD8F4976E8B593 MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: machine assigned for product: 01E4D47B48861030 MSI (s) (A4:90) [19:20:59:078]: Using cached product context: machine assigned for product: 01E4D47B48861030 MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: machine assigned for product: 0B54E49C6AA69BF4A9EA2280F5368108 MSI (s) (A4:90) [19:20:59:078]: Using cached product context: machine assigned for product: 0B54E49C6AA69BF4A9EA2280F5368108 MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: machine assigned for product: 0B79C053C7D38EE4AB9A00CB3B5D2472 MSI (s) (A4:90) [19:20:59:078]: Using cached product context: machine assigned for product: 0B79C053C7D38EE4AB9A00CB3B5D2472 MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: machine assigned for product: 0CDFA50527E582943886D5AE83E56374 MSI (s) (A4:90) [19:20:59:078]: Using cached product context: machine assigned for product: 0CDFA50527E582943886D5AE83E56374 MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: machine assigned for product: 104C2FB8EC20D424CB62C6F4F94B646B MSI (s) (A4:90) [19:20:59:078]: Using cached product context: machine assigned for product: 104C2FB8EC20D424CB62C6F4F94B646B MSI (s) (A4:90) [19:20:59:078]: Setting cached product context: machine assigned for product: 166A73803CEF3B8478C6197E3D02849A MSI (s) (A4:90) [19:20:59:078]: Using cached product context: machine assigned for product: 166A73803CEF3B8478C6197E3D02849A MSI (s)
Re: [HACKERS] advisory locks and permissions
On Sep 22, 2006, at 12:46 , Merlin Moncure wrote: On 9/22/06, AgentM [EMAIL PROTECTED] wrote: I would be more worried about accidental collisions between applications. The lock ranges will now need to be in their respective i dont think this argument has merit because the lock is scoped to the current database. this would only be a problem if two applications used the same database...not likely. Since we have schemas, I have several applications running in one database- sometimes several versions of the same application. This makes it easier to shuffle data around. application's configuration file in case of collision with another app once developers really start using locks for IPC. Ideally, the user-level lock functions would take strings instead of integers and hash them appropriately, no? Otherwise, someone will end up maintaining a registry of lock numbers in use. LISTEN doesn't use integers. application can translate the locks to strings via a very simple translation table. there is no downside to this besides a index lookup on a small table, which is more or less what the listen/notify does internally. advisory locks work off of the internal lock system which is an integer only system. the whole point is to get at these locks while bypassing the transaction system. you are suggesting something which does not fit into the current lock system. I didn't suggest using lookup tables; I suggested that the lock functions should perform the string hashing itself- the applications will write wrappers for this anyway to prevent collisions or they will have to provide some configuration element to change the lock range in case of collision- which will be extraordinarily difficult to debug in the first place. -M ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] advisory locks and permissions
On Fri, Sep 22, 2006 at 12:56:37PM -0400, Merlin Moncure wrote: On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: This is why I suggested we set aside some range of numbers that should not be used. Doing so would allow adding a better-managed numbering/naming scheme in the future. the whole point about advisory locks is that the provided lock space is unmanaged. for example, in the ISAM system I wrote which hooked into the acucobol virtual file system interface, I used a global sequence for row level pessimistic locking but reserved the 48th bit for table level locks. This system was extremely effective. on the current system I'm working on I use them to lock sequence oid's plus a high bit indicator for what i am doing. in short, advisory locks are application-defined in concept. Yes, but if you get two pieces of code written by different people using them in the same database, you can get hosed. As PostgreSQL becomes more popular and more people start developing software for it, this is more likely to occur. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] 8.3 Development Cycle
On Fri, Sep 22, 2006 at 02:16:53PM +0100, Dave Page wrote: Following the recent discussion on this list and another on pgsql-core, we have decided that we would like to aim to meet the following schedule for the release of PostgreSQL 8.3: April 1st 2007 - Feature freeze ^ We should probably move this forward or back one day. Cheers, D -- David Fetter [EMAIL PROTECTED] http://fetter.org/ phone: +1 415 235 3778AIM: dfetter666 Skype: davidfetter Remember to vote! ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Is there any utility to update the table whenever text file gets changed?
On Thu, Sep 14, 2006 at 03:41:06 -0700, Dhanaraj M [EMAIL PROTECTED] wrote: Is there any utility in postgresql which can do the following? The utility must update the table whenever there is any change in the text file. COPY command helps to do that, though this is not straight forward. Can it be automated? You would either need to have the application which changes the text file do soemthing or have another program watching the text file to see when it changes and then take action. You probably don't want to use COPY as that essentially does inserts, not updates. ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Getting a move on for 8.2 beta
On Fri, Sep 22, 2006 at 11:58:04AM -0500, Bruno Wolff III wrote: On Wed, Sep 13, 2006 at 22:22:12 -0700, Tom Dunstan [EMAIL PROTECTED] wrote: That's a worthwhile point. How many patches come from the general community vs out of the blue? Patches from regulars could probably get a free pass, which might cut down the review burden substantially. And how were you planning to tell if a patch cam from a regular? Hopefully you weren't planning on blindly trusting the from header. Misuse of the build farm in a way the effects other sites could get the project a big black eye, so you want to be very careful building and executing code from the patch queue. Of course not, but there's any number of ways we could handle that problem. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] advisory locks and permissions
On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: On Fri, Sep 22, 2006 at 12:56:37PM -0400, Merlin Moncure wrote: the whole point about advisory locks is that the provided lock space is unmanaged. for example, in the ISAM system I wrote which hooked into the acucobol virtual file system interface, I used a global sequence for row level pessimistic locking but reserved the 48th bit for table level locks. This system was extremely effective. on the current system I'm working on I use them to lock sequence oid's plus a high bit indicator for what i am doing. in short, advisory locks are application-defined in concept. Yes, but if you get two pieces of code written by different people using them in the same database, you can get hosed. As PostgreSQL becomes more popular and more people start developing software for it, this is more likely to occur. imo, that is no more or less likely than having two pieces of code store the same table in the same database. I think what you are describing would only be a concern if the locks were shared across databases, however this is not the case. the purpose of advisory locks is to be 'appplication-defined'. how the application is written is not part of that concept. we are simply granting the ability to create a mutex with a number for a name, that is all. merlin ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] Timezone doc patch
Joachim Wieland [EMAIL PROTECTED] writes: Appended is a doc patch that removes tables B-4 and B-5 from Appendix B and integrates information from there into other parts, mostly into section 8.5.3. Applied with a few minor editorializations. I still havent gotten a reply to http://archives.postgresql.org/pgsql-hackers/2006-09/msg01590.php so I didn't change those parts. I don't feel a strong need to do anything about that ... regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] advisory locks and permissions
On Fri, Sep 22, 2006 at 01:21:57PM -0400, Merlin Moncure wrote: On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: On Fri, Sep 22, 2006 at 12:56:37PM -0400, Merlin Moncure wrote: the whole point about advisory locks is that the provided lock space is unmanaged. for example, in the ISAM system I wrote which hooked into the acucobol virtual file system interface, I used a global sequence for row level pessimistic locking but reserved the 48th bit for table level locks. This system was extremely effective. on the current system I'm working on I use them to lock sequence oid's plus a high bit indicator for what i am doing. in short, advisory locks are application-defined in concept. Yes, but if you get two pieces of code written by different people using them in the same database, you can get hosed. As PostgreSQL becomes more popular and more people start developing software for it, this is more likely to occur. imo, that is no more or less likely than having two pieces of code store the same table in the same database. I think what you are describing would only be a concern if the locks were shared across databases, however this is not the case. the purpose of advisory locks is to be 'appplication-defined'. how the application is written is not part of that concept. we are simply granting the ability to create a mutex with a number for a name, that is all. Ok, here's a real-world example. RRS (http://rrs.decibel.org) will make use of userlocks if available. RRS by itself isn't very interesting at all; you'd want to use it with something else. Because there's no standard at all for carving up the numbers, I did the best I could by using the OID of one of my functions, because at least back then there was standard OID support. I'm not sure if that even made it into the current version. Using named locks is possibly overkill, but it would be good to at least set aside some chunk of numbers so that it can be done. Likewise I suggested setting aside OIDs above 10k (or whatever a normal database starts numbering at) so that you could at least do per-schema numbering. I'm not asking for a defined solution to how to support multiple different users of locks within the same database. I just want us to set aside (as in, recommend they not be used) some set of numbers so that in the future we could recommend a means of picking lock numbers that will avoid collisions. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 8.3 Development Cycle
Josh Berkus wrote: I'm obviously thinking of enums which was ready (for review at least) a few weeks ago, but has probably bitrotted slightly since then given the number of patches that have landed in the tree. I intended to brush it up as soon as the 8.3 tree was open and resubmit it. Will that be a waste of time? Ooops. Are you sure these weren't committed? Pretty sure. :) Why the oops? They haven't been mentioned in some PR material or something have they? Cheers Tom ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 8.3 Development Cycle
Josh Berkus wrote: Tom, I'm obviously thinking of enums which was ready (for review at least) a few weeks ago, but has probably bitrotted slightly since then given the number of patches that have landed in the tree. I intended to brush it up as soon as the 8.3 tree was open and resubmit it. Will that be a waste of time? Ooops. Are you sure these weren't committed? It's not committed - it was submitted long after feature freeze (in fact, coding didn't begin until after freeze). But it's fairly liable to bitrot, since it touches the catalog in a significant way. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] advisory locks and permissions
On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: I'm not asking for a defined solution to how to support multiple different users of locks within the same database. I just want us to set aside (as in, recommend they not be used) some set of numbers so that in the future we could recommend a means of picking lock numbers that will avoid collisions. you pretty much already have this, current advisory lock exposes 64 bits of locktag storage. there is 112 bits (3 int4 and 1 int2) available. this is since 8.1 when locktag was reorganized. I was actually going to suggest esposing these fields but had second thoughts due to future proofing issues. note i am not arguing that advisory lock should not be expanded in the future or do string maps, just that at present talking about reserved ranges would just confuse people since the lock space is intentionally generic. merlin ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] 8.3 Development Cycle
Tom Lane wrote: Tom Dunstan [EMAIL PROTECTED] writes: Joshua's original mail suggested that only certain features would go in. Is that still on the cards, or will other features be considered if they're ready? You'll note that Dave's mail said no such thing. No, but it did explicitly mention features that just missed 8.2, so I just wanted some clarification, which you and Dave have now provided. Thanks. Tom ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] advisory locks and permissions
Merlin Moncure [EMAIL PROTECTED] writes: On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: This is why I suggested we set aside some range of numbers that should not be used. Doing so would allow adding a better-managed numbering/naming scheme in the future. the whole point about advisory locks is that the provided lock space is unmanaged. I think we forgot to document that the lock space is per-database; also, wouldn't it be a good idea to specifically recommend that advisory locks be used only in databases that are used just by one application, or a few cooperating applications? The lack of any permissions checks makes them fairly unsafe in databases that are used by multiple users. I don't actually have a problem with the lack of security checks or key range limitations --- I see advisory locks as comparable to large objects, which are likewise permissions-free. It's an optional feature and you just won't use it in databases where permission constraints are a critical need. The thing that's bothering me is the relative ease of accidental DoS to applications in *other* databases in the same cluster. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] advisory locks and permissions
On Fri, Sep 22, 2006 at 01:42:48PM -0400, Merlin Moncure wrote: On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: I'm not asking for a defined solution to how to support multiple different users of locks within the same database. I just want us to set aside (as in, recommend they not be used) some set of numbers so that in the future we could recommend a means of picking lock numbers that will avoid collisions. you pretty much already have this, current advisory lock exposes 64 bits of locktag storage. there is 112 bits (3 int4 and 1 int2) available. this is since 8.1 when locktag was reorganized. I was actually going to suggest esposing these fields but had second thoughts due to future proofing issues. note i am not arguing that advisory lock should not be expanded in the future or do string maps, just that at present talking about reserved ranges would just confuse people since the lock space is intentionally generic. Ahh, ok, I didn't realize that the total lock space was larger than what's being exposed today. That means we can easily add that stuff in the future and not break anything, which is all I was looking for. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Fwd: Is the fsync() fake on FreeBSD6.1?
I thought folks might be interested in this... note in particular the comment about linux. Begin forwarded message: From: Greg 'groggy' Lehey [EMAIL PROTECTED] Date: June 26, 2006 11:34:12 PM EDT To: leo huang [EMAIL PROTECTED] Cc: freebsd-performance@freebsd.org Subject: Re: Is the fsync() fake on FreeBSD6.1? On Tuesday, 27 June 2006 at 10:18:47 +0800, leo huang wrote: Hi, I benchmarked MySQL 4.1.18 on FreeBSD 6.1 and Debian 3.1 using Super Smack 1.3 some days ago. ... The result surprise me. The MySQL Performance on FreeBSD6.1 is about 10 times of on Debian3.1??and the output of iostat also shows it. I know that MySQL uses fsync() to flush both the data and log files at default when using innodb engine( http://dev.mysql.com/doc/refman/4.1/en/innodb-parameters.html). Our evaluating computer only has a 1RPM SCSI hard disk. I think it can do about 200 sequential fsync() calls per second if the fsync() is real. Is the fsync() on FreeBSD6.1 fake? My understanding from the last time I looked at the code was that fsync does the right thing: The fsync() system call causes all modified data and attributes of fd to be moved to a permanent storage device. This normally results in all in- core modified copies of buffers for the associated file to be written to a disk. This is not the case for Linux, where fsync syncs the entire file system. That could explain some of the performance difference, but not all of it. I suppose it's worth noting that, in general, people report much better performance with MySQL on Linux than on FreeBSD. I mean than the data is only written to the drives memory and so can be lost if power goes down. I don't believe that fsync is required to flush the drive buffers. It would be nice to have a function that did, though. And how I can confirm this? Trial and error? Greg -- See complete headers for address and phone numbers. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] advisory locks and permissions
On 9/22/06, Tom Lane [EMAIL PROTECTED] wrote: Merlin Moncure [EMAIL PROTECTED] writes: On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: This is why I suggested we set aside some range of numbers that should not be used. Doing so would allow adding a better-managed numbering/naming scheme in the future. the whole point about advisory locks is that the provided lock space is unmanaged. I think we forgot to document that the lock space is per-database; also, ok. (ditto user lock legacy) wouldn't it be a good idea to specifically recommend that advisory locks be used only in databases that are used just by one application, or a few cooperating applications? The lack of any permissions checks makes them fairly unsafe in databases that are used by multiple users. yes and no. so long as it is understood as lock space inside a single database is shared by all sessions? what does it matter how many applications are connecting? applications sharing a single database implied that there is some negotiation of sharing of resources that the server is not aware of. There is no specific objection to n applictions using them on a shared database except for: 1. you must understand that the lock 'namespace' is at database level 2. memory for the lock table is sized for the database cluster, and is shared with standard locks. use it up, and its game over. imo, documentational thrust should be reinforcing those points and not making any specific recommendations which are not derived from them. I dont understand how having one or more applications has anything to do with namespace conflicts, either you have a centrally managed way of managing that namespace or you don't. the idea is just to make sure you have one. I would rather suggest, 'if you have multiple apps connection to the same database, take care to...' etc. going the in the mvcc area which deals (lightly) with locking strategies. overall, the documentation is extremely light on strategies for dealing with concurrency. however, something of a best practices might be in order in light of these considerations. I don't actually have a problem with the lack of security checks or key range limitations --- I see advisory locks as comparable to large objects, which are likewise permissions-free. It's an optional feature and you just won't use it in databases where permission constraints are a critical need. The thing that's bothering me is the relative ease of accidental DoS to applications in *other* databases in the same cluster. you have a point there. merlin ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] initdb ignores invalid locale names
When initdb is given an invalid (possibly mistyped) locale name, it just prints a warning and proceeds with the default locale from the environment. Someone already wondered about this before: /* should we exit here? */ if (!ret) fprintf(stderr, _(%s: invalid locale name \%s\\n), progname, locale); return ret; I obviously think we should. Why shouldn't we? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Release Notes: Major Changes in 8.2
Great, all added. --- Simon Riggs wrote: On Thu, 2006-09-21 at 21:45 -0400, Bruce Momjian wrote: Let me know how it looks. Very Good Very last, Minor change thoughts: * Continuous archiving enhancements change: Warm Standby enhancements The improvements to Continuous Archiving relate directly to the creation of Warm Standby servers, so it would be better to mention Warm Standby, not Continuous Archiving (and definitely not PITR) * Monitoring and logging additions add to end of line: improve performance tuning capability * COPY support for SELECT statements change: COPY TO support ... add to end of line: enhances data unload * Array and aggregate improvements add to end of line: , plus SQL:2003 statistical functions -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] advisory locks and permissions
On Sep 22, 2006, at 14:11 , Jim C. Nasby wrote: On Fri, Sep 22, 2006 at 01:21:57PM -0400, Merlin Moncure wrote: On 9/22/06, Jim C. Nasby [EMAIL PROTECTED] wrote: On Fri, Sep 22, 2006 at 12:56:37PM -0400, Merlin Moncure wrote: the whole point about advisory locks is that the provided lock space is unmanaged. for example, in the ISAM system I wrote which hooked into the acucobol virtual file system interface, I used a global sequence for row level pessimistic locking but reserved the 48th bit for table level locks. This system was extremely effective. on the current system I'm working on I use them to lock sequence oid's plus a high bit indicator for what i am doing. in short, advisory locks are application-defined in concept. Yes, but if you get two pieces of code written by different people using them in the same database, you can get hosed. As PostgreSQL becomes more popular and more people start developing software for it, this is more likely to occur. imo, that is no more or less likely than having two pieces of code store the same table in the same database. I think what you are describing would only be a concern if the locks were shared across databases, however this is not the case. the purpose of advisory locks is to be 'appplication-defined'. how the application is written is not part of that concept. we are simply granting the ability to create a mutex with a number for a name, that is all. Except you can put tables (and pretty much all your other objects) in a schema, one that's presumably named after your application. That greatly removes the odds of conficts. Indeed. In our development environment, we store development, integration, and testing schemas in the same database. This makes it trivial to move testing data to development, for example. If I want to use these locks, it seems I will have to hard-code some offset into each app or hash the schema name and use that as an offset :( In any case, I can't imagine the wtf? nightmares an accidental collision would induce. -M ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Release Notes: Major Changes in 8.2
On Thu, Sep 21, 2006 at 03:05:36PM -0500, Jim C. Nasby wrote: Regardless, I think we should include a section of major new projects/developments from pgFoundry, because they ultimately make PostgreSQL a more useful database. Maybe this list should only be in the I like that. New enhancement products or something? A -- Andrew Sullivan | [EMAIL PROTECTED] I remember when computers were frustrating because they *did* exactly what you told them to. That actually seems sort of quaint now. --J.D. Baldwin ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Getting a move on for 8.2 beta
Jim C. Nasby wrote: On Fri, Sep 22, 2006 at 11:58:04AM -0500, Bruno Wolff III wrote: On Wed, Sep 13, 2006 at 22:22:12 -0700, Tom Dunstan [EMAIL PROTECTED] wrote: That's a worthwhile point. How many patches come from the general community vs out of the blue? Patches from regulars could probably get a free pass, which might cut down the review burden substantially. And how were you planning to tell if a patch cam from a regular? Hopefully you weren't planning on blindly trusting the from header. Misuse of the build farm in a way the effects other sites could get the project a big black eye, so you want to be very careful building and executing code from the patch queue. Of course not, but there's any number of ways we could handle that problem. pgp signed patches? -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] silent install: silent error (even using the manual)
Good afternoon, dear PostgreSQL developers! First I want to thank all of you for the great DB Server! It's really useful, thank you! Second, here is my question(guys from #postgresql channel advised to write to dev list): I want to make a silent install of PostgreSQL version 8.1 using your recommendations on http://pginstaller.projects.postgresql.org/silent.html . But when I use this command: msiexec /i postgresql-8.1.msi /qb! /l*vx c:\pg_inst1.log INTERNALLAUNCH=1 ADDLOCAL=server,psql,pgadmin,jdbc,docs DOSERVICE=1 SERVICEDOMAIN=%COMPUTERNAME% SERVICEPASSWORD=suntek CREATESERVICEUSER=1 NOSHORTCUTS=1 I have nothing happened except appearing of window for 1 second with words: Preparing to install Then I can find msiexec.exe in processes with 0% CPU usage. I pasted full msi log below. What's the problem, can you tell me? You're not quite following the manual. You are executing postgresql-8.1.msi, not postgresql-8.1-int.msi. Small change, should make all the difference you need. //Magnus ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] advisory locks and permissions
Tom Lane wrote: Merlin Moncure [EMAIL PROTECTED] writes: advisory locks still show up as 'userlock' in the pg_locks view. does this matter? I'm disinclined to change that, because it would probably break existing client-side code for little gain. I think clarity suggests we should make the heading match the feature, i.e call it advisory rather than userlock. We changed the API, I don't see why keeping the heading makes sense. I think we should leave it unprotected unless we find out that there are unique security problems with advisory locks. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Release Notes: Major Changes in 8.2
Bruce Momjian wrote: I created a major features list for 8.2 and put it into CVS. Instead of going into detail (meaning the item would not appear in the Changes section below, I just highlighted some of the big stuff, and was purposely vague about the details, so people just have an overview of what is below. Let me know how it looks. Some of these just look rather vague. For example: * More control over creating/dropping objects and inheritance If I did not know what the features were, that item would convey nothing to me. The fact that you can add/drop the inheritance characteristics of a table after its creation isn't something I would just lump under more control - it's a major new feature that will possibly revolutionize the way people use inheritance, especially for partitioning. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Fwd: Is the fsync() fake on FreeBSD6.1?
On Fri, Sep 22, 2006 at 01:52:02PM -0400, Jim Nasby wrote: I thought folks might be interested in this... note in particular the comment about linux. ... From: Greg 'groggy' Lehey [EMAIL PROTECTED] Date: June 26, 2006 11:34:12 PM EDT To: leo huang [EMAIL PROTECTED] Cc: freebsd-performance@freebsd.org Subject: Re: Is the fsync() fake on FreeBSD6.1? ... My understanding from the last time I looked at the code was that fsync does the right thing: The fsync() system call causes all modified data and attributes of fd to be moved to a permanent storage device. This normally results in all in- core modified copies of buffers for the associated file to be written to a disk. This is not the case for Linux, where fsync syncs the entire file system. That could explain some of the performance difference, but not all of it. I suppose it's worth noting that, in general, people report much better performance with MySQL on Linux than on FreeBSD. I see Greg's comment as contradictory. People see better performance with MySQL on Linux than on FreeBSD, fsync() on Linux syncs the whole file system? I don't believe that fsync() on Linux syncs the whole file system either. This sounds made up, or a confusion with 'sync'. Perhaps people @FreeBSD.org are not as familiar with Linux. Cheers, mark -- [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] __ . . _ ._ . . .__. . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/|_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] advisory locks and permissions
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I'm disinclined to change that, because it would probably break existing client-side code for little gain. I think clarity suggests we should make the heading match the feature, i.e call it advisory rather than userlock. We changed the API, I don't see why keeping the heading makes sense. (a) we changed a *different* part of the API; I don't see how that licenses us to whack around anything that's marginally related. (b) we put up that pgfoundry module so that there would be a backward compatible solution. Won't be very backward compatible if the locks look different in pg_locks. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] advisory locks and permissions
Jim C. Nasby [EMAIL PROTECTED] writes: Ahh, ok, I didn't realize that the total lock space was larger than what's being exposed today. That means we can easily add that stuff in the future and not break anything, which is all I was looking for. Yeah --- in particular, we can always add more LOCKTAG values, or make use of field4 values that are not possible with the current API. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] advisory locks and permissions
* Tom Lane ([EMAIL PROTECTED]) wrote: Stephen Frost [EMAIL PROTECTED] writes: * Tom Lane ([EMAIL PROTECTED]) wrote: An admin who is concerned about this can revoke public access on the functions for himself ... but should that be the default out-of-the-box configuration? I feel more comfortable with saying you have to turn on this potentially-dangerous feature than with saying you have to turn it off. I agree with having it turned off by default, at least in 8.2. Do we have a consensus to do this for 8.2? Or are we going to leave it as is? Those are the only two realistic short-term options ... I'm still of the opinion it'd be better disabled by default, but it seems that the majority is going the other way. I guess in the end I'd like to see most of these patched up in such a way that a given user would be reasonably limited in their ability to DoS the server. That's not going to happen today though. Thanks, Stephen signature.asc Description: Digital signature
Re: [HACKERS] advisory locks and permissions
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: I'm disinclined to change that, because it would probably break existing client-side code for little gain. I think clarity suggests we should make the heading match the feature, i.e call it advisory rather than userlock. We changed the API, I don't see why keeping the heading makes sense. (a) we changed a *different* part of the API; I don't see how that licenses us to whack around anything that's marginally related. (b) we put up that pgfoundry module so that there would be a backward compatible solution. Won't be very backward compatible if the locks look different in pg_locks. But is anyone going to know what userlocks is in 1-2 years? We have few people using /contrib/userlocks, but in the future, I bet we have a lot more people using advisory locks, and being confused. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Fwd: Is the fsync() fake on FreeBSD6.1?
[EMAIL PROTECTED] writes: I don't believe that fsync() on Linux syncs the whole file system either. Indeed. I'd disregard this as coming from someone who knows much less than he thinks. (The most likely explanation for his results, I expect, is that FreeBSD is trying to fsync and the disk drive is lying to it, whereas on his comparison Linux machine the drive is not configured to lie about write-complete.) regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Fwd: Is the fsync() fake on FreeBSD6.1?
On Sep 22, 2006, at 15:00 , [EMAIL PROTECTED] wrote: On Fri, Sep 22, 2006 at 01:52:02PM -0400, Jim Nasby wrote: I thought folks might be interested in this... note in particular the comment about linux. ... From: Greg 'groggy' Lehey [EMAIL PROTECTED] Date: June 26, 2006 11:34:12 PM EDT To: leo huang [EMAIL PROTECTED] Cc: freebsd-performance@freebsd.org Subject: Re: Is the fsync() fake on FreeBSD6.1? ... My understanding from the last time I looked at the code was that fsync does the right thing: The fsync() system call causes all modified data and attributes of fd to be moved to a permanent storage device. This normally results in all in- core modified copies of buffers for the associated file to be written to a disk. This is probably the same issue that the hackers encountered on Darwin- namely fsync() flushes the kernel cache, but a further function call was needed to flush the hard drive buffers. This meets the standard's definition of fsync because the data is indeed moved to the device, but it happens to just be the device's buffer instead of non-volatile storage. -M ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] initdb ignores invalid locale names
Peter Eisentraut [EMAIL PROTECTED] writes: When initdb is given an invalid (possibly mistyped) locale name, it just prints a warning and proceeds with the default locale from the environment. Someone already wondered about this before: /* should we exit here? */ if (!ret) fprintf(stderr, _(%s: invalid locale name \%s\\n), progname, locale); I obviously think we should. Why shouldn't we? That would make it impossible to install at all on a machine with broken locale support, which seems a bit of an overreaction, especially for the noncritical locale items like LC_MESSAGES (which also happens to be the one that fails most often). I could go with erroring for LC_COLLATE and LC_CTYPE, but I'd prefer to see us fall back to C for any of the other LC_ items that fail. Those guys are easy to fix after the fact, if the user cares enough (or at all). BTW, it would also be a good idea if the message specified which LC_ item we failed to set. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] advisory locks and permissions
AgentM [EMAIL PROTECTED] writes: If I want to use these locks, it seems I will have to hard-code some offset into each app or hash the schema name and use that as an offset :( In any case, I can't imagine the wtf? nightmares an accidental collision would induce. That depends entirely on how you are choosing to assign the lock key numbers. If you use something involving table OID, for example, there is not a risk of collision from schema considerations. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] advisory locks and permissions
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: (b) we put up that pgfoundry module so that there would be a backward compatible solution. Won't be very backward compatible if the locks look different in pg_locks. But is anyone going to know what userlocks is in 1-2 years? We have few people using /contrib/userlocks, but in the future, I bet we have a lot more people using advisory locks, and being confused. The reason they're advisory is that the current set of functions for accessing them doesn't enforce anything. That doesn't make the locks themselves any more or less user-defined than they were before --- certainly the pg_locks view has got nothing to do with whether they are advisory or enforced. I do not see a good reason to change it. It might be worth mentioning in the description of the pg_xxx_lock functions that the locks they acquire are shown as userlock in pg_locks, but that seems sufficient. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] advisory locks and permissions
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: (b) we put up that pgfoundry module so that there would be a backward compatible solution. Won't be very backward compatible if the locks look different in pg_locks. But is anyone going to know what userlocks is in 1-2 years? We have few people using /contrib/userlocks, but in the future, I bet we have a lot more people using advisory locks, and being confused. The reason they're advisory is that the current set of functions for accessing them doesn't enforce anything. That doesn't make the locks themselves any more or less user-defined than they were before --- certainly the pg_locks view has got nothing to do with whether they are advisory or enforced. I do not see a good reason to change it. It might be worth mentioning in the description of the pg_xxx_lock functions that the locks they acquire are shown as userlock in pg_locks, but that seems sufficient. My point is that if you are going to call them user locks, you then are going to have to call them userlocks in the documentation, which seems pointless, considering that 99% of people who use pg_locks are not applicadtions but users monitoring the system. I just don't see a problem with making it consistent. I don't see the column rename as an API change issue. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] advisory locks and permissions
On 9/22/06, AgentM [EMAIL PROTECTED] wrote:
Except you can put tables (and pretty much all your other objects)
in a
schema, one that's presumably named after your application. That
greatly
removes the odds of conficts.
Indeed. In our development environment, we store development,
integration, and testing schemas in the same database. This makes it
trivial to move testing data to development, for example.
If I want to use these locks, it seems I will have to hard-code some
offset into each app or hash the schema name and use that as an
offset :( In any case, I can't imagine the wtf? nightmares an
accidental collision would induce.
i think you are obsuring something here. advisory_lock is a mutex
with a numeric name...thats it :) any meaning you impart into that
name is your problem. listen/notify is a similar construct in that
way.
I ran an erp system, one company per schema, using userlock module for
pessimistic row locking with no problems. I used bit shifting to
strip off the high bit (out of 48) for special table locks and other
things. key mechasim was to use a sequence to provide lock id which
was shared by all lockable objects. a domain could be appropriate
here:
create sequence lock_provider;
create domain lockval as bigint default nextval('lock_provider');
and the following becomes standard practice:
create table foo (lv lockval); --no need for index here
select pg_advisory_lock(lv) from foo where [..];
for bit shifting or special cases you can wrap the lock function, which i did.
merlin
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] advisory locks and permissions
Bruce Momjian [EMAIL PROTECTED] writes: I don't see the column rename as an API change issue. How can you possibly claim it's not an API change? If you're insistent on this, my recommendation would be to add a new LOCKTAG value for advisory locks instead of re-using LOCKTAG_USERLOCK. This would take a little bit more code space but it would preserve the same pg_locks display for people using the old contrib code, while we could use advisory for locks created by the new code. (Which I still maintain is a pretty bad way of describing the locks themselves, but obviously I'm failing to get through to you on that.) regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] 8.3 Development Cycle
Tom, Pretty sure. :) Why the oops? They haven't been mentioned in some PR material or something have they? No, I'd just been confused and thought the patch was submitted before feature freeze. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Release Notes: Major Changes in 8.2
Andrew Sullivan wrote: On Thu, Sep 21, 2006 at 03:05:36PM -0500, Jim C. Nasby wrote: Regardless, I think we should include a section of major new projects/developments from pgFoundry, because they ultimately make PostgreSQL a more useful database. Maybe this list should only be in the I like that. New enhancement products or something? In that case, what about things on gborg too? I just updated PL/R for 8.2 compatibility (and finally changed the status from alpha to beta). BTW, I'm happy to move PL/R over to pgFoundry, but became a little concerned about doing that after seeing the lengthy thread regarding pgFoundry concerns (but admittedly, I didn't have time to read the thread in detail, because I'm back over in Germany on a long business trip again). Joe ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Fwd: Is the fsync() fake on FreeBSD6.1?
On 2006-09-22, Jim Nasby [EMAIL PROTECTED] wrote: I thought folks might be interested in this... note in particular the comment about linux. I don't believe that either person in that discussion knows what they are really talking about. fsync() on FreeBSD does, as is required, force any modified data for the file, plus any metadata, plus any modifications to any parent directories, to the underlying disk device and waits for that device to report the write as complete. Whether the underlying device lies about the write completion is another matter. All current SCSI disks have WCE enabled by default, which means that they will lie about write completion if FUA was not set in the request, which FreeBSD never sets. (It's not possible to get correct results by having fsync() somehow selectively set FUA, because that would leave previously-completed requests in the cache.) WCE can be disabled on either a temporary or permanent basis by changing the appropriate modepage. It's possible that Linux does this automatically, or sets FUA on all writes, though that would surprise me considerably; however I disclaim any knowledge of Linux internals. On FreeBSD, this command will disable WCE permanently on a SCSI drive: echo 'WCE: 0' | camcontrol modepage daXX -m 8 -P3 -e (use -P0 to disable it only temporarily, or you can use just the second of those commands alone to interactively edit the mode page) -- Andrew, Supernews http://www.supernews.com - individual and corporate NNTP services ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] advisory locks and permissions
On 9/22/06, Tom Lane [EMAIL PROTECTED] wrote: Bruce Momjian [EMAIL PROTECTED] writes: I don't see the column rename as an API change issue. How can you possibly claim it's not an API change? i dunno, i agree with bruce here. we are just changing the output of pg_locks a bit reflecting the change in moving contrib to core. nobody cares about the literal output of pg_locks for userlocks except the old contrib users. compatiblity could be supplied in the pgfoundry module for this as well. i say to leave the lock tables alone and change to 'advsiory'. it just seems odd the way it is. merlin ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] advisory locks and permissions
Merlin Moncure wrote: On 9/22/06, Tom Lane [EMAIL PROTECTED] wrote: Bruce Momjian [EMAIL PROTECTED] writes: I don't see the column rename as an API change issue. How can you possibly claim it's not an API change? i dunno, i agree with bruce here. we are just changing the output of pg_locks a bit reflecting the change in moving contrib to core. nobody cares about the literal output of pg_locks for userlocks except the old contrib users. compatiblity could be supplied in the pgfoundry module for this as well. i say to leave the lock tables alone and change to 'advsiory'. it just seems odd the way it is. Agreed. I just don't imagine many current user applications referencing userlocks, and I do imagine confusion in the future by users using the new API which call them advisory. I guess it is a compatibility change, but weighing compatibility against clarity, I am leaning toward clarity. I assume it is this line that would be changed: _(user lock [%u,%u,%u,%u]), By my reading of that, that string is language-local, so anyone trying to parse that directly is going to have a larger problem than our renaming it for 8.2. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] advisory locks and permissions
Bruce Momjian [EMAIL PROTECTED] writes: I guess it is a compatibility change, but weighing compatibility against clarity, I am leaning toward clarity. I assume it is this line that would be changed: _(user lock [%u,%u,%u,%u]), You assume wrong ... that has nothing to do with what appears in pg_locks. Sigh. I'll go break up the locktag into two. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Bitmap index status
Jie Zhang wrote: Hi Heikki and all, I just sent the latest bitmap index patch to the list. I am not sure if there is any size limit for this mailing list. If you have received my previous email, please let me know. Hi Jie, I know I said I was going to get testing on this months ago but I've been juggling between 3 systems due to disk failures and other hardware configuration issues. Anyways, I've take a baseline run of only the power test using a 1GB database with the patch 09-17 patch against a snapshot of pgsql from 2006-09-17: http://dbt.osdl.org/dbt/dbt3testing/results/dev8-007/2/ Do you think the 1GB scale factor will be sufficient for testing as it will certainly be faster? Do you think testing with just a power test will be sufficient for now? I really don't have a good reason why I didn't run a throughput test other than to save time. :) I also wanted to get your opinion again on which indexes we will want to try first. Thanks, Mark ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] Traveling to Oxford, England
FYI, I am traveling to Oxford, England next week, Monday to Friday, to meet with Simon and Heikki. My Internet connectivity will be irregular. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 8.3 Development Cycle
On Friday 22 September 2006 12:40, Tom Lane wrote:
Tom Dunstan [EMAIL PROTECTED] writes:
Joshua's original mail suggested that only certain features would go in.
Is that still on the cards, or will other features be considered if
they're ready?
You'll note that Dave's mail said no such thing.
I'd like to see some type of statement from core what level of changes thier
willing to allow in the core for this short release (system catalog changes
seem like a must, and looks like on disk format changes is ok too?) Of
course my angle is making the upgrade from 8.2-8.3 as painless as
possible... if we can avoid a dump/reload cycle then people are less likely
to have to choose between 8.2 and 8.3, which would make everyone happy I
imagine.
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] 8.3 Development Cycle
Robert Treat wrote: On Friday 22 September 2006 12:40, Tom Lane wrote: Tom Dunstan [EMAIL PROTECTED] writes: Joshua's original mail suggested that only certain features would go in. Is that still on the cards, or will other features be considered if they're ready? You'll note that Dave's mail said no such thing. I'd like to see some type of statement from core what level of changes thier willing to allow in the core for this short release (system catalog changes seem like a must, and looks like on disk format changes is ok too?) Of course my angle is making the upgrade from 8.2-8.3 as painless as possible... if we can avoid a dump/reload cycle then people are less likely to have to choose between 8.2 and 8.3, which would make everyone happy I imagine. Agreed, but my guess is that we are going to introduce shorter varlena headers for 8.3. It will hard to reject an optimization like that, and that will probably change the disk format for most columns. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 8.3 Development Cycle
Bruce Momjian [EMAIL PROTECTED] writes: Agreed, but my guess is that we are going to introduce shorter varlena headers for 8.3. It will hard to reject an optimization like that, and that will probably change the disk format for most columns. Well, several of the proposals that have been made would not cause existing disk images to become broken --- in particular, the idea of introducing separate short datatypes without touching the existing ones would have that merit. So we might want to factor that point into our choices about what to do. This is all pretty pointless unless someone actually writes a pg_upgrade tool, but maybe we'll see one for 8.3. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] PostgreSQL 8.2beta1 Now Available
Just a short note that the first Beta is now available on ftp.postgresql.org, and, shortly, on the mirrors ... This isn't a full announce, which will be on Monday ... but please run a few tests, make sure everything looks okay ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
