Re: [HACKERS] remove flatfiles.c
On Mon, 2009-08-31 at 18:53 -0400, Alvaro Herrera wrote: Regarding sync commits that previously happen and now won't, I think the only case worth worrying about is the one in vacuum.c. Do we need a ForceSyncCommit() in there? I'm not sure if vacuum itself already forces sync commit. VACUUM FULL requires ForceSyncCommit(). Not sure why removing them elsewhere is important? Getting robustness wrong is a big, bad thing and this opens us to future error. We already tuned VACUUM so it does very little if it has no work to do, why would one extra I/O improve things so much? If it ain't broke... VACUUM does so many things that I'd rather have it all safely on disk. I'd feel happier with the rule VACUUM always sync commits, so we all remember it and can rely upon it to be the same from release to release. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] binary compat
Hi, I've been idly thinking about binary COPY and recent performance efforts spent on it. The main problem I have with binary output is that I never know when it'll be any useful (except very same hardware and PostgreSQL setup)... useful meaning I get to read it back into some database. Would providing the following two functions help? SELECT * FROM pg_binary_compat(); - version, timestamp ondisk format, architecture, you name it SELECT pg_binary_compat(version, timestamp ondisk format, ...) - bool Equiped with this I would first setup a (somewhat portable) virtual machine where I'm sure I'm able to load the binary files, then get to use COPY BINARY everywhere possible: I have a way to easily tell when not to use it. First example would be to teach londiste about this... Now writing the function doesn't sound a lot of fun, but I don't know what to compare to be able to decide whether COPY output will get read correctly on input. It even looks to me like in some cases this would work accross major PostgreSQL versions? Or would that depend on used types, so we'd need a variant taking those into consideration? Anyone interrested in spelling out the checks to get done to implement the functions, or to implementing it, provided there's no obvious reason why it can not be made trustworthy? Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] autovacuum launcher using InitPostgres
On Mon, Aug 31, 2009 at 21:49, Tom Lanet...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Tom Lane wrote: Well, I'm not sure the average user knows or cares about the difference between the launcher and the workers. The thing that was in the back of my mind was that we would now have the option to have the launcher show up in pg_stat_activity. If we were to do that then the case for counting it in the user-visible number-of-processes parameter would get a lot stronger (enough to justify renaming the parameter, if you insist that the launcher isn't a worker). I don't however have any strong opinion on whether we *should* include it in pg_stat_activity --- comments? The user may not care about the difference, but there's a point in having the limit be the simpler concept of this is the maximum amount of processes running vacuum at any time. The launcher is very uninteresting to users. I committed things that way, but I'm still not convinced that we shouldn't expose the launcher in pg_stat_activity. The thing that is bothering me is that it is now able to take locks and potentially could block some other process or even participate in a deadlock. Do we really want to have entries in pg_locks that don't match any entry in pg_stat_activity? At first I'd have voted that we don't have it in pg_stat_activity, but the argument about pg_locks really is the killer on. IMHO, it *has* to go there then. I would also suggest that we add a separate column to pg_stat_activity indicating what type of process it is, so that monitoring tools can figure that out without having to resort to string matching on the current query field. This field would indicate if it's a backend, autovac worker, autovac launcher. And perhaps we should even add the other utility processes to pg_stat_activity, for completeness? Another thought along a similar line is some way to detect the idle and idle-in-transaction states without doing string matching as well. Perhaps this could be overloaded on said extra field, or should it have a separate one? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] binary compat
On 9/1/09, Dimitri Fontaine dfonta...@hi-media.com wrote: I've been idly thinking about binary COPY and recent performance efforts spent on it. The main problem I have with binary output is that I never know when it'll be any useful (except very same hardware and PostgreSQL setup)... useful meaning I get to read it back into some database. Would providing the following two functions help? SELECT * FROM pg_binary_compat(); - version, timestamp ondisk format, architecture, you name it SELECT pg_binary_compat(version, timestamp ondisk format, ...) - bool Equiped with this I would first setup a (somewhat portable) virtual machine where I'm sure I'm able to load the binary files, then get to use COPY BINARY everywhere possible: I have a way to easily tell when not to use it. First example would be to teach londiste about this... Now writing the function doesn't sound a lot of fun, but I don't know what to compare to be able to decide whether COPY output will get read correctly on input. It even looks to me like in some cases this would work accross major PostgreSQL versions? Or would that depend on used types, so we'd need a variant taking those into consideration? Anyone interrested in spelling out the checks to get done to implement the functions, or to implementing it, provided there's no obvious reason why it can not be made trustworthy? Based on plproxy experience with binary i/o, the requirements are: - same major.minor [Perhaps checking catversion would be better?] - same integer_datetimes - same server_encoding This seems to cover most (?) builtin types. Arch details (32/64 bit, little/big endian) should not matter, as binary i/o functions are supposed to use arch-independent format. As plproxy uses explicit casting ($2::typname) it did not need to worry about any oid differences. I have not looked at binary COPY details, whether it can also ignore oids. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] binary compat
Dimitri Fontaine wrote: Hi, I've been idly thinking about binary COPY and recent performance efforts spent on it. The main problem I have with binary output is that I never know when it'll be any useful (except very same hardware and PostgreSQL setup)... useful meaning I get to read it back into some database. If you want a binary cross compatible, how about something well known, standardized, simple and cross platform like XDR? http://www.faqs.org/rfcs/rfc1014.html -- Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] binary compat
I've been idly thinking about binary COPY and recent performance efforts spent on it. The main problem I have with binary output is that I never know when it'll be any useful (except very same hardware and PostgreSQL setup)... useful meaning I get to read it back into some database. I posted about patching binary copy to improve speed some time ago... I'm a bit out of time right now to work on it, but I should have the time in September to submit a patch. Would providing the following two functions help? SELECT * FROM pg_binary_compat(); - version, timestamp ondisk format, architecture, you name it SELECT pg_binary_compat(version, timestamp ondisk format, ...) - bool Equiped with this I would first setup a (somewhat portable) virtual machine where I'm sure I'm able to load the binary files, then get to use COPY BINARY everywhere possible: I have a way to easily tell when not to use it. First example would be to teach londiste about this... Now writing the function doesn't sound a lot of fun, but I don't know what to compare to be able to decide whether COPY output will get read correctly on input. It even looks to me like in some cases this would work accross major PostgreSQL versions? Or would that depend on used types, so we'd need a variant taking those into consideration? Anyone interrested in spelling out the checks to get done to implement the functions, or to implementing it, provided there's no obvious reason why it can not be made trustworthy? Actually, I've been surprised to discover that if you COPY a table which contains integers to a file in binary format, you can reload it into a table which contains dates (still using binary format) and Postgres will not complain, since a date is an integer too, of the same length... so you'll get completely garbled data, of course, and no error message. Well, the data isn't really garbled, but it's just the interpretation of it that is different (you could dump it again and reload it into an integer table, and you'd find your original integers, interpreted as integers this time). You could probably load a bunch of 4-byte text strings into an INT column, too. Thinking about it again, I wonder why I was surprised, since the binary format doesn't include any column type information. But given how careful postgres is with correctness, this could be considered a bug... it is very careful on parsing text data, and not at all on binary. What makes text copy slow is all the parsing : turning text into datums is very complex (look at the source code of strtod() in glibc...) Validation is simple : including it in binary copy shouldn't slow it down. So, I believe the COPY binary file format should include this information : 1- everything you proposed - version - timestamp format - etc (or perhaps a binary format version which is known for each type included in the table) 2- list of column names and types - check at COPY FROM time and issue appropriate errors : cannot copy a column of type DATE from binary data of type INTEGER - have a special command or a little tool (anything, even a simple python script) that can look at a COPY header and write the appropriate CREATE TABLE statement for your convenience : have you ever grumbled, when developing an app, hm, I want to reload this dump I did a week ago for testing... what was the table like at this time ?... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release notes idea
Bruce Momjian írta: Greg Sabino Mullane wrote: [ There is text before PGP section. ] -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Just noticed in the release notes for 8.4, there are two items accredited to just Greg (but there are three of us Gregs who contributed to 8.4 and are in the notes). While this is very likely Greg Stark, due to the context, I think at this point in the project we simply should spell out everyone's complete name every time, rather than the hit or miss style we are developing. (Enjoy while you can, Bruce, we'll get another Bruce to contribute someday! Zoltan might be harder... :) Believe it or not, there's another guy with my name in Hungary who is a poet and writer and is more famous than me. :-) http://www.boszormenyizoltan.info/ ( he's not me :-) ) That is an interesting suggestion. I have tried to use short names where possible to avoid having the usernames become too prominent. -- Bible has answers for everything. Proof: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. (Matthew 5:37) - basics of digital technology. May your kingdom come - superficial description of plate tectonics -- Zoltán Böszörményi Cybertec Schönig Schönig GmbH http://www.postgresql.at/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Linux LSB init script
On mån, 2009-08-31 at 20:54 -0400, Greg Smith wrote: On Mon, 31 Aug 2009, Kevin Grittner wrote: My counter-argument to that would be that the SuSE distribution's version of PostgreSQL is so out-of-date that we don't install it. Given that it's also RPM based, is it possible to get SuSE in sync so that it shares the same init script as RHEL? Devrim is in the middle of a major overhaul of the RHEL/Fedora init script lately, adding better support for multiple postmasters and the like, and I'd think that SuSE users would like to take advantage of that work too. Well, the init script of the SUSE RPM is not the problem. (It was, arguably, one of the bright spots for a long time, since they removed lots of legacy wackyness accumulated from the Red Hat era.) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Linux LSB init script
On mån, 2009-08-31 at 15:17 -0500, Kevin Grittner wrote: My counter-argument to that would be that the SuSE distribution's version of PostgreSQL is so out-of-date that we don't install it. It also doesn't enable debug info or integer date times. So we switched to build from source before we actually got as far as loading any data. I'm inclined to recommend the same to others. Fixes and help are welcome, btw. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Bison crashes postgresql
The thing is I was in a project to develop a Fuzzy Database Management System. We have to bring fuzzy logic to postgresql, there was a team developing a software in java and the other developing in the postgresql core. But I always think these were wrong, so I'm trying to develop a library to do that, but I guess I'm moving forward, the bug was solved and I have a google code project: http://code.google.com/p/fuzzyquery/. Now I hope to give more features and have a fully functional library. 2009/9/1 Hans-Juergen Schoenig -- PostgreSQL postg...@cybertec.at: Andrew Dunstan wrote: Werner Echezuria wrote: Hi, I have a code in which I translate some code from sqlf to sql, but when it comes to yy_parse the server crashes, I have no idea why, because it works fine in other situations. I don't understand why you're doing what you're doing this way. Wouldn't it be better to patch the main postgres parser and make your functionality first class rather than having it run via an SQL string and a function that calls a secondary parser? cheers andrew yes, this is the thing i had in mind as well. what is your ultimate goal? many thanks, hans -- Cybertec Schoenig Schoenig GmbH Reyergasse 9 / 2 A-2700 Wiener Neustadt Web: www.postgresql-support.de -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: [COMMITTERS] pgsql: Move processing of startup-packet switches and GUC settings into
Tom Lane wrote: Log Message: --- Move processing of startup-packet switches and GUC settings into InitPostgres, to fix the problem that SetClientEncoding needs to be done before InitializeClientEncoding, as reported by Zdenek Kotala. We get at least the small consolation of being able to remove the bizarre API detail that had InitPostgres returning whether user is a superuser. So, I think InitPostgres API could be deuglified a bit more by having the launcher pass the database name instead of the OID. There's the small race condition that the launcher could pass a database name and have it be renamed before the worker starts, but I think it's small enough to not matter much (note that it's already true that the launcher could pass a database OID and have it be dropped before the worker starts). Is anyone opposed to doing that change? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
Simon Riggs si...@2ndquadrant.com writes: VACUUM does so many things that I'd rather have it all safely on disk. I'd feel happier with the rule VACUUM always sync commits, so we all remember it and can rely upon it to be the same from release to release. Non-FULL vacuum has *never* done a sync commit, except in the unusual corner case that it moves the database's datfrozenxid, which is a corner case that didn't even exist until fairly recently. I think the argument that we should have it force sync for no reason whatsoever is silly. We get beat up on a regular basis about spikes in response time; why would you want to have vacuum creating one when it doesn't need to? As for the FULL case, the sync commit is to try to protect a horribly unsafe kluge that should go away entirely (if vacuum full itself doesn't go away entirely). That's hardly something I want to institutionalize either. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Move processing of startup-packet switches and GUC settings into
Alvaro Herrera alvhe...@commandprompt.com writes: So, I think InitPostgres API could be deuglified a bit more by having the launcher pass the database name instead of the OID. Huh? You mean workers? Where are they going to get the name from? The AV shmem structures carry OIDs no? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Linux LSB init script
Peter Eisentraut pete...@gmx.net wrote: On mån, 2009-08-31 at 15:17 -0500, Kevin Grittner wrote: the SuSE distribution's version of PostgreSQL is so out-of-date that we don't install it. It also doesn't enable debug info or integer date times. Fixes and help are welcome, btw. We're using SLES10. If we installed the latest from the SuSE site, we would be at PostgreSQL version 8.1. Now, to their credit, that is 8.1.17; but we're looking at when to upgrade from 8.3 to 8.4, not at whether to go back to 8.1. And unless I'm remembering incorrectly, the configure options are not what we would want. I don't see any reason the packaged build shouldn't be with --enable-debug on a platform where that has no performance hit. I never understood why anyone converting to PostgreSQL would want the floating point approximate timestamps rather than using --enable-integer-datetimes. We don't have a need for multiple languages, so I figure it's best to use --diable-nls, although I doubt that one's a huge deal. Since we have much XML in our database, I'm building with --with-libxml, just in case someone decides they want to use xpath. And since we often need more than one version of PostgreSQL on a server, we always use a prefix of /usr/local/pgsql-version and use a symlink to map one version to the default we put on our PATH. (Our init scripts always point to the version-specific locations.) I wouldn't expect a packaged SuSE build to cater to all of that; but it would be nice if they donated their init script to the PostgreSQL project, so that those of us who have a reason to build from source on SuSE have have a convenient starting point in the source distribution of PostgreSQL. I seem to remember that SuSE has an abstraction layer similar to the functions I defined in my linux-lsb script. I couldn't use those in a portable LSB submission, but if there's anything in what I did which is of any use to SuSE, there's no reason for SuSE not to use it. I'm putting it under the same license as the rest of PostgreSQL. Although, now that I think of it, I don't think I actually put any copyright or license message into the file yet. In any event, I think I've come up with something which should work pretty well on any LSB conforming implementation, including SuSE. If we get the pg_ping functionality which is occassionally discussed, it can be made really bulletproof. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
On Tue, 2009-09-01 at 09:58 -0400, Tom Lane wrote: We get beat up on a regular basis about spikes in response time; why would you want to have vacuum creating one when it doesn't need to? If one I/O on a background utility can cause such a spike, we are in serious shitake. I would be more comfortable if the various important things VACUUM does were protected by sync commit. I see no reason to optimise away one I/O just because we might theoretically do so. Any mistake in the theory and we are exposed. Why take the risk? We do many things to check and secure our data, why not this one? If this was suggested separately it as an optimisation you'd laugh and say why bother? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] \d+ for long view definitions?
On 8/31/09 1:13 PM, Peter Eisentraut wrote: I guess my premise is that if I use \d, I'm primarily interested in the column names and types. The view definition is secondary. If the view definition is a single line or uses a single table, it's interesting because it might describe something about the schema design, but if it's 20 lines it's an implementation detail. I agree with Peter here. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
Simon Riggs si...@2ndquadrant.com writes: On Tue, 2009-09-01 at 09:58 -0400, Tom Lane wrote: We get beat up on a regular basis about spikes in response time; why would you want to have vacuum creating one when it doesn't need to? If one I/O on a background utility can cause such a spike, we are in serious shitake. I would be more comfortable if the various important things VACUUM does were protected by sync commit. I see no reason to optimise away one I/O just because we might theoretically do so. Any mistake in the theory and we are exposed. Why take the risk? *WHAT* risk? Most vacuums do not do a sync commit, and never have. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] \d+ for long view definitions?
On Tue, Sep 1, 2009 at 1:31 PM, Josh Berkusj...@agliodbs.com wrote: On 8/31/09 1:13 PM, Peter Eisentraut wrote: I guess my premise is that if I use \d, I'm primarily interested in the column names and types. The view definition is secondary. If the view definition is a single line or uses a single table, it's interesting because it might describe something about the schema design, but if it's 20 lines it's an implementation detail. I agree with Peter here. I think we should always or never show the view definition, not sometimes. And I also agree with Tom's point that we should fix the pager. The way that it works now is really annoying. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release notes idea
g...@turnstep.com (Greg Sabino Mullane) writes: (Enjoy while you can, Bruce, we'll get another Bruce to contribute someday! Zoltan might be harder... :) My middle name is Bruce; there wouldn't be any legal humps to overcome for me to use the moniker Bruce Browne :-). -- output = reverse(ofni.sailifa.ac @ enworbbc) Christopher Browne Bother, said Pooh, Eeyore, ready two photon torpedoes and lock phasers on the Heffalump, Piglet, meet me in transporter room three -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] \d+ for long view definitions?
Robert Haas robertmh...@gmail.com writes: I think we should always or never show the view definition, not sometimes. Yeah. I can live with \d not showing it and \d+ showing it --- as Peter already mentioned, that would be consistent with \df behavior. Making it depend on the length is just weird. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Building under Visual Studio 2008 - pqcomm.c compile error
I'm getting this same error and wonder if Tom's fix is sufficient. IPPROTO_IPV6 is defined in wd2def.h IF _WIN32_WINNT = 0x0501, but in pg_config_os.h _WIN32_WINNT is DEFINED as 0x0500 so IPPROTO_IPV6 is left undefined. Regards, Dave --- Unfortunately I had to uninstall VS2005 to install VS2008 due to limited space, or I'd search the old libraries to see where the symbol used to be. Anyone care to search their VS2005 includes for IPPROTO_IPV6 so I can compare to the VS2008? Also, following the usual build procedure (using build.bat/build.pl), I encountered errors saying I needed to do vcbuild /upgrade on postgres.vcproj, libpgport.vcproj, and pgevent.vcproj. After consulting with one of our QA people who builds stuff in Windows all the time, I came up with the following procedure to build postgres under VS2008 (This includes changing pqcomm.c per Tom's comment below): (in msvc) mkvcbuild.pl cd ..\..\.. vcbuild /upgrade postgres.vcproj vcbuild /upgrade libpgport.vcproj vcbuild /upgrade pgevent.vcproj (actually, I wound up creating a script that did vcbuild /upgrade to ALL of the vcproj files here) msbuild pgsql.sln Note that after doing all the vcbuild /upgrade commands, I used msbuild referencing the pgsql.sln file instead of vcbuild to do the actual build. So, aside from the issue of the missing IPPROTO_IPV6 symbol, its built under VS2008. It will probably be some time before I can try running postgres built this way, as we're under the gun to get a release out this week. I'm not an MS developer by any means, so if someone has a better way to do this feel free to say so. This is what worked for me. Doug -Original Message- From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us] Sent: Monday, July 14, 2008 12:05 PM To: Knight, Doug Subject: Re: [HACKERS] Building under Visual Studio 2008 - pqcomm.c compile error Knight, Doug dknight(at)wsi(dot)com writes: Since I am primarily a Linux-based coder, do you know where I would find the header files under VS2005 or 2008? No idea, I don't use MSVC. Also, it looks like the IPPOROTO_IPV6 is only used within a ifdef check for IPV6_ONLY. Is there some way I could undefined it to prevent this part of the code from being compiled? Well, you could just change #ifdef IPV6_V6ONLY to #if defined(IPPROTO_IPV6) defined(IPV6_V6ONLY) and then it would compile --- but whether it would work right is less clear, unless your machine doesn't do IPV6 anyway. Since we know this code compiles under VS2005, I'm inclined to recommend that you look for the real solution, which is to find out where that symbol went... regards, tom lane This electronic mail message is intended exclusively for the individual(s) or entity to which it is addressed. This message, together with any attachment, is confidential and may contain privileged information. Any unauthorized review, use, printing, retaining, copying, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email message to the sender and delete all copies of this message. THIS E-MAIL IS NOT AN OFFER OR ACCEPTANCE: Notwithstanding the Uniform Electronic Transactions Act or any other law of similar import, absent an express statement to the contrary contained in this e-mail, neither this e-mail nor any attachments are an offer or acceptance to enter into a contract, and are not intended to bind the sender, LeTourneau Technologies, Inc., or any of its subsidiaries, affiliates, or any other person or entity. WARNING: Although the company has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
Re: [HACKERS] \d+ for long view definitions?
Robert Haas robertmh...@gmail.com wrote: I think we should always or never show the view definition, not sometimes. +1 And I also agree with Tom's point that we should fix the pager. The way that it works now is really annoying. +1 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Adding \ev view editor?
All, I've grown to adore the new \ef function editor. It doesn't seem like it would be that difficult to add a view editor as \ev. While editors would also be good for other objects, I don't think we can do \et or \er etc. because those objects don't support CREATE OR REPLACE. Opinions? Other objects which could take \e? -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] \d+ for long view definitions?
On Tue, Sep 01, 2009 at 02:29:12PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I think we should always or never show the view definition, not sometimes. Yeah. I can live with \d not showing it and \d+ showing it --- as Peter already mentioned, that would be consistent with \df behavior. Making it depend on the length is just weird. As I see it, there are two independent issues here: * Smart display based on number of columns in the query and the psql window * Whether to display the view definition. I'm thinking on the second, \d should not display the definition, and \d+ should. On the first...could we go to \x-type display if the columns will overflow the terminal? Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding \ev view editor?
On Tue, Sep 01, 2009 at 11:41:31AM -0700, Josh Berkus wrote: All, I've grown to adore the new \ef function editor. It doesn't seem like it would be that difficult to add a view editor as \ev. While editors would also be good for other objects, I don't think we can do \et or \er etc. because those objects don't support CREATE OR REPLACE. Opinions? Other objects which could take \e? For objects that don't take CREATE OR REPLACE, \e could include the object definition at the top in a multi-line comment, then start with ALTER [OBJECT] Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] \d+ for long view definitions?
On Tue, Sep 1, 2009 at 3:01 PM, David Fetterda...@fetter.org wrote: On Tue, Sep 01, 2009 at 02:29:12PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: I think we should always or never show the view definition, not sometimes. Yeah. I can live with \d not showing it and \d+ showing it --- as Peter already mentioned, that would be consistent with \df behavior. Making it depend on the length is just weird. As I see it, there are two independent issues here: * Smart display based on number of columns in the query and the psql window * Whether to display the view definition. I'm thinking on the second, \d should not display the definition, and \d+ should. On the first...could we go to \x-type display if the columns will overflow the terminal? I don't understand exactly what you're proposing, but I don't think flipping into \x mode based on the window size is ever a good idea. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release notes idea
Christopher Browne wrote: g...@turnstep.com (Greg Sabino Mullane) writes: (Enjoy while you can, Bruce, we'll get another Bruce to contribute someday! Zoltan might be harder... :) My middle name is Bruce; there wouldn't be any legal humps to overcome for me to use the moniker Bruce Browne :-). I'm an Aussie. We're all called Bruce! http://uncyclopedia.wikia.com/wiki/Aussies cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 8.5 release notes idea
On Mon, Aug 31, 2009 at 3:32 PM, Greg Sabino Mullaneg...@turnstep.com wrote: Just noticed in the release notes for 8.4, there are two items accredited to just Greg (but there are three of us Gregs who contributed to 8.4 and are in the notes). While this is very likely Greg Stark, due to the context, I think at this point in the project we simply should spell out everyone's complete name every time, rather than the hit or miss style we are developing. I think that's probably a good idea... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding \ev view editor?
Josh Berkus wrote: All, I've grown to adore the new \ef function editor. It doesn't seem like it would be that difficult to add a view editor as \ev. While editors would also be good for other objects, I don't think we can do \et or \er etc. because those objects don't support CREATE OR REPLACE. Opinions? Other objects which could take \e? That would make it even more desirable to make my proposed change to pg_get_viewdef to put line breaks between each target item. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding \ev view editor?
On tis, 2009-09-01 at 11:41 -0700, Josh Berkus wrote: Opinions? Other objects which could take \e? All of them. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
On Tue, Sep 1, 2009 at 2:58 PM, Tom Lanet...@sss.pgh.pa.us wrote: We get beat up on a regular basis about spikes in response time; why would you want to have vacuum creating one when it doesn't need to? Isn't this sync commit just going to do the same thing that the wal writer is going to do in at most 200ms anyways? As for the FULL case, the sync commit is to try to protect a horribly unsafe kluge that should go away entirely (if vacuum full itself doesn't go away entirely). I'm all for throwing away VACUUM FULL btw. I was thinking of proposing that we replace it with something like CLUSTER which just rewrites the tuples in the order it finds them. The use cases where VACUUM FULL wins currently are where storing two copies of the table and its indexes concurrently just isn't practical. Also perhaps tables where there are too many large indexes to make rebuilding them all in one maintenance window practical. I don't see any way to address these problems without something as complex as xvac and moved_in/moved_off and without the index bloat problems. I think we could improve the i/o access patterns we have currently which make vacuum full so slow, but the fundamental problems would remain. So the question is whether those use cases are worth keeping our existing vacuum full for or whether we could do without it and just recommend partitioning for people with tables large enough to make table rewrites impractical. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
Greg Stark wrote: The use cases where VACUUM FULL wins currently are where storing two copies of the table and its indexes concurrently just isn't practical. Yeah, but then do you really need to use VACUUM FULL? If that's really a problem then there ain't that many dead tuples around. Also perhaps tables where there are too many large indexes to make rebuilding them all in one maintenance window practical. If that's the concern maybe we oughta do something about concurrently re-creating those indexes somehow. Plain REINDEX doesn't work of course, but maybe we can do some trick with creating a new index and dropping the original one afterwards. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
On Wed, Sep 2, 2009 at 12:01 AM, Alvaro Herreraalvhe...@commandprompt.com wrote: The use cases where VACUUM FULL wins currently are where storing two copies of the table and its indexes concurrently just isn't practical. Yeah, but then do you really need to use VACUUM FULL? If that's really a problem then there ain't that many dead tuples around. That's what I want to believe. But picture if you have, say a 1-terabyte table which is 50% dead tuples and you don't have a spare 1-terabytes to rewrite the whole table. Also perhaps tables where there are too many large indexes to make rebuilding them all in one maintenance window practical. If that's the concern maybe we oughta do something about concurrently re-creating those indexes somehow. Plain REINDEX doesn't work of course, but maybe we can do some trick with creating a new index and dropping the original one afterwards. Well that doesn't really work if you want to rewrite the table. CLUSTER has to rebuild all the indexes when it's done. I think the solution for both of these is actually partitioning. The bottom line is that having a single table which contains very large amounts of data is awkward to maintain. -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
Greg Stark gsst...@mit.edu writes: On Wed, Sep 2, 2009 at 12:01 AM, Alvaro Herreraalvhe...@commandprompt.com wrote: The use cases where VACUUM FULL wins currently are where storing two copies of the table and its indexes concurrently just isn't practical. Yeah, but then do you really need to use VACUUM FULL? If that's really a problem then there ain't that many dead tuples around. That's what I want to believe. But picture if you have, say a 1-terabyte table which is 50% dead tuples and you don't have a spare 1-terabytes to rewrite the whole table. But trying to VACUUM FULL that table is going to be horridly painful too, and you'll still have bloated indexes afterwards. You might as well just live with the 50% waste, especially since if you did a full-table update once you'll probably do it again sometime. I'm having a hard time believing that VACUUM FULL really has any interesting use-case anymore. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Linux LSB init script
On tis, 2009-09-01 at 12:04 -0500, Kevin Grittner wrote: We're using SLES10. If we installed the latest from the SuSE site, we would be at PostgreSQL version 8.1. Now, to their credit, that is 8.1.17; but we're looking at when to upgrade from 8.3 to 8.4, not at whether to go back to 8.1. Well, that is not surprising. At the time SLES 10 was released, PostgreSQL 8.1 was the stable PostgreSQL release. If you want major upgrades, you shouldn't be looking for the SLES 10 upgrade channel to provide you that. If you want to have out-of-line, unsupported updates, you can get them from the OpenSuse build service. I think you will find exactly the same/analogous situation for any enterprise Linux distribution or other operating system. And unless I'm remembering incorrectly, the configure options are not what we would want. I don't see any reason the packaged build shouldn't be with --enable-debug on a platform where that has no performance hit. Debatable, but it's not upstream default, so why should it be downstream default? I never understood why anyone converting to PostgreSQL would want the floating point approximate timestamps rather than using --enable-integer-datetimes. Backward compatibility. Same issue with RHEL; discussion in archives. We don't have a need for multiple languages, so I figure it's best to use --diable-nls, although I doubt that one's a huge deal. Well, that's not a reason to rebuild from source, although it could be a reason to rebuild the RPM. I wouldn't expect a packaged SuSE build to cater to all of that; but it would be nice if they donated their init script to the PostgreSQL project, so that those of us who have a reason to build from source on SuSE have have a convenient starting point in the source distribution of PostgreSQL. Well, no one needs to donate anything. If you are interested in maintaining it, just take it and put it under contrib/start-scripts. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
Greg Stark wrote: That's what I want to believe. But picture if you have, say a 1-terabyte table which is 50% dead tuples and you don't have a spare 1-terabytes to rewrite the whole table. Could one hypothetically do update bigtable set pk = pk where ctid in (select ctid from bigtable order by ctid desc limit 100); vacuum; and repeat until max(ctid) is small enough? Sure, it'll take longer than vacuum full; but at first glance it seems lightweight enough to do even on a live, heavily accessed table. IIRC I tried something like this once, and it worked to some extent, but after a few loops didn't shrink the table as much as I had expected. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Linux LSB init script
Peter Eisentraut pete...@gmx.net writes: On tis, 2009-09-01 at 12:04 -0500, Kevin Grittner wrote: And unless I'm remembering incorrectly, the configure options are not what we would want. I don't see any reason the packaged build shouldn't be with --enable-debug on a platform where that has no performance hit. Debatable, but it's not upstream default, so why should it be downstream default? FWIW, that particular issue is invariably a matter of distro policy; they could care less what upstream's default is. For instance Red Hat *always* builds all RPMs with debug enabled, and then splits the debug data off into separate debuginfo RPMs, which are not installed by default for space/bandwidth reasons. But you can get debug symbols when you need 'em. I don't know what Debian or SUSE do, but I'm sure they do it consistently across all their packages. A lot of other packaging choices are likewise driven by distro-wide policy and not what a particular upstream package might choose as default. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A bug in scan.l
The previous change should follow with this.
uescape[uU][eE][sS][cC][aA][pP][eE]{space}*{quote}[^']{quote}
Thanks,
Gokul.
On Wed, Sep 2, 2009 at 7:35 AM, Gokulakannan Somasundaram
gokul...@gmail.com wrote:
There is a rule like this in scan.l
uescapefail
(-|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*-|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*{quote}[^']|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*{quote}|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*|[uU][eE][sS][cC][aA][pP]|[uU][eE][sS][cC][aA]|[uU][eE][sS][cC]|[uU][eE][sS]|[uU][eE]|[uU])
I think this should be corrected to
uescapefail
(-|[uU][eE][sS][cC][aA][pP][eE]{space}*-|[uU][eE][sS][cC][aA][pP][eE]{space}*{quote}[^']|[uU][eE][sS][cC][aA][pP][eE]{space}*{quote}|[uU][eE][sS][cC][aA][pP][eE]{space}*|[uU][eE][sS][cC][aA][pP]|[uU][eE][sS][cC][aA]|[uU][eE][sS][cC]|[uU][eE][sS]|[uU][eE]|[uU])
I have replaced whitespace with space. This has to be done because
whitespace allows comments. This would cause conflict between some of the
alternatives. I found this, while trying to make this rule work with LL(1).
Just thought, it might be useful.
Thanks,
Gokul.
Re: [HACKERS] Linux LSB init script
On Sep 1, 2009, at 6:43 PM, Tom Lane wrote: Debatable, but it's not upstream default, so why should it be downstream default? FWIW, that particular issue is invariably a matter of distro policy; they could care less what upstream's default is. Well, they could, but do they? /me offers Tom a not. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] make installcheck is broken in HEAD on mingw
make installcheck seems to be broken in HEAD on mingw for a few days, though it ran sucessfully on Linux. $ cd contrib/citext/ $ make installcheck make -C ../../src/test/regress pg_regress.exe make[1]: Entering directory `/d/projects/head/src/test/regress' make[1]: `pg_regress.exe' is up to date. make[1]: Leaving directory `/d/projects/head/src/test/regress' ../../src/test/regress/pg_regress --inputdir=. --psqldir=/usr/local/pgsql/bin --dbname=contrib_regression citext (using postmaster on localhost, default port) == dropping database contrib_regression == psql: FATAL: attempted change of parameter port ignored DETAIL: This parameter cannot be changed after server start. command failed: C:/mingw/local/pgsql/bin/psql -X -c DROP DATABASE IF EXISTS \contrib_regression\ postgres make: *** [installcheck] Error 2 Regards, --- ITAGAKI Takahiro NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Linux LSB init script
Kevin Grittner wrote: We're using SLES10. If we installed the latest from the SuSE site, we would be at PostgreSQL version 8.1. Now, to their credit, that is 8.1.17; but we're looking at when to upgrade from 8.3 to 8.4, not at whether to go back to 8.1. I recently build 8.4 RPMs for SLES10/SP2 and they are running very nicely for a very happy customer. IIRC I got the SRPM from the OpenSuse site. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] A bug in scan.l
There is a rule like this in scan.l
uescapefail
(-|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*-|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*{quote}[^']|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*{quote}|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*|[uU][eE][sS][cC][aA][pP]|[uU][eE][sS][cC][aA]|[uU][eE][sS][cC]|[uU][eE][sS]|[uU][eE]|[uU])
I think this should be corrected to
uescapefail
(-|[uU][eE][sS][cC][aA][pP][eE]{space}*-|[uU][eE][sS][cC][aA][pP][eE]{space}*{quote}[^']|[uU][eE][sS][cC][aA][pP][eE]{space}*{quote}|[uU][eE][sS][cC][aA][pP][eE]{space}*|[uU][eE][sS][cC][aA][pP]|[uU][eE][sS][cC][aA]|[uU][eE][sS][cC]|[uU][eE][sS]|[uU][eE]|[uU])
I have replaced whitespace with space. This has to be done because
whitespace allows comments. This would cause conflict between some of the
alternatives. I found this, while trying to make this rule work with LL(1).
Just thought, it might be useful.
Thanks,
Gokul.
Re: [HACKERS] remove flatfiles.c
Ron Mayer wrote: Greg Stark wrote: That's what I want to believe. But picture if you have, say a 1-terabyte table which is 50% dead tuples and you don't have a spare 1-terabytes to rewrite the whole table. Could one hypothetically do update bigtable set pk = pk where ctid in (select ctid from bigtable order by ctid desc limit 100); vacuum; and repeat until max(ctid) is small enough? I remember Hannu Krosing said they used something like that to shrink really bloated tables. Maybe we should try to explicitely support a mechanism that worked in that fashion. I think I tried it at some point and found that the problem with it was that ctid was too limited in what it was able to do. The neat thing is that now that we have the visibility fork, each vacuum needn't scan the whole table each time. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] make installcheck is broken in HEAD on mingw
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes: make installcheck seems to be broken in HEAD on mingw for a few days, though it ran sucessfully on Linux. The mingw buildfarm machines seem to be happy. Are you sure you have a clean build? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
On Tue, Sep 1, 2009 at 7:42 PM, Tom Lanet...@sss.pgh.pa.us wrote: Greg Stark gsst...@mit.edu writes: On Wed, Sep 2, 2009 at 12:01 AM, Alvaro Herreraalvhe...@commandprompt.com wrote: The use cases where VACUUM FULL wins currently are where storing two copies of the table and its indexes concurrently just isn't practical. Yeah, but then do you really need to use VACUUM FULL? If that's really a problem then there ain't that many dead tuples around. That's what I want to believe. But picture if you have, say a 1-terabyte table which is 50% dead tuples and you don't have a spare 1-terabytes to rewrite the whole table. But trying to VACUUM FULL that table is going to be horridly painful too, and you'll still have bloated indexes afterwards. You might as well just live with the 50% waste, especially since if you did a full-table update once you'll probably do it again sometime. I'm having a hard time believing that VACUUM FULL really has any interesting use-case anymore. What if your large table doesn't have an index? Then there's no way to cluster. I'm a bit skeptical about partitioning as a solution, too. The planner is just not clever enough with partitioned tables, yet. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
On Tue, Sep 1, 2009 at 9:29 PM, Alvaro Herreraalvhe...@commandprompt.com wrote: Ron Mayer wrote: Greg Stark wrote: That's what I want to believe. But picture if you have, say a 1-terabyte table which is 50% dead tuples and you don't have a spare 1-terabytes to rewrite the whole table. Could one hypothetically do update bigtable set pk = pk where ctid in (select ctid from bigtable order by ctid desc limit 100); vacuum; and repeat until max(ctid) is small enough? I remember Hannu Krosing said they used something like that to shrink really bloated tables. Maybe we should try to explicitely support a mechanism that worked in that fashion. I think I tried it at some point and found that the problem with it was that ctid was too limited in what it was able to do. I think a way to incrementally shrink large tables would be enormously beneficial. Maybe vacuum could try to do a bit of that each time it runs. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
Robert Haas escribió: On Tue, Sep 1, 2009 at 7:42 PM, Tom Lanet...@sss.pgh.pa.us wrote: But trying to VACUUM FULL that table is going to be horridly painful too, and you'll still have bloated indexes afterwards. You might as well just live with the 50% waste, especially since if you did a full-table update once you'll probably do it again sometime. I'm having a hard time believing that VACUUM FULL really has any interesting use-case anymore. What if your large table doesn't have an index? Then there's no way to cluster. But there's nothing saying we cannot provide a version of CLUSTER that does not follow any index and just copies the live tuples. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
On Tue, Sep 1, 2009 at 10:58 PM, Alvaro Herreraalvhe...@commandprompt.com wrote: Robert Haas escribió: On Tue, Sep 1, 2009 at 7:42 PM, Tom Lanet...@sss.pgh.pa.us wrote: But trying to VACUUM FULL that table is going to be horridly painful too, and you'll still have bloated indexes afterwards. You might as well just live with the 50% waste, especially since if you did a full-table update once you'll probably do it again sometime. I'm having a hard time believing that VACUUM FULL really has any interesting use-case anymore. What if your large table doesn't have an index? Then there's no way to cluster. But there's nothing saying we cannot provide a version of CLUSTER that does not follow any index and just copies the live tuples. Agreed. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A bug in scan.l
Gokulakannan Somasundaram gokul...@gmail.com writes: I have replaced whitespace with space. This has to be done because whitespace allows comments. This would cause conflict between some of the alternatives. I found this, while trying to make this rule work with LL(1). Um, if it's ambiguous, why doesn't flex complain? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
On Tue, Sep 1, 2009 at 19:34, Greg Stark gsst...@mit.edu wrote: On Wed, Sep 2, 2009 at 12:01 AM, Alvaro Herreraalvhe...@commandprompt.com wrote: The use cases where VACUUM FULL wins currently are where storing two copies of the table and its indexes concurrently just isn't practical. Yeah, but then do you really need to use VACUUM FULL? If that's really a problem then there ain't that many dead tuples around. That's what I want to believe. But picture if you have, say a 1-terabyte table which is 50% dead tuples and you don't have a spare 1-terabytes to rewrite the whole table. It would be interesting if there was something between VACUUM FULL and CLUSTER which could, say, work on a single 1GB segment at a time in a manner similar to cluster. You would still end up with index bloat like vacuum full, though perhaps not as bad, but shuffling around the tuples should be faster. The idea here is that the files can be truncated individually. Two 500MB files is pretty much the same as a single 1GB file on disk. Of course, I'm hand waving and don't have the technical expertise to figure out if it can be done easily within PostgreSQL.
[HACKERS] community decision-making 8.5
I posted a message a little over a week ago discussing the timetable for 8.5: http://archives.postgresql.org/pgsql-hackers/2009-08/msg01576.php That thread went off on a number of interesting tangents which I found pretty informative. Probably the most interesting one to me personally was about a need for more efficient decision-making. http://archives.postgresql.org/pgsql-hackers/2009-08/msg02149.php It's interesting to note that the original purpose of this thread was to get a decision about the timetable for 8.5 development, and that of the original responses to that message only one person (Peter) expression a clear opinion about the topic in question. Everyone else, so far as I can see, said some variant of on the one hand... but then on the other hand Eventually, Josh Berkus retracted his original endorsement for the 3-CF plan and suggested that we go with four. http://archives.postgresql.org/pgsql-hackers/2009-08/msg01651.php http://archives.postgresql.org/pgsql-hackers/2009-08/msg01983.php Josh's schedule was subsequently endorsed by Simon Riggs. So by my count we now have four votes for a 4-CF schedule and one for a 3-CF schedule (me), maybe two if you count Tom, who I think was leaning in that direction - so I guess that settles the matter? I think this is a good illustration of the problems with decision-making in a community environment - given choices 3 and 4 most of the votes were somewhere between 3.25 and 3.75. I think, in general, that when people weigh in with clear opinions, we're pretty good about moving in the direction that most people want to go. Even two votes can be enough for a consensus, if they both go in the same direction. However, when the responses aren't clearly in favor of one option or the other, or when no-one writes back at all, I think we tend to flounder. It's worth thinking about how we could improve this. I think the ideas that were floated on the previous thread of having a beta-mom and/or an open-items-fest are good ones, and we might want to have both: someone to work with beta testers, and someone to coordinate volunteers to propose solutions to the open items. Those proposals are specific to getting a release out the door, though, and that may not be the only context in which this problem comes up. Still, it's a start - any other ideas? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] remove flatfiles.c
On Tue, Sep 1, 2009 at 9:55 PM, Robert Haasrobertmh...@gmail.com wrote: I'm a bit skeptical about partitioning as a solution, too. The planner is just not clever enough with partitioned tables, yet. analyze and vacuum a *very* big table and even scan a huge index is not a joke neither... and yes the planner is not very clever about partitioning and certainly that is something we need to fix not something we have to live with... no that that will be easy but hey! we have very brilliant people here (you being one of them) -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil - Ecuador Cel. +59387171157 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A bug in scan.l
Well, i am at a very beginner level with Flex. I could see how flex works with it even if it is a ambiguity. Since it matches the rule with the maximum text and we don't allow a new line character in the rule, it works fine. Even in LL(1), it works fine, but throws warnings. So i just thought of suggesting to remove the ambiguity. But do we need to allow comments as part of unicode escapes? Thanks, Gokul. On Wed, Sep 2, 2009 at 8:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: Gokulakannan Somasundaram gokul...@gmail.com writes: I have replaced whitespace with space. This has to be done because whitespace allows comments. This would cause conflict between some of the alternatives. I found this, while trying to make this rule work with LL(1). Um, if it's ambiguous, why doesn't flex complain? regards, tom lane
Re: [HACKERS] make installcheck is broken in HEAD on mingw
Tom Lane t...@sss.pgh.pa.us wrote:
Itagaki Takahiro itagaki.takah...@oss.ntt.co.jp writes:
make installcheck seems to be broken in HEAD on mingw for a few days,
though it ran sucessfully on Linux.
The mingw buildfarm machines seem to be happy. Are you sure you
have a clean build?
Yes, but distclean and deleting auto-generated files won't work.
Here is a result of printf-debug for process_postgres_switches() in postgres.c.
WARNING: postgres: pid=3360
WARNING: startup packet[0] = postgres
WARNING: startup packet[1] = postgres
WARNING: postgres: pid=3360
WARNING: startup packet[0] = postgres
WARNING: startup packet[1] = -c
WARNING: startup packet[2] = intervalstyle=postgres_verbose
WARNING: getopt(p) = ostgres
FATAL: attempted change of parameter port ignored
The first argument 'postgres' was interpreted as -p ostgres by getopt().
We might need to re-initilaize variables for getopt() because we call
process_postgres_switches twice(). So getopt() is also called twice.
But optind=1 has no effect in my environment and optind=0 crashes
backend with stack-overflow...
(Are there any known bugs in mingw's getopt?)
BTW, the following code seems to be a bit storange.
The part of { argv++; argc--; } removes the first arugment,
but '--single' argument (argv[1]) is not removed, no?
if (secure)
{
gucsource = PGC_S_ARGV; /* switches came from
command line */
/* Ignore the initial --single argument, if present */
if (argc 1 strcmp(argv[1], --single) == 0)
{
argv++;
argc--;
}
}
Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
