Re: [HACKERS] storing TZ along timestamps
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, May 27, 2011 at 04:43:28PM -0400, Alvaro Herrera wrote: Hi, One of our customers is interested in being able to store original timezone along with a certain timestamp. I've felt that pain here and there too... So the first thing is cataloguing timezone names, and assigning an ID to each (maybe an OID). If we do that, then we can store the OID of the timezone name along the int64/float8 of the actual timestamp value. Right now we rely on the tzdata files on disk for things like pg_timezone_names and other accesses of TZ data; so the files are the authoritative source of TZ info. So we need to ensure that whenever the files are updated, the catalogs are updated as well. Problem with this approach (mapping external time zone names to OIDs) is: dump/restore would only be meaningful if you carry over the time zone data, right? That is: two independent systems are likely to have different mappings (even if at some point they have the same TZ data?) What would be a solution to that? (a) A central, official catalog, with only additions, never deletions (perhaps with some space carved out for local additions, to minimize conflicts)? (b) A hash of the time zone name? Both not very good ideas, I know. Although (a) might be less bad than it seems. Most Unixoids (including OSX) seem to have basically Olson's. Don't know about Windows, but it might seem feasible to make some mapping (or union). Only important rule: no backtrack :-) Regards - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFN4JWtBcgs9XrR2kYRAt+mAJ0atx3u6pll50+s4vVwCKZUjqmnSQCffWNe gzSFgRCFUvsd8pbH1Qm/ho4= =FVhO -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Allow ALTER TABLE name {OF type | NOT OF}.
2011/5/28 Alvaro Herrera alvhe...@commandprompt.com: Excerpts from Cédric Villemain's message of vie may 27 18:37:05 -0400 2011: 2011/4/21 Robert Haas rh...@postgresql.org: Modified Files -- doc/src/sgml/ref/alter_table.sgml | 26 +++ src/backend/commands/tablecmds.c | 277 +++-- I noticed 2 warnings on unused variables from gcc in tablecmds.c The attached patch fix that by removing those 2 variables. My compiler wasn't complaining, but since the variable is clearly unused I went ahead and pushed this. Thanks I have a recent gcc wich probably help here: gcc-4.6.real (Debian 4.6.0-7) 4.6.1 20110507 (prerelease) -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Cédric Villemain 2ndQuadrant http://2ndQuadrant.fr/ PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] patch integration
Sir, i am developing a patch for postgresql in c language. i want to know that how can i integrate my patch with postgresql. regards emman
[HACKERS] install softwares
sir, i am developnig a patch and as per instructionsdescribed on this site http://wiki.postgresql.org/wiki/Developer_FAQ it is specifed on the link that along with unix platform we have to use GCC, GNU Make, GDB, Autoconf but i dont know how to install these softwares in linux ,because the commandas given on the web site are not working. please guide me for this.regards emman
Re: [HACKERS] How can I check the treatment of bug fixes?
On 05/28/2011 05:47 AM, MauMau wrote: From: Peter Eisentraut pete...@gmx.net On fre, 2011-05-27 at 13:55 -0400, Robert Haas wrote: Also, I think it's about time we got ourselves some kind of bug tracker. I have no idea how to make that work without breaking workflow that works now, but a quick survey of my pgsql-bugs email suggests that this is far from the only thing slipping through the cracks. The problem is finding a usable bug tracking software. I think JIRA is very good. Almost all projects in Apache Software Foundation (ASF) including Tomcat, Hadoop, Apache HTTP server, use JIRA. With JIRA, we can know various counts such as the number of bugs per major/minor release, not-fixed bugs, new features in each major release, well that is rather basic functionality of a tracker software and i would expect those to be a given, but I don't think that is where the problems are with implementing a tracker for postgresql.org... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How can I check the treatment of bug fixes?
On 05/27/2011 07:55 PM, Robert Haas wrote: On Fri, May 27, 2011 at 12:21 PM, Tom Lanet...@sss.pgh.pa.us wrote: Joshua D. Drakej...@commandprompt.com writes: You have done what you need to do to check the status. Someone who knows something about the bug should speak up at some point. That patch is waiting for a committer who knows something about Windows to pick it up. It might be useful, in this situation, for the OP to add this patch to the CommitFest application. https://commitfest.postgresql.org/action/commitfest_view/open Also, I think it's about time we got ourselves some kind of bug tracker. I have no idea how to make that work without breaking workflow that works now, but a quick survey of my pgsql-bugs email suggests that this is far from the only thing slipping through the cracks. well as for just keeping track of -bugs I guess a very simple schema would go pretty far: * have some tool monitor the list and if it sees a new bug# make it a ticket/bugreport * if that bug number is mentioned in a commit close it * provide a dashboard of: a) bugs that never got a response b) bugs that got a response but never have been mentioned in a commit c) bugs that got mentioned in a commit but no stable release was done yet * provide a trivial interface (either mail or simple web interface - maybe in CF style) to make issues as not a bug or not postgresql-core product (which seems to be the top two non-big related inquiries we get on -bugs) this is more or less exactly what I hacked up back in early 2008 based on bugzilla (without actually exposing the BZ User-Interface at all - just using it as a tracker core and talking to it using the API it provides). Independent of whether we want to do a full tracker or not anywhere in the future we could at least start by prototyping with better automatic monitoring of -bugs. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How can I check the treatment of bug fixes?
On Sat, May 28, 2011 at 10:02 AM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: well as for just keeping track of -bugs I guess a very simple schema would go pretty far: * have some tool monitor the list and if it sees a new bug# make it a ticket/bugreport The bug numbers come from a database backed web form anyway - seems it would be a lot easier to just have that script write a record to a table. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How can I check the treatment of bug fixes?
On 05/28/2011 12:19 PM, Dave Page wrote: On Sat, May 28, 2011 at 10:02 AM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: well as for just keeping track of -bugs I guess a very simple schema would go pretty far: * have some tool monitor the list and if it sees a new bug# make it a ticket/bugreport The bug numbers come from a database backed web form anyway - seems it would be a lot easier to just have that script write a record to a table. maybe - but for a poc it was much easier to have something that had no dependency on any modification of the webinfrastructure(all it needed was an email subscription to the list), you also get some stuff like rss feeds, XML/CSV aggregation output, a commit log parser (and a GUI for playing even if you don't use it for anything officially) for free if you use some existing framework ;) For a real implemenation based on an existing tool you would probably modify the bug reporting form to post the bug report to the tracker and have that one send the report on behalf and with the sender address of the original reporter, that way the -pgsql-bugs list could exactly stay as it is now and if you wished to be able to use it as a not-only bugreport-form triggered tracker you could do that as well. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_terminate_backend and pg_cancel_backend by not administrator user
On Fri, Mar 11, 2011 at 8:54 AM, Bruce Momjian br...@momjian.us wrote: I have added it to the next commit fest. Hi Torello, I have volunteered (more accurately, Greg Smith volunteered me :-) to be a reviewer for this patch. I know you're a bit new here, so I thought I'd outline where this patch stands and what's expected if you'd like to move it along. We organize patch reviews via commitfests lasting a month or so. Some more information about this process: http://wiki.postgresql.org/wiki/CommitFest Each commitfest is a period wherein you can expect to receive some feedback on your patch and advice on things which might need to be improved (in this case, it's my job to provide you this feedback). Your patch is in the upcoming commitfest, scheduled to run from June 15 to July 14. So if you're interested in being responsible for this patch, or some variant of it, eventually making its way into PostgreSQL 9.2, you should be willing to update your patch based on feedback, request advice, etc. during this period. If you're not interested in getting sucked into this process that's OK -- just please advise us if that's the case, and maybe someone else will be willing to take charge of the patch. Anssi and I posted some initial feedback on the patch's goals earlier. I would like to ultimately see users have the capability to pg_cancel_backend() their own queries. But I could at least conceive of others not wanting this behavior enabled by default. So perhaps this patch's approach of granting extra privs to the database owner could work as a first attempt. And maybe a later version could introduce a GUC allowing the DBA to control whether users can cancel/terminate their backends, or we could instead have an option flag to CREATE/ALTER ROLE, allowing per-user configuration. It would be helpful to hear from others whether this patch's goals would work as a first pass at this problem, so that Torello doesn't waste time on a doomed approach. Also, it might be helpful to add an entry on the Todo list for 'allow non-superusers to use pg_cancel_backend()', in case this patch gets sunk. Now, a few technical comments about the patch: 1.) This bit looks dangerous: +backend = pgstat_fetch_stat_beentry(i); +if (backend-st_procpid == pid) { Since pgstat_fetch_stat_beentry() might return NULL. I'm a bit suspicious about whether looping through pgstat_fetch_stat_beentry() is the best way to determine the database owner for a given backend PID, but I haven't dug in enough yet to suggest a better alternative. 2.) The way the code inside pg_signal_backend() is structured, doing: select pg_cancel_backend(12345); as non-superuser, where '12345' is a fictitious PID, can now give you the incorrect error message: ERROR: must be superuser or target database owner to signal other server processes 3.) No documentation adjustments, and the comments need some cleaup. Torello: I'll be happy to handle comments/documentation for you as a native English speaker, so you don't have to worry about this part. That's it for now. Torello, I look forward to hearing back from you, and hope that you have some time to work on this patch further. Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [ADMIN] pg_class reltuples/relpages not updated by autovacuum/vacuum
Robert Haas robertmh...@gmail.com writes: Still, maybe we don't have a better option. If it were me, I'd add an additional safety valve: use your formula if the percentage of the relation scanned is above some threshold where there's unlikely to be too much skew. But if the percentage scanned is too small, then don't use that formula. Instead, only update relpages/reltuples if the relation is now larger; set relpages to the new actual value, and scale up reltuples proportionately. However, I just work here. It's possible that I'm worrying about a problem that won't materialize in practice. Attached is a proposed patch to fix these issues. Experimentation convinced me that including a fudge factor for VACUUM's results made things *less* accurate, not more so. The reason seems to be bound up in Greg Stark's observation that the unmodified calculation is equivalent to assuming that the old average tuple density still applies to the unscanned pages. In a VACUUM, we know that the unscanned pages are exactly those that have had no changes since (at least) the last vacuum, which means that indeed the old density ought to be a good estimate. Now, this reasoning can break down if the table's tuple density is nonuniform, but what I found in my testing is that if you vacuum after a significant change in a table (such as deleting a lot of rows), and you don't apply the full unfudged correction, you get a badly wrong result. I think that's a more significant issue than the possibility of drift over time. I also found that Greg was right in thinking that it would help if we tweaked lazy_scan_heap to not always scan the first SKIP_PAGES_THRESHOLD-1 pages even if they were all_visible_according_to_vm. That seemed to skew the results if those pages weren't representative. And, for the case of a useless manual vacuum on a completely clean table, it would cause the reltuples value to drift when there was no reason to change it at all. Lastly, this patch removes a bunch of grotty interconnections between VACUUM and ANALYZE that were meant to prevent ANALYZE from updating the stats if VACUUM had done a full-table scan in the same command. With the new logic it's relatively harmless if ANALYZE does that, and anyway autovacuum frequently fires the two cases independently anyway, making all that logic quite useless in the normal case. (This simplification accounts for the bulk of the diff, actually.) Comments? regards, tom lane diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 0568a1bcf86281a9b1086d343e7027557295065c..fa84989fc6fa8be90d4eecb9c33e94a232d79880 100644 *** a/src/backend/commands/analyze.c --- b/src/backend/commands/analyze.c *** static MemoryContext anl_context = NULL; *** 84,91 static BufferAccessStrategy vac_strategy; ! static void do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, ! bool update_reltuples, bool inh); static void BlockSampler_Init(BlockSampler bs, BlockNumber nblocks, int samplesize); static bool BlockSampler_HasMore(BlockSampler bs); --- 84,90 static BufferAccessStrategy vac_strategy; ! static void do_analyze_rel(Relation onerel, VacuumStmt *vacstmt, bool inh); static void BlockSampler_Init(BlockSampler bs, BlockNumber nblocks, int samplesize); static bool BlockSampler_HasMore(BlockSampler bs); *** static bool std_typanalyze(VacAttrStats *** 115,132 /* * analyze_rel() -- analyze one relation - * - * If update_reltuples is true, we update reltuples and relpages columns - * in pg_class. Caller should pass false if we're part of VACUUM ANALYZE, - * and the VACUUM didn't skip any pages. We only have an approximate count, - * so we don't want to overwrite the accurate values already inserted by the - * VACUUM in that case. VACUUM always scans all indexes, however, so the - * pg_class entries for indexes are never updated if we're part of VACUUM - * ANALYZE. */ void ! analyze_rel(Oid relid, VacuumStmt *vacstmt, ! BufferAccessStrategy bstrategy, bool update_reltuples) { Relation onerel; --- 114,122 /* * analyze_rel() -- analyze one relation */ void ! analyze_rel(Oid relid, VacuumStmt *vacstmt, BufferAccessStrategy bstrategy) { Relation onerel; *** analyze_rel(Oid relid, VacuumStmt *vacst *** 238,250 /* * Do the normal non-recursive ANALYZE. */ ! do_analyze_rel(onerel, vacstmt, update_reltuples, false); /* * If there are child tables, do recursive ANALYZE. */ if (onerel-rd_rel-relhassubclass) ! do_analyze_rel(onerel, vacstmt, false, true); /* * Close source relation now, but keep lock so that no one deletes it --- 228,240 /* * Do the normal non-recursive ANALYZE. */ ! do_analyze_rel(onerel, vacstmt, false); /* * If there are child tables, do recursive ANALYZE. */ if
Re: [HACKERS] eviscerating the parser
On Sun, May 22, 2011 at 3:10 PM, Robert Haas robertmh...@gmail.com wrote: ... However, in this case, there was only one client, so that's not the problem. I don't really see how to get a big win here. If we want to be 4x faster, we'd need to cut time per query by 75%. That might require 75 different optimizations averaging 1% a piece, most likely none of them trivial. I do confess I'm a bit confused as to why prepared statements help so much. That is increasing the throughput by 80%, which is equivalent to decreasing time per query by 45%. That is a surprisingly big number, and I'd like to better understand where all that time is going. On my old 32-bit linux box, the difference is even bigger, 150% increase in throughput (4000 vs 9836 tps) with using prepared statements. By gprof, over half of that extra time is going to planning, specifically standard_planner and its children. Unfortunately once you dig down beyond that level, the time is spread all over the place, so there is no one hot spot to focus on. I've don't trust gprof all that much, so I've also poked at tcop/postgres.c a bit to make it do silly things like parse the statement repeatedly, and throw away all results but the last one (and similar things with analyze/rewriting, and planning) and see how much slower that makes things. Here too the planner is the slow part. But by extrapolating backwards; parsing, analyzing, and planning all together only account for 1/3 of the extra time of not using -M prepared. I don't know where the other 2/3 of the time is lost. It could be, for example, that parsing the command twice does not take twice as long doing it once, due to L1 and instruction caching, in which extrapolation backwards is not very reliable But by both methods, the majority of the extra time that can be accounted for is going to the planner. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] storing TZ along timestamps
On fre, 2011-05-27 at 16:57 -0700, Steve Crawford wrote: And the second case is already well handled. In fact calendaring is a great example. I enter the time for the teleconference and PG nicely uses my default timezone to store the point-in-time. When you retrieve it, it is shown in your timezone and we both pick up the phone at the correct time. And if I know I'll be somewhere else at that time, I just ask for the data in that zone. Altering the data type gains nothing. How about a recurring appointment that happens every Tuesday whenever it is 9:00am in California, independent of DST (in California or where ever the participant actually is). I'm not sure how to solve that within the SQL framework. You might need to use time with time zone with a placeholder timezone, and then a rule that date + time with time zone creates a timestamp with time zone that resolves the time zone for that particular day. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] eviscerating the parser
On Sat, May 28, 2011 at 5:51 PM, Jeff Janes jeff.ja...@gmail.com wrote: But by both methods, the majority of the extra time that can be accounted for is going to the planner. Sounds like an argument for a plan cache. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How can I check the treatment of bug fixes?
On Fri, May 27, 2011 at 5:54 PM, Andres Freund and...@anarazel.de wrote: If I see a bug in a region I know something about and its on a platform I care about (i.e. likely only linux) I try to do this. But its hard, in most situations one of you already did it. Tom and you are just to goddamn fast in many, many cases. Which is totally great, don't get me wrong, but makes it hard to beat you as a mere mortal ;) It's funny to be lumped in with Tom, who leaves me in the dust! But the problem is really with the bugs that never get a response, not the ones that do. There are no shortage of things that neither Tom nor I nor anyone else is working on. Do you like separate patches for the back branches or is that basically useless work? If it doesn't apply cleanly, yes. It's also quite helpful to identify how far the back-patch can reasonably go, and why. Related to doing stuff like that is that I really find it hard to write a patch that happens to be liked by Tom or you so it does not have to be mostly rewritten. For that to change for one I would like to have the Coding Style to be expanded because I think there are loads of rules that exist only in bits and bits on the mailing lists. For another I would like to get a patch back instead of rewritten because without knowing the individual reasons for the changes its sometimes rather hard to know what the reason for a specific change was. I do realize thats quite a bit of work for you which is why I hesitated writing that... Well, frankly, I think you're doing pretty well. I find it's quite helpful to have a patch to start with, even if I don't agree with the approach, because it gives me an idea of what portions of the code need to be changed and often makes it easier to understand what is broken. But in your particular case, your recent patches have gone in with minimal changes. I tend to avoid spelling out all the details on-list because I don't want to be seen as nit-picking. If something is a logic error or one or more places that needed to be changed were altogether ignored, then I usually mention that, because those are, well, important. But if I reindented the code to make pg_indent mangle it less or corrected a typo in a comment or simplified something like: if (something) { do stuff; } else break; more things; to: if (!something) break; do stuff; more things; ...then I don't tend to mention that, first because it's sort of self-evident that the second one is clearer, second because I don't want to demoralize people who have done basically good work by pointing out trivial flaws, and third because it's a bit time-consuming. But that really is third. If you want to know why I did something, feel free to ask. I have been really pleased to see that there is a growing group of people who I can rely on to submit good stuff most of the time, stuff that I can apply without spending a lot of time on it. If I were less busy, I might spend more time hacking on patches that were marginal, as I know Tom still does sometimes. But I just don't have the cycles for it. It's far faster for me to read the patch and list the issues than it is to fix them, unless the issues are trivial cosmetic stuff. If there were fewer patches, I might spend more time hacking on marginal patches, but as it is I mostly do that when I think that the patch won't go in any other way. Actually, I think it's kind of good that the volume is such as to preclude my doing that very often. It's not so good for the patches that get bounced for lack of attention, but I think overall the average quality of patches is improving (perhaps partly for that reason?), and I expect that some of the better and more prolific submitters will eventually get commit bits of their own. I can only hope that some of those people will be interested in helping with the CF work. It is easy to find people who are willing to commit their own patches. Finding people who are willing to commit other people's patches is the tough part. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] switch UNLOGGED to LOGGED
On Fri, May 27, 2011 at 6:19 AM, Noah Misch n...@leadboat.com wrote: So, it's ok to have a log item that is replayed only if WalRcvInProgress() is true? No, that checks for WAL streaming in particular. A log-shipping standby needs the same treatment. Is it a correct approach? I couldn't find any other way to find out if we are in a standby or a master... InArchiveRecovery looks like the right thing, but it's currently static to xlog.c. Perhaps exporting that is the way to go. Why is it necessary to replay the operation only on the slave? Can we just use XLOG_HEAP_NEWPAGE? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] psql: missing tab completions for COMMENT ON
Hi all, psql's auto-complete support for COMMENT ON was missing support for a few object types: 1.) EXTENSION and PROCEDURAL LANGUAGE are now auto-complete candidates for COMMENT ON [TAB]. Lists of extensions and procedural languages should also be filled in when a user types COMMENT ON EXTENSION [TAB] COMMENT ON PROCEDURAL LANGUAGE [TAB] 2.) This part of tab-complete.c looked like a spurious leftover: *** psql_completion(char *text, int start, i *** 1580,1592 COMPLETE_WITH_LIST(list_TRANS2); } else if ((pg_strcasecmp(prev4_wd, COMMENT) == 0 pg_strcasecmp(prev3_wd, ON) == 0) || (pg_strcasecmp(prev6_wd, COMMENT) == 0 ! pg_strcasecmp(prev5_wd, ON) == 0) || !(pg_strcasecmp(prev5_wd, ON) == 0 ! pg_strcasecmp(prev4_wd, TEXT) == 0 ! pg_strcasecmp(prev3_wd, SEARCH) == 0)) COMPLETE_WITH_CONST(IS); Since we want these choices to be filled in for COMMENT ON TEXT SEARCH [TAB]: {CONFIGURATION, DICTIONARY, PARSER, TEMPLATE, NULL}; which were already being handled correctly in an above block. One piece that I gave up on trying to fix is the auto-completion for {OPERATOR, OPERATOR CLASS, OPERATOR FAMILY}, since getting it working correctly would be a real hassle. There's the trouble of whether to auto-complete operators for OPERATOR [TAB], or whether to fill in {CLASS, FAMILY} instead. Plus the auto-completes for 'USING index_method'. While wasting time on OPERATOR [TAB], I realized we're being a bit overeager with this bit: else if ((pg_strcasecmp(prev4_wd, COMMENT) == 0 pg_strcasecmp(prev3_wd, ON) == 0) || (pg_strcasecmp(prev6_wd, COMMENT) == 0 pg_strcasecmp(prev5_wd, ON) == 0)) COMPLETE_WITH_CONST(IS); which will auto-complete e.g. COMMENT ON AGGREGATE avg [TAB] with 'IS', when instead we'd want the possible argument types to avg, or nothing at all. Same deal with a few other object types, but it's probably not worth worrying about (at least, I'm not worrying about it at the moment). Barring objections, I can add this patch to the CF. Josh tab_complete.v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [ADMIN] pg_class reltuples/relpages not updated by autovacuum/vacuum
On Sat, May 28, 2011 at 12:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: I also found that Greg was right in thinking that it would help if we tweaked lazy_scan_heap to not always scan the first SKIP_PAGES_THRESHOLD-1 pages even if they were all_visible_according_to_vm. That seemed to skew the results if those pages weren't representative. And, for the case of a useless manual vacuum on a completely clean table, it would cause the reltuples value to drift when there was no reason to change it at all. You fixed the logic only for the first 32 pages which helps with the skew. But really the logic is backwards in general. Instead of counting how many missed opportunities for skipped pages we've seen in the past we should read the bits for the next 32 pages in advance and decide what to do before we read those pages. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Getting a bug tracker for the Postgres project
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 well that is rather basic functionality of a tracker software and i would expect those to be a given, but I don't think that is where the problems are with implementing a tracker for postgresql.org... Right, the problem has been the lukewarm response from the hackers who would be using it every day, and without whose buy-in using a bug tracker would be possible, but much more difficult. Bug tracking software is definitely religious war territory; most people have a bug tracker they use and tolerate, and pretty much everyone has a bug tracker that they absolutely despise (hi JIRA!). Therefore, I suggest we adopt the first one that someone takes the time to build and implement, along with a plan for keeping it up to date. My own bare bones wish list for such a tracker is: * Runs on Postgres * Has an email interface Make no mistake, whichever we choose, the care of feeding of such a beast will require some precious resources in time from at least two people, probably more. If there is anyone in the community that wants to help the project but hasn't found a way, this is your chance to step up! :) - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201105282322 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAk3hvCgACgkQvJuQZxSWSsi8gwCfQq/2WRhtnN8HJKoup5KxTrI6 S6QAn1rhm5QIr5cLplhz6U67ZSv6njK8 =oU4a -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting a bug tracker for the Postgres project
On Sat, May 28, 2011 at 11:23 PM, Greg Sabino Mullane g...@turnstep.com wrote: My own bare bones wish list for such a tracker is: * Runs on Postgres * Has an email interface Make no mistake, whichever we choose, the care of feeding of such a beast will require some precious resources in time from at least two people, probably more. If there is anyone in the community that wants to help the project but hasn't found a way, this is your chance to step up! :) Yeah, agreed. My basic requirements are: 1. Given a bug number, find the pgsql-bugs emails that mention it in the subject line. Note that the archives would actually MOSTLY do this ,but for the stupid month-boundary problem which we seem unable to fix despite having some of the finest engineers in the world. 2. Associate some kind of status like OPEN, FIXED, NOTABUG, WONTFIX, etc. with each such bug via web interface. I'm not asking for a lot. In fact, less may be more. We don't want to have to do a lot of work to keep something up to date. But for the love of pity, there should be some way to get a list of which bugs we haven't fixed yet. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting a bug tracker for the Postgres project
Robert Haas robertmh...@gmail.com writes: On Sat, May 28, 2011 at 11:23 PM, Greg Sabino Mullane g...@turnstep.com wrote: My own bare bones wish list for such a tracker is: * Runs on Postgres * Has an email interface Make no mistake, whichever we choose, the care of feeding of such a beast will require some precious resources in time from at least two people, probably more. If there is anyone in the community that wants to help the project but hasn't found a way, this is your chance to step up! :) Yeah, agreed. My basic requirements are: 1. Given a bug number, find the pgsql-bugs emails that mention it in the subject line. Note that the archives would actually MOSTLY do this ,but for the stupid month-boundary problem which we seem unable to fix despite having some of the finest engineers in the world. Many, many, many bug issues are not associated with a bug report submitted through the web interface. People mail stuff to pgsql-bugs manually, or issues turn up in threads on other lists. If a tracker can only find things submitted through the web interface, that is not going to lead to everyone filing bugs that way; it's going to lead to the tracker being ignored as useless. 2. Associate some kind of status like OPEN, FIXED, NOTABUG, WONTFIX, etc. with each such bug via web interface. Anything that even pretends to be a bug tracker will do that. The real question is, who is going to keep it up to date? GSM has the right point of view here: we need at least a couple of people who are willing to invest substantial amounts of time, or it's not going to go anywhere. Seeing that we can barely manage to keep the mailing list moderator positions staffed, I'm not hopeful. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting a bug tracker for the Postgres project
On Sun, May 29, 2011 at 12:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: 1. Given a bug number, find the pgsql-bugs emails that mention it in the subject line. Note that the archives would actually MOSTLY do this ,but for the stupid month-boundary problem which we seem unable to fix despite having some of the finest engineers in the world. Many, many, many bug issues are not associated with a bug report submitted through the web interface. People mail stuff to pgsql-bugs manually, or issues turn up in threads on other lists. If a tracker can only find things submitted through the web interface, that is not going to lead to everyone filing bugs that way; it's going to lead to the tracker being ignored as useless. 2. Associate some kind of status like OPEN, FIXED, NOTABUG, WONTFIX, etc. with each such bug via web interface. Anything that even pretends to be a bug tracker will do that. The real question is, who is going to keep it up to date? GSM has the right point of view here: we need at least a couple of people who are willing to invest substantial amounts of time, or it's not going to go anywhere. Seeing that we can barely manage to keep the mailing list moderator positions staffed, I'm not hopeful. The issues that you raise are real ones, but doing nothing isn't better. Right now we have no organized tracking of ANY bugs, and if someone were hypothetically willing to help with that they would have nowhere to start. This is a big enough problem that we should at least TRY to get our arms around it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting a bug tracker for the Postgres project
On 29 May 2011 14:04, Tom Lane t...@sss.pgh.pa.us wrote: Anything that even pretends to be a bug tracker will do that. The real question is, who is going to keep it up to date? GSM has the right point of view here: we need at least a couple of people who are willing to invest substantial amounts of time, or it's not going to go anywhere. Seeing that we can barely manage to keep the mailing list moderator positions staffed, I'm not hopeful. Well the good news is that first-pass triage of bug reports can be done by pretty much anybody who is a moderately experienced postgres user; they don't even need to be a hacker. They just need to know when to send back a RTFM link, when to say you didn't tell us your PG version / post your query / post your explain analyse / post your show all, and when to kick the bug report up to a sage hacker. It's not glamorous work, but it is a very accessible way to contribute, without the need to block out hours at a time. A bug wrangler could very readily log in, sort out reports for 20 minutes and then go do something else with the rest of their day. Cheers, BJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [ADMIN] pg_class reltuples/relpages not updated by autovacuum/vacuum
Greg Stark gsst...@mit.edu writes: On Sat, May 28, 2011 at 12:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: I also found that Greg was right in thinking that it would help if we tweaked lazy_scan_heap to not always scan the first SKIP_PAGES_THRESHOLD-1 pages even if they were all_visible_according_to_vm. You fixed the logic only for the first 32 pages which helps with the skew. But really the logic is backwards in general. Instead of counting how many missed opportunities for skipped pages we've seen in the past we should read the bits for the next 32 pages in advance and decide what to do before we read those pages. OK, do you like the attached version of that logic? (Other fragments of the patch as before.) regards, tom lane diff --git a/src/backend/commands/vacuumlazy.c b/src/backend/commands/vacuumlazy.c index 9393fa0727aaad7508e1163623322b4066412257..231447b31223bc5350ce49a136cffafaa53bc5fb 100644 *** a/src/backend/commands/vacuumlazy.c --- b/src/backend/commands/vacuumlazy.c *** lazy_scan_heap(Relation onerel, LVRelSta *** 311,317 int i; PGRUsageru0; Buffer vmbuffer = InvalidBuffer; ! BlockNumber all_visible_streak; pg_rusage_init(ru0); --- 305,312 int i; PGRUsageru0; Buffer vmbuffer = InvalidBuffer; ! BlockNumber next_not_all_visible_block; ! boolskipping_all_visible_blocks; pg_rusage_init(ru0); *** lazy_scan_heap(Relation onerel, LVRelSta *** 329,340 nblocks = RelationGetNumberOfBlocks(onerel); vacrelstats-rel_pages = nblocks; vacrelstats-nonempty_pages = 0; vacrelstats-latestRemovedXid = InvalidTransactionId; lazy_space_alloc(vacrelstats, nblocks); ! all_visible_streak = 0; for (blkno = 0; blkno nblocks; blkno++) { Buffer buf; --- 324,369 nblocks = RelationGetNumberOfBlocks(onerel); vacrelstats-rel_pages = nblocks; + vacrelstats-scanned_pages = 0; vacrelstats-nonempty_pages = 0; vacrelstats-latestRemovedXid = InvalidTransactionId; lazy_space_alloc(vacrelstats, nblocks); ! /* !* We want to skip pages that don't require vacuuming according to the !* visibility map, but only when we can skip at least SKIP_PAGES_THRESHOLD !* consecutive pages. Since we're reading sequentially, the OS should be !* doing readahead for us, so there's no gain in skipping a page now and !* then; that's likely to disable readahead and so be counterproductive. !* Also, skipping even a single page means that we can't update !* relfrozenxid, so we only want to do it if we can skip a goodly number !* of pages. !* !* Before entering the main loop, establish the invariant that !* next_not_all_visible_block is the next block number = blkno that's !* not all-visible according to the visibility map, or nblocks if there's !* no such block. Also, we set up the skipping_all_visible_blocks flag, !* which is needed because we need hysteresis in the decision: once we've !* started skipping blocks, we may as well skip everything up to the next !* not-all-visible block. !* !* Note: if scan_all is true, we won't actually skip any pages; but we !* maintain next_not_all_visible_block anyway, so as to set up the !* all_visible_according_to_vm flag correctly for each page. !*/ ! for (next_not_all_visible_block = 0; !next_not_all_visible_block nblocks; !next_not_all_visible_block++) ! { ! if (!visibilitymap_test(onerel, next_not_all_visible_block, vmbuffer)) ! break; ! } ! if (next_not_all_visible_block = SKIP_PAGES_THRESHOLD) ! skipping_all_visible_blocks = true; ! else ! skipping_all_visible_blocks = false; ! for (blkno = 0; blkno nblocks; blkno++) { Buffer buf; *** lazy_scan_heap(Relation onerel, LVRelSta *** 347,387 OffsetNumber frozen[MaxOffsetNumber]; int nfrozen; Sizefreespace; ! boolall_visible_according_to_vm = false; boolall_visible; boolhas_dead_tuples; ! /* !* Skip pages that don't require vacuuming according to the visibility !* map. But only if we've seen a streak of at least !* SKIP_PAGES_THRESHOLD pages marked as clean. Since we're reading !* sequentially, the OS should be doing readahead for us and there's !
Re: [HACKERS] Getting a bug tracker for the Postgres project
Brendan Jurd dire...@gmail.com writes: On 29 May 2011 14:04, Tom Lane t...@sss.pgh.pa.us wrote: Anything that even pretends to be a bug tracker will do that. The real question is, who is going to keep it up to date? GSM has the right point of view here: we need at least a couple of people who are willing to invest substantial amounts of time, or it's not going to go anywhere. Seeing that we can barely manage to keep the mailing list moderator positions staffed, I'm not hopeful. It's not glamorous work, but it is a very accessible way to contribute, without the need to block out hours at a time. A bug wrangler could very readily log in, sort out reports for 20 minutes and then go do something else with the rest of their day. Yup, you're right. But the same comments can be made about mailing list moderation, and I've lost count of the number of fails we've seen in that domain. Anyway, as I said earlier, I'm not standing in the way of anybody who wants to volunteer. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers