Re: [HACKERS] LDC - Load Distributed Checkpoints with PG8.3b2 on Solaris
Jignesh K. Shah wrote: Since its really writes that I am having trouble.. the auto vacuum message tells me 11 pages were removed and so many tuples were removed.. I am guessing its writes. Do you keep track of I/O to WAL and data separately? WAL bandwidth will spike up when a checkpoint starts, because of full page writes. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] LDC - Load Distributed Checkpoints with PG8.3b2 on Solaris
Tom Lane wrote: Looking at the autovacuum log output, 2007-11-13 09:21:19.830 PST 9458 LOG: automatic vacuum of table specdb.public.txn_log_table: index scans: 1 pages: 11 removed, 105 remain tuples: 3147 removed, 40 remain system usage: CPU 0.11s/0.09u sec elapsed 6.02 sec it seems like a serious omission that this gives you no hint how many pages were scanned. Isn't it pages removed + remain? 116 in this case. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] psql -f doesn't complain about directories
Martijn van Oosterhout napsal(a): On Wed, Nov 14, 2007 at 09:33:17PM +0100, Zdenek Kotala wrote: Sure, why not. To be honest I think that psql shouldn't be ignoring the EISDIR error the kernel is returning. But it works when you open directory in read-only mode. See posix definition: [EISDIR] The named file is a directory and oflag includes O_WRONLY or O_RDWR. $ strace psql -f /tmp snip open(/tmp, O_RDONLY|O_LARGEFILE) = 4 snip read(4, 0xb7f1b000, 4096) = -1 EISDIR (Is a directory) Which is subsequently ignored. I'm hoping it doesn't ignore other errors, like EIO or EPIPE, Yes, you have right I checked only open command which works fine, but read fails. Zdenek ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] psql -f doesn't complain about directories
Am Mittwoch, 14. November 2007 schrieb Martijn van Oosterhout:
It's not the fopen that fails, it's the fgets that returns NULL. We
don't subsequently check if that's due to an I/O error or EISDIR or if
it's an end-of-file.
Here is a patch for this.
--
Peter Eisentraut
http://developer.postgresql.org/~petere/
diff -ru ../../../../cvs-pgsql/src/bin/psql/input.c ./input.c
--- ../../../../cvs-pgsql/src/bin/psql/input.c 2007-01-05 23:19:49.0 +0100
+++ ./input.c 2007-11-15 13:59:22.0 +0100
@@ -179,7 +179,7 @@
/* Disable SIGINT again */
sigint_interrupt_enabled = false;
- /* EOF? */
+ /* EOF or error? */
if (result == NULL)
break;
diff -ru ../../../../cvs-pgsql/src/bin/psql/mainloop.c ./mainloop.c
--- ../../../../cvs-pgsql/src/bin/psql/mainloop.c 2007-01-05 23:19:49.0 +0100
+++ ./mainloop.c 2007-11-15 13:57:36.0 +0100
@@ -129,7 +129,14 @@
line = gets_interactive(get_prompt(prompt_status));
}
else
+ {
line = gets_fromFile(source);
+ if (!line ferror(source))
+ {
+psql_error(could not read from input file: %s\n, strerror(errno));
+successResult = EXIT_FAILURE;
+ }
+ }
/*
* query_buf holds query already accumulated. line is the malloc'd
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] Simplifying Text Search
Am Mittwoch, 14. November 2007 schrieb Trevor Talbot: But that coversion itself is fundamentally flawed, is the problem. I know it's incorrect, but with a different parser and/or dictionary you could make it work. In practice, the search pattern will mostly be provided dynamically from some user input, so you could conceivably be able to modify the search patterns more readily than the entire queries in your application. Anyway, it's just an idea for those who need it. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] LDC - Load Distributed Checkpoints with PG8.3b2 on Solaris
Yes I separate out as follows: PGDATA + 1 TABLE which needs to be cached (also workaround CLOG read problem) LOGS DATABASE TABLES DATABASE INDEX to get a good view of IOs out I have full_page_writes=off in my settings I dont see spikes of increase on WAL during checkpoints (maybe due to my setting) but the constant load which is in the range of about 2-2.5MB/sec which is not low but my load is high. In my current run I do have async wal on with wal_writer_delay=100ms and commit_delay off. -Jignesh Heikki Linnakangas wrote: Jignesh K. Shah wrote: Since its really writes that I am having trouble.. the auto vacuum message tells me 11 pages were removed and so many tuples were removed.. I am guessing its writes. Do you keep track of I/O to WAL and data separately? WAL bandwidth will spike up when a checkpoint starts, because of full page writes. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Simplifying Text Search
On 11/15/07, Peter Eisentraut [EMAIL PROTECTED] wrote: In practice, the search pattern will mostly be provided dynamically from some user input, so you could conceivably be able to modify the search patterns more readily than the entire queries in your application. Anyway, it's just an idea for those who need it. Ah, I see what you mean, like for a simple web forum that only knows LIKE searches now. It may be easier to adjust the DB to do the intended thing instead of trying to change the entire forum. I wasn't thinking of those narrow cases. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Simplifying Text Search
Peter Eisentraut [EMAIL PROTECTED] writes: Am Mittwoch, 14. November 2007 schrieb Trevor Talbot: But that coversion itself is fundamentally flawed, is the problem. I know it's incorrect, but with a different parser and/or dictionary you could make it work. No, I don't think so. Trevor's killer point is that the token boundaries chosen by the parser --- no matter *what* they are --- might not line up with the substrings needed by a given LIKE pattern. There isn't any mechanism in tsearch that will find the stored word foobar if the search is for foo bar; nor vice versa. It might be possible to use a pg_trgm index in this way, since AFAICT from the documents pg_trgm just chops up the substrings blindly rather than trying to have smarts about word boundaries. (And on third thought, I suppose you could emulate pg_tgrm with a suitable parser and a lobotomized dictionary ... but it would be pretty wasteful to use the tsearch mechanisms for that.) regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] psql -f doesn't complain about directories
Peter Eisentraut [EMAIL PROTECTED] writes: Am Mittwoch, 14. November 2007 schrieb Martijn van Oosterhout: It's not the fopen that fails, it's the fgets that returns NULL. We don't subsequently check if that's due to an I/O error or EISDIR or if it's an end-of-file. Here is a patch for this. This seems too far removed from the scene of the crime --- I don't have a lot of confidence that errno will still be unchanged back in the main loop. I'd rather see the psql_error printout occur immediately after the failed fgets call. Either that or you need to be a bit more proactive about ensuring errno is returned undamaged. Also, I think you overlooked the case where we get a read error after having already loaded some data into gets_fromFile's result buffer. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] psql -f doesn't complain about directories
Am Donnerstag, 15. November 2007 schrieb Tom Lane: This seems too far removed from the scene of the crime Yeah, my zeroth attempt was to place this in gets_fromFile(), but there you don't have any opportunity to report failure to the main loop. We'd need to change the function signature to be able to pass that around. Maybe that's better overall. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] LDC - Load Distributed Checkpoints with PG8.3b2 on Solaris
Tom Lane wrote: 2007-11-13 09:21:19.830 PST 9458 LOG: automatic vacuum of table specdb.public.txn_log_table: index scans: 1 pages: 11 removed, 105 remain tuples: 3147 removed, 40 remain system usage: CPU 0.11s/0.09u sec elapsed 6.02 sec it seems like a serious omission that this gives you no hint how many pages were scanned. Hmm, right. I'm not sure how to fix it; the simplest idea is to count the number of heap page accesses in lazy_scan_heap, but this wouldn't count index pages so it wouldn't be real. (However, we already report index scans so maybe this is not all that important). Another, more complex idea would be to use the already existing infrastructure for counting buffer accesses, as in ShowBufferUsage. However, just calling ResetBufferUsage and then get the counts would make the counters useless for the outer reporter (the callers in postgres.c). We could have a separate set of save counters; so when vacuum starts, save the current counters and reset them; do the vacuum, report the counters; and finally, restore the save counters by adding the current counters. Is this too complex? -- Alvaro Herrera http://www.amazon.com/gp/registry/CTMLCN8V17R4 La tristeza es un muro entre dos jardines (Khalil Gibran) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Heads up: 8.3beta3 to be wrapped this evening
Any last-minute fixes out there? With luck this will be the last beta --- we are thinking RC1 in about two weeks and final in early December, if no showstopper bugs are reported. So get out there and test it ... I will not have time to fix the default TS parser before then. There's a faint chance I can get to it this weekend. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] LDC - Load Distributed Checkpoints with PG8.3b2 on Solaris
Jignesh K. Shah [EMAIL PROTECTED] writes: I will turn on checkpoint_logging to get more idea as Heikki suggested Did you find out anything? Did this happen on every checkpoint, or only some of them? The bug Itagaki-san pointed out today in IsCheckpointOnSchedule might account for some checkpoints being done at full speed, but not all ... regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] LDC - Load Distributed Checkpoints with PG8.3b2 on Solaris
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: it seems like a serious omission that this gives you no hint how many pages were scanned. Hmm, right. I'm not sure how to fix it; the simplest idea is to count the number of heap page accesses in lazy_scan_heap, but this wouldn't count index pages so it wouldn't be real. (However, we already report index scans so maybe this is not all that important). Another, more complex idea would be to use the already existing infrastructure for counting buffer accesses, as in ShowBufferUsage. However, just calling ResetBufferUsage and then get the counts would make the counters useless for the outer reporter (the callers in postgres.c). We could have a separate set of save counters; so when vacuum starts, save the current counters and reset them; do the vacuum, report the counters; and finally, restore the save counters by adding the current counters. Is this too complex? Too complex for my taste, anyway. I would be satisfied if the log entries just indicated how big the table and indexes were. Heikki pointed out that the message does tell you how many heap pages there were, if you know to add removed + remain; but unless I'm still missing something there's no way to tell how many index pages. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] LDC - Load Distributed Checkpoints with PG8.3b2 on Solaris
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: it seems like a serious omission that this gives you no hint how many pages were scanned. Too complex for my taste, anyway. I would be satisfied if the log entries just indicated how big the table and indexes were. Heikki pointed out that the message does tell you how many heap pages there were, if you know to add removed + remain; but unless I'm still missing something there's no way to tell how many index pages. Maybe we can save RelationGetNumberOfBlocks() at the start of vacuum for heap and indexes, and report it as two numbers (total heap pages: X, total index pages: Y). We would still be missing the TOAST rel and index ... -- Alvaro Herrera Developer, http://www.PostgreSQL.org/ Cuando miro a alguien, más me atrae cómo cambia que quién es (J. Binoche) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] psql -f doesn't complain about directories
Peter Eisentraut [EMAIL PROTECTED] writes: Am Donnerstag, 15. November 2007 schrieb Tom Lane: This seems too far removed from the scene of the crime Yeah, my zeroth attempt was to place this in gets_fromFile(), but there you don't have any opportunity to report failure to the main loop. We'd need to change the function signature to be able to pass that around. Maybe that's better overall. Well, you could still handle that the same as in your patch: on NULL return, check ferror. It's just that I don't trust errno to stay unchanged for very long. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Terminal width for help output
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 15 Nov 2007 21:58:28 +0100 Peter Eisentraut [EMAIL PROTECTED] wrote: Do we care to maintain a maximum width for programs' --help output (and psql's \?)? I think 79 characters was once a recommendation (or perhaps 72), but we have a couple of violations either way, which I'd like to fix, but what to? Why not just have the terminal control the wrap? It isn't just \? that does it. Sincerely, Joshua D. Drake - -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHPLPqATb/zqfZUUQRAuu9AKCIBLRm0ehiWWW7RYi9zjles97/nACcCQu3 5aIzQRH0navadQQzCE6Xn64= =tTOa -END PGP SIGNATURE- ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Terminal width for help output
Peter Eisentraut wrote: Do we care to maintain a maximum width for programs' --help output (and psql's \?)? I think 79 characters was once a recommendation (or perhaps 72), but we have a couple of violations either way, which I'd like to fix, but what to? Yea, I went over with the pg_ctl --timeout parameter and had to trim a little. I would like us to keep it 80 myself. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] Terminal width for help output
Do we care to maintain a maximum width for programs' --help output (and psql's \?)? I think 79 characters was once a recommendation (or perhaps 72), but we have a couple of violations either way, which I'd like to fix, but what to? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Terminal width for help output
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 15 Nov 2007 16:04:46 -0500 Tom Lane [EMAIL PROTECTED] wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Do we care to maintain a maximum width for programs' --help output (and psql's \?)? I think 79 characters was once a recommendation (or perhaps 72), but we have a couple of violations either way, which I'd like to fix, but what to? I think 79 is still a reasonable maximum. AFAIK 80 columns is still a pretty standard terminal window width, but if you try to print in the last column you may get unexpected extra blank lines. O.k. this might be offtopic if it is feel free to smack me... but I have noticed that psql really breaks on terminals that are wide.. \df works fine, but \df+ is completely broke. Can't we just ask the terminal? Sincerely, Joshua D. Drake regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate - -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHPLW4ATb/zqfZUUQRAtH2AJ0cm3Od8SibuI79osf8wG94pqGGnwCeNVjQ qe+JxVIPris0akcrlCLeAO4= =8pTZ -END PGP SIGNATURE- ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Dave Page [EMAIL PROTECTED] writes: hold the presses... Narwhal just broke :-( Grumble ... looks like we have to use ftruncate to keep Windows happy. Will fix. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Terminal width for help output
Alvaro Herrera [EMAIL PROTECTED] writes: Also it would rock if translation kept the alignment in various output. Theoretically the translators are supposed to do that already ... regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Tom Lane wrote: Marc G. Fournier [EMAIL PROTECTED] writes: Let me know when ready to proceed ... I just fixed a second bug in that patch :-( But I think we're good to go now. Note that none of the version-stamping has been done yet. Right, I believe only Marc does that. I have never touched it for beta. Also, I'm not sure where Bruce is with his pgindent stuff. I am done with pgindent, thanks. I was even able to pgindent files I had to skip in the past. Should we wait longer for the buildfarm to become more green? -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Joshua D. Drake [EMAIL PROTECTED] writes: Since we are waiting anyway, something I brought up to Dave about this exact problem was the idea of a freeze :). E.g; All animals must go green and stay green with zero additional commits for 24 hours before we wrap. Is that something that sounds reasonable? Sounds like a good way to waste a day. Even more likely, people would hold off fixing problems till after a beta, because it would reset the counter; and then commit the changes later, guaranteeing that they'd get *less* testing. That's not even mentioning the problem that animals fail on a near-daily basis for reasons beyond our control. I'm not in favor of arbitrary rules. The buildfarm is a tool to serve us, not vice versa. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 'k, then I'd say let's wait ... I'll aim for 00:00 AST to do the build, which gives ~2.25hrs from now, which should be loads of time, right? - --On Thursday, November 15, 2007 17:35:32 -0800 Devrim GÜNDÜZ [EMAIL PROTECTED] wrote: Hi, On Thu, 2007-11-15 at 21:26 -0400, Marc G. Fournier wrote: Any idea on how often narwhal will do a build? It looks that it builds -HEAD every 6 hours: http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=narwhalbr=HEAD and the next build is 2 hours later. Regards, -- Devrim GÜNDÜZ , RHCE PostgreSQL Replication, Consulting, Custom Development, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/ - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHPPa/4QvfyHIvDvMRAheAAKCJrdG0kuJO+yS/kx2DqrUrUTpJcACgxdKg lcw0W1Fon15eZu/gfpEPsRI= =KOkj -END PGP SIGNATURE- ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Terminal width for help output
Bruce Momjian wrote: Peter Eisentraut wrote: Do we care to maintain a maximum width for programs' --help output (and psql's \?)? I think 79 characters was once a recommendation (or perhaps 72), but we have a couple of violations either way, which I'd like to fix, but what to? Yea, I went over with the pg_ctl --timeout parameter and had to trim a little. I would like us to keep it 80 myself. +1. wrapping is annoying. cheers andrew ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= [EMAIL PROTECTED] writes: On Thu, 2007-11-15 at 21:26 -0400, Marc G. Fournier wrote: Any idea on how often narwhal will do a build? It looks that it builds -HEAD every 6 hours: http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=narwhalbr=HEAD and the next build is 2 hours later. Yeah. Theoretically we should be OK because we have a couple of green results from MSVC animals, but I wouldn't mind waiting two hours to see one from a Windows/gcc build. I already asked Dave if he could force a rebuild from home, no go :-( regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Hi, On Thu, 2007-11-15 at 21:26 -0400, Marc G. Fournier wrote: Any idea on how often narwhal will do a build? It looks that it builds -HEAD every 6 hours: http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=narwhalbr=HEAD and the next build is 2 hours later. Regards, -- Devrim GÜNDÜZ , RHCE PostgreSQL Replication, Consulting, Custom Development, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/ signature.asc Description: This is a digitally signed message part
Re: [HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Tom Lane wrote: Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= [EMAIL PROTECTED] writes: On Thu, 2007-11-15 at 21:26 -0400, Marc G. Fournier wrote: Any idea on how often narwhal will do a build? It looks that it builds -HEAD every 6 hours: http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=narwhalbr=HEAD and the next build is 2 hours later. Yeah. Theoretically we should be OK because we have a couple of green results from MSVC animals, but I wouldn't mind waiting two hours to see one from a Windows/gcc build. I already asked Dave if he could force a rebuild from home, no go :-( I will force one in a few minutes. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PERFORM] [HACKERS] 8.3beta1 testing on Solaris
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Jignesh K. Shah wrote: I changed CLOG Buffers to 16 Running the test again: # ./read.d dtrace: script './read.d' matched 2 probes CPU IDFUNCTION:NAME 0 1027 :tick-5sec /export/home0/igen/pgdata/pg_clog/0024 -27530282192961 /export/home0/igen/pgdata/pg_clog/0025 -27530282111041 # ./read.d dtrace: script './read.d' matched 2 probes CPU IDFUNCTION:NAME 1 1027 :tick-5sec # ./read.d dtrace: script './read.d' matched 2 probes CPU IDFUNCTION:NAME 1 1027 :tick-5sec # ./read.d dtrace: script './read.d' matched 2 probes CPU IDFUNCTION:NAME 0 1027 :tick-5sec /export/home0/igen/pgdata/pg_clog/0025 -27530281947201 So Tom seems to be correct that it is a case of CLOG Buffer thrashing. But since I saw the same problem with two different workloads, I think people hitting this problem is pretty high. Also I am bit surprised that CLogControlFile did not show up as being hot.. Maybe because not much writes are going on .. Or maybe since I did not trace all 500 users to see their hot lock status.. Dmitri has another workload to test, I might try that out later on to see if it causes similar impact or not. Of course I havent seen my throughput go up yet since I am already CPU bound... But this is good since the number of IOPS to the disk are reduced (and hence system calls). If I take this as my baseline number.. I can then proceed to hunt other bottlenecks Whats the view of the community? Hunt down CPU utilizations or Lock waits next? Your votes are crucial on where I put my focus. Another thing Josh B told me to check out was the wal_writer_delay setting: I have done two settings with almost equal performance (with the CLOG 16 setting) .. One with 100ms and other default at 200ms.. Based on the runs it seemed that the 100ms was slightly better than the default .. (Plus the risk of loosing data is reduced from 600ms to 300ms) Thanks. Regards, Jignesh Tom Lane wrote: Jignesh K. Shah [EMAIL PROTECTED] writes: So the ratio of reads vs writes to clog files is pretty huge.. It looks to me that the issue is simply one of not having quite enough CLOG buffers. Your first run shows 8 different pages being fetched and the second shows 10. Bearing in mind that we pin the latest CLOG page into buffers, there are only NUM_CLOG_BUFFERS-1 buffers available for older pages, so what we've got here is thrashing for the available slots. Try increasing NUM_CLOG_BUFFERS to 16 and see how it affects this test. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Terminal width for help output
Peter Eisentraut [EMAIL PROTECTED] writes: Do we care to maintain a maximum width for programs' --help output (and psql's \?)? I think 79 characters was once a recommendation (or perhaps 72), but we have a couple of violations either way, which I'd like to fix, but what to? I think 79 is still a reasonable maximum. AFAIK 80 columns is still a pretty standard terminal window width, but if you try to print in the last column you may get unexpected extra blank lines. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: I already asked Dave if he could force a rebuild from home, no go :-( I will force one in a few minutes. narwhal and dawn_bat are both showing green, so I think we're ready to wrap at Marc's convenience. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: I already asked Dave if he could force a rebuild from home, no go :-( I will force one in a few minutes. narwhal and dawn_bat are both showing green, so I think we're ready to wrap at Marc's convenience. Agreed. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Terminal width for help output
Peter Eisentraut wrote: Do we care to maintain a maximum width for programs' --help output (and psql's \?)? I think 79 characters was once a recommendation (or perhaps 72), but we have a couple of violations either way, which I'd like to fix, but what to? 79 is perfect IMHO. It would be great to ask translators to preserve the constraint too. Also it would rock if translation kept the alignment in various output. For example, avoid stuff like $ pg_controldata Numéro de version de pg_control : 833 Numéro de version du catalogue : 200710232 Identifiant du système de base de données : 5125738698090412341 Etat du groupe de base de données : arrêt Dernière modification de pg_control : jeu 15 nov 2007 08:17:22 CLST Dernière localisation du point de contrôle: 0/4D06F8 Précédente localisation du point de contrôle: 0/4B9818 Dernière localisation de reprise du point de contrôle : 0/4D06F8 Dernier TimeLineID du point de vérification : 1 Dernier NextXID du point de contrôle : 0/395 Dernier NextXOID du point de contrôle : 16407 Dernier NextMultiXactId du point de contrôle : 1 Dernier NextMultiOffset du point de contrôle : 0 Heure du dernier point de contrôle : jeu 15 nov 2007 08:17:22 CLST Emplacement de fin de la récupération minimale: 0/0 Alignement maximale de la donnée :8 Taille de bloc de la base de données : 8192 Blocs par segment pour une relation importante : 131072 Taille de bloc du WAL : 8192 Octets par segment WAL : 16777216 Taille maximale des identifiants :64 Nombre de colonnes maximum d'un index: 32 Maximum size of a TOAST chunk:1996 Stockage du type date/heure : nombres à virgule flottante Taille maximum d'un nom local : 128 LC_COLLATE : fr_CA.UTF-8 LC_CTYPE :fr_CA.UTF-8 -- Alvaro Herrera http://www.amazon.com/gp/registry/DXLWNGRJD34J Those who use electric razors are infidels destined to burn in hell while we drink from rivers of beer, download free vids and mingle with naked well shaved babes. (http://slashdot.org/comments.pl?sid=44793cid=4647152) ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
Marc G. Fournier [EMAIL PROTECTED] writes: Let me know when ready to proceed ... I just fixed a second bug in that patch :-( But I think we're good to go now. Note that none of the version-stamping has been done yet. Also, I'm not sure where Bruce is with his pgindent stuff. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] Re: [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Let me know when ready to proceed ... - --On Thursday, November 15, 2007 16:30:00 -0500 Tom Lane [EMAIL PROTECTED] wrote: Dave Page [EMAIL PROTECTED] writes: hold the presses... Narwhal just broke :-( Grumble ... looks like we have to use ftruncate to keep Windows happy. Will fix. regards, tom lane - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHPOy/4QvfyHIvDvMRAvp8AJ99cHVDw0p3Nwq1vlQsYGDMbGPOrgCfXBUH gtfoN8cw8oKLiFnWN5kMOuQ= =3FXh -END PGP SIGNATURE- ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 'k, 2.59 isn't even available in FreeBSD ports anymore, only 2.13 and 2.61, so can someone else please run autoconf and commit, and I'll re-tag ... - --On Thursday, November 15, 2007 23:37:22 -0500 Tom Lane [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] (Marc G. Fournier) writes: configure (r1.570 - r1.571) (http://developer.postgresql.org/cvsweb.cgi/pgsql/configure?r1=1.570 r2=1.571) It appears that Marc has got autoconf 2.61 installed now, instead of the 2.59 that we've been using for some time. I'm a bit concerned about the implications of switching to a version that's got zero track record for us, and that AFAIK no other committers have installed. I'd rather see a switch happen at the start of a devel cycle than at beta3; and in any case it's got to be coordinated so that what is in the release doesn't vary depending on who committed last. regards, tom lane - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHPSCd4QvfyHIvDvMRApMlAJsFhy+DGsSXMzy6bH0FAEQl11zBKwCfbIQ6 66Upa6SjHDUccKm+Mun/l+g= =rNQj -END PGP SIGNATURE- ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
Tom Lane wrote: [EMAIL PROTECTED] (Marc G. Fournier) writes: configure (r1.570 - r1.571) (http://developer.postgresql.org/cvsweb.cgi/pgsql/configure?r1=1.570r2=1.571) It appears that Marc has got autoconf 2.61 installed now, instead of the 2.59 that we've been using for some time. I'm a bit concerned about the implications of switching to a version that's got zero track record for us, and that AFAIK no other committers have installed. I'd rather see a switch happen at the start of a devel cycle than at beta3; and in any case it's got to be coordinated so that what is in the release doesn't vary depending on who committed last. O.k., so what do you want to do? Or more to the point, is there something I can help with? I have machines with 2.59... Marc do you have any left with 2.59? Sincerely, Joshua D. Drake regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
Marc G. Fournier wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 'k, 2.59 isn't even available in FreeBSD ports anymore, only 2.13 and 2.61, so can someone else please run autoconf and commit, and I'll re-tag ... I can't commit but I can give access to a 2.59 version... Sincerely, Joshua D. Drake - --On Thursday, November 15, 2007 23:37:22 -0500 Tom Lane [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] (Marc G. Fournier) writes: configure (r1.570 - r1.571) (http://developer.postgresql.org/cvsweb.cgi/pgsql/configure?r1=1.570 r2=1.571) It appears that Marc has got autoconf 2.61 installed now, instead of the 2.59 that we've been using for some time. I'm a bit concerned about the implications of switching to a version that's got zero track record for us, and that AFAIK no other committers have installed. I'd rather see a switch happen at the start of a devel cycle than at beta3; and in any case it's got to be coordinated so that what is in the release doesn't vary depending on who committed last. regards, tom lane - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHPSCd4QvfyHIvDvMRApMlAJsFhy+DGsSXMzy6bH0FAEQl11zBKwCfbIQ6 66Upa6SjHDUccKm+Mun/l+g= =rNQj -END PGP SIGNATURE- ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
Marc G. Fournier wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Thursday, November 15, 2007 20:49:04 -0800 Joshua D. Drake [EMAIL PROTECTED] wrote: Marc G. Fournier wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 'k, 2.59 isn't even available in FreeBSD ports anymore, only 2.13 and 2.61, so can someone else please run autoconf and commit, and I'll re-tag ... I can't commit but I can give access to a 2.59 version... Well, easiest is for Tom to run autoconf 2.59 and commit ... or Bruce ... *shrug* :) helping where I can... number is on pmt if you need me :) Joshua D. Drake - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHPSIc4QvfyHIvDvMRAoZCAJ9MF5wdAcB0aUTyT8qo62+DF61wywCfQLJF kSsl+ZTYu9SC+OEuA2NGPfU= =EDTa -END PGP SIGNATURE- ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
Marc G. Fournier [EMAIL PROTECTED] writes: [EMAIL PROTECTED] wrote: I can't commit but I can give access to a 2.59 version... Well, easiest is for Tom to run autoconf 2.59 and commit ... or Bruce ... Locally I've got several autoconf versions installed so that I can update back-branch configure scripts properly. It'd probably be a good idea to improve your release scripts so that they select the right autoconf version for each release branch. You'll need multiple local installations though, instead of depending on freebsd ports for the one true autoconf. Either that or we try to move up all supported back branches to the latest autoconf version; which might be a good idea but it scares me a bit. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Thursday, November 15, 2007 21:21:59 -0800 Joshua D. Drake [EMAIL PROTECTED] wrote: Marc G. Fournier wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 That would be a good idea, and really simply things ... FreeBSD seems to have drop'd off support for all but 2.13 and 2.61 ... If we do that, (I honestly don't know) what happens on versions that are running an older version of autoconf? I mean, if everything is put together with 2.61, are 2.59 versions going to have an issue? I believe Tom was suggesting upgrading and testing them to make sure of that ... instead of having 7.3 using one version, 7.4 and 8.0 usnig another, and 8.1 and beyond using a third ... I know right now we have three different versions 'required', just can't recall which fall under which ... - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHPSnf4QvfyHIvDvMRAjZSAJ9D28LaQ3abmr6Mb4/vWpwACR602ACgvBP9 18xCapgj6muv2SuYNrLzv+8= =HvQG -END PGP SIGNATURE- ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Friday, November 16, 2007 00:03:46 -0500 Tom Lane [EMAIL PROTECTED] wrote: Marc G. Fournier [EMAIL PROTECTED] writes: [EMAIL PROTECTED] wrote: I can't commit but I can give access to a 2.59 version... Well, easiest is for Tom to run autoconf 2.59 and commit ... or Bruce ... Locally I've got several autoconf versions installed so that I can update back-branch configure scripts properly. It'd probably be a good idea to improve your release scripts so that they select the right autoconf version for each release branch. You'll need multiple local installations though, instead of depending on freebsd ports for the one true autoconf. Either that or we try to move up all supported back branches to the latest autoconf version; which might be a good idea but it scares me a bit. That would be a good idea, and really simply things ... FreeBSD seems to have drop'd off support for all but 2.13 and 2.61 ... - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHPShG4QvfyHIvDvMRAm48AJ9D7FOT0EyASLJuBmxeLbE+464HdgCg54fJ xQOk7rf3xBmwEreHKzlk3C4= =6M0k -END PGP SIGNATURE- ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
Marc G. Fournier [EMAIL PROTECTED] writes: - --On Friday, November 16, 2007 00:03:46 -0500 Tom Lane [EMAIL PROTECTED] wrote: Either that or we try to move up all supported back branches to the latest autoconf version; which might be a good idea but it scares me a bit. That would be a good idea, and really simply things ... FreeBSD seems to have drop'd off support for all but 2.13 and 2.61 ... Perhaps so, but it'd cost us a fair amount of up-front work to verify that we don't break the back branches by updating their configure scripts. Not something I want to touch on a last-minute basis ;-) [ digs for a moment... ] According to my notes we are using autoconf 2.53 for versions 7.3-8.0 and 2.59 for the later branches. So 2.13 is already out of the picture. It might be that 2.53 to 2.59 to 2.61 is not all that big a jump in reality, but I've got to say that it scares me when I read commit-log entries that report ten thousand lines worth of diffs in a 20K-line script ... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Friday, November 16, 2007 00:40:31 -0500 Tom Lane [EMAIL PROTECTED] wrote: Marc G. Fournier [EMAIL PROTECTED] writes: - --On Friday, November 16, 2007 00:03:46 -0500 Tom Lane [EMAIL PROTECTED] wrote: Either that or we try to move up all supported back branches to the latest autoconf version; which might be a good idea but it scares me a bit. That would be a good idea, and really simply things ... FreeBSD seems to have drop'd off support for all but 2.13 and 2.61 ... Perhaps so, but it'd cost us a fair amount of up-front work to verify that we don't break the back branches by updating their configure scripts. Not something I want to touch on a last-minute basis ;-) Wasn't suggesting 'last-minute', but maybe post 8.3 release, while things are a bit quiet? - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFHPS7E4QvfyHIvDvMRAvgVAKDZ4OlC6dcwIeTu4APXYTfQ6XArlwCdEtBz ApsX48H6dalG+KJDVy0m+rA= =RFnE -END PGP SIGNATURE- ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] Heads up: 8.3beta3 to be wrapped this evening
Any last-minute fixes out there? With luck this will be the last beta --- we are thinking RC1 in about two weeks and final in early December, if no showstopper bugs are reported. So get out there and test it ... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] GiST crash recovery (potential problems?)
Hi, In GiST, I found that after the crash recovery, NSN and right page link are initialized. We can search all the records in this case but performance may become a little worse because we cannot traverse leaves. I'm not sure if it is preffered behavior. -- Koichi Suzuki ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [COMMITTERS] pgsql: update files for beta3
Marc G. Fournier [EMAIL PROTECTED] writes: 'k, 2.59 isn't even available in FreeBSD ports anymore, only 2.13 and 2.61, so can someone else please run autoconf and commit, and I'll re-tag ... Done regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] [Fwd: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure]
hold the presses... Narwhal just broke :-( Original Message Subject: PGBuildfarm member narwhal Branch HEAD Status changed from OK to Make failure Date: Thu, 15 Nov 2007 13:23:06 -0800 (PST) From: PG Build Farm [EMAIL PROTECTED] To: [EMAIL PROTECTED], [EMAIL PROTECTED] The PGBuildfarm member narwhal had the following event on branch HEAD: Status changed from OK to Make failure The snapshot timestamp for the build that triggered this notification is: 2007-11-15 21:00:00 The specs of this machine are: OS: Windows Server 2003 R2 / 5.2.3790 Arch: i686 Comp: GCC / 3.4.2 (mingw-special) For more information, see http://www.pgbuildfarm.org/cgi-bin/show_history.pl?nm=narwhalbr=HEAD ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
