Re: [HACKERS][PATCHES] odd output in restore mode

2008-07-28 Thread Greg Smith
Windows as well, at least enough to cover this particular issue. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches

Re: [PATCHES] pg_dump additional options for performance

2008-07-25 Thread Greg Sabino Mullane
re "before" data.) Is that a small enough corner case > to live with in order to gain implementation simplicity and robustness? I'm not comfortable with corner cases for pg_restore backwards compatibility. What exactly would happen (worse case) in that scenario? - -- Greg Sabino

Re: [PATCHES] Sorting writes during checkpoint

2008-07-09 Thread Greg Smith
results were reported on, but so far I don't see any significant difference after applying the sorted writes patch. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subs

Re: [PATCHES] pgbench minor fixes

2008-07-05 Thread Greg Smith
then, then runs VACCUM ANALYZE. There is no notion of what script you'll end up executing yet. If you have a truly custom script that works against other data instead of the pgbench tables, you won't even be executing this initialization bit. -- * Greg Smith [EMAIL

Re: [PATCHES] posix advises ...

2008-06-19 Thread Greg Smith
a less popular API I expect a round of portability concerns, too. Where did Marc's patch come from? I'd like to be able to separate out that change from the rest if necessary. Also, if you have any specific test cases you ran that I could start by trying to replicate a speedup on, t

Re: [PATCHES] Better formatting of functions in pg_dump

2008-06-12 Thread Greg Sabino Mullane
witching between newline-before and newline-after > styles? Please be consistent. I thought they were all "after". On second glance, they still seem all after? - -- Greg Sabino Mullane [EMAIL PROTECTED] End Point Corporation PGP Key: 0x14964AC8 200806122044 http://biglumber.com/x/web

[PATCHES] Better formatting of functions in pg_dump

2008-06-12 Thread Greg Sabino Mullane
Attached patch puts the "metadata" about a function, especially the language name, at the top of the CREATE FUNCTION statement, above the possibly long, multi-line function definition. -- Greg Sabino Mullane Index:

Re: [PATCHES] LOCK_DEBUG documentation

2008-05-27 Thread Greg Sabino Mullane
self and others in production systems for debugging purposes, and the lack of detail in that section while doing so led to the patch. The docs also has the advantage of being more available, searchable, and found via the web. - -- Greg Sabino Mullane [EMAIL PROTECTED] End Point Corporation PGP Key:

[PATCHES] LOCK_DEBUG documentation

2008-05-20 Thread Greg Sabino Mullane
Documentation patch by Kevin L. McBride explaining LOCK_DEBUG options in detail. -- Greg Sabino Mullane End Point Corporation Index: config.sgml === RCS file: /projects/cvsroot/pgsql/doc/src/sgml/config.sgml,v retrieving revision

Re: [PATCHES] posix advises ...

2008-05-13 Thread Greg Smith
, and leave out the resulting configure file if you didn't use the same version of Autoconf. I find the concept behind this patch very useful and I'd like to see a useful one re-submitted. I'm in the middle of setting up some new hardware this month and was planning to test the

Re: [PATCHES] [GENERAL] pgbench not setting scale size correctly?

2008-05-09 Thread Greg Smith
On Wed, 7 May 2008, Bruce Momjian wrote: Tom Lane wrote: Greg Smith <[EMAIL PROTECTED]> writes: The way the option parsing code is done would make complaining in the case where your parameter is ignored a bit of a contortion. Yeah. But couldn't we have that part issue a warnin

Re: [PATCHES] [GENERAL] pgbench not setting scale size correctly?

2008-05-07 Thread Greg Smith
re it should go now that I look at this again. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches

Re: [PATCHES] Sorting writes during checkpoint

2008-05-04 Thread Greg Smith
iginal source to re-open with more information. That keeps the unprocessed part of the queue always shrinking, and as long as people know that they can get it reconsidered by submitting new results it's not unfair to them. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

Re: [PATCHES] Sorting writes during checkpoint

2008-05-04 Thread Greg Smith
for this situation might be for you to bounce this into the rejected pile as "Returned for testing" immediately, to clearly remove it from the main queue. A reasonable expectation there is that you might consider it again during May if someone gets back with said testing results befor

Re: [PATCHES] Sorting writes during checkpoint

2008-05-04 Thread Greg Smith
then (I'm stuck sorting out a number of OS level issue right now before my testing system is online again). Was planning to take a longer look at Greg Stark's prefetching work at that point as well. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via p

Re: [PATCHES] configure option for XLOG_BLCKSZ

2008-05-02 Thread Greg Smith
ocks to match the underlying flash better, and you don't pay as much of a penalty for writing that way because lining up with the spinning disk isn't important. Someone who put one of DB/WAL on SSD and the other on traditional disk might end up with very different DB/WAL block sizes t

Re: [PATCHES] Sorting writes during checkpoint

2008-04-16 Thread Greg Smith
emory for it, RelFileNode in BufferTag could be hashed and packed into an integer I personally don't feel it's worth making the code any more complicated than it needs to be just to save a fraction of a percent of the total memory used by the buffer pool. -- * Greg Smith [EMAIL PRO

Re: [PATCHES] Sorting writes during checkpoint

2008-04-15 Thread Greg Smith
BufAndTag is a relatively small structure (5 ints). Let's call it 40 bytes; even that's only a 0.5% overhead relative to the shared buffer allocation. If we can speed checkpoints significantly with that much overhead it sounds like a good tradeoff to me. -- * Greg Smith [EMAIL PROTEC

Re: [PATCHES] Expose checkpoint start/finish times into SQL.

2008-04-04 Thread Greg Smith
collected some actual useful information here. That's the theory at least. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-patches

Re: [PATCHES] Expose checkpoint start/finish times into SQL.

2008-04-03 Thread Greg Smith
27;t working anymore, and that wasn't being used. Given some of the other corruption found later and the bad memory issues discovered, a bit flipping in the "do I need to checkpoint now?" code or data seems just as likely as any other explanation. -- * Greg Smith [EMAIL PROTECTED

Re: [PATCHES] Expose checkpoint start/finish times into SQL.

2008-04-03 Thread Greg Smith
then start sprinkling exports of those probe points in some popular places people would like to look at. I will apologize now for suggesting this, followed by not having enough time to code it in the near future. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- S

Re: [PATCHES] Expose checkpoint start/finish times into SQL.

2008-04-03 Thread Greg Smith
cause they do still vary even in 8.3. I'm just not sure if the current methods available for that really aren't good enough, or if it's just the case that not everyone is aware of all of them. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via p

Re: [PATCHES] Expose checkpoint start/finish times into SQL.

2008-04-03 Thread Greg Smith
table so you can compute statistics about how the checkpoints are doing there. I don't see why that sort of stuff should go into core when it's now easy to do outside of it. I have a whole stack of scripts in that area I plan to release over the summer. -- * Greg Smith [EMAIL PROTECTED] http:

Re: [PATCHES] Expose checkpoint start/finish times into SQL.

2008-04-03 Thread Greg Smith
t easier to answer with a simple command which has some value; a little SQL in a cron job would be good enough to trigger an alert rather than needing a real monitoring probe. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-patches mailing list (pgs

Re: [PATCHES] Consistent \d commands in psql

2008-04-02 Thread Greg Sabino Mullane
> the second. Agreed, we should respect the search path like the other commands do. Although I wonder if a long-term idea would be to at least indicate that there are other same-named things in your path? - -- Greg Sabino Mullane [EMAIL PROTECTED] [EMAIL PROTECTED] End Point Corporation 610-

Re: [PATCHES] Friendly help for psql

2008-04-02 Thread Greg Sabino Mullane
Index: mainloop.c === RCS file: /projects/cvsroot/pgsql/src/bin/psql/mainloop.c,v retrieving revision 1.87 diff -c -r1.87 mainloop.c *** mainloop.c 1 Jan 2008 19:45:56 - 1.87 --- mainloop.c 2 Apr 2008 12:51:36 - **

Re: [PATCHES] CopyReadLineText optimization

2008-03-06 Thread Greg Smith
gression that shows up with >8 escapes per line. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org) To make changes to your subscription: http://mail.postgresql.org/mj/mj_wwwusr?domain=postgresql.org&

Re: [PATCHES] Proposing correction to posix_fadvise() usage in xlog.c

2008-03-01 Thread Greg Smith
disk cache. I've never heard of a SATA drive that had its write cache disabled by default. They're always on unless you force them off, and even then they can turn themselves back on again if there's a device reset and you didn't change the drive's default using t

Re: [PATCHES] Better default_statistics_target

2008-02-03 Thread Greg Sabino Mullane
ot; number was made without any such analysis either, and I can assure you it has caused lots of real-world problems. Going from 10 to 100 adds a small amount of planner overhead. The 99/100 change adds an order of magnitude speed difference to SELECT queries. I still cannot see that as anything

Re: [PATCHES] Friendly help for psql

2008-01-28 Thread Greg Sabino Mullane
ot;. It's simple, standard, and points you to anything else you may need or want to do. > What would be interesting would be if the _server_ could send back some > message about "Use the help facility of your client application" but it > would have to have a trailing se

Re: [PATCHES] Friendly help for psql

2008-01-21 Thread Greg Sabino Mullane
ql prompt, and sends someone else in to it, or comes back later on. Same thing: the login note is no longer available ('screen' has a small scrollback), and even if you've used it before, "backslash question mark" is, let's be honest, very obscure, unintuitive, and

Re: [PATCHES] Friendly help for psql

2008-01-20 Thread Greg Sabino Mullane
g to > ever cause conflicts with a SQL command but perhaps it's better to be > prepared), one idea would be to respond with "please execute \help > instead", and then \help would emit the verbose output. Ugh. Why make people do two steps? - -- Greg Sabino Mullane [

[PATCHES] Friendly help for psql

2008-01-20 Thread Greg Sabino Mullane
Why not run help when someone enters "help" (or "HELP ME!") on the command line? \? is hardly an easy thing to remember (and some people can't be bothered to actually read the screen...) ? NOTES ? README ? SUBMIT ? backslash_consistency.patch ? ddproblem.sql ? dfs.20071104 ? pop ? psql ? scrap ? t

Re: [PATCHES] [HACKERS] SSL over Unix-domain sockets

2008-01-18 Thread Greg Smith
d ever find a short symlink it can delete if I'm understanding this correctly. I left behind the link I was just playing with and I'll see if I can get tmpwatch to eat it tomorrow, that seems like the most appropriate test here. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmit

Re: [PATCHES] pgbench - startup delay

2007-12-10 Thread Greg Smith
nch will use PGOPTIONS. In addition to that, the current documentation is less clear than it could be on the subject of what you can usefully put into PGOPTIONS. That's two small documentation patches I should be able to handle. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.

Re: [PATCHES] pgbench - startup delay

2007-12-10 Thread Greg Smith
n imagine pgbench users might also want to use -S (sort memory) or -f (forbid scan/join types). If I can get someone to clarify what is supported there I can put together a pgbench doc patch that addresses this topic. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

Re: [PATCHES] Better default_statistics_target

2007-12-05 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Simon spoke: > The choice of 100 is because of the way the LIKE estimator is > configured. Greg is not suggesting he measured it and found 100 to be > best, he is saying that the LIKE operator is hard-coded at 100 and so > the

Re: [PATCHES] Re: [HACKERS] [GENERAL] plperl and regexps with accented characters - incompatible?

2007-12-01 Thread Greg Sabino Mullane
d our energy is better spent improving Pl/Perl itself at this point rather than tweaking things for old versions of Perl. I don't even think I have a pre 5.8 version around anymore. Would such a requirement cause any problems with packagers? I imagine a perl 5.8 prereq is a common thing

Re: [PATCHES] Proposed patch for operator lookup caching

2007-11-27 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 > Since Simon seems intent on hacking something in there, here is a patch > that I think is actually sane for improving operator lookup speed. +1 on the patch (reviewed and tested), and +1 for rolling it into RC. - -- Greg Sabino M

[PATCHES] Better default_statistics_target

2007-11-13 Thread Greg Sabino Mullane
Per a recent bug in which the planner can behave very differently at < 100, and accounting for the fact that analyze is still plenty fast on today's systems even at a tenfold increase, attached is a patch to change default_statistics_target from 10 to 100. Index: doc/src/sgml/config.sgml =

[PATCHES] Consistent \d commands in psql

2007-11-04 Thread Greg Sabino Mullane
on the vine, despite strong support from -general, hence I'm going to try again, as I really want a way to view my functions without querying the pg_proc tables directly. :) -- Greg Sabino Mullane [EMAIL PROTECTED] PGP Key: 0x14964AC8 200711042003 http://biglumber.com/x/web?pk=2529DF6AB8F79407

[PATCHES] Better psql tab-completion support for schemas and tables

2007-10-21 Thread Greg Sabino Mullane
quot;foo" where select * from "Uppercase".lower where select * from "gtsm.com"."foo.Bar" where select * from "GTSM.com".foo where Also applies to other places that get lists of columns: insert into, alter table, create index, etc. -- Greg Sabino Mul

[PATCHES] Better psql tab-completion support for schemas and tables

2007-10-21 Thread Greg Sabino Mullane
ot;Uppercase".lower where select * from "gtsm.com"."foo.Bar" where select * from "GTSM.com".foo where Also applies to other places that get lists of columns: insert into, alter table, create index, etc. -- Greg Sabino Mullane [EMAIL PROTECTED] PG

Re: [PATCHES] HOT patch - version 15

2007-09-06 Thread Greg Smith
for bgwriter_delay feasible. None of the tunables *should* have to be adjusted if you need to run the bgwriter much more often to support some new HOT-related activity in there as well; this is actually my next area I wanted to do a bit more testing on myself. -- * Greg Smith [EMAIL PROTECTED

Re: [PATCHES] Async Commit, v21 (now: v22)

2007-07-24 Thread Greg Smith
tinct walwriter. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

Re: [PATCHES] HOT latest patch - version 8

2007-07-15 Thread Greg Smith
ssume this is a typo on Benford's Law: http://en.wikipedia.org/wiki/Benford's_law which notes there are far more ones in real-world data sets. If there were a Benfold's Law, it would surely involve the number 5 instead. -- * Greg Smith [EMAIL PROTECTED] http://www.gr

Re: [PATCHES] Maintaining cluster order on insert

2007-07-10 Thread Greg Smith
ually trying to draw conclusions from was probably real enough. Agreed, just figuring out the test ramp-up situation, and your explanation for that quirk sounds reasonable. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)-

Re: [PATCHES] Maintaining cluster order on insert

2007-07-10 Thread Greg Smith
2.449248 sec (1st) ... executed INSERT in 12.212027 sec (2nd) results-patch: ... executed DELETE in 18.062664 sec executed VACUUM in 28.487570 sec executed INSERT in 25.638022 sec (1st) ... executed INSERT in 40.759404 sec (2nd) -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmi

[PATCHES] Tracking buffer allocation stats (new LRU method base code)

2007-07-02 Thread Greg Smith
me. 3) The recent strategy changes in freelist.c left me unsure how to count some of what it does; I marked the section I'm concerned about with an XXX comment. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD buf-alloc-stats-2.patch

Re: [PATCHES] Checkpoint logging, revised patch

2007-06-30 Thread Greg Smith
(most of this patch was done against 8.2 and then ported forward). If that already shows clearly the start and end of each autovacuum section, ignore that I even brought this up. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

Re: [PATCHES] Checkpoint logging, revised patch

2007-06-29 Thread Greg Smith
(int) pid, port->sock))); Little stuff, but all things I've found valuable on several occasions, which suggests eventually someone else may appreciate them as well. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PATCHES] Load Distributed Checkpoints, final patch

2007-06-28 Thread Greg Smith
ts are built back up again 3) You ran an early manual checkpoint which doesn't seem to recycle as many segments usefully Any change that would be more proactive about creating segments in these situations than the current code would be a benefit, even though these are not common path

Re: [PATCHES] Load Distributed Checkpoints, final patch

2007-06-26 Thread Greg Smith
On Tue, 26 Jun 2007, Heikki Linnakangas wrote: I'm scheduling more DBT-2 tests at a high # of warehouses per Greg Smith's suggestion just to see what happens, but I doubt that will change my mind on the above decisions. I don't either, at worst I'd expect a small document

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-26 Thread Greg Smith
nt process. That's OK for now, but I'd like it to be the case that one day the database's I/O scheduling would eventually get to those, in order to optimize performance in the kind of bursty scenarios I've been mentioning lately. -- * Greg Smith [EMAIL PROTECTED] http://www.gre

Re: [PATCHES] Load Distributed Checkpoints, final patch

2007-06-26 Thread Greg Smith
ill yourself exactly this way right now though, and the fact that LDC gives you a parameter to aim this particular foot-gun more precisely isn't a big deal IMHO as long as the documentation is clear. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-26 Thread Greg Smith
hat there are corner cases floating around this area is what makes me feel that removing it altogether is still a bit premature. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-26 Thread Greg Smith
will have moved along. I'll add this idea to my list of things that would be nice to have as part of a larger rewriter, I think it's more trouble than it's worth to chase right now. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD --

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-25 Thread Greg Smith
ore UPDATEs relative to the other types of transactions; having more of those seemed to aggrevate my LDC-related issues because they leave a different pattern of dirty+used buffers around than other operations. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-25 Thread Greg Smith
ts to get expensive CPU-wise. In addition to being a CPU pig, that still won't necessarily work because the way the LRU writer ignores things with a non-zero usage_count. If it's dirty, it's probably been used recently as well. -- * Greg Smith [EMAIL PROTECTED] http://www.gre

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-25 Thread Greg Smith
On Mon, 25 Jun 2007, Heikki Linnakangas wrote: Greg, is this the kind of workload you're having, or is there some other scenario you're worried about? The way transitions between completely idle and all-out bursts happen were one problematic area I struggled with. Since the

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-24 Thread Greg Smith
you think that the current code had enormous amounts of testing before it went in, I've got to disillusion you :-( It's having been on the painful receiving end of that fact that makes me so paranoid now :) -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-24 Thread Greg Smith
On Sun, 24 Jun 2007, Simon Riggs wrote: Greg can't choose to use checkpoint_segments as the limit and then complain about unbounded recovery time, because that was clearly a conscious choice. I'm complaining only because everyone seems content to wander in a direction where the

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-24 Thread Greg Smith
presented so far is extrapolation. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-23 Thread Greg Smith
7;s most recent results? The way I remember that, it was just me pushing to expose that problem, because I knew it was there from my unfortunately private tests, but it was difficult to encounter the issue on other types of benchmarks (thanks again to Greg Stark and Heikki for helping with that

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-22 Thread Greg Smith
On Fri, 22 Jun 2007, Tom Lane wrote: Greg had worried about being able to turn this behavior off, so we'd still need at least a bool, and we might as well expose the fraction instead. I agree with removing the non-LRU part of the bgwriter's write logic though If you accept that

Re: [PATCHES] Load Distributed Checkpoints, take 3

2007-06-22 Thread Greg Smith
e the rest of the xlog writes do? That write goes through the regular buffered path instead. The checkpoint logging patch I submitted logs when this happens specifically because that particular issue messed with some operations and I found it important to be awar

Re: [PATCHES] trace_checkpoint parameter patch

2007-06-12 Thread Greg Smith
chives for the discussion this week on the topic "Controlling Load Distributed Checkpoints", and check out the "Automatic adjustment of bgwriter_lru_maxpages" patch whose latest version is at http://archives.postgresql.org/pgsql-patches/2007-05/msg00142.php -- * Greg Smith [E

Re: [PATCHES] COPY-able csv log outputs

2007-06-01 Thread Greg Smith
On Fri, 1 Jun 2007, Andrew Dunstan wrote: Greg Smith wrote: Since the rotation size feature causes other issues anyway that make importing more complicated, documenting the issue seemed sufficient. What are the other issues? I'm not happy about producing files with split lines. Just

Re: [PATCHES] Logging checkpoints and other slowdown causes

2007-05-29 Thread Greg Smith
range, and I wouldn't want to see that truncated too much. Anyway, I have a bunch of data on this subject being collected at this moment, and I'll rescale the results based on what I see after analyzing that this week. -- * Greg Smith [EMAIL PROTECTED

Re: [PATCHES] WIP: 2nd-generation buffer ring patch

2007-05-29 Thread Greg Smith
kind of an unreasonable situation to go out of your way to accommodate. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [PATCHES] Logging checkpoints and other slowdown causes

2007-05-28 Thread Greg Smith
t server this week to confirm whether's it's worthless precision or worth capturing. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore

Re: [PATCHES] Seq scans status update

2007-05-28 Thread Greg Smith
the usage_count regardless. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [PATCHES] COPY-able csv log outputs

2007-05-28 Thread Greg Smith
cts the dynamics of how downstream applications will have to work with this data, so you can see that my contrary preference isn't completely random here. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)---

Re: [PATCHES] COPY-able csv log outputs

2007-05-21 Thread Greg Smith
, so I'll add another column for the line number to the output. Thanks for pointing that out, I can finish cleaning up all the functional implementation work on this patch myself now. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(e

Re: [PATCHES] COPY-able csv log outputs

2007-05-20 Thread Greg Smith
fix that too when I'm revising. I plan to have a version free of obvious bugs to re-submit ready by next weekend. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 1: if posting/reading throug

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread Greg Smith
to what's already been done here, I sent in what I thought was ready to go because I didn't want to hold up reviewing the bulk of the code over some of these fine details. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of br

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread Greg Smith
y're allocating more than that during a period, they should shrink the delay instead. The kinds of systems where 1000 isn't high enough for bgwriter_lru_maxpages are going to be compelled to adjust these parameters anyway for good performance. -- * Greg Smith [EMAIL PROTECTED] htt

Re: [PATCHES] [DOCS] OS/X startup scripts

2007-05-14 Thread Greg Smith
w.php/140/4/PGBuildFarm-HOWTO.txt and those aren't quite focused right if the goal is to work on new patches while keeping in sync with the repository. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)-

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-14 Thread Greg Smith
easier to write that if you can say "You can safely set bgwriter_lru_maxpages high because it only writes what it needs to based on your usage". -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)---

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-13 Thread Greg Smith
f back into the callers would end up being a cut and paste job that would duplicate many lines. That's on top of the fact that the buffer is cleanly locked/unlocked all in one section of code right now, and I didn't see how to move any parts of that to the c

Re: [PATCHES] Automatic adjustment of bgwriter_lru_maxpages

2007-05-12 Thread Greg Smith
the LRU limiting behavior of the second patch, at which point it's strictly internals code with no expected functional impact that alternate approaches might be built on. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MDIndex: src/backend/storage/buffer/bufmgr.c ==

Re: [PATCHES] Logging checkpoints and other slowdown causes

2007-05-11 Thread Greg Smith
the shoes of a log file parser the other way around is easier to work with. Piecing together log entries is a pain, splitting them is easy. If I had to only keep one line out of this, it would be the one with the summary. It would be nice to have it logged at INFO. -- * Greg Smith [EMAIL

Re: [PATCHES] Load distributed checkpoint V4

2007-04-23 Thread Greg Smith
ible to make a reasonable estimate. Any less information than all that and you really have very little basis on which to guess how long it's going to take. Other operating systems are going to give completely different behavior here, which of course makes the problem ev

Re: [PATCHES] Load distributed checkpoint V3

2007-04-06 Thread Greg Smith
a discussion on the patches list about a feature useful *right now* in this area. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Greg Smith
ould come from things like AIO--it gets real messy. Good luck drumming up support for all that when the initial benchmarks suggest it's going to be a big step back. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)-

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Greg Smith
using a moving-average approach (as I did in my auto-tuning writer prototype and as Tom just suggested here) will be sufficient to fix all of them. Was already on my to-do list to investigate that further. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Greg Smith
h a redesign there, maybe you can get away with it at this late moment. But I ask that you please don't remove anything from the current design until you have significant test results to back up that change. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD --

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Greg Smith
ious portions of the checkpoint, to tell when it was bogged down with slow writes versus being held up in sync for various (possibly fixed with your CheckpointStartLock) issues. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

Re: [PATCHES] Logging checkpoints and other slowdown causes

2007-04-03 Thread Greg Smith
ting I consider a bonus rather than the main focus here. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PATCHES] bgwriter stats

2007-04-01 Thread Greg Smith
a partial result because I haven't had enough time track all the code paths involved to prove otherwise. Glad to hear it was never an issue. Doesn't change what I submitted though. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -

Re: [PATCHES] bgwriter stats

2007-03-31 Thread Greg Smith
On Thu, 29 Mar 2007, Magnus Hagander wrote: I've included a couple of more counters per ideas from Greg Smith in his logging patch. I just submitted a patch that logs the remaining things of value from my original that couldn't be derived from the information you're collecting

[PATCHES] Logging checkpoints and other slowdown causes

2007-03-31 Thread Greg Smith
g a dependency on its current structure to here. Ideally, it would be nice to call something like WillLog(DEBUG2) and get a boolean back saying whether something at that log level will be output anywhere; I don't know enough about the logging code to add such a thing to it mys

Re: [PATCHES] Add usage counts to pg_buffercache

2007-03-31 Thread Greg Smith
uffer cache before they could get evicted, which would take quite a while. My bet is that bufferid gets expanded from 32 bits before that happens, which would break pg_buffercache similarly and isn't being checked for either. -- * Greg Smith [EMAIL PROTECTED] http://www

[PATCHES] Add usage counts to pg_buffercache

2007-03-31 Thread Greg Smith
massive cost in overhead, of course. I'll be passing along all that code once I get it ready for other people to use. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MDIndex: README.pg_buffercache === RCS f

[PATCHES] pgbench transaction timestamps

2007-03-31 Thread Greg Smith
un variation in TPS (checkpoints are particularly visible in the graphs), after further testing I concluded running a VACUUM VERBOSE and CHECKPOINT in a script afterwards and analyzing the results was more useful than integrating something into pgbench itself. -- * Greg Smith [EMAIL

Re: [PATCHES] Load distributed checkpoint V3

2007-03-25 Thread Greg Smith
the example graphs there aren't from the production system I mentioned above, they're from my server at home, which is similar to the system your results came from). -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broad

Re: [PATCHES] Load distributed checkpoint V3

2007-03-25 Thread Greg Smith
slow checkpoint issues even worse on my Linux system. I'm trying to evaluate this fairly. -This code operates on the assumption you have a good value for the checkpoint timeout. Have you tested its behavior when checkpoints are being triggered by checkpoint_segments being reached instea

[PATCHES] Have \da in psql show return type

2007-03-14 Thread Greg Sabino Mullane
Using \da in psql should show the return type. Index: doc/src/sgml/ref/psql-ref.sgml === RCS file: /projects/cvsroot/pgsql/doc/src/sgml/ref/psql-ref.sgml,v retrieving revision 1.188 diff -c -r1.188 psql-ref.sgml *** doc/src/sgml/ref/

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-21 Thread Greg Smith
ley related mechanics. But the SQL logs are be based on my requirements, which is to include close enough to everything that it might as well be the whole set, in case I forgot something I find I need later. The SQL logs are *completely* different from the syslog setup. -- * Greg Smith [EMAIL

Re: [PATCHES] WIP patch - INSERT-able log statements

2007-02-20 Thread Greg Smith
ir logs probably has an installation that can support the overhead. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

  1   2   3   >