Re: [HACKERS] DB Tuning Notes for comment...

2002-12-10 Thread Philip Warner
At 03:54 PM 9/12/2002 -0500, Tom Lane wrote: I have some uncommitted patches concerning the FSM management heuristics from Stephen Marshall, which I deemed too late/risky for 7.3, but we should get something done for 7.4. Anyone interested in playing around in this area? I'd be interested in se

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Philip Warner
At 09:10 PM 9/12/2002 -0500, Robert Treat wrote: Even this little bit would be a step in the right direction. What I would find really useful is a 'VACUUM...WITH HISORY' which wrote the underlying details of VACUUM VERBOSE to a 'pg_vacuum_history' table. -

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Robert Treat
On Mon, 09 Dec 2002 19:10:23 -0500, Tom Lane wrote: > Philip Warner <[EMAIL PROTECTED]> writes: >> I think it would be worth looking at removing max_fsm_tables as a >> tuning option, and adding a 'relhasfsm' flag to pg_class for those >> tables that should not be mapped. Default to 't'. Then, make

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Philip Warner
At 08:39 PM 9/12/2002 -0500, Tom Lane wrote: A VACUUM done just after startup does not have any historical info to base this decision on. The actual order is: start delete vacuum; insert <- does not use free space vacuum; insert <- does not use free space vacuum; vacuum; insert <- uses free spa

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Scott Shattuck
Tom Lane wrote: Scott Shattuck <[EMAIL PROTECTED]> writes: Willing to learn here but skipping a vacuum full has caused some issues for us. Here's some data from a recent 3 day test run that was done with regular vacuums but not vacuum fulls. When running with vacuum full the indexes remain in

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: > Secondly, an empty database contains 98 tables, so the default setting of > max_fsm_pages to 100 is way too low. Only 37 of them need FSM entries, but still a good point; we should probably bump it up to 1000 to be more realistic. > oddly (bug? edge be

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Philip Warner
At 12:17 PM 10/12/2002 +1100, Philip Warner wrote: Secondly, an empty database contains 98 tables, Corrected based on Tom's later mail; from the FSM PoV, it contains 37 (indices don't count). So it is exhausted when more than two DBs are created. -

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Philip Warner
At 07:01 PM 9/12/2002 -0500, Tom Lane wrote: We could make the constraint be on total space for relation entries + page entries rather than either individually, but I think that'd mostly make it harder to interpret the config setting rather than offer any real ease of administration. Perhaps doi

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: > At 03:54 PM 9/12/2002 -0500, Tom Lane wrote: >> FSM entries aren't needed for sequences either, so more correct is >> >> select count(*) from pg_class where relkind in ('r', 't'); > presumably: > select count(*) from pg_class where relkind in ('r

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Philip Warner
At 03:54 PM 9/12/2002 -0500, Tom Lane wrote: FSM entries aren't needed for sequences either, so more correct is select count(*) from pg_class where relkind in ('r', 't'); presumably: select count(*) from pg_class where relkind in ('r', 't', 'i');

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: > I think it would be worth looking at removing max_fsm_tables as a tuning > option, and adding a 'relhasfsm' flag to pg_class for those tables that > should not be mapped. Default to 't'. Then, make the table grow dynamically > as tables are added, or w

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Philip Warner
At 02:46 PM 9/12/2002 -0500, Robert Treat wrote: getting this info into the standard documentation and/or Bruce's tuning guide. I'd vote for the standard docs since it is sufficiently basic as to be needed by most users. We either need a tuning chapter or a new section in runtime configuration

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Philip Warner
At 03:54 PM 9/12/2002 -0500, Tom Lane wrote: However, I suspect that the present FSM code is not very effective at deciding *which* tables to track if it has too few slots, You are definitely right there. I think it would be worth looking at removing max_fsm_tables as a tuning option, and addi

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Tom Lane
Scott Shattuck <[EMAIL PROTECTED]> writes: > Willing to learn here but skipping a vacuum full has caused some issues > for us. Here's some data from a recent 3 day test run that was done with > regular vacuums but not vacuum fulls. When running with vacuum full the > indexes remain in line: > n

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Scott Shattuck
Tom Lane wrote: Scott Shattuck <[EMAIL PROTECTED]> writes: Robert Treat wrote: I don't think this is entirely true. On tables that have large numbers of inserts, but no updates or deletes, you do not need to run vacuum. In my experience I've seen tables with numerous indexes continue to

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Tom Lane
Scott Shattuck <[EMAIL PROTECTED]> writes: > Robert Treat wrote: >> I don't think this is entirely true. On tables that have large numbers >> of inserts, but no updates or deletes, you do not need to run vacuum. > In my experience I've seen tables with numerous indexes continue to > benefit great

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Tom Lane
Robert Treat <[EMAIL PROTECTED]> writes: > On Sun, 2002-12-08 at 09:41, Philip Warner wrote: >> First of all, the free space manager is useless at managing free space if >> it can not map all relations (including system relations and toast >> relations). The following query should give the correc

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Scott Shattuck
Robert Treat wrote: On Sun, 2002-12-08 at 09:41, Philip Warner wrote: Any comments or suggestions would be welcome. first and foremost, this is really excellent work! We need to look into getting this info into the standard documentation and/or Bruce's tuning guide. Seconded! Tuning =

Re: [HACKERS] DB Tuning Notes for comment...

2002-12-09 Thread Robert Treat
On Sun, 2002-12-08 at 09:41, Philip Warner wrote: > Any comments or suggestions would be welcome. > first and foremost, this is really excellent work! We need to look into getting this info into the standard documentation and/or Bruce's tuning guide. > > Tuning > == > > 1. max_fsm_relation

[HACKERS] DB Tuning Notes for comment...

2002-12-08 Thread Philip Warner
The notes below are the results of various tuning issues experienced recently on a large database (several GB) that has many tables and a high transient data flow (ie. thousands of records added, updated, and deleted every hour) on a few tables. This kind of data flow is not at all well handle