Simon Riggs wrote:
>>old dead tuple list. If the system manages the dead tuple list we may
>>need to keep such files around for long periods, which doesn't sound
>>great either.
The system manages such files. The files are kept in location like
$PGDATA/pg_vacuum. They are removed when CLUSTER, DR
"Joshua D. Drake" <[EMAIL PROTECTED]> writes:
> Correct me if I am wrong, but would the way that HOT has been handled be
> a good way for the SoC people to do things?
Well, the HOT discussion hasn't yet led to an accepted patch ... and I'd
say its authors still did way too much work before getting
On Mon, Feb 26, 2007 at 06:23:22PM -0500, Matthew T. O'Connor wrote:
> Alvaro Herrera wrote:
> >Matthew T. O'Connor wrote:
> >>How can you determine what tables can be vacuumed within
> >>autovacuum_naptime?
> >
> >My assumption is that
> >pg_class.relpages * vacuum_cost_page_miss * vacuum_cost_de
Jim C. Nasby wrote:
> The advantage to keying this to autovac_naptime is that it means we
> don't need another GUC, but after I suggested that before I realized
> that's probably not the best idea. For example, I've seen clusters that
> are running dozens-hundreds of databases; in that environment
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> On Mon, Feb 26, 2007 at 09:22:42PM -0500, Tom Lane wrote:
>> I'm not liking any of these very much, as they seem critically dependent
>> on impossible-to-tune parameters. I think it'd be better to design this
>> around having the first worker explicitly
[ oh, I forgot to respond to this: ]
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> Isn't there a special lock acquired on a relation by vacuum? Can't we
> just check for that?
I think you're thinking that ConditionalLockRelation solves the problem,
but it does not, because it will fail if someone
Tom Lane wrote:
> I think an absolute minimum requirement for a sane design is that no two
> workers ever try to vacuum the same table concurrently, and I don't see
> where that behavior will emerge from your proposal; whereas it's fairly
> easy to make it happen if non-first workers pay attention
This is the second proposal for Dead Space Map (DSM).
Here is the previous discussion:
http://archives.postgresql.org/pgsql-hackers/2006-12/msg01188.php
I'll post the next version of the Dead Space Map patch to -patches.
I've implemented 2bits/page bitmap and new vacuum commands.
Memory management
Jeff Davis wrote:
> On Mon, 2007-02-26 at 22:56 +, Simon Riggs wrote:
> > Proposal: Implement a new option for COMMIT, for enhancing performance,
> > providing a MySQL-like trade-off between performance and robustness for
> > *only* those that want it.
> >
> > COMMIT NOWAIT
> >
> > Th
On Mon, Feb 26, 2007 at 02:57:03PM -0500, Andrew Dunstan wrote:
> Robert Treat wrote:
> >FWIW ClearCase also offers a command line version of its merge tool, where
> >it shows three columns (a la diff --side-by-side) and allows you to pick
> >which column you want to merge in (repo, change1, or c
Galy Lee <[EMAIL PROTECTED]> writes:
> For example, there is one table:
>- The table is a hundreds GBs table.
>- It takes 4-8 hours to vacuum such a large table.
>- Enabling cost-based delay may make it last for 24 hours.
>- It can be vacuumed during night time for 2-4 hours.
> It
Jim C. Nasby wrote:
On Mon, Feb 26, 2007 at 06:23:22PM -0500, Matthew T. O'Connor wrote:
I'm not sure how pg_class.relpages is maintained but what happens to a
bloated table? For example, a 100 row table that is constantly updated
and hasn't been vacuumed in a while (say the admin disabled aut
Alvaro Herrera <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> I think an absolute minimum requirement for a sane design is that no two
>> workers ever try to vacuum the same table concurrently,
> FWIW, I've always considered this to be a very important and obvious
> issue, and I think I've negle
Tom Lane wrote:
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
The real problem is trying to set that up in such a fashion that keeps
hot tables frequently vacuumed;
Are we assuming that no single worker instance will vacuum a given table
more than once? (That's not a necessary assumption, certai
On Mon, 2007-02-26 at 18:14 -0500, Tom Lane wrote:
> "Simon Riggs" <[EMAIL PROTECTED]> writes:
> > A prototype patch is posted to -patches, which is WORK IN PROGRESS.
> > [This patch matches discussion thread on -hackers.]
>
> What does this accomplish other than adding syntactic sugar over a
> fe
Josh Berkus wrote:
> Can I have a copy of the patch to add to the Sun testing queue?
This is the revised version of the patch. Delay factors in checkpoints
can be specified by checkpoint_write_percent, checkpoint_nap_percent
and checkpoint_sync_percent. They are relative to checkpoint_timeout.
Tom Lane wrote:
BTW, to what extent might this whole problem be simplified if we adopt
chunk-at-a-time vacuuming (compare current discussion with Galy Lee)?
If the unit of work has a reasonable upper bound regardless of table
size, maybe the problem of big tables starving small ones goes away.
"Matthew T. O'Connor" writes:
> Tom Lane wrote:
>> I'm inclined to propose an even simpler algorithm in which every worker
>> acts alike;
> That is what I'm proposing except for one difference, when you catch up
> to an older worker, exit.
No, that's a bad idea, because it means that any large
Hi,
I didn't see this on the TODO list, but if it is my apologies. Is it
in the cards to expand the functionality of DELETE/UPDATE returning
to be able to sort the output of the rows returned? Or allow delete
and update to be used in sub-queries?
Some examples:
create temp table t1 (id
"Matthew T. O'Connor" writes:
> That does sounds simpler. Is chunk-at-a-time a realistic option for 8.3?
It seems fairly trivial to me to have a scheme where you do one
fill-workmem-and-scan-indexes cycle per invocation, and store the
next-heap-page-to-scan in some handy place (new pg_class colum
Tom Lane wrote:
"Matthew T. O'Connor" writes:
Tom Lane wrote:
I'm inclined to propose an even simpler algorithm in which every worker
acts alike;
That is what I'm proposing except for one difference, when you catch up
to an older worker, exit.
No, that's a bad idea, because it means that
Tom Lane wrote:
"Matthew T. O'Connor" writes:
That does sounds simpler. Is chunk-at-a-time a realistic option for 8.3?
It seems fairly trivial to me to have a scheme where you do one
fill-workmem-and-scan-indexes cycle per invocation, and store the
next-heap-page-to-scan in some handy place (
"Simon Riggs" <[EMAIL PROTECTED]> writes:
> COMMIT NOWAIT can co-exist with the normal form of COMMIT and does not
> threaten the consistency or robustness of other COMMIT modes. Read that
> again and think about it, before we go further, please.
I read that, and thought about it, and don't think
On Tue, February 27, 2007 06:06, Joshua D. Drake wrote:
>
>> Why do we want this?? Because some apps have *lots* of data and many
>> really don't care whether they lose a few records. Honestly, I've met
>> people that want this, even after 2 hours of discussion and
>> understanding. Plus probably l
"Simon Riggs" <[EMAIL PROTECTED]> writes:
> On Mon, 2007-02-26 at 18:14 -0500, Tom Lane wrote:
>> What does this accomplish other than adding syntactic sugar over a
>> feature that really doesn't work well anyway?
> This patch doesn't intend to implement group commit. I've changed the
> meaning of
Rusty Conover <[EMAIL PROTECTED]> writes:
> I didn't see this on the TODO list, but if it is my apologies. Is it
> in the cards to expand the functionality of DELETE/UPDATE returning
> to be able to sort the output of the rows returned?
No.
> Or allow delete
> and update to be used in sub-
On Mon, Feb 26, 2007 at 09:10:38PM -0500, Tom Lane wrote:
> Andrew Dunstan <[EMAIL PROTECTED]> writes:
> > Well, here's a question. Given the recent discussion re full
> > disjunction, I'd like to know what sort of commitment we are going to
> > give people who work on proposed projects.
>
> Um,
On Mon, Feb 26, 2007 at 10:56:58PM +, Simon Riggs wrote:
> 2. remove fsync parameter
Why? Wouldn't fsync=off still speed up checkpoints? ISTM you'd still
want this for things like database restores.
--
Jim Nasby[EMAIL PROTECTED]
EnterpriseDB ht
On Tue, Feb 27, 2007 at 12:05:57PM +0900, ITAGAKI Takahiro wrote:
> Each heap pages have 4 states for dead space map; HIGH, LOW, UNFROZEN and
> FROZEN. VACUUM uses the states to reduce the number of target pages.
>
> - HIGH : High priority to vacuum. Maybe many dead tuples in the page.
> - LOW
On Mon, Feb 26, 2007 at 11:14:01PM -0500, Tom Lane wrote:
> Rusty Conover <[EMAIL PROTECTED]> writes:
> > I didn't see this on the TODO list, but if it is my apologies. Is
> > it in the cards to expand the functionality of DELETE/UPDATE
> > returning to be able to sort the output of the rows ret
On Mon, Feb 26, 2007 at 11:14:01PM -0500, Tom Lane wrote:
> Rusty Conover <[EMAIL PROTECTED]> writes:
> > I didn't see this on the TODO list, but if it is my apologies. Is it
> > in the cards to expand the functionality of DELETE/UPDATE returning
> > to be able to sort the output of the rows r
On Tue, Feb 27, 2007 at 12:00:41AM -0300, Alvaro Herrera wrote:
> Jim C. Nasby wrote:
>
> > The advantage to keying this to autovac_naptime is that it means we
> > don't need another GUC, but after I suggested that before I realized
> > that's probably not the best idea. For example, I've seen clu
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> On Mon, Feb 26, 2007 at 11:14:01PM -0500, Tom Lane wrote:
>> Rusty Conover <[EMAIL PROTECTED]> writes:
>>> Or allow delete and update to be used in sub-queries?
>>
>> That's been discussed but the implementation effort seems far from
>> trivial. One bi
On Mon, Feb 26, 2007 at 10:48:49PM -0500, Tom Lane wrote:
> "Matthew T. O'Connor" writes:
> > That does sounds simpler. Is chunk-at-a-time a realistic option for 8.3?
>
> It seems fairly trivial to me to have a scheme where you do one
> fill-workmem-and-scan-indexes cycle per invocation, and stor
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> The proposal to save enough state to be able to resume a vacuum at
> pretty much any point in it's cycle might work; we'd have to benchmark
> it. With the default maintenance_work_mem of 128M it would mean writing
> out 64M of state every minute on aver
On Tue, Feb 27, 2007 at 11:05:45AM +0700, Jeroen T. Vermeulen wrote:
> On Tue, February 27, 2007 06:06, Joshua D. Drake wrote:
> >
> >> Why do we want this?? Because some apps have *lots* of data and many
> >> really don't care whether they lose a few records. Honestly, I've met
> >> people that wa
On Mon, Feb 26, 2007 at 10:18:36PM -0500, Matthew T. O'Connor wrote:
> Jim C. Nasby wrote:
> >On Mon, Feb 26, 2007 at 06:23:22PM -0500, Matthew T. O'Connor wrote:
> >>I'm not sure how pg_class.relpages is maintained but what happens to a
> >>bloated table? For example, a 100 row table that is con
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> Yes, but the list being discussed is SoC projects that the community
> would like to see done, which means most people would assume that #1
> isn't an issue.
> We need to make sure that every project on the list of SoC ideas is
> supported by the commun
On Tue, Feb 27, 2007 at 12:37:42AM -0500, Tom Lane wrote:
> "Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> > The proposal to save enough state to be able to resume a vacuum at
> > pretty much any point in it's cycle might work; we'd have to benchmark
> > it. With the default maintenance_work_mem of
Jim C. Nasby wrote:
On Mon, Feb 26, 2007 at 10:18:36PM -0500, Matthew T. O'Connor wrote:
Jim C. Nasby wrote:
Here is a worst case example: A DB with 6 tables all of which are highly
active and will need to be vacuumed constantly. While this is totally
hypothetical, it is how I envision things
Tom Lane wrote:
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
The proposal to save enough state to be able to resume a vacuum at
pretty much any point in it's cycle might work; we'd have to benchmark
it. With the default maintenance_work_mem of 128M it would mean writing
out 64M of state every min
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> Yes, DSM would make FSM recovery more important, but I thought it was
> recoverable now? Or is that only on a clean shutdown?
Currently we throw away FSM during any non-clean restart. This is
probably overkill but I'm quite unclear what would be a safe
"Matthew T. O'Connor" writes:
> I'm not sure what you are saying here, are you now saying that partial
> vacuum won't work for autovac? Or are you saying that saving state as
> Jim is describing above won't work?
I'm saying that I don't like the idea of trying to "stop on a dime" by
saving the
Tom Lane wrote:
"Matthew T. O'Connor" writes:
I'm not sure what you are saying here, are you now saying that partial
vacuum won't work for autovac? Or are you saying that saving state as
Jim is describing above won't work?
I'm saying that I don't like the idea of trying to "stop on a dime"
On 2/26/07, ITAGAKI Takahiro <[EMAIL PROTECTED]> wrote:
Josh Berkus wrote:
> Can I have a copy of the patch to add to the Sun testing queue?
This is the revised version of the patch. Delay factors in checkpoints
can be specified by checkpoint_write_percent, checkpoint_nap_percent
and checkpoi
On Mon, 2007-02-26 at 22:50 -0600, Jim C. Nasby wrote:
> On Mon, Feb 26, 2007 at 10:56:58PM +, Simon Riggs wrote:
> > 2. remove fsync parameter
>
> Why? Wouldn't fsync=off still speed up checkpoints? ISTM you'd still
> want this for things like database restores.
Well, it seemed to be part of
On Monday 26 February 2007 10:13, Bruce Momjian wrote:
> Warren Turkal wrote:
> > On Saturday 24 February 2007 16:47, Chad Wagner wrote:
> > > head pgsql/src/interfaces/perl5/Attic/test.pl.oldstyle,v
> > > head ? ?1.3;
> > > access;
> > > symbols
> > > ? ? ? ? Release-1-6-0:1.1.1.1
> > > ? ? ? ? cr
On Mon, 2007-02-26 at 21:20 -0300, Alvaro Herrera wrote:
> Simon Riggs wrote:
>
> > The interesting point is you can have a huge data grinding app, yet with
> > other tables alongside that hold more important data. In that scenario,
> > 90% of the data would be COMMIT NOWAIT, whilst the small impo
On Tue, 2007-02-27 at 12:05 +0900, ITAGAKI Takahiro wrote:
> I think it's better to get DSM and HOT together. DSM is good at
> complex updated cases but not at heavily updated cases. HOT has
> opposite aspects, as far as I can see. I think they can cover each
> other.
Very much agreed.
I'll be a
101 - 149 of 149 matches
Mail list logo
