Re: [HACKERS] Dead Space Map version 2

2007-02-26 Thread Simon Riggs
On Tue, 2007-02-27 at 12:05 +0900, ITAGAKI Takahiro wrote: > I think it's better to get DSM and HOT together. DSM is good at > complex updated cases but not at heavily updated cases. HOT has > opposite aspects, as far as I can see. I think they can cover each > other. Very much agreed. I'll be a

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Simon Riggs
On Mon, 2007-02-26 at 21:20 -0300, Alvaro Herrera wrote: > Simon Riggs wrote: > > > The interesting point is you can have a huge data grinding app, yet with > > other tables alongside that hold more important data. In that scenario, > > 90% of the data would be COMMIT NOWAIT, whilst the small impo

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-26 Thread Warren Turkal
On Monday 26 February 2007 10:13, Bruce Momjian wrote: > Warren Turkal wrote: > > On Saturday 24 February 2007 16:47, Chad Wagner wrote: > > > head pgsql/src/interfaces/perl5/Attic/test.pl.oldstyle,v > > > head ? ?1.3; > > > access; > > > symbols > > > ? ? ? ? Release-1-6-0:1.1.1.1 > > > ? ? ? ? cr

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Simon Riggs
On Mon, 2007-02-26 at 22:50 -0600, Jim C. Nasby wrote: > On Mon, Feb 26, 2007 at 10:56:58PM +, Simon Riggs wrote: > > 2. remove fsync parameter > > Why? Wouldn't fsync=off still speed up checkpoints? ISTM you'd still > want this for things like database restores. Well, it seemed to be part of

Re: [PATCHES] [HACKERS] Load distributed checkpoint

2007-02-26 Thread Inaam Rana
On 2/26/07, ITAGAKI Takahiro <[EMAIL PROTECTED]> wrote: Josh Berkus wrote: > Can I have a copy of the patch to add to the Sun testing queue? This is the revised version of the patch. Delay factors in checkpoints can be specified by checkpoint_write_percent, checkpoint_nap_percent and checkpoi

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" writes: I'm not sure what you are saying here, are you now saying that partial vacuum won't work for autovac? Or are you saying that saving state as Jim is describing above won't work? I'm saying that I don't like the idea of trying to "stop on a dime"

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Tom Lane
"Matthew T. O'Connor" writes: > I'm not sure what you are saying here, are you now saying that partial > vacuum won't work for autovac? Or are you saying that saving state as > Jim is describing above won't work? I'm saying that I don't like the idea of trying to "stop on a dime" by saving the

Re: [HACKERS] Dead Space Map version 2

2007-02-26 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Yes, DSM would make FSM recovery more important, but I thought it was > recoverable now? Or is that only on a clean shutdown? Currently we throw away FSM during any non-clean restart. This is probably overkill but I'm quite unclear what would be a safe

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Jim C. Nasby" <[EMAIL PROTECTED]> writes: The proposal to save enough state to be able to resume a vacuum at pretty much any point in it's cycle might work; we'd have to benchmark it. With the default maintenance_work_mem of 128M it would mean writing out 64M of state every min

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Jim C. Nasby wrote: On Mon, Feb 26, 2007 at 10:18:36PM -0500, Matthew T. O'Connor wrote: Jim C. Nasby wrote: Here is a worst case example: A DB with 6 tables all of which are highly active and will need to be vacuumed constantly. While this is totally hypothetical, it is how I envision things

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 12:37:42AM -0500, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > The proposal to save enough state to be able to resume a vacuum at > > pretty much any point in it's cycle might work; we'd have to benchmark > > it. With the default maintenance_work_mem of

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Yes, but the list being discussed is SoC projects that the community > would like to see done, which means most people would assume that #1 > isn't an issue. > We need to make sure that every project on the list of SoC ideas is > supported by the commun

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Jim C. Nasby
On Mon, Feb 26, 2007 at 10:18:36PM -0500, Matthew T. O'Connor wrote: > Jim C. Nasby wrote: > >On Mon, Feb 26, 2007 at 06:23:22PM -0500, Matthew T. O'Connor wrote: > >>I'm not sure how pg_class.relpages is maintained but what happens to a > >>bloated table? For example, a 100 row table that is con

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 11:05:45AM +0700, Jeroen T. Vermeulen wrote: > On Tue, February 27, 2007 06:06, Joshua D. Drake wrote: > > > >> Why do we want this?? Because some apps have *lots* of data and many > >> really don't care whether they lose a few records. Honestly, I've met > >> people that wa

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > The proposal to save enough state to be able to resume a vacuum at > pretty much any point in it's cycle might work; we'd have to benchmark > it. With the default maintenance_work_mem of 128M it would mean writing > out 64M of state every minute on aver

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Jim C. Nasby
On Mon, Feb 26, 2007 at 10:48:49PM -0500, Tom Lane wrote: > "Matthew T. O'Connor" writes: > > That does sounds simpler. Is chunk-at-a-time a realistic option for 8.3? > > It seems fairly trivial to me to have a scheme where you do one > fill-workmem-and-scan-indexes cycle per invocation, and stor

Re: [HACKERS] Expanding DELETE/UPDATE returning

2007-02-26 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > On Mon, Feb 26, 2007 at 11:14:01PM -0500, Tom Lane wrote: >> Rusty Conover <[EMAIL PROTECTED]> writes: >>> Or allow delete and update to be used in sub-queries? >> >> That's been discussed but the implementation effort seems far from >> trivial. One bi

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 12:00:41AM -0300, Alvaro Herrera wrote: > Jim C. Nasby wrote: > > > The advantage to keying this to autovac_naptime is that it means we > > don't need another GUC, but after I suggested that before I realized > > that's probably not the best idea. For example, I've seen clu

Re: [HACKERS] Expanding DELETE/UPDATE returning

2007-02-26 Thread Jim C. Nasby
On Mon, Feb 26, 2007 at 11:14:01PM -0500, Tom Lane wrote: > Rusty Conover <[EMAIL PROTECTED]> writes: > > I didn't see this on the TODO list, but if it is my apologies. Is it > > in the cards to expand the functionality of DELETE/UPDATE returning > > to be able to sort the output of the rows r

Re: [HACKERS] Expanding DELETE/UPDATE returning

2007-02-26 Thread David Fetter
On Mon, Feb 26, 2007 at 11:14:01PM -0500, Tom Lane wrote: > Rusty Conover <[EMAIL PROTECTED]> writes: > > I didn't see this on the TODO list, but if it is my apologies. Is > > it in the cards to expand the functionality of DELETE/UPDATE > > returning to be able to sort the output of the rows ret

Re: [HACKERS] Dead Space Map version 2

2007-02-26 Thread Jim C. Nasby
On Tue, Feb 27, 2007 at 12:05:57PM +0900, ITAGAKI Takahiro wrote: > Each heap pages have 4 states for dead space map; HIGH, LOW, UNFROZEN and > FROZEN. VACUUM uses the states to reduce the number of target pages. > > - HIGH : High priority to vacuum. Maybe many dead tuples in the page. > - LOW

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Jim C. Nasby
On Mon, Feb 26, 2007 at 10:56:58PM +, Simon Riggs wrote: > 2. remove fsync parameter Why? Wouldn't fsync=off still speed up checkpoints? ISTM you'd still want this for things like database restores. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB ht

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Jim C. Nasby
On Mon, Feb 26, 2007 at 09:10:38PM -0500, Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > Well, here's a question. Given the recent discussion re full > > disjunction, I'd like to know what sort of commitment we are going to > > give people who work on proposed projects. > > Um,

Re: [HACKERS] Expanding DELETE/UPDATE returning

2007-02-26 Thread Tom Lane
Rusty Conover <[EMAIL PROTECTED]> writes: > I didn't see this on the TODO list, but if it is my apologies. Is it > in the cards to expand the functionality of DELETE/UPDATE returning > to be able to sort the output of the rows returned? No. > Or allow delete > and update to be used in sub-

Re: [HACKERS] [PATCHES] COMMIT NOWAIT Performance Option (patch)

2007-02-26 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > On Mon, 2007-02-26 at 18:14 -0500, Tom Lane wrote: >> What does this accomplish other than adding syntactic sugar over a >> feature that really doesn't work well anyway? > This patch doesn't intend to implement group commit. I've changed the > meaning of

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Jeroen T. Vermeulen
On Tue, February 27, 2007 06:06, Joshua D. Drake wrote: > >> Why do we want this?? Because some apps have *lots* of data and many >> really don't care whether they lose a few records. Honestly, I've met >> people that want this, even after 2 hours of discussion and >> understanding. Plus probably l

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > COMMIT NOWAIT can co-exist with the normal form of COMMIT and does not > threaten the consistency or robustness of other COMMIT modes. Read that > again and think about it, before we go further, please. I read that, and thought about it, and don't think

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" writes: That does sounds simpler. Is chunk-at-a-time a realistic option for 8.3? It seems fairly trivial to me to have a scheme where you do one fill-workmem-and-scan-indexes cycle per invocation, and store the next-heap-page-to-scan in some handy place (

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Matthew T. O'Connor" writes: Tom Lane wrote: I'm inclined to propose an even simpler algorithm in which every worker acts alike; That is what I'm proposing except for one difference, when you catch up to an older worker, exit. No, that's a bad idea, because it means that

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Tom Lane
"Matthew T. O'Connor" writes: > That does sounds simpler. Is chunk-at-a-time a realistic option for 8.3? It seems fairly trivial to me to have a scheme where you do one fill-workmem-and-scan-indexes cycle per invocation, and store the next-heap-page-to-scan in some handy place (new pg_class colum

[HACKERS] Expanding DELETE/UPDATE returning

2007-02-26 Thread Rusty Conover
Hi, I didn't see this on the TODO list, but if it is my apologies. Is it in the cards to expand the functionality of DELETE/UPDATE returning to be able to sort the output of the rows returned? Or allow delete and update to be used in sub-queries? Some examples: create temp table t1 (id

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Tom Lane
"Matthew T. O'Connor" writes: > Tom Lane wrote: >> I'm inclined to propose an even simpler algorithm in which every worker >> acts alike; > That is what I'm proposing except for one difference, when you catch up > to an older worker, exit. No, that's a bad idea, because it means that any large

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: BTW, to what extent might this whole problem be simplified if we adopt chunk-at-a-time vacuuming (compare current discussion with Galy Lee)? If the unit of work has a reasonable upper bound regardless of table size, maybe the problem of big tables starving small ones goes away.

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: "Jim C. Nasby" <[EMAIL PROTECTED]> writes: The real problem is trying to set that up in such a fashion that keeps hot tables frequently vacuumed; Are we assuming that no single worker instance will vacuum a given table more than once? (That's not a necessary assumption, certai

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I think an absolute minimum requirement for a sane design is that no two >> workers ever try to vacuum the same table concurrently, > FWIW, I've always considered this to be a very important and obvious > issue, and I think I've negle

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Jim C. Nasby wrote: On Mon, Feb 26, 2007 at 06:23:22PM -0500, Matthew T. O'Connor wrote: I'm not sure how pg_class.relpages is maintained but what happens to a bloated table? For example, a 100 row table that is constantly updated and hasn't been vacuumed in a while (say the admin disabled aut

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-26 Thread Tom Lane
Galy Lee <[EMAIL PROTECTED]> writes: > For example, there is one table: >- The table is a hundreds GBs table. >- It takes 4-8 hours to vacuum such a large table. >- Enabling cost-based delay may make it last for 24 hours. >- It can be vacuumed during night time for 2-4 hours. > It

Re: [HACKERS] SCMS question

2007-02-26 Thread mark
On Mon, Feb 26, 2007 at 02:57:03PM -0500, Andrew Dunstan wrote: > Robert Treat wrote: > >FWIW ClearCase also offers a command line version of its merge tool, where > >it shows three columns (a la diff --side-by-side) and allows you to pick > >which column you want to merge in (repo, change1, or c

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Alvaro Herrera
Jeff Davis wrote: > On Mon, 2007-02-26 at 22:56 +, Simon Riggs wrote: > > Proposal: Implement a new option for COMMIT, for enhancing performance, > > providing a MySQL-like trade-off between performance and robustness for > > *only* those that want it. > > > > COMMIT NOWAIT > > > > Th

[HACKERS] Dead Space Map version 2

2007-02-26 Thread ITAGAKI Takahiro
This is the second proposal for Dead Space Map (DSM). Here is the previous discussion: http://archives.postgresql.org/pgsql-hackers/2006-12/msg01188.php I'll post the next version of the Dead Space Map patch to -patches. I've implemented 2bits/page bitmap and new vacuum commands. Memory management

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Alvaro Herrera
Tom Lane wrote: > I think an absolute minimum requirement for a sane design is that no two > workers ever try to vacuum the same table concurrently, and I don't see > where that behavior will emerge from your proposal; whereas it's fairly > easy to make it happen if non-first workers pay attention

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Tom Lane
[ oh, I forgot to respond to this: ] "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Isn't there a special lock acquired on a relation by vacuum? Can't we > just check for that? I think you're thinking that ConditionalLockRelation solves the problem, but it does not, because it will fail if someone

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > On Mon, Feb 26, 2007 at 09:22:42PM -0500, Tom Lane wrote: >> I'm not liking any of these very much, as they seem critically dependent >> on impossible-to-tune parameters. I think it'd be better to design this >> around having the first worker explicitly

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Alvaro Herrera
Jim C. Nasby wrote: > The advantage to keying this to autovac_naptime is that it means we > don't need another GUC, but after I suggested that before I realized > that's probably not the best idea. For example, I've seen clusters that > are running dozens-hundreds of databases; in that environment

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Jim C. Nasby
On Mon, Feb 26, 2007 at 06:23:22PM -0500, Matthew T. O'Connor wrote: > Alvaro Herrera wrote: > >Matthew T. O'Connor wrote: > >>How can you determine what tables can be vacuumed within > >>autovacuum_naptime? > > > >My assumption is that > >pg_class.relpages * vacuum_cost_page_miss * vacuum_cost_de

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Tom Lane
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > Correct me if I am wrong, but would the way that HOT has been handled be > a good way for the SoC people to do things? Well, the HOT discussion hasn't yet led to an accepted patch ... and I'd say its authors still did way too much work before getting

Re: [HACKERS] Resumable vacuum proposal and design overview

2007-02-26 Thread Galy Lee
Simon Riggs wrote: >>old dead tuple list. If the system manages the dead tuple list we may >>need to keep such files around for long periods, which doesn't sound >>great either. The system manages such files. The files are kept in location like $PGDATA/pg_vacuum. They are removed when CLUSTER, DR

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Jeff Davis
On Mon, 2007-02-26 at 22:56 +, Simon Riggs wrote: > Proposal: Implement a new option for COMMIT, for enhancing performance, > providing a MySQL-like trade-off between performance and robustness for > *only* those that want it. > > COMMIT NOWAIT > > This form of COMMIT will *not* perfo

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Jim C. Nasby
On Mon, Feb 26, 2007 at 08:11:44PM -0300, Alvaro Herrera wrote: > Matthew T. O'Connor wrote: > > Alvaro Herrera wrote: > > > >The second mode is the "hot table worker" mode, enabled when the worker > > >detects that there's already a worker in the database. In this mode, > > >the worker is limite

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Jim C. Nasby
On Mon, Feb 26, 2007 at 09:22:42PM -0500, Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Matthew T. O'Connor wrote: > >> I'm not sure it's a good idea to tie this to the vacuum cost delay > >> settings either, so let me as you this, how is this better than just > >> allowing the

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: Matthew T. O'Connor wrote: I'm not sure it's a good idea to tie this to the vacuum cost delay settings either, so let me as you this, how is this better than just allowing the admin to set a new GUC variable like autovacuum_hot_table_

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Joshua D. Drake
Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> writes: >> Well, here's a question. Given the recent discussion re full >> disjunction, I'd like to know what sort of commitment we are going to >> give people who work on proposed projects. > > Um, if you mean are we going to promise to accep

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Matthew T. O'Connor wrote: >> I'm not sure it's a good idea to tie this to the vacuum cost delay >> settings either, so let me as you this, how is this better than just >> allowing the admin to set a new GUC variable like >> autovacuum_hot_table_size_

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Matthew T. O'Connor wrote: I'm not sure how pg_class.relpages is maintained but what happens to a bloated table? For example, a 100 row table that is constantly updated and hasn't been vacuumed in a while (say the admin disabled autovacuum for a while), now that small 10

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Well, here's a question. Given the recent discussion re full > disjunction, I'd like to know what sort of commitment we are going to > give people who work on proposed projects. Um, if you mean are we going to promise to accept a patch in advance of s

Re: [HACKERS] Load distributed checkpoint

2007-02-26 Thread ITAGAKI Takahiro
Josh Berkus wrote: > Can I have a copy of the patch to add to the Sun testing queue? This is the revised version of the patch. Delay factors in checkpoints can be specified by checkpoint_write_percent, checkpoint_nap_percent and checkpoint_sync_percent. They are relative to checkpoint_timeout.

Re: [HACKERS] Load distributed checkpoint

2007-02-26 Thread Josh Berkus
Itagaki, > Thank you for testing! Yes, I'm cleaning the patch. I changed > configuration parameters to delay each phase in checkpoints from setting > absolute times (checkpoint_xxx_duration) to setting relative to > checkpoint_timeout (checkpoint_xxx_percent). Delay factors strongly > depend on to

Re: [HACKERS] Proposal for Implenting read-only queries during wal replay (SoC 2007)

2007-02-26 Thread Merlin Moncure
getting back on topic (ahem), florian: are you comfortable going ahead with this? is there anything you need help with? merlin ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [HACKERS] Load distributed checkpoint

2007-02-26 Thread ITAGAKI Takahiro
"Inaam Rana" <[EMAIL PROTECTED]> wrote: > Did you had a chance to look into this any further? We, at EnterpriseDB, > have done some testing on this patch (dbt2 runs) and it looks like we are > getting the desired results, particularly so when we spread out both sync > and write phases. Thank you

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread A.M.
On Feb 26, 2007, at 18:58 , Simon Riggs wrote: On Mon, 2007-02-26 at 23:25 +, Richard Huxton wrote: Simon Riggs wrote: Proposal: Implement a new option for COMMIT, for enhancing performance, providing a MySQL-like trade-off between performance and robustness for *only* those that want

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Alvaro Herrera
Simon Riggs wrote: > The interesting point is you can have a huge data grinding app, yet with > other tables alongside that hold more important data. In that scenario, > 90% of the data would be COMMIT NOWAIT, whilst the small important data > is safe. Does this means that the regular COMMIT is s

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Alvaro Herrera
Matthew T. O'Connor wrote: > Alvaro Herrera wrote: > >Matthew T. O'Connor wrote: > >>How can you determine what tables can be vacuumed within > >>autovacuum_naptime? > > > >My assumption is that > >pg_class.relpages * vacuum_cost_page_miss * vacuum_cost_delay = time to > >vacuum > > > >This is of

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Simon Riggs
On Mon, 2007-02-26 at 23:25 +, Richard Huxton wrote: > Simon Riggs wrote: > > Proposal: Implement a new option for COMMIT, for enhancing performance, > > providing a MySQL-like trade-off between performance and robustness for > > *only* those that want it. > > > > COMMIT NOWAIT > > >

Re: [HACKERS] [PATCHES] COMMIT NOWAIT Performance Option (patch)

2007-02-26 Thread Simon Riggs
On Mon, 2007-02-26 at 18:14 -0500, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > A prototype patch is posted to -patches, which is WORK IN PROGRESS. > > [This patch matches discussion thread on -hackers.] > > What does this accomplish other than adding syntactic sugar over a > fe

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Josh Berkus
Demian, > Could you also please share your thoughts on what would be a good > student profile- for instance, how much theoretical background and > practical experience, for working on a SoC project? Well, it shouldn't be the student's first year writing code. Basically, they're committing to de

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Richard Huxton
Simon Riggs wrote: Proposal: Implement a new option for COMMIT, for enhancing performance, providing a MySQL-like trade-off between performance and robustness for *only* those that want it. COMMIT NOWAIT This form of COMMIT will *not* perform XLogFlush(), but will rely on a special back

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Matthew T. O'Connor wrote: How can you determine what tables can be vacuumed within autovacuum_naptime? My assumption is that pg_class.relpages * vacuum_cost_page_miss * vacuum_cost_delay = time to vacuum This is of course not the reality, because the delay is not how lo

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Demian Lessa
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Could you also please share your thoughts on what would be a good student profile- for instance, how much theoretical background and practical experience, for working on a SoC project? Demian > The next Summer of Code is just around the corner. > >

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Alvaro Herrera
Matthew T. O'Connor wrote: > Alvaro Herrera wrote: > >The second mode is the "hot table worker" mode, enabled when the worker > >detects that there's already a worker in the database. In this mode, > >the worker is limited to those tables that can be vacuumed in less than > >autovacuum_naptime, s

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-26 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > Reading through the design, I see the following > - bgwriter performs XLogWrite, not each backend > - WAL fsync is only performed when WAL file fills > - no checkpoints are performed until shutdown > Not checkpointing at all is not a good plan, since th

Re: [HACKERS] Bitmap index stuff

2007-02-26 Thread Gavin Sherry
On Mon, 26 Feb 2007, Heikki Linnakangas wrote: > Hi, > > How are you doing with the bitmap indexes? I need to send of a patch fixing the last bug you pointed out. The code needs a merge of HEAD. > > I've been trying to get my head around the patch a couple of times to > add the vacuum support, b

Re: [HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Joshua D. Drake
> Why do we want this?? Because some apps have *lots* of data and many > really don't care whether they lose a few records. Honestly, I've met > people that want this, even after 2 hours of discussion and > understanding. Plus probably lots of MySQLers also. Most users will take speed over data l

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Andrew Dunstan
Warren Turkal wrote: On Monday 26 February 2007 14:22, Andrew Dunstan wrote: Is there a list of projects? Or can we suggest some? Temporal database support * base data types - verify current for ISO compliance (timestamp, interval) - implement period datatype * operators for those

[HACKERS] COMMIT NOWAIT Performance Option

2007-02-26 Thread Simon Riggs
Proposal: Implement a new option for COMMIT, for enhancing performance, providing a MySQL-like trade-off between performance and robustness for *only* those that want it. COMMIT NOWAIT This form of COMMIT will *not* perform XLogFlush(), but will rely on a special background process to per

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Warren Turkal
On Monday 26 February 2007 14:22, Andrew Dunstan wrote: > Is there a list of projects? Or can we suggest some? Temporal database support * base data types - verify current for ISO compliance (timestamp, interval) - implement period datatype * operators for those datatypes * add support for val

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread markwkm
On 2/26/07, Josh Berkus wrote: Andrew, > Is there a list of projects? Or can we suggest some? Suggest away, please! I'm going to update the website soon, would appreciate new content. I can also volunteer to mentor continuing work on a TPC-E kit, for C stored procedures and improved results

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Andrew Dunstan
Josh Berkus wrote: Andrew, Is there a list of projects? Or can we suggest some? Suggest away, please! I'm going to update the website soon, would appreciate new content. here are a few ideas to be going on with (none of these are new): buildfarm: 1. Buildfarm web app: is in u

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread markwkm
On 2/26/07, Andrew Dunstan <[EMAIL PROTECTED]> wrote: Josh Berkus wrote: > The next Summer of Code is just around the corner. > > Last year, we had 46 submissions and seven we accepted. Out of the SoC we got > two ongoing contributors, several good patches, two code refactors and even > an emplo

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Josh Berkus
Andrew, > Is there a list of projects? Or can we suggest some? Suggest away, please! I'm going to update the website soon, would appreciate new content. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco ---(end of broadcast)--- TIP 2: Don't

Re: [HACKERS] [Monotone-devel] Re: SCMS question

2007-02-26 Thread Warren Turkal
On Monday 26 February 2007 13:50, Robert Treat wrote: > It's worth keeping in mind that one of the primary reasons we don't have a > different usage pattern is because CVS makes such a thing painful.  Given > how much of development is done now, I have a feeling that the community > might well adop

Re: [HACKERS] Acclerating INSERT/UPDATE using UPS

2007-02-26 Thread Simon Riggs
On Mon, 2007-02-26 at 09:44 -0500, Bruce Momjian wrote: > Joshua D. Drake wrote: > > Christopher Browne wrote: > > > A long time ago, in a galaxy far, far away, [EMAIL PROTECTED] (Hideyuki > > > Kawashima) wrote: > > >> I appreciate your great suggestion! > > >> It is great honor for me if Sigres

Re: [HACKERS] Proposal for Implenting read-only queries during wal replay (SoC 2007)

2007-02-26 Thread Joshua D. Drake
Jonah H. Harris wrote: > On 2/26/07, Bruce Momjian <[EMAIL PROTECTED]> wrote: >> Jonah, I have no idea what "fault" you are trying to blame on the >> community in the above statement. The author didn't discuss the idea >> with the community before spending months on it so we have no obligation >>

Re: [HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Andrew Dunstan
Josh Berkus wrote: The next Summer of Code is just around the corner. Last year, we had 46 submissions and seven we accepted. Out of the SoC we got two ongoing contributors, several good patches, two code refactors and even an employee for a PostgreSQL company. I'd like to see us do the same

Re: [HACKERS] [Monotone-devel] Re: SCMS question

2007-02-26 Thread Robert Treat
On Sunday 25 February 2007 01:11, Tom Lane wrote: > "Joshua D. Drake" <[EMAIL PROTECTED]> writes: > > Lastly, who really cares? Does it really matter? No. I would much rather > > Warren (if he has the skills) put some effort into Patch Review. > > That's pretty much the bottom line. CVS is not so

Re: [HACKERS] SCMS question

2007-02-26 Thread Robert Treat
On Monday 26 February 2007 14:57, Andrew Dunstan wrote: > Robert Treat wrote: > > FWIW ClearCase also offers a command line version of its merge tool, > > where it shows three columns (a la diff --side-by-side) and allows you to > > pick which column you want to merge in (repo, change1, or change2

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Matthew T. O'Connor
Alvaro Herrera wrote: Jim C. Nasby wrote: That's why I'm thinking it would be best to keep the maximum size of stuff for the second worker small. It probably also makes sense to tie it to time and not size, since the key factor is that you want it to hit the high-update tables every X number of

[HACKERS] Seeking Google SoC Mentors

2007-02-26 Thread Josh Berkus
The next Summer of Code is just around the corner. Last year, we had 46 submissions and seven we accepted. Out of the SoC we got two ongoing contributors, several good patches, two code refactors and even an employee for a PostgreSQL company. I'd like to see us do the same this year! Therefo

Re: [HACKERS] Proposal for Implenting read-only queries during wal replay (SoC 2007)

2007-02-26 Thread Jonah H. Harris
On 2/26/07, Bruce Momjian <[EMAIL PROTECTED]> wrote: Jonah, I have no idea what "fault" you are trying to blame on the community in the above statement. The author didn't discuss the idea with the community before spending months on it so we have no obligation to accept it in the core. You're

Re: [HACKERS] autovacuum next steps, take 2

2007-02-26 Thread Alvaro Herrera
Jim C. Nasby wrote: > That's why I'm thinking it would be best to keep the maximum size of > stuff for the second worker small. It probably also makes sense to tie > it to time and not size, since the key factor is that you want it to hit > the high-update tables every X number of seconds. > > If

Re: [HACKERS] Simple Column reordering

2007-02-26 Thread Andrew Dunstan
Josh Berkus wrote: For my part, I continue to the interested in this proposal and would like to see some performance benchmarks on it. If there is enough performance gain, I think it would be possible to implement a "logical" order which was different from the "physical" order. Such a feature

Re: [HACKERS] Developer TODO List as a PostgreSQL DB

2007-02-26 Thread Josh Berkus
Gottfried, > Just wondering after reading so many mails from Hackers List.(its 2.15 AM > now!!) Is there anybody working on something to create a DB from > a) The TODO list http://www.postgresql.org/docs/faqs.TODO.html > b) The sourcecode of PostgreSQL > c) The relevant Mailings from the developer

Re: [HACKERS] Simple Column reordering

2007-02-26 Thread Josh Berkus
Bruce, > True, but usually I don't see the breakage. What concerned me is you > saw some of the breakage, but still went ahead with the proposal. That's completely unfair, Bruce. This is a *discussion list*, and hackers are free to propose and discuss even far-out improbable ideas in the hopes

Re: [HACKERS] Compile libpq for pg8.2.3 with vc7

2007-02-26 Thread Jeff McKenna
Please ignore this question. I was compiling with the /src/interfaces/libpq/win32.mak file. I later noticed that the proper way to compile libpq is to use the /src/win32.mak file. thanks. jeff Jeff McKenna wrote: Trying to compile 8.2.3 with VC 7.10.3077 (2003) on Win32, I get the follo

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-02-26 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > This is not the first GUC that has needed this. Exactly. I think that we simply made a mistake in the initial implementation of log_min_error_statement: we failed to think about whether it should use client or server priority ordering, and the easy-to-c

Re: [HACKERS] Simple Column reordering

2007-02-26 Thread Bruce Momjian
Simon Riggs wrote: > > > wondered why that proposal had been overlooked, so I started a separate > > > thread to ensure that the idea was discussed. That seems very similar to > > > many of your own posts. > > > > True, but usually I don't see the breakage. > > Sorry, I just meant you summarise i

Re: [PATCHES] [HACKERS] Deadlock with pg_dump?

2007-02-26 Thread Bruce Momjian
Simon Riggs wrote: > On Mon, 2007-02-26 at 13:34 -0500, Bruce Momjian wrote: > > > I am a little concerned about a log_* setting that is INFO. I understand > > why you used INFO (for log_min_error_messages), but INFO is inconsistent > > with the log* prefix, and by default INFO doesn't appear in t

Re: [HACKERS] Deadlock with pg_dump?

2007-02-26 Thread Simon Riggs
On Mon, 2007-02-26 at 14:52 -0500, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > On Mon, 2007-02-26 at 14:28 -0500, Tom Lane wrote: > >> "Simon Riggs" <[EMAIL PROTECTED]> writes: > >>> The idea of the patch is that it generates a log message which then > >>> invokes log_min_error_

Re: conversion efforts (Re: [HACKERS] SCMS question)

2007-02-26 Thread Joshua D. Drake
Alvaro Herrera wrote: > Joshua D. Drake wrote: >> Joshua D. Drake wrote: > I imagine the problems are caused by manual mangling of the files in the > early days, like the perl5 dir stuff you found. Hmm, if you only checked using the Trac interface, maybe this is an issue with re-c

Re: [HACKERS] SCMS question

2007-02-26 Thread Andrew Dunstan
Robert Treat wrote: FWIW ClearCase also offers a command line version of its merge tool, where it shows three columns (a la diff --side-by-side) and allows you to pick which column you want to merge in (repo, change1, or change2 for example). It's a nice attempt at doing it on the command li

Re: [HACKERS] Simple Column reordering

2007-02-26 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > The order of the columns is *arbitrary* in relational theory; SQL is very far from being relational theory... regards, tom lane ---(end of broadcast)--- TIP 7: You can help support

  1   2   >