Added to TODO:
* Delay fsync() when other backends are about to commit too
[ Charset ISO-8859-1 unsupported, converting... ]
> > > BUT, do we know for sure that sleep(0) is not optimized in
> > > the library to just return?
> >
> > We can only do our best here. I think guessing wheth
At 07:05 PM 11/19/00 +0100, [EMAIL PROTECTED] wrote:
>Cam I ask what BAR is ?
Backup and recovery, presumably...
- Don Baccus, Portland OR <[EMAIL PROTECTED]>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.
> > > Ok, so with CHECKPOINTS, we could move the offline log files to
> > > somewhere else so that we could archive them, in my
> > > undertstanding. Now question is, how we could recover from disaster
> > > like losing every table files except log files. Can we do this with
> > > WAL? If so, how
"Mikheev, Vadim" wrote:
>
> > > > > No. Checkpoints are to speedup after crash recovery and
> > > > > to remove/archive log files. With WAL server doesn't write
> > > > > any datafiles on commit, only commit record goes to log
> > > > > (and log fsync-ed). Dirty buffers remains in memory long
> >
* Bruce Momjian <[EMAIL PROTECTED]> [001117 11:39]:
> > * Bruce Momjian <[EMAIL PROTECTED]> [001117 11:23]:
> > > > > sleep(3) should conform to POSIX specification, if anyone has the
> > > > > reference they can check it to see what the effect of sleep(0)
> > > > > should be.
> > > >
> > > > Y
* Bruce Momjian <[EMAIL PROTECTED]> [001117 11:23]:
> > > sleep(3) should conform to POSIX specification, if anyone has the
> > > reference they can check it to see what the effect of sleep(0)
> > > should be.
> >
> > Yes, but Posix also specifies sched_yield() which rather explicitly
> > allow
> > >> Ewe, so we have this 1/200 second delay for every transaction. Seems
> > >> bad to me.
> > >
> > >I think as long as it becomes a tunable this isn't a bad idea at
> > >all. Fixing it at 1/200 isn't so great because people not wrapping
> > >large amounts of inserts/updates with transactio
On Thu, 16 Nov 2000, Alfred Perlstein wrote:
> * Larry Rosenman <[EMAIL PROTECTED]> [001116 12:09] wrote:
> > * Bruce Momjian <[EMAIL PROTECTED]> [001116 14:02]:
> > > > > This sounds like an interesting approach, yes.
> > > > Question: Is sleep(0) guaranteed to at least give up control?
> > >
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes:
> A long ago you, Bruce, made me gift - book about transaction processing
> (thanks again -:)). This sleeping before fsync in commit is described
> there as standard technique. And the reason is cleanest.
> Men, cost of fsync is very high! { write (64 b
Alfred Perlstein <[EMAIL PROTECTED]> writes:
>> That's the hard way to do it. We just need to keep track of the
>> endpoint of the log as of the last fsync.
> Well that breaks when you move to a overwriting storage manager,
No, because the log is just a series of records written sequentially --
> > > > No. Checkpoints are to speedup after crash recovery and
> > > > to remove/archive log files. With WAL server doesn't write
> > > > any datafiles on commit, only commit record goes to log
> > > > (and log fsync-ed). Dirty buffers remains in memory long
>
> Ok, so with CHECKPOINTS, we could
> > You are going to kernel call/yield anyway to fsync, so why
> > not try and if someone does the fsync, we don't need to do it.
> > I am suggesting re-checking the need for fsync after the return
> > from sleep(0).
>
> It might make more sense to keep a private copy of the last time
> the file
* Bruce Momjian <[EMAIL PROTECTED]> [001116 12:31] wrote:
> > > In OS kernel design, you try to avoid process herding bottlenecks.
> > > Here, we want them herded, and giving up the CPU may be the best way to
> > > do it.
> >
> > Yes, but if everyone yeilds you're back where you started, and wit
* Tom Lane <[EMAIL PROTECTED]> [001116 13:31] wrote:
> Alfred Perlstein <[EMAIL PROTECTED]> writes:
> > It might make more sense to keep a private copy of the last time
> > the file was modified per-backend by that particular backend and
> > a timestamp of the last fsync shared globally so one can
Alfred Perlstein <[EMAIL PROTECTED]> writes:
> It might make more sense to keep a private copy of the last time
> the file was modified per-backend by that particular backend and
> a timestamp of the last fsync shared globally so one can forgo the
> fsync if "it hasn't been dirtied by me since the
> > BUT, do we know for sure that sleep(0) is not optimized in
> > the library to just return?
>
> We can only do our best here. I think guessing whether other backends
> are _about_ to commit is pretty shaky, and sleeping every time is a
> waste. This seems the cleanest.
A long ago you, Bruc
Bruce Momjian writes:
> > The way I read my UnixWare 7's man page, it might not, since alarm(0)
> > just cancels the alarm...
>
> Well, it certainly is a kernel call, and most OS's re-evaluate on kernel
> call return.
In glibc, sleep(0) just does "return 0;", so if the compiler has a good
day t
* Larry Rosenman <[EMAIL PROTECTED]> [001116 12:09] wrote:
> * Bruce Momjian <[EMAIL PROTECTED]> [001116 14:02]:
> > > > This sounds like an interesting approach, yes.
> > > Question: Is sleep(0) guaranteed to at least give up control?
> > >
> > > The way I read my UnixWare 7's man page, it migh
> > In OS kernel design, you try to avoid process herding bottlenecks.
> > Here, we want them herded, and giving up the CPU may be the best way to
> > do it.
>
> Yes, but if everyone yeilds you're back where you started, and with
> 128 or more backends do you really want to cause possibly that m
* Bruce Momjian <[EMAIL PROTECTED]> [001116 11:59] wrote:
> > At 02:13 PM 11/16/00 -0500, Bruce Momjian wrote:
> >
> > >> I think the default should probably be no delay, and the documentation
> > >> on enabling this needs to be clear and obvious (i.e. hard to miss).
> > >
> > >I just talked to T
* Bruce Momjian <[EMAIL PROTECTED]> [001116 14:02]:
> > > This sounds like an interesting approach, yes.
> > Question: Is sleep(0) guaranteed to at least give up control?
> >
> > The way I read my UnixWare 7's man page, it might not, since alarm(0)
> > just cancels the alarm...
>
> Well, it cer
> At 02:13 PM 11/16/00 -0500, Bruce Momjian wrote:
>
> >> I think the default should probably be no delay, and the documentation
> >> on enabling this needs to be clear and obvious (i.e. hard to miss).
> >
> >I just talked to Tom Lane about this. I think a sleep(0) just before
> >the flush would
* Don Baccus <[EMAIL PROTECTED]> [001116 13:46]:
> At 02:13 PM 11/16/00 -0500, Bruce Momjian wrote:
>
> >> I think the default should probably be no delay, and the documentation
> >> on enabling this needs to be clear and obvious (i.e. hard to miss).
> >
> >I just talked to Tom Lane about this.
At 02:13 PM 11/16/00 -0500, Bruce Momjian wrote:
>> I think the default should probably be no delay, and the documentation
>> on enabling this needs to be clear and obvious (i.e. hard to miss).
>
>I just talked to Tom Lane about this. I think a sleep(0) just before
>the flush would be the best.
> At 09:32 AM 11/16/00 -0800, Alfred Perlstein wrote:
> >* Bruce Momjian <[EMAIL PROTECTED]> [001116 08:59] wrote:
>
> >> Ewe, so we have this 1/200 second delay for every transaction. Seems
> >> bad to me.
> >
> >I think as long as it becomes a tunable this isn't a bad idea at
> >all. Fixing i
At 09:32 AM 11/16/00 -0800, Alfred Perlstein wrote:
>* Bruce Momjian <[EMAIL PROTECTED]> [001116 08:59] wrote:
>> Ewe, so we have this 1/200 second delay for every transaction. Seems
>> bad to me.
>
>I think as long as it becomes a tunable this isn't a bad idea at
>all. Fixing it at 1/200 isn't
* Bruce Momjian <[EMAIL PROTECTED]> [001116 08:59] wrote:
> [ Charset ISO-8859-1 unsupported, converting... ]
> > > > > Earlier, Vadim was talking about arranging to share fsyncs of the WAL
> > > > > log file across transactions (after writing your commit record to the
> > > > > log, sleep a few m
> > Earlier, Vadim was talking about arranging to share fsyncs of the WAL
> > log file across transactions (after writing your commit record to the
> > log, sleep a few milliseconds to see if anyone else fsyncs before you
> > do; if not, issue the fsync yourself). That would offer less-than-
> >
> > > Earlier, Vadim was talking about arranging to share fsyncs of the WAL
> > > log file across transactions (after writing your commit record to the
> > > log, sleep a few milliseconds to see if anyone else fsyncs before you
> > > do; if not, issue the fsync yourself). That would offer less-th
> > I am just suggesting that instead of flushing the log on every
> > transaction end, just do it every X seconds.
>
> Or maybe more practical is, when the log buffer fills.
> And of course during checkpoints.
Also before backend's going to write dirty buffer from pool
to system cache - change
> Earlier, Vadim was talking about arranging to share fsyncs of the WAL
> log file across transactions (after writing your commit record to the
> log, sleep a few milliseconds to see if anyone else fsyncs before you
> do; if not, issue the fsync yourself). That would offer less-than-
> one-fsync-
[ Charset ISO-8859-1 unsupported, converting... ]
>
> > > Or maybe more practical is, when the log buffer fills.
> > > And of course during checkpoints.
> >
> > Log filling is too abritrary. If I commit something, and nothing
> > happens for 2 hours, we should commit that transaction.
>
> Che
> > Or maybe more practical is, when the log buffer fills.
> > And of course during checkpoints.
>
> Log filling is too abritrary. If I commit something, and nothing
> happens for 2 hours, we should commit that transaction.
Checkpoint, as I said ?
Andreas
[ Charset ISO-8859-1 unsupported, converting... ]
>
> > I am just suggesting that instead of flushing the log on every
> > transaction end, just do it every X seconds.
>
> Or maybe more practical is, when the log buffer fills.
> And of course during checkpoints.
Log filling is too abritrary.
> I am just suggesting that instead of flushing the log on every
> transaction end, just do it every X seconds.
Or maybe more practical is, when the log buffer fills.
And of course during checkpoints.
Andreas
> I have to agree with Alfred here: this does not sound like a feature,
> it sounds like a horrid hack. You're giving up *all* consistency
> guarantees for a performance gain that is really going to be pretty
> minimal in the WAL context.
The "buffered log" still guarantees consistency within t
* Tom Lane <[EMAIL PROTECTED]> [00 12:06] wrote:
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> >> I have to agree with Alfred here: this does not sound like a feature,
> >> it sounds like a horrid hack. You're giving up *all* consistency
> >> guarantees for a performance gain that is really g
Bruce Momjian <[EMAIL PROTECTED]> writes:
>> I have to agree with Alfred here: this does not sound like a feature,
>> it sounds like a horrid hack. You're giving up *all* consistency
>> guarantees for a performance gain that is really going to be pretty
>> minimal in the WAL context.
> It does n
Bruce Momjian <[EMAIL PROTECTED]> writes:
>> Not really, I thought an ack on a commit would mean that the data
>> is actually in stable storage, breaking that would be pretty bad
>> no?
> The default is to sync on commit, but we need to give people options of
> several seconds delay for performan
* Bruce Momjian <[EMAIL PROTECTED]> [00 00:16] wrote:
> > * Tatsuo Ishii <[EMAIL PROTECTED]> [001110 18:42] wrote:
> > > >
> > > > Yes, though we can change this. We also can implement now
> > > > feature that Bruce wanted so long and so much -:) -
> > > > fsync log not on each commit but eac
> * Tatsuo Ishii <[EMAIL PROTECTED]> [001110 18:42] wrote:
> > >
> > > Yes, though we can change this. We also can implement now
> > > feature that Bruce wanted so long and so much -:) -
> > > fsync log not on each commit but each ~ 5sec, if
> > > losing some recent commits is acceptable.
> >
>
* Tatsuo Ishii <[EMAIL PROTECTED]> [001110 18:42] wrote:
> >
> > Yes, though we can change this. We also can implement now
> > feature that Bruce wanted so long and so much -:) -
> > fsync log not on each commit but each ~ 5sec, if
> > losing some recent commits is acceptable.
>
> Sounds great.
> > Can you tell me how to use CHECKPOINT please?
>
> You shouldn't normally use it - postmaster will start backend
> each 3-5 minutes to do this automatically.
Oh, I see.
> > > > Is this the same as a SAVEPOINT?
> > >
> > > No. Checkpoints are to speedup after crash recovery and
> > > to remo
> > > > New CHECKPOINT command.
> > > > Auto removing of offline log files and creating new file
> > > > at checkpoint time.
>
> Can you tell me how to use CHECKPOINT please?
You shouldn't normally use it - postmaster will start backend
each 3-5 minutes to do this automatically.
> > > Is this t
> > > New CHECKPOINT command.
> > > Auto removing of offline log files and creating new file
> > > at checkpoint time.
Can you tell me how to use CHECKPOINT please?
> > Is this the same as a SAVEPOINT?
>
> No. Checkpoints are to speedup after crash recovery and
> to remove/archive log files. Wi
> > New CHECKPOINT command.
> > Auto removing of offline log files and creating new file
> > at checkpoint time.
>
> Is this the same as a SAVEPOINT?
No. Checkpoints are to speedup after crash recovery and
to remove/archive log files. With WAL server doesn't write
any datafiles on commit, only c
[EMAIL PROTECTED] writes:
> New CHECKPOINT command.
> Auto removing of offline log files and creating new file
> at checkpoint time.
Is this the same as a SAVEPOINT?
--
Peter Eisentraut [EMAIL PROTECTED] http://yi.org/peter-e/
> We seem to be missing a file "src/include/access/xlogutils.h".
Ops, sorry - please create empty file, it will work
(I have no its src at office comp -:().
Vadim
48 matches
Mail list logo
