Tom Lane wrote:
> "Curtis Faith" <[EMAIL PROTECTED]> writes:
> > After some research I still hold that fsync blocks, at least on
> > FreeBSD. Am I missing something?
>
> > Here's the evidence:
> > [ much snipped ]
> > vp = (struct vnode *)fp->f_data;
> > vn_lock(vp, LK_EXC
"Curtis Faith" <[EMAIL PROTECTED]> writes:
> After some research I still hold that fsync blocks, at least on
> FreeBSD. Am I missing something?
> Here's the evidence:
> [ much snipped ]
> vp = (struct vnode *)fp->f_data;
> vn_lock(vp, LK_EXCLUSIVE | LK_RETRY, p);
Hm, I ta
I resent this since it didn't seem to get to the list.
After some research I still hold that fsync blocks, at least on
FreeBSD. Am I missing something?
Here's the evidence:
Code from: /usr/src/sys/syscalls/vfs_syscalls
int
fsync(p, uap)
struct proc *p;
struct fsync_args /* {
Neil Conway <[EMAIL PROTECTED]> writes:
> "Curtis Faith" <[EMAIL PROTECTED]> writes:
>> It looks to me like BufferAlloc will simply result in a call to
>> BufferReplace > smgrblindwrt > write for md storage manager objects.
>>
>> This means that a process will block while the write of dirty cache
Curtis Faith writes:
> I'm no Unix filesystem expert but I don't see how the OS can handle
> multiple writes and fsyncs to the same file descriptors without
> blocking other processes from writing at the same time.
Why not? Other than the necessary synchronisation for attributes such
as file s
On Fri, 2002-10-04 at 18:03, Neil Conway wrote:
> "Curtis Faith" <[EMAIL PROTECTED]> writes:
> > It looks to me like BufferAlloc will simply result in a call to
> > BufferReplace > smgrblindwrt > write for md storage manager objects.
> >
> > This means that a process will block while the write of
> Hmmm ... if you were willing to dedicate a half meg or meg of shared
> memory for WAL buffers, that's doable.
Yup, configuring Informix to three 2 Mb buffers (LOGBUF 2048) here.
> However, this would only be a win if you had few and large transactions.
> Any COMMIT will force a write of what
"Curtis Faith" <[EMAIL PROTECTED]> writes:
> It looks to me like BufferAlloc will simply result in a call to
> BufferReplace > smgrblindwrt > write for md storage manager objects.
>
> This means that a process will block while the write of dirty cache
> buffers takes place.
I think Tom was sugge
I wrote:
> > ... most file systems can't process fsync's
> > simultaneous with other writes, so those writes block because the file
> > system grabs its own internal locks.
>
tom lane replies:
> Oh? That would be a serious problem, but I've never heard that asserted
> before. Please provide som
"Zeugswetter Andreas SB SD" <[EMAIL PROTECTED]> writes:
> To make this competitive, the WAL writes would need to be improved to
> do more than one block (up to 256k or 512k per write) with one write call
> (if that much is to be written for this tx to be able to commit).
> This should actually n
> > ... most file systems can't process fsync's
> > simultaneous with other writes, so those writes block because the file
> > system grabs its own internal locks.
>
> Oh? That would be a serious problem, but I've never heard that asserted
> before. Please provide some evidence.
>
> On a file
"Curtis Faith" <[EMAIL PROTECTED]> writes:
> ... most file systems can't process fsync's
> simultaneous with other writes, so those writes block because the file
> system grabs its own internal locks.
Oh? That would be a serious problem, but I've never heard that asserted
before. Please provide
Bruce Momjian wrote:
> I am again confused. When we do write(), we don't have to lock
> anything, do we? (Multiple processes can write() to the same file just
> fine.) We do block the current process, but we have nothing else to do
> until we know it is written/fsync'ed. Does aio more easily
Curtis Faith wrote:
> > Yes, I can see some contention, but what does aio solve?
> >
>
> Well, theoretically, aio lets the file system handle the writes without
> requiring any locks being held by the processes issuing those reads.
> The disk i/o scheduler can therefore issue the writes using s
I wrote:
> > I'm no Unix filesystem expert but I don't see how the OS can
> > handle multiple writes and fsyncs to the same file descriptors without
> > blocking other processes from writing at the same time. It may be that
> > there are some clever data structures they use but I've not seen huge
Curtis Faith wrote:
> Bruce Momjian wrote:
> > I may be missing something here, but other backends don't block while
> > one writes to WAL.
>
> I don't think they'll block until they get to the fsync or XLogWrite
> call while another transaction is fsync'ing.
>
> I'm no Unix filesystem expert bu
I wrote:
> > The REAL issue and the one that will greatly affect total system
> > throughput is that of contention on the file locks. Since
> fsynch needs to
> > obtain a write lock on the file descriptor, as does the write
> calls which
> > originate from XLogWrite as the writes are written to th
]
> Sent: Friday, October 04, 2002 12:44 AM
> To: Curtis Faith
> Cc: Tom Lane; Pgsql-Hackers
> Subject: Re: [HACKERS] Potential Large Performance Gain in WAL synching
>
>
> Curtis Faith wrote:
> > The method I propose does not result in any blocking because of writes
"Curtis Faith" <[EMAIL PROTECTED]> writes:
> The REAL issue and the one that will greatly affect total system
> throughput is that of contention on the file locks. Since fsynch needs to
> obtain a write lock on the file descriptor, as does the write calls which
> originate from XLogWrite as the wr
Curtis Faith wrote:
> The method I propose does not result in any blocking because of writes
> other than the final commit's write and it has the very significant
> advantage of allowing other transactions (from other back-ends) to
> continue until they enter commit (and blocking waiting for their
r's huge
database test by at least a factor of two or three, perhaps even an
order of magnitude. :-)
> -Original Message-
> From: Tom Lane [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, October 03, 2002 7:17 PM
> To: Curtis Faith
> Cc: Pgsql-Hackers
> Subject: Re: [HACKERS] Pot
"Curtis Faith" <[EMAIL PROTECTED]> writes:
> So, why don't we use files opened with O_DSYNC | O_APPEND for the WAL log
> and then use aio_write for all log writes?
We already offer an O_DSYNC option. It's not obvious to me what
aio_write brings to the table (aside from loss of portability).
You
I've been looking at the TODO lists and caching issues and think there may
be a way to greatly improve the performance of the WAL.
I've made the following assumptions based on my reading in the manual and
the WAL archives since about November 2000:
1) WAL is currently fsync'd before commit succe
23 matches
Mail list logo
