Re: msync() differences between Linux and FreeBSD

[Peter Jeremy]

On 2008-Jul-02 05:00:02 -0700, Marcus Reid [EMAIL PROTECTED] wrote:
  It seems that in FreeBSD, msync() waits for bits to be
committed to disk even when MS_ASYNC is specified.

Your previous ktrace output suggests that, at least for the way
rrdtool is using mmap(2), physical I/O is being performed by msync(2).
It's not clear whether FreeBSD is ignoring the MS_ASYNC flag (the code
suggests it isn't), is blocking on previously queued I/O or is blocking
for some other reason.

First off, I don't know how frequently msync() is used, and whether changing
its behavior would impact the performance of many things.

The behaviour of msync(2) is defined in the Single Unix Specification
and FreeBSD adheres to SUS unless there is a very good reason.

media... i.e. issue real I/O.  So msync() can't be a NOP if you go by
the OpenGroup specification.

Is there a spec that FreeBSD is adhering to that prevents msync() with
MS_ASYNC from being a NOP, seeing as munmap() does the job?

As per Matt's response that you quoted, yes.

  And does this
really matter for the real-world performance of some apps?

IMO, rrdtool is using mmap()/madvise()/msync()/munmap() in an unusual
fashion and it should be fixed, rather than changing FreeBSD to match
rrdtool.  I believe a more usual approach would be to mmap() a file
(or part thereof), optionally call madvise(), perform a series of
accesses and maybe msync()s of any updated regions then a single
munmap() before exiting.  Performing mmap()/msync()/munmap() (where
the msync() specifies the entire file) for each update maximises
system overheads for no obvious benefit.

Also, you mentioned hitting a brick wall between 940MB and 1161MB.
That straddles 1GB.  You may also be running into system or process
boundary conditions (how much RAM do you have and what tuning have you
done).  You might like to write a tool to simulate the rrdtool
behaviour with varying DB sizes and identify exactly what you are
hitting.

-- 
Peter Jeremy
Please excuse any delays as the result of my ISP's inability to implement
an MTA that is either RFC2821-compliant or matches their claimed behaviour.


pgpy7HVw3YMNp.pgp
Description: PGP signature

msync() differences between Linux and FreeBSD

[Marcus Reid]

Hi,

I recently had a patch accepted to the ports collection that took out an msync()
call that seriously detrimented performance for rrdtool updates.  It seems that
in FreeBSD, msync() waits for bits to be committed to disk even when MS_ASYNC
is specified.  Under Linux, there is not such a wait for msync().

First off, I don't know how frequently msync() is used, and whether changing
its behavior would impact the performance of many things.

It's possible that FreeBSD msync() behavior is more correct in some ways. From
a message from Matt Dillon on this same list:

It used to be that msync() only synced VM pages to the underlying
file, making them consistent with read()'s and write()'s against
the underlying file.  Since FreeBSD uses a unified VM page cache
this is always true.  However, the Open Group specification now
requires that the dirty pages actually be written out to the underlying
media... i.e. issue real I/O.  So msync() can't be a NOP if you go by
the OpenGroup specification.

Is there a spec that FreeBSD is adhering to that prevents msync() with
MS_ASYNC from being a NOP, seeing as munmap() does the job?  And does this
really matter for the real-world performance of some apps?

Thanks,

Marcus
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]