Re: [PATCH] Win32: Why explicitly futz with the file pointer?
On Sun, Oct 20, 2002 at 06:39:36PM -0500, William A. Rowe, Jr. wrote: > >I like the idea, but yeah, to have a global mutex on windows you have > >to give it a filename (so the non-related processes can rendezvous on > >the same mutex). > > Doesn't it make more sense to use a file lock instead of a mutex? > [They should be effectively the same construct, performance wise.] Umm...good point. :) -aaron
Re: [PATCH] Win32: Why explicitly futz with the file pointer?
At 12:46 PM 10/20/2002, Aaron Bannert wrote: >On Sun, Oct 20, 2002 at 11:30:35AM -0400, Jeff Trawick wrote: >> Maybe APR_APPEND needs to be cheap/simple append a la stdio append: we >> seek to the end of the file at open time and forget about it after >> that. >> >> Then we need new APR_WRITE_AT_END or something better named which is >> the expensive atomic append. For Unix, this enables O_APPEND on the >> file and the kernel handles the details. For Win32, this enables >> acquire-global-mutex + setfileptr + release-global-mutex prior to >> every write. But then that has issues with non-related processes >> sharing the mutex. > >I like the idea, but yeah, to have a global mutex on windows you have >to give it a filename (so the non-related processes can rendezvous on >the same mutex). Doesn't it make more sense to use a file lock instead of a mutex? [They should be effectively the same construct, performance wise.] Bill
Re: [PATCH] Win32: Why explicitly futz with the file pointer?
On 20 Oct 2002 11:30:35 -0400, Jeff Trawick wrote: >"William A. Rowe, Jr." <[EMAIL PROTECTED]> writes: > >> At 07:08 AM 10/20/2002, Bill Stoddard wrote: >> >Why do we need to call SetFilePointer to each call of apr_file_write()? In >> >the common case where only threads in a single process write to a file, >> >calling SetFilePointer is a waste of cycles. If threads from multiple >> >processes are writing to a file, then we are broken unless the application >> >explicitly serializes access to apr_file_write() >> >> I'll take a wild guess... >> >> Unix append mode -always- writes to the end of the file, even when it is >> in use by multiple processes. Win32 has no absolute, atomic append >> mode for it's file open. >> >> But you are correct, there is a race here that even the 'emulation' falls >> down. > >Maybe APR_APPEND needs to be cheap/simple append a la stdio append: we >seek to the end of the file at open time and forget about it after >that. > >Then we need new APR_WRITE_AT_END or something better named which is >the expensive atomic append. For Unix, this enables O_APPEND on the >file and the kernel handles the details. For Win32, this enables >acquire-global-mutex + setfileptr + release-global-mutex prior to >every write. But then that has issues with non-related processes >sharing the mutex. Or you can write-lock the file around the seek/write pair. That's what I ended up doing for OS/2 which has the same behaviour. The lock & seek could be avoided if the file was opened in "deny write" sharing mode but apr_file_open() doesn't have any way to specify that at the moment & it wouldn't help with logs shared between processes. -- __ | Brian Havard | "He is not the messiah! | | [EMAIL PROTECTED] | He's a very naughty boy!" - Life of Brian | --
Re: [PATCH] Win32: Why explicitly futz with the file pointer?
On Sun, 20 Oct 2002, Aaron Bannert wrote: > On Sun, Oct 20, 2002 at 11:30:35AM -0400, Jeff Trawick wrote: > > Maybe APR_APPEND needs to be cheap/simple append a la stdio append: we > > seek to the end of the file at open time and forget about it after > > that. APR_APPEND should always remain what it is right now. Namely, "Always write to the end of the file". There are a couple of reasons for that. 1) That is the POSIX definition of append, which is why APR_APPEND was chosen. This removes obstacles to writing APR code. 2) It is what the current code that uses APR expects. As for creating a new flag that opens a file and seeks to the end, do we really need that? I would much rather suggest to programmers who want that functionality to write the following code: apr_file_open(&fd,...); apr_file_seek(fd, 0, APR_FILE_END); I don't like continuing to add flags to functions. It makes the API harder to grasp quickly. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: [PATCH] Win32: Why explicitly futz with the file pointer?
On Sun, 20 Oct 2002, Bill Stoddard wrote: > Why do we need to call SetFilePointer to each call of apr_file_write()? In > the common case where only threads in a single process write to a file, > calling SetFilePointer is a waste of cycles. If threads from multiple > processes are writing to a file, then we are broken unless the application > explicitly serializes access to apr_file_write() We only make that call if it we opened the file in append mode. The rule for append is that all data is written to the end of the file. Unfortunately, Windows doesn't have this concept, so we have to hack it in. While the app may serialize access to the file, that won't do anything, unless the app also seeks to the end of the file. However, this is only a problem on Windows, which is why APR_APPEND is an option. So, IMNSHO, Windows is totally borked when it comes to having threads share write access to a file, and this is the best solution we have for fixing it. BTW, removing this will almost definately break the web server on Windows, if you ever get multiple processes serving data at the same time. As for having the app serialize and seek to the end of the file, that is generally a much larger waste of cycles for non-windows platforms. If you want to remove the cycles from your app, the best thing you can do, is to stop opening the file for append mode on Windows. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: [PATCH] Win32: Why explicitly futz with the file pointer?
Aaron Bannert <[EMAIL PROTECTED]> writes: > On Sun, Oct 20, 2002 at 11:30:35AM -0400, Jeff Trawick wrote: > > Maybe APR_APPEND needs to be cheap/simple append a la stdio append: we > > seek to the end of the file at open time and forget about it after > > that. > > > > Then we need new APR_WRITE_AT_END or something better named which is > > the expensive atomic append. For Unix, this enables O_APPEND on the > > file and the kernel handles the details. For Win32, this enables > > acquire-global-mutex + setfileptr + release-global-mutex prior to > > every write. But then that has issues with non-related processes > > sharing the mutex. > > I like the idea, but yeah, to have a global mutex on windows you have > to give it a filename (so the non-related processes can rendezvous on > the same mutex). at the very least (if not practical for APR to handle the mutex) we need to set APR_WRITE_AT_END_IS_ATOMIC0 or 1 depending on the platform characteristics so that apps know when to compile in code for a mutex. nasty... -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: [PATCH] Win32: Why explicitly futz with the file pointer?
On Sun, Oct 20, 2002 at 11:30:35AM -0400, Jeff Trawick wrote: > Maybe APR_APPEND needs to be cheap/simple append a la stdio append: we > seek to the end of the file at open time and forget about it after > that. > > Then we need new APR_WRITE_AT_END or something better named which is > the expensive atomic append. For Unix, this enables O_APPEND on the > file and the kernel handles the details. For Win32, this enables > acquire-global-mutex + setfileptr + release-global-mutex prior to > every write. But then that has issues with non-related processes > sharing the mutex. I like the idea, but yeah, to have a global mutex on windows you have to give it a filename (so the non-related processes can rendezvous on the same mutex). -aaron
Re: [PATCH] Win32: Why explicitly futz with the file pointer?
"William A. Rowe, Jr." <[EMAIL PROTECTED]> writes: > At 07:08 AM 10/20/2002, Bill Stoddard wrote: > >Why do we need to call SetFilePointer to each call of apr_file_write()? In > >the common case where only threads in a single process write to a file, > >calling SetFilePointer is a waste of cycles. If threads from multiple > >processes are writing to a file, then we are broken unless the application > >explicitly serializes access to apr_file_write() > > I'll take a wild guess... > > Unix append mode -always- writes to the end of the file, even when it is > in use by multiple processes. Win32 has no absolute, atomic append > mode for it's file open. > > But you are correct, there is a race here that even the 'emulation' falls > down. Maybe APR_APPEND needs to be cheap/simple append a la stdio append: we seek to the end of the file at open time and forget about it after that. Then we need new APR_WRITE_AT_END or something better named which is the expensive atomic append. For Unix, this enables O_APPEND on the file and the kernel handles the details. For Win32, this enables acquire-global-mutex + setfileptr + release-global-mutex prior to every write. But then that has issues with non-related processes sharing the mutex. -- Jeff Trawick | [EMAIL PROTECTED] Born in Roswell... married an alien...
Re: [PATCH] Win32: Why explicitly futz with the file pointer?
At 07:08 AM 10/20/2002, Bill Stoddard wrote: >Why do we need to call SetFilePointer to each call of apr_file_write()? In >the common case where only threads in a single process write to a file, >calling SetFilePointer is a waste of cycles. If threads from multiple >processes are writing to a file, then we are broken unless the application >explicitly serializes access to apr_file_write() I'll take a wild guess... Unix append mode -always- writes to the end of the file, even when it is in use by multiple processes. Win32 has no absolute, atomic append mode for it's file open. But you are correct, there is a race here that even the 'emulation' falls down.
