Re: How to open file with exclusive lock?
On 07/12/2016 03:54 PM, H. S. Teoh via Digitalmars-d-learn wrote: On Tue, Jul 12, 2016 at 03:40:36PM -0700, Charles Hixson via Digitalmars-d-learn wrote: [...] OK. It's not possible without OS support. Agreed. And I don't want to get into C calls, but rather to use the mechanisms that D provides. And this *probably* won't cause any problems. But how should I tell D that that's the way I want to access to work? It's not clear to me that locking beyond the current EOF is valid, or whether it will decide that it needs to reserve the space before I use it (in which case locking from 0->ulong.max bytes is a bad idea, and would totally fill my hard disk). Here's another approach. Since it's almost a given that we're talking about cooperating processes sharing an advisory lock here (since a global mandatory file lock seems to have poor support, at least on Posix), we don't *have* to lock the actual bytes we're writing to. We can simply reserve a couple of bytes (perhaps even just one byte) at the beginning of the file, and agree among all processes that we will always try to acquire lock on that byte before writing to (the rest of) the file. Essentially it would serve as a file equivalent of a mutex. Then wrap all your file primitives in a little wrapper that acquires this mutex before performing file I/O, have your program only do I/O through this wrapper, and you're all set. (IIRC, this is how the SQLite3 library implements database read/write locks. Each sqlite3 database file has a few bytes, IIRC 6 bytes, at a known file offset, that it uses to control various levels of access to the database. Depending on which level of locking was needed, it would acquire a lock on one or more of these bytes, before accessing the rest of the file. It doesn't actually acquire a lock on the bytes being read / modified.) T That's probably the most viable approach. It does mean there's a lag between file open and file lock (unless I was to mess around with concurrency locks), but it's quite unlikely that anyone else will access the file while I have it open. (Well, possibly baloo or nepomuk or some such, but that's probably harmless.)
Re: How to open file with exclusive lock?
On Tue, Jul 12, 2016 at 03:40:36PM -0700, Charles Hixson via Digitalmars-d-learn wrote: [...] > OK. It's not possible without OS support. Agreed. And I don't want > to get into C calls, but rather to use the mechanisms that D provides. > And this *probably* won't cause any problems. But how should I tell D > that that's the way I want to access to work? It's not clear to me > that locking beyond the current EOF is valid, or whether it will > decide that it needs to reserve the space before I use it (in which > case locking from 0->ulong.max bytes is a bad idea, and would totally > fill my hard disk). Here's another approach. Since it's almost a given that we're talking about cooperating processes sharing an advisory lock here (since a global mandatory file lock seems to have poor support, at least on Posix), we don't *have* to lock the actual bytes we're writing to. We can simply reserve a couple of bytes (perhaps even just one byte) at the beginning of the file, and agree among all processes that we will always try to acquire lock on that byte before writing to (the rest of) the file. Essentially it would serve as a file equivalent of a mutex. Then wrap all your file primitives in a little wrapper that acquires this mutex before performing file I/O, have your program only do I/O through this wrapper, and you're all set. (IIRC, this is how the SQLite3 library implements database read/write locks. Each sqlite3 database file has a few bytes, IIRC 6 bytes, at a known file offset, that it uses to control various levels of access to the database. Depending on which level of locking was needed, it would acquire a lock on one or more of these bytes, before accessing the rest of the file. It doesn't actually acquire a lock on the bytes being read / modified.) T -- If you're not part of the solution, you're part of the precipitate.
Re: How to open file with exclusive lock?
On 07/12/2016 12:05 PM, H. S. Teoh via Digitalmars-d-learn wrote: On Tue, Jul 12, 2016 at 11:54:18AM -0700, Charles Hixson via Digitalmars-d-learn wrote: I want to open a file with an exclusive lock. It would be important that no other thread be able to access the file in write mode, and desirable that no other thread be able to access the file in read mode. (Ditto for other processes.) stdio.file.lock (or is it stdio.file.File.lock?) seems to demand a range of chunks to lock, but the file is not fixed in length. Is it appropriate to just specify the maximum possible number of bytes (i.e. ulong.max)? Whether this is even possible depends on your OS. I don't know of any cross-platform way of file-locking that is guaranteed to work everywhere. In Posix there's fcntl(F_RDLCK / F_WRLCK), which I believe is the underlying OS call for File.lock. However, this is only an *advisory* lock, i.e., any other processes accessing the file must also call flock(), otherwise none of its protections apply. It only works between cooperating processes. Linux does have a mandatory locking feature, but it requires the kernel to be specially configured for it, and the filesystem must also support it (and must be mounted with the correct options). Unfortunately it's Linux-specific and probably won't work for any other OS. Windows may have some file-locking mechanism that does what you want, but I'm not familiar with the intricacies of how it works. T OK. It's not possible without OS support. Agreed. And I don't want to get into C calls, but rather to use the mechanisms that D provides. And this *probably* won't cause any problems. But how should I tell D that that's the way I want to access to work? It's not clear to me that locking beyond the current EOF is valid, or whether it will decide that it needs to reserve the space before I use it (in which case locking from 0->ulong.max bytes is a bad idea, and would totally fill my hard disk).
Re: How to open file with exclusive lock?
On Tue, Jul 12, 2016 at 11:54:18AM -0700, Charles Hixson via Digitalmars-d-learn wrote: > I want to open a file with an exclusive lock. It would be important > that no other thread be able to access the file in write mode, and > desirable that no other thread be able to access the file in read > mode. (Ditto for other processes.) > > stdio.file.lock (or is it stdio.file.File.lock?) seems to demand a > range of chunks to lock, but the file is not fixed in length. Is it > appropriate to just specify the maximum possible number of bytes (i.e. > ulong.max)? Whether this is even possible depends on your OS. I don't know of any cross-platform way of file-locking that is guaranteed to work everywhere. In Posix there's fcntl(F_RDLCK / F_WRLCK), which I believe is the underlying OS call for File.lock. However, this is only an *advisory* lock, i.e., any other processes accessing the file must also call flock(), otherwise none of its protections apply. It only works between cooperating processes. Linux does have a mandatory locking feature, but it requires the kernel to be specially configured for it, and the filesystem must also support it (and must be mounted with the correct options). Unfortunately it's Linux-specific and probably won't work for any other OS. Windows may have some file-locking mechanism that does what you want, but I'm not familiar with the intricacies of how it works. T -- Why waste time reinventing the wheel, when you could be reinventing the engine? -- Damian Conway
Re: How to open file with exclusive lock?
On Tuesday, 12 July 2016 at 18:54:18 UTC, Charles Hixson wrote: I want to open a file with an exclusive lock. It would be important that no other thread be able to access the file in write mode, and desirable that no other thread be able to access the file in read mode. (Ditto for other processes.) stdio.file.lock (or is it stdio.file.File.lock?) seems to demand a range of chunks to lock, but the file is not fixed in length. Is it appropriate to just specify the maximum possible number of bytes (i.e. ulong.max)? From the lock[1] description: If both start and length are zero, the entire file is locked. [1] https://dlang.org/phobos/std_stdio.html#.File.lock
How to open file with exclusive lock?
I want to open a file with an exclusive lock. It would be important that no other thread be able to access the file in write mode, and desirable that no other thread be able to access the file in read mode. (Ditto for other processes.) stdio.file.lock (or is it stdio.file.File.lock?) seems to demand a range of chunks to lock, but the file is not fixed in length. Is it appropriate to just specify the maximum possible number of bytes (i.e. ulong.max)?
