Re: close call in a device ?
To add to this, the close calls can be forces; there is a flag
in the device structure wich can force notification. I'm not
sure what it does over a fork(), though: I think you really want
open notification.
You mean that when I register my device/kernel module, I can
explicitely request that all close calls will notify my module?
That is exactly what I need.
Add D_TRACKCLOSE to d_flags for your device. When the d_close()
of your device is call, the first arg is the dev.
Unfortunately, vcount() is used to see if the close is really a
final close, or not, and the vp is not passed into the close
itself. You will have to track closes yourself. One kludge to
get around having to so this is to modify spec_close() to do:
} else if (devsw(dev)-d_flags D_TRACKCLOSE) {
/* Keep device updated on status */
if (vcount(vp) 1) {
/* clear flag to signal driver of last close*/
devsw(dev)-d_flags = ~D_TRACKCLOSE;
}
} else if (vcount(vp) 1) {
and then as the _first_ thing in your close code:
if( !(devsw(dev)-d_flags D_TRACKCLOSE)) {
/* magic: final close: add flag back in to keep sane*/
devsw(dev)-d_flags |= D_TRACKCLOSE;
...
}
You can find spec_close() in /sys/miscfs/specfs/spec_vnops.c.
You really probably ought to just add the flag back in in the
first open. The thing that makes this a kludge is that it very
evilly unsets a flag it shouldn't unset, and it makes it the job
of the device to fix up the damage (the interface isn't reflexive).
A secondary nit is that this is not really reentrant, while the
flag is clear, so you have to be careful.
Really, since you will be doing per-open instance housekeeping
anyway, you ought to just add a list pointer to the per-open
instance data, and keep the open instances on a linked list;
you will have tolook up the per-open instance data somehow,
anyway, and it might as well be a list traversal. When list
membership goes from 1-0, you'll know it's the last close, and
you can free global (non per-open instance) resources.
Traditionally, this is done using a minor number, but you
can't just naievely allocate an unused one, since you might
not get called.
What do you mean by open notification ? I do get all "open" calls to my
device, just not all the "close"
For each open, d_open() gets called. This is where you will
be creating your per open instance data. You should look at
how fd's are handled over a fork() or other call. Without a
look at this code in depth, I can't tell you today whether or
not your d_open() code will get called again for each open fd,
or not. If it doesn't, this could be a problem for you. It
used to get called, FWIW.
The main problem with per process resources is that the VFS that
implements devices, specfs, doesn't own its own vnodes.
Could you develop a little ? I don't know about VFS, specfs and vnodes...
When you perform an operation on an open file, the vnode pointer
is dereferenced out of the per process open file table. The kernel
internally doesn't know from file handles (architectural bug, IMO),
and so not only is it hard to do file I/O in the kernel, but you
have this opaque handle called a vnode.
When you do an ioctl() or something else, then because this is
a special device, there is a dereference into a function table
pointed to by the vnode. This table is called "struct fileops",
and the table for special devices is spec_fileops.
So you give it a vnode, it dereferences the fcntl() function
pointer out of this table to make the call, and passes the
vnode pointer as an argument.
In the spec_fileops version of fcntl(), the device specific data
is derefenced out of the vnode; it can do this, because it knows
that any vnode of type VCHR will have one of these structures on
it. This is used by specfs to locate the correct device entry
point to call: your device.
Your device driver function is then called, with the device
private data pointer from the vnode, called "dev". It's a
pointer to your device private data.
Because the specfs does not own its own vnodes, each time you
open a device, you get the same vnode back from specfs. It
can't give you a different one, because you asked for the same
one: by the time it gets to the open, all it has is the vnode
of the parent directory, a major number, and a minor number.
So there's no way for the open to return a unique instance of
the device each time you open it, because it can only return
one vnode. This gets worse, because of fork() and other fd
managemetn behaviour. The kernel likes to give back the same
vnode to a user space process as often as possible. If one of
these calls to do this returns another reference to an existing
open isntance (say you open the same device twice from the same
program, or you call dup() or dup2()), then you may not get to
call all the
Re: close call in a device ?
Hello, Luigi Rizzo wrote: you could do something like this: + open allocates a descriptor which stores the PID of the process requesting access to the "device" doing that now + each I/O operation uses the descriptor matching the PID passed to the read/write/ioctl that too + you could implement an ioctl() to dispose the storage, a "well behaved" process would have to invoke this ioctl before terminating; I could, but my problem is that the process is not under my control. I can't modify the program to have it do something more. Maybe I can have a module that captures the close call and make it issue my special cleanup ioctl ? Or maybe that call won't be intercepted precisely because of the so far observed behavior ? back to studying the kernel module stuff + and a timeout as you suggested could be used to purge entries that have been idle for some time, or you could also purge them basing on usage patterns assumning there are clearly identifiable ones. yes, but that is the problem again, there may not be clearly identifiable patterns. At least, in the worst case, I could just check the process table and when a process goes away, I clean its associated resources. And I have to hope that a given process does not reopen and close the device several times. Did I miss something in your suggestion ? Or were you suggesting that I can create same name device entries, differing only by their minor number ? But then you cannot use the same name unless the entries are in different directories. My suggestion was to use /dev/foo.00 /dev/foo.01 /dev/foo.02 and so on. yep, I tried and tested that now :-) well there may not be a simple solution, but I think the idea of trapping "close" calls may work if I can trap the actual call before the kernel does its job of deciding where to route the call (which driver) and if the driver "close" should be called. Thanks for the idea. Any comments anyone ? bruno ### Bruno Schwander Senior Software Engineer Worldgate Communications, Inc email: [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
when a process closes the device, I do not get a "close" call for each process closing the device. I instead get a close only on the last process closing the device. the reason for this is that you might have a process fork() after it has opened the device, and you do not want to get to the device all close calls from all processes generated by the original one, but really only the last instance. The thing is, i think your model (allocating per-user resources on open) is wrong. It cannot protect you from a process forking and then having two instances using the same device. If you want multiple instances of the device, one option could be to use the minor number and really create multiple instances of the device, and open them in exclusive way so you know that there can be only one open per device (you should scan available devices in a similar way as the one is used for scanning pty's). To add to this, the close calls can be forces; there is a flag in the device structure wich can force notification. I'm not sure what it does over a fork(), though: I think you really want open notification. The main problem with per process resources is that the VFS that implements devices, specfs, doesn't own its own vnodes. This is actually the primary reason that VMWARE can only run one instance at a time of a virtual machine: there is no way to have per open instance resources, which are not shared. If you were to use the TFS flag (grep for TFS in the headers, that's a substring), you could make specfs own its own vnodes. The way you would handle your problem then is by returning a different instance of the device, with a different instance of per process attached storage. It's pretty simple to do this: just return a different vnode for the next open of the same device, instead of the same vnode with an additional reference. You will have to worry about close tracking in this case, as well. Other potential pitfalls are the directory name lookup cache returning a cached reference to the vnode from the last open (you will need to disable it), and the fork() and decriptor passing (using UNIX domain sockets) mechanisms, since what they return is a reference to the same reference, instead of a reference to a seperate, new reference (look at the per process open file table code to understand this). NB: If you are trying to do this for VMWARE or some other binary code, there's no way that the pty opening soloution suggested in the previous posting will be able to work for you, since the code will expect a particular behaviour. Right now, FreeBSD doesn't support this behaviour (cloning devices), but as pointed out above, it's not hard to implement, it's mostly just labor intensive. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
If I understand you correctly, you have multiple processes all of which are going to try to open /dev/foo, and you want them to behave as though they have each opened a unique device? You can't do this with FreeBSD, or with many other Unixes. Any SVR4 system can support this. So can AIX. Arguably, this is a defect with the device model. If you are trying to fake up concurrent access to a device, and the client processes are only going to read and write (no ioctls) to this device, then you can use a fifo and a multiplexor process. Alternatively, and this would be recommended; fix the client program. Or fix the device model. You can't have multiple VMWARE sessions in FreeBSD today because of this defect in the device model. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
Terry Lambert wrote: If I understand you correctly, you have multiple processes all of which are going to try to open /dev/foo, and you want them to behave as though they have each opened a unique device? You can't do this with FreeBSD, or with many other Unixes. Any SVR4 system can support this. So can AIX. Arguably, this is a defect with the device model. If you are trying to fake up concurrent access to a device, and the client processes are only going to read and write (no ioctls) to this device, then you can use a fifo and a multiplexor process. Alternatively, and this would be recommended; fix the client program. Or fix the device model. You can't have multiple VMWARE sessions in FreeBSD today because of this defect in the device model. I believe that this is not the reason.. it's another resource that we only have globally.. (i.e. once) but I forget what it is.. the problem here can be solved by using Poul's 'cloning device' interface in the driver. I don't think he has it quite completed but it is partly there.. maybe enough.. only in -current at the moment and you need to have devfs turned on. each time you open it, you actually get a different device and vnode.. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message -- __--_|\ Julian Elischer / \ [EMAIL PROTECTED] ( OZ) World tour 2000 --- X_.---._/ presently in: Budapest v To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
Terry Lambert wrote: when a process closes the device, I do not get a "close" call for each process closing the device. I instead get a close only on the last process closing the device. devices in a similar way as the one is used for scanning pty's). To add to this, the close calls can be forces; there is a flag in the device structure wich can force notification. I'm not sure what it does over a fork(), though: I think you really want open notification. You mean that when I register my device/kernel module, I can explicitely request that all close calls will notify my module ? That is exactly what I need. What do you mean by open notification ? I do get all "open" calls to my device, just not all the "close" The main problem with per process resources is that the VFS that implements devices, specfs, doesn't own its own vnodes. Could you develop a little ? I don't know about VFS, specfs and vnodes... What I did is make a module that defines a struct cdevsw with the open/read/ etc callbacks, then I register my calls for various devices entries with make_dev(), and at the end used the DEV_MODULE() macro to declare it to the system. I modeled that after the example in /usr/src/share/examples/kld of FreeBSD 4 Is there a different driver/module architecture ? This is actually the primary reason that VMWARE can only run one instance at a time of a virtual machine: there is no way to have per open instance resources, which are not shared. If you were to use the TFS flag (grep for TFS in the headers, that's a substring), you could make specfs own its own vnodes. Where should I look for this ? I looked into /usr/src/ and only some references to NTFS and TFS filesystems turned up ? Would I have to roll out a custom filesystem to have this running ? The way you would handle your problem then is by returning a different instance of the device, with a different instance of per process attached storage. It's pretty simple to do this: just return a different vnode for the next open of the same device, instead of the same vnode with an additional reference. this is really confusing me... in the example I had, the only thing I return from my open routine is an int telling success or errors happened... any pointers for the vnode stuff ? if it could apply to what I am trying to do ? Am I basing my driver on the wrong stuff ? NB: If you are trying to do this for VMWARE or some other binary code, there's no way that the pty opening soloution suggested in the previous posting will be able to work for you, since the code Yes, I came to that conclusion too. will expect a particular behaviour. Right now, FreeBSD doesn't support this behaviour (cloning devices), but as pointed out above, it's not hard to implement, it's mostly just labor intensive. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. -- ### Bruno Schwander Senior Software Engineer Worldgate Communications, Inc email: [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
Julian Elischer wrote: the problem here can be solved by using Poul's 'cloning device' interface in the driver. I don't think he has it quite completed but it is partly there.. maybe enough.. this seems very promising. Any pointers toward more info on this ? Thanks bruno only in -current at the moment and you need to have devfs turned on. each time you open it, you actually get a different device and vnode.. Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message -- __--_|\ Julian Elischer / \ [EMAIL PROTECTED] ( OZ) World tour 2000 --- X_.---._/ presently in: Budapest v -- ### Bruno Schwander Senior Software Engineer Worldgate Communications, Inc tel:(408) 378-7800 x116 fax: (408) 378-8018 email: [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
close call in a device ?
Hello everybody, I am writing a pseudo-device driver (as a kernel module) that needs to be opened in write mode by several processes. The problem I am having is that I do get all the "open" calls when a process opens the device, and I am able to process data written, etc. on a per-process basis; however, when a process closes the device, I do not get a "close" call for each process closing the device. I instead get a close only on the last process closing the device. This is a problem since I need to allocate/free resources for each process accessing the device, at the time a process closes the device. Is there a way to make sure my driver gets all "close" calls ? I could possibly get around this by using timeouts, but the unpredictability of the accessing processes may make this very difficult and suboptimal, so getting the "close" calls would be way better Thank you all for any information on this bruno -- ### Bruno Schwander Senior Software Engineer Worldgate Communications, Inc email: [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
when a process closes the device, I do not get a "close" call for each process closing the device. I instead get a close only on the last process closing the device. the reason for this is that you might have a process fork() after it has opened the device, and you do not want to get to the device all close calls from all processes generated by the original one, but really only the last instance. The thing is, i think your model (allocating per-user resources on open) is wrong. It cannot protect you from a process forking and then having two instances using the same device. If you want multiple instances of the device, one option could be to use the minor number and really create multiple instances of the device, and open them in exclusive way so you know that there can be only one open per device (you should scan available devices in a similar way as the one is used for scanning pty's). cheers luigi --+- Luigi RIZZO, [EMAIL PROTECTED] . ACIRI/ICSI (on leave from Univ. di Pisa) http://www.iet.unipi.it/~luigi/ . 1947 Center St, Berkeley CA 94704 Phone: (510) 666 2927 --+- This is a problem since I need to allocate/free resources for each process accessing the device, at the time a process closes the device. Is there a way to make sure my driver gets all "close" calls ? I could possibly get around this by using timeouts, but the unpredictability of the accessing processes may make this very difficult and suboptimal, so getting the "close" calls would be way better Thank you all for any information on this bruno -- ### Bruno Schwander Senior Software Engineer Worldgate Communications, Inc email: [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
Hi, The reason I am doing this, is precisely because I need to virtualize accesses from several processes to _one_ _predefined_ device. I have no control over that device name from the client process point of view, so I can not have multiple devices. I pretty much need to be able to lie to the process, telling it that it has full control of the device it wants, and actually allow other processes to open and use the device in the same way at the same time. Did I miss something in your suggestion ? Or were you suggesting that I can create same name device entries, differing only by their minor number ? But then I still hit the problem that the processes are maybe not going to open the device in exclusive mode ? the reason for this is that you might have a process fork() after it has opened the device, and you do not want to get to the device all close calls from all processes generated by the original one, but really only the last instance. The thing is, i think your model (allocating per-user resources on open) is wrong. It cannot protect you from a process forking and then having two instances using the same device. I can walk the process tree to figure whose parent process allocated the resource and thus use the same resource for the child, so this is not a problem (I think...). Besides, in my context, I doubt the process will fork after opening the device. If you want multiple instances of the device, one option could be to use the minor number and really create multiple instances of the device, and open them in exclusive way so you know that there can be only one open per device (you should scan available devices in a similar way as the one is used for scanning pty's). cheers luigi To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
The reason I am doing this, is precisely because I need to virtualize accesses from several processes to _one_ _predefined_ device. I have no control over that device name from the client process point of view, so I can not have multiple devices. I pretty much need to be able to lie to the process, telling it that it has full control of the device it wants, and actually allow other processes to open and use the device in the same way at the same time. If I understand you correctly, you have multiple processes all of which are going to try to open /dev/foo, and you want them to behave as though they have each opened a unique device? You can't do this with FreeBSD, or with many other Unixes. Arguably, this is a defect with the device model. If you are trying to fake up concurrent access to a device, and the client processes are only going to read and write (no ioctls) to this device, then you can use a fifo and a multiplexor process. Alternatively, and this would be recommended; fix the client program. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: close call in a device ?
Hi, The reason I am doing this, is precisely because I need to virtualize accesses from several processes to _one_ _predefined_ device. I have no control over that device name from the client process point of view, so I can not have multiple devices. I pretty much need to be able to lie to the process, telling it that it has full control of the device it wants, and actually allow other processes to open and use the device in the same way at the same time. you could do something like this: + open allocates a descriptor which stores the PID of the process requesting access to the "device" + each I/O operation uses the descriptor matching the PID passed to the read/write/ioctl + you could implement an ioctl() to dispose the storage, a "well behaved" process would have to invoke this ioctl before terminating; + and a timeout as you suggested could be used to purge entries that have been idle for some time, or you could also purge them basing on usage patterns assumning there are clearly identifiable ones. Did I miss something in your suggestion ? Or were you suggesting that I can create same name device entries, differing only by their minor number ? But then you cannot use the same name unless the entries are in different directories. My suggestion was to use /dev/foo.00 /dev/foo.01 /dev/foo.02 and so on. cheers luigi --+- Luigi RIZZO, [EMAIL PROTECTED] . ACIRI/ICSI (on leave from Univ. di Pisa) http://www.iet.unipi.it/~luigi/ . 1947 Center St, Berkeley CA 94704 Phone: (510) 666 2927 --+- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
