Re: Posix Semaphores in -CURRENT
Joe Kelsey wrote: OK, this is a bug. The semantics don't conform to POSIX. ... I rather imagine the correct thing to do is to root it in the FS, and, without a leading '/', treat it as relative to the process current directory. Basically, this is not a two line fix... it's a lot of work, to get a filesystem object to use. I think that it *is* a two-line fix. Remove the maximum length (or impose a maximum length of MAX_PATHNAMELEN), and simply remove the whole '/' checking. Then, the private namespace correctly emulates posix semantics, except for the rooted versus relative stuff, which would be *really* hard to do in a private namespace and of questionable value anyway. Sorry, but Garrett was right about the implementation: it should not be using a private namespace in the first place. Making it work for a private namespace that shouldn't be there in the first place seems really questionable, to me... -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Terry Lambert wrote:
Joe Kelsey wrote:
/* name must start with a '/' but not contain one. */
if (*name != '/' || len 2 || index(name + 1, '/') != NULL) {
free(ret, M_SEM);
return (EINVAL);
}
The comment makes it look like this code allows a 14-character named
semaphore which *must* start with a slash and cannot contain embedded
slashes. In other words, it does *not* conform to pathname semantics.
OK, this is a bug. The semantics don't conform to POSIX.
...
I rather imagine the correct thing to do is to root it in the FS,
and, without a leading '/', treat it as relative to the process
current directory.
Basically, this is not a two line fix... it's a lot of work, to
get a filesystem object to use.
I think that it *is* a two-line fix. Remove the maximum length (or
impose a maximum length of MAX_PATHNAMELEN), and simply remove the whole
'/' checking. Then, the private namespace correctly emulates posix
semantics, except for the rooted versus relative stuff, which would be
*really* hard to do in a private namespace and of questionable value anyway.
/Joe
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Joe Kelsey wrote:
/* name must start with a '/' but not contain one. */
if (*name != '/' || len 2 || index(name + 1, '/') != NULL) {
free(ret, M_SEM);
return (EINVAL);
}
The comment makes it look like this code allows a 14-character named
semaphore which *must* start with a slash and cannot contain embedded
slashes. In other words, it does *not* conform to pathname semantics.
OK, this is a bug. The semantics don't conform to POSIX.
It looks like FreeBSD's version doesn't use filesystem objects, and
makes up its own namespace (a flat one) in which to create the
semaphores. Even if it's not an FS thing, the POSIX semantics
pretty much guarantee at *least* the ability to create your own
named namespace, one deep, with the '/' as a namespace component
seperator, even if it's not a path component seperator.
I rather imagine the correct thing to do is to root it in the FS,
and, without a leading '/', treat it as relative to the process
current directory.
Basically, this is not a two line fix... it's a lot of work, to
get a filesystem object to use.
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Joe Kelsey wrote: I have been looking at the implementation of POSIX semaphores in -CURRENT. I noticed that there are several missing pieces, specifically the man pages and the removal of uthread_sem.c from libc_r. I suppose the man pages are not critical, but it seems silly to keep uthread_sem.c in libc_r if there is a completely new kernel-based implementation of the sem_* routines in -CURRENT. First, realize that there is no such kernel implementation, in the non-KSE case, and that the default in 5.0 is going to be non-KSE (which will wait until at least 5.1). Second, realize that even if there were a shiny, happy kernel implementation, it would be a bad idea to use it directly from a threaded program. This is because, even in the KSE world, the thread scheduler is a hybrid, and it's possible to have a multiplicity of threads in user space backed by a single kernel context. When this happens, crossing the user/kernel boundary does nothing for you... except to make your program run slower. There are some other things about uipc_sem.c that seem unnecessarily restrictive. For one thing, it *requires* named semaphores to begin with a '/' but not contain embedded '/'. This seems like a severe portabliity issue as TRU64 (at least) allows arbitrary pathnames as sempahores, probably storing some sort of marker in the directories (I get this only from examining the TRU64 online manual pages at http://www.tru64unix.compaq.com/docs/base_doc/DOCUMENTATION/HTML/APS33DTE_html/DOCU_010.HTM I guess the question to ask at this point is whether it's talking about disallowing embedding a '/' in a component, or disallowing it as a component seperator. I think you will find that it's the former. There seems to be some broad interpretation of the POSIX semantice in TRU64 to allow for negative semaphore values, one of their questionable choices, although their example code is instructive). Since the semaphore names are merely stored in kernel structures, what difference does it make if they contain embedded '/' characters or not? And why not allow names without leading '/'? I don't think that there is any wording in POSIX preventing this. In general, an application should not assume interpretation of standards. If a standard is written such that it can be interpreted in more than one way, then the application writer should assume that some OS vendor has interpreted it in every possible way, and should not depend on behaviour that requires a single interpretation to be implemented by everyone. Sucks, I know, but the only way around it is to write your POSIX congressman. In general, I think these types of ambiguities arise *intentionally*: the standard is written so that existing implementations already comply with it, so that vendors don't have to change things, and can still claim compliance. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Terry Lambert wrote: Joe Kelsey wrote: I have been looking at the implementation of POSIX semaphores in -CURRENT. I noticed that there are several missing pieces, specifically the man pages and the removal of uthread_sem.c from libc_r. I suppose the man pages are not critical, but it seems silly to keep uthread_sem.c in libc_r if there is a completely new kernel-based implementation of the sem_* routines in -CURRENT. First, realize that there is no such kernel implementation, in the non-KSE case, and that the default in 5.0 is going to be non-KSE (which will wait until at least 5.1). I am not interested in threading. Named POSIX semaphores are inter-*process* synchronization more than they are inter-*thread* synchronization. I do not care about unnamed semaphores. Second, realize that even if there were a shiny, happy kernel implementation, it would be a bad idea to use it directly from a threaded program. This is because, even in the KSE world, the thread scheduler is a hybrid, and it's possible to have a multiplicity of threads in user space backed by a single kernel context. When this happens, crossing the user/kernel boundary does nothing for you... except to make your program run slower. Did I say anything about threads? Why would named posix semaphores depend on threads? There are some other things about uipc_sem.c that seem unnecessarily restrictive. For one thing, it *requires* named semaphores to begin with a '/' but not contain embedded '/'. This seems like a severe portabliity issue as TRU64 (at least) allows arbitrary pathnames as sempahores, probably storing some sort of marker in the directories (I get this only from examining the TRU64 online manual pages at http://www.tru64unix.compaq.com/docs/base_doc/DOCUMENTATION/HTML/APS33DTE_html/DOCU_010.HTM I guess the question to ask at this point is whether it's talking about disallowing embedding a '/' in a component, or disallowing it as a component seperator. I think you will find that it's the former. So, is there some mechanism I am missing? Is there a layer between the application calling sem_open and the kernel receiving the parameters that strips it down to the last component? If there is a higher level involved here, why is the low-level ksem_create function worrying about embedded '/' characters? There seems to be some broad interpretation of the POSIX semantice in TRU64 to allow for negative semaphore values, one of their questionable choices, although their example code is instructive). Since the semaphore names are merely stored in kernel structures, what difference does it make if they contain embedded '/' characters or not? And why not allow names without leading '/'? I don't think that there is any wording in POSIX preventing this. In general, an application should not assume interpretation of standards. If a standard is written such that it can be interpreted in more than one way, then the application writer should assume that some OS vendor has interpreted it in every possible way, and should not depend on behaviour that requires a single interpretation to be implemented by everyone. Sucks, I know, but the only way around it is to write your POSIX congressman. In general, I think these types of ambiguities arise *intentionally*: the standard is written so that existing implementations already comply with it, so that vendors don't have to change things, and can still claim compliance. Yes, I realize this, but it seems that from my cursory inspection of uipc_sem.c that the check for embedded '/' characters is unnecessary and much too restrictive according to the posix standard. The standard only talks about whether or not the semaphore name begins with '/' and has nothing to say about embedded slashes except that the name must conform to filename requirements. It seems to me that either you take the TRU64 approach and place a marker in the file system (special directory entry?) or you allow all strings, only looking at the first character and not caring about anything else, except for whatever maximum pathname requirements you choose to impose. Maybe I am missing something in the system call semantics? /Joe To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
On Fri, 13 Dec 2002 08:41:16 -0800, Joe Kelsey [EMAIL PROTECTED] said: So, is there some mechanism I am missing? Is there a layer between the application calling sem_open and the kernel receiving the parameters that strips it down to the last component? If there is a higher level involved here, why is the low-level ksem_create function worrying about embedded '/' characters? I find this rather puzzling. Speaking as a standards person, I can state with some certainty that *the name of a POSIX semaphore is intended to have path name semantics*. It is not required to be an actual path name, but there is a clear expectation that a quality implementation will do so. The POSIX developers saw these IPC objects as being analogous to shared memory objects or fifos, and did not see a compelling reason to invent an entirely new namespace for them. Stevens actually suggests an implementation of named semaphores in which the semaphore is represented by a file which contains the name (``key'') of an SVID semaphore. -GAWollman To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Garrett Wollman wrote: On Fri, 13 Dec 2002 08:41:16 -0800, Joe Kelsey [EMAIL PROTECTED] said: So, is there some mechanism I am missing? Is there a layer between the application calling sem_open and the kernel receiving the parameters that strips it down to the last component? If there is a higher level involved here, why is the low-level ksem_create function worrying about embedded '/' characters? I find this rather puzzling. Speaking as a standards person, I can state with some certainty that *the name of a POSIX semaphore is intended to have path name semantics*. It is not required to be an actual path name, but there is a clear expectation that a quality implementation will do so. The POSIX developers saw these IPC objects as being analogous to shared memory objects or fifos, and did not see a compelling reason to invent an entirely new namespace for them. What are you puzzled about? I am asking why uipc_sem.c checks for embedded slashes. I agree with you that it should allow arbitrary pathname strings as semaphores and it appears to me that it does not, in fact, allow arbitrary pathname strings as named semaphores. I am asking where my misunderstanding of uipc_sem.c is, or if I do understand it correctly, why does it place this restriction on named semaphores. Stevens actually suggests an implementation of named semaphores in which the semaphore is represented by a file which contains the name (``key'') of an SVID semaphore. That would be a truly horrid implementation of named posix semaphores. I was hoping to get away from all SVID-related anachronisms here. /Joe To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
On Fri, Dec 13, 2002 at 08:41:16AM -0800, Joe Kelsey wrote: portabliity issue as TRU64 (at least) allows arbitrary pathnames as sempahores, probably storing some sort of marker in the directories (I get this only from examining the TRU64 online manual pages at http://www.tru64unix.compaq.com/docs/base_doc/DOCUMENTATION/HTML/APS33DTE_html/DOCU_010.HTM If you are going to argue about what FreeBSD should do, then in addition to checking with the TRU64 manual (TRU64 is probably in its sunset stage: http://news.com.com/2100-1001-976211.html), you should also refer to the POSIX standard: http://www.opengroup.org/onlinepubs/007904975/ For the man page for sem_open(), there is this: sem_t *sem_open(const char *name, int oflag, ...); The name argument points to a string naming a semaphore object. It is unspecified whether the name appears in the file system and is visible to functions that take pathnames as arguments. The name argument conforms to the construction rules for a pathname. If name begins with the slash character, then processes calling sem_open() with the same value of name shall refer to the same semaphore object, as long as that name has not been removed. If name does not begin with the slash character, the effect is implementation-defined. The interpretation of slash characters other than the leading slash character in name is implementation-defined. So apart from the leading slash character, nothing is mentioned about embedded slashes in the semaphore name. What's the right behavior for FreeBSD then? -- Craig Rodrigues http://www.gis.net/~craigr [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Joe Kelsey wrote: I have been looking at the implementation of POSIX semaphores in -CURRENT. I noticed that there are several missing pieces, specifically the man pages and the removal of uthread_sem.c from libc_r. [ ... ] I am not interested in threading. Named POSIX semaphores are inter-*process* synchronization more than they are inter-*thread* synchronization. I do not care about unnamed semaphores. [ ... ] Did I say anything about threads? Why would named posix semaphores depend on threads? My mistake. I suggest you use System V semaphores, instead, if you do not need threads interoperability. Your code will be more portable that way, too. I guess the question to ask at this point is whether it's talking about disallowing embedding a '/' in a component, or disallowing it as a component seperator. I think you will find that it's the former. So, is there some mechanism I am missing? Is there a layer between the application calling sem_open and the kernel receiving the parameters that strips it down to the last component? If there is a higher level involved here, why is the low-level ksem_create function worrying about embedded '/' characters? The semaphores are technically hierachical object references; you should think of them as file references, as an implementation detail. It's a namespace issue, more than anything else(*). Yes, I realize this, but it seems that from my cursory inspection of uipc_sem.c that the check for embedded '/' characters is unnecessary and much too restrictive according to the posix standard. I would need an exact version of the source files you are referencing, at this point, to be able to say anything more meaningful, sorry. The standard only talks about whether or not the semaphore name begins with '/' and has nothing to say about embedded slashes except that the name must conform to filename requirements. It seems to me that either you take the TRU64 approach and place a marker in the file system (special directory entry?) or you allow all strings, only looking at the first character and not caring about anything else, except for whatever maximum pathname requirements you choose to impose. Maybe I am missing something in the system call semantics? Actually, I believe they are real file references. The issue with making them imaginary, as you seem to imply (e.g. the special directory entry) is that the path names are actually redesvous, and you don't want them being duplicated, and you want them to be look-uppable. IMO, they will probably be 0 length regular files, unless they have to persist across reboots with stte, in which case, they will contain state information. Get me the exact file you are concerned about, and I will stare at it with you. I think, though, that if there is a problem, it's just that you are catching things in mid-implementation (POSIX semaphores were supported in the other scope, but not system, until very recently; there's even magic POSIX manifests that tell you not to use them). -- Terry (*) True for the implementations whose source I have examined; probably not true for every implementation, and possibly not true for the final FreeBSD implementation. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Joe Kelsey [EMAIL PROTECTED] writes: Yes, I realize this, but it seems that from my cursory inspection of uipc_sem.c that the check for embedded '/' characters is unnecessary and much too restrictive according to the posix standard. The standard only talks about whether or not the semaphore name begins with '/' and has nothing to say about embedded slashes except that the name must conform to filename requirements. It seems to me that either you take the TRU64 approach and place a marker in the file system (special directory entry?) or you allow all strings, only looking at the first character and not caring about anything else, except for whatever maximum pathname requirements you choose to impose. Maybe I am missing something in the system call semantics? Sounds like a bug to me. Could you open a PR? Best regards, Mike Barcroft To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
On Fri, 13 Dec 2002 12:02:47 -0500, Craig Rodrigues [EMAIL PROTECTED] said: So apart from the leading slash character, nothing is mentioned about embedded slashes in the semaphore name. What's the right behavior for FreeBSD then? The reason why the standard is written that way was to allow for RTOS implementations which do not have a filesystem. A quality implementation should represent these objects in the filesystem when this is possible. The semaphore itself need not reside in the filesystem; the path name could be used only as a rendezvous point. -GAWollman To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Terry Lambert wrote:
Get me the exact file you are concerned about, and I will stare
at it with you. I think, though, that if there is a problem, it's
just that you are catching things in mid-implementation (POSIX
semaphores were supported in the other scope, but not system,
until very recently; there's even magic POSIX manifests that tell
you not to use them).
src/sys/kern/uipc_sem.c, version 1.3:
...
#define SEM_MAX_NAMELEN 14
...
static int
sem_create(td, name, ksret, mode, value)
...
if (value SEM_VALUE_MAX)
return (EINVAL);
ret = malloc(sizeof(*ret), M_SEM, M_WAITOK | M_ZERO);
if (name != NULL) {
len = strlen(name);
if (len SEM_MAX_NAMELEN) {
free(ret, M_SEM);
return (ENAMETOOLONG);
}
/* name must start with a '/' but not contain one. */
if (*name != '/' || len 2 || index(name + 1, '/') != NULL) {
free(ret, M_SEM);
return (EINVAL);
}
The comment makes it look like this code allows a 14-character named
semaphore which *must* start with a slash and cannot contain embedded
slashes. In other words, it does *not* conform to pathname semantics.
If some other part of the system already verifies that the named
semaphore is actually an empty directory entry, then why is there a name
length restriction and why is there a '/' check?
If this is a bug, I will happily submit a PR. I wanted some feedback
before doing so. I am interested in playing with the semantics of posix
semaphores. It looks to me as though they could be implemented more
resource-efficiently than SVID semaphores and really do have different
semantics. I really want to try to understand what will and won't be
released in 5.0. How will I enable these for testing/use and will they
work in a portable way? The only way to answer those questions right
now is to examine the source. I am trying to understand the source,
especially in how it may or may not restrict use of semaphores.
/Joe
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: Posix Semaphores in -CURRENT
Mike Barcroft wrote: Sounds like a bug to me. Could you open a PR? http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/46239 /Joe To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Posix Semaphores in -CURRENT
I have been looking at the implementation of POSIX semaphores in -CURRENT. I noticed that there are several missing pieces, specifically the man pages and the removal of uthread_sem.c from libc_r. I suppose the man pages are not critical, but it seems silly to keep uthread_sem.c in libc_r if there is a completely new kernel-based implementation of the sem_* routines in -CURRENT. There are some other things about uipc_sem.c that seem unnecessarily restrictive. For one thing, it *requires* named semaphores to begin with a '/' but not contain embedded '/'. This seems like a severe portabliity issue as TRU64 (at least) allows arbitrary pathnames as sempahores, probably storing some sort of marker in the directories (I get this only from examining the TRU64 online manual pages at http://www.tru64unix.compaq.com/docs/base_doc/DOCUMENTATION/HTML/APS33DTE_html/DOCU_010.HTM There seems to be some broad interpretation of the POSIX semantice in TRU64 to allow for negative semaphore values, one of their questionable choices, although their example code is instructive). Since the semaphore names are merely stored in kernel structures, what difference does it make if they contain embedded '/' characters or not? And why not allow names without leading '/'? I don't think that there is any wording in POSIX preventing this. I would like to take advantage of the posix semaphore implementation when 5.0 comes out. I think the libc_r issue at least needs looking at. Or maybe I misunderstand the reading of src/lib/libc_r/uthread/Makefile HEAD branch. /Joe To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
