RE: How to create a ksh93 package...
I'd prefer a seperate dir not hidden too deep in the tree, where all the ast utilities (including ksh) get installed, e.g. /usr/ast/ bin fun lib man Bad. With one exception, we've decided not to clutter the top level /usr directory with a bunch of package-specific subdirs. OK, I see. BTW, I thought you weren't installing the man pages? Man pages will not be in my ksh93 package. ATT have man pages in their ast-ksh package. For people who want to fully fledged ast-open package, man pages will be useful. That would be another package, though. Maybe we need a top-level /opt directory? OTOH, I see no need for /usr/ast/* instead of ksh, and the DLLs go in /usr/bin; stub executables that are ksh-replacements for standard unix tools -- regardless of whether cygwin has normal ports yet -- go in /usr/bin/ksh/ or /usr/libexec/ksh/ or wherever. I'd prefer the name /usr/libexec/ast then. Would other executables that are not stub executables but alternative version to existing commands go there, too? ATT have own versions of dd, df, du, ed, expand, file, find, grep, od, pr, sed, sort, strings, etc. The other tools that have no Cygwin pendant, like cql, ditto, iffe, look, mamake, nmake, ratz, etc., they would go into /usr/bin? There are 150 DLLs, executables and scripts altogether in the full ast-open package /bin directory, and I have to distribute them into two different directories? No thanks. Chances that I would overwrite something existing by error are high. The ksh93 package would make a symlink /bin/ksh -- /usr/ast/bin/ksh so that ksh is available for everyone without changing the PATH. The accompanying DLLs cygast54.dll, cygcmd12.dll, cygdll10.dll and cygshell11.dll reside in /usr/ast/bin, too. This is a bad idea. You can't use symlinks for DLLs, and DLLs must be in the path I know. -- so you need to put /usr/ast/bin (or /usr/bin/ksh/bin or /usr/libexec/ksh/ in the PATH somewhere. But it needs to be at the END of the path for normal cygwin, and at the beginning of the path for ksh-centric usage. Blech. (okay, if .dll is in same dir as .exe, it'll work...but geez) That's why I wanted to put all the ast .exe files into a seperate */bin dir along with the DLLs. Why not let ksh be a full-fledged cygwin package? /usr/bin/ksh.exe /usr/bin/*.dll /usr/libexec/ksh/*.exe /usr/lib/* (what is fun?) A directory for auto-load shell functions. and you already said you weren't installing the man pages... See above. Anyone who wants to use ksh with the standalone version of the cygcmd12.dll commands would have to download another package and add /usr/ast/bin to the PATH. The remaining tools would go into yet another package. (And we might want to have a developer package with import libs, headers etc. but that's a different thing to be discussed later...). /usr/include/* /usr/lib/* We might have conflicts here with headers, too. I have to check. Regarding the sources, I will build only with the full ast-open package which is about 8MB compressed tar. I cannot split up the sources into seperate archives that would reflect the actual structure of the binary packages. The source archive may be distributed, this is not the problem. The problem is that I have a n:1 relationship between binary and source packages. How should I deal with this? Use dummy -src packages, for now. See ncurses/libncurses, gettext/libintl1, etc. OK. Karsten
RE: cygwin1.dll bug in ftime
Also notes the usage of unspecified. Unspecified means the standard does not say anything about the implementation, and, IMHO, the implementors are free to choose the best practices. ... or to not implement anything at all. I think it is obviously a good way to follow BSD. Possibly. The better way for application developers is to follow the Single UNIX Specification. Any application relying on results that are explicitely marked as unspecified in the SUS standard can be considered non-portable. From the SUSv3 documentation: APPLICATION USAGE For applications portability, the time() function should be used to determine the current time instead of ftime(). Karsten -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: OT: possible project/research project -- ksh has it all and i s now available for cygwin
BTW, BIG THANKS to all the people out there who already worked on a Cygwin port of aforementioned packages. A grep -il cygwin on the sources almost always gave me a clue where to put my hands on for the Uwin port. I guess this is one of the benefits of open source development. Is Uwin going to be open sourced sometime? Or has that already happened? David Korn wants to make U/Win open source. But I think it's not easy to convince the intellectual property folks at ATT of this. I am going to meet David personally in June, and, be sure, this is a point I'll be discussing with him extensively. Karsten -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: ksh on cygwin
And, I'm sorry but it really looks to me like you'd need a release from ATT indicating that any patches you provided to us are unemcumbered by this license. I don't see how you can sign away the rights to any patches that you make if you have been working on code that is covered by this license. Actually, having reviewed my patches, it's only patch based on AST - the $SHELL patch. I'm imitating the AST function pathshell() there. Now, since Corinna has made clear to me that there's no real super user on Cygwin, half of the patch is nonsense anyway and can be removed. Two other patches mimic UWIN behavior. That can not be a problem, since Cygwin also has adopted the UWIN symbolics links. I found something interesting in the archives, see http://www.cygwin.com/ml/cygwin/2001-02/msg00417.html. He didn't need a release from ATT, did he? -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: ksh on cygwin
If we just left out that patch we won't have a problem. OK. Two other patches mimic UWIN behavior. That can not be a problem, since Cygwin also has adopted the UWIN symbolics links. Mimicing isn't a problem as long as you didn't look into the sources and get the idea from there. If you just looked how it works from examining the behaviour of the binaries, that's ok. Again, I cannot have had a look into the sources. They are not available to the public. I got the ideas a) from the UWIN mailing list (the add .exe on close patch, see http://www.research.att.com/lists/uwin-users/2001/08/msg00043.html) b) from playing with the binaries (rename() and link() logic) Glenn and David commented a bit on those on request. Their comments were something like: Don't add the .exe extension to b on rename(a.exe, b)! Specifying an .exe extension indicates that the users knows what he's doing. [I had first added .exe to b, because I thought UWIN was wrong in this case.] Again, no code and not even implementation details. I believe that wouldn't have helped me either, as I don't know their underlying data structures and they do not know Cygwin's. I've invented my own implementations. There are no other UWIN related patches than those. All other patches are bug fixes or small corrections to make Cygwin behave more consistent. I found something interesting in the archives, see http://www.cygwin.com/ml/cygwin/2001-02/msg00417.html. He didn't need a release from ATT, did he? There's a difference. He didn't contribute to Cygwin beyond May 2000 and his contributions weren't AST or U/Win related at all. I was just trying to be picky :-) Licensing issues are really a big *@#$ but we have to be careful. We may not even take any code which smells GPL. It would infect the Cygwin Library License. For that reason we're most happy about completely self-written code or copies/ports of BSD licensed source. Are the issues cleared now? Karsten -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: ksh on cygwin
I know about that. Ok. Then that's the way to go. Just follow the procedures in http://cygwin.com/contrib.html . If your fix is big you'll need to fill out an assignment form as that web page mentions. It's a whole bunch of small fixes. I think I need to fill out the assignment form. Is it OK to send patches to 1.3.3-2 or should I move them to 1.3.6 first? So can we get down to discuss my fixes now? If you want to start a discussion, then... start it. You don't need permission to begin. It would be nice if, before you start making observations or suggesting fixes, you checked the mailing list archives to see if you are covering old ground or not. I think I've done that thoroughly enough. I've done some fixes to mmap() that might be superseded by other patches now. I haven't checked this yet. Also be advised that we can't break backwards compatibility in the DLL. That's one of the reasons I haven't been coming up with my patches until now. I wanted to test it carefully. If you provide patches, small well-defined ones are the best. Long patches which do many different things are less likely to be inspected or accepted. They're quite small. And, if you are really going to be engaged in a discussion you probably should subscribe to the cygwin mailing list. I don't think we're going to be able to remember to include your two email addresses in every response to you. Done. I'll give a short summary of what I've fixed: - pathconf() doesn't check existance of the path - getpagesize() should return a value compatible with mmap(), that is dwAllocGranularity (65536) instead of dwPageSize (1024). - use .exe extension detection consequently in all syscalls - automatically add the .exe extension to a newly created file on close() when the first four bytes are 0x4d, 0x5a, 0x90, 0x00 (MS Executable magic bytes) and the file name didn't have an extension already. (This is from UWIN, I think it's really nice). - exec*() functions would always invoke a /bin/sh on a file that isn't a valid executable. Only execlp()/execvp() should do so, others must return with an ENOEXEC. - the exec*() fix revealed a bug with vfork() in ash - use the contents of $SHELL instead of /bin/sh for execvp()/execlp() and system() (with some additional checks, e.g. do not use a csh, use only 'trusted' shells from /bin, /usr/bin, /usr/local/bin etc.). This allows the user to select his favorite shell manually, so no more copy /bin/bash to /bin/sh troubles. (This is also from UWIN). - some mmap() problems have been fixed. - utime() doesn't mark st_ctime for update Does this sound OK for you, especially the UWIN things I'm mimicing? Karsten -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: ksh on cygwin
It's a whole bunch of small fixes. I think I need to fill out the assignment form. Yeah, please send it as soon as possible since you'll have to send it by snail mail. Sometimes it takes two to three weeks for some reason. OK, I'll fill it out later today. Is it OK to send patches to 1.3.3-2 or should I move them to 1.3.6 first? I would suggest to move them to the latest from CVS. If you're always working against the latest from CVS you don't get hit too much by changes from other people. I'll try that. I'll give a short summary of what I've fixed: I'll compare your below listing against current from CVS. You should evaluate my answers in that light. - pathconf() doesn't check existance of the path It does. However, there's an error in _PC_PATH_MAX. It doesn't validate the incoming path which could result in a SEGV. I've checked in a fix. Good. - getpagesize() should return a value compatible with mmap(), that is dwAllocGranularity (65536) instead of dwPageSize (1024). We discussed that months ago. I think we're not going to change that (it's 4096, not 1024, btw.). Oops, I was writing this summary from memory, I don't have the patches at hand now... It will result in dubious problems when a process mmaps a file. For instance, the latest gcc expects to be able to read over the end of an mmaped file if the size is not a multiple of getpagesize(). Now think of a file which is coincidentally exactly 1 page long... Glenn found some test cases where mmap() failed and has also written a nice test program. I will get this to you later. He also states that the value returned by getpagesize() must conform to mmap() alignment by definition in the SUSv2. I'm not quite sure about that, though. - use .exe extension detection consequently in all syscalls You mean unlink() etc.? Yes. chmod(), link(), open(), pathconf(), rename(), truncate() and unlink() (have I missed one?) didn't check for the .exe extension. - automatically add the .exe extension to a newly created file on close() when the first four bytes are 0x4d, 0x5a, 0x90, 0x00 (MS Executable magic bytes) and the file name didn't have an extension already. (This is from UWIN, I think it's really nice). Ugh. I'm not quite sure if I like that. This eliminates the need to fix up programs which produce executables, e.g. ld. Also you don't need to take care of the .exe extension in cp. It's there automatically. - exec*() functions would always invoke a /bin/sh on a file that isn't a valid executable. Only execlp()/execvp() should do so, others must return with an ENOEXEC. Sounds like a bug. Yes. A big one. - the exec*() fix revealed a bug with vfork() in ash We have some vfork() changes in the meantime and even ash had an related error which should be fixed. Maybe we fixed the same error. I'll send you the details. - use the contents of $SHELL instead of /bin/sh for execvp()/execlp() and system() (with some additional checks, e.g. do not use a csh, use only 'trusted' shells from /bin, /usr/bin, /usr/local/bin etc.). This allows the user to select his favorite shell manually, so no more copy /bin/bash to /bin/sh troubles. (This is also from UWIN). Hmm, interesting idea... OK, more detailed. I allow only absolute pathes in $SHELL and don't allow any *csh. If superuser then only shells from [/usr][/local]/bin are considered trusted shells. If not superuser shells from other directories are allowed, but if uid != euid or gid != egid the shell and the directory where it resides must not be writable. Fall back value is /bin/sh. - some mmap() problems have been fixed. Since I have changed so much in mmap() you should actually first compare your changes to the current version. I'll do that. - utime() doesn't mark st_ctime for update Really? I would never think so when inspecting the source code. Has this been fixed meanwhile? Also other calls like chmod() must mark st_ctime for update. My patches are not complete here. Karsten -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: ksh on cygwin
Glenn found some test cases where mmap() failed and has also written a nice test program. I will get this to you later. He also states that the value returned by getpagesize() must conform to mmap() alignment by definition in the SUSv2. I'm not quite sure about that, though. See my reply to Robert. It's just an example. I don't have another reason at hand now but we already considered that change and we actually *had* reasons to avoid it. Perhaps Chris can help out here. OK. But, uhm, what exactly is a `superuser' from your point of view? We don't have that concept except for SYSTEM as _the_ user which is able to change user context w/o changing security policies. And on 9x/Me... Does the SYSTEM user have uid == 0? Does any user have an uid == 0? If not then it does not matter anyway. I can just leave it as it is. If in future some superuser concept might find it's way into Cygwin, this $SHELL stuff is safe already. Oh, I forgot to mention that I changed the rename() logic a bit. rename(a, b): If a is really a.exe it is renamed to b.exe rename(a, b.suffix): If a is really a.exe it is nevertheless renamed to b.suffix. The .suffix implies that the user knows what she's doing. rename(a.exe, b): The .exe suffix implies that the user knows what she's doing, too, so a.exe is renamed to b This also holds for link(). I've taken that from UWIN, too. Karsten -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: ksh on cygwin
The problem is that by default the Everyone group has the uid and gid 0. The user can change that in the passwd and group files. OK, I'll take that out again then. You just should stick with uid/gid 18 for the user SYSTEM. Are you familar with the NT security concept? If you want to have a rough insight how that's used in Cygwin, I suggest reading http://cygwin.com/cygwin-ug-net/ntsec.html It's rather old and a bit badly maintained but it's basically still correct. I've read it a long time ago... One general question, though. How do these changes to handle things like U/WIN collide with the propietary U/WIN license? We don't want to have problems with ATT suddenly. Especially we don't want to have sources taken from U/WIN. No sources were taken from UWIN or from the AST libraries. I'm using UWIN quite a lot here at Ford, because I needed to use the MSVC++ compiler. I've found many bugs in UWIN and I have email contact with David and Glenn on a regular basis. They asked me if I could help out porting AST to Cygwin, because they didn't want to touch (or even read) the Cygwin sources for obvious copyright problems. The concepts I've taken from UWIN were explained to me by them verbally, so no source code involved here. Other portions are rewritten from the AST sources (which are open source, see http://www.research.att.com/sw/license/ast-open.html). I don't know if this license and the GPL collide, so I've rewritten the code from memory after looking at the sources. The differences are substancial, so no problem here either. Karsten -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: ksh on cygwin
If you've actually looked at the UWIN sources, this is not enough. IANAL either, but I believe that this means you've been tainted. That means that we can't use your patches. Sorry. I've never had the chance to look at the UWIN sources. It's proprietary. As I said before, the UWIN developers explained the concepts verbally to me, no source code involved. The AST tools and libraries, which form the basis for the UWIN _tools_ (not the UNIX emulation itself) are open source. I rewritten some things from those sources (but from memory). I hope I am misinterpreting what you said incorrectly... :-P Karsten -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
