Hi! ----
Here comes round "one" of the pre-review: I created a couple of webrevs in various flavours based on the 2007-04-19 ksh93-integration sources. Test binaries compiled from the same sources as this webrev can be found at http://www.opensolaris.org/os/project/ksh93-integration/downloads/2007-04-19/ ** Webrevs: * Webrev over all files: http://www.nrubsig.org/people/gisburn/work/solaris/ksh93_integration/ksh93_integration_prototype005_webrev_20070419/allfiles/webrev/ * Webrev only over files which match the substring "makefile": http://www.nrubsig.org/people/gisburn/work/solaris/ksh93_integration/ksh93_integration_prototype005_webrev_20070419/makefile_files/webrev/ * Webrev over all non-AST files (this includes the files in usr/src/cmd/ast/msgcc/ by accident): http://www.nrubsig.org/people/gisburn/work/solaris/ksh93_integration/ksh93_integration_prototype005_webrev_20070419/non_ast_files/webrev/ * Webrev over all AST sources (e.g. all files not listed in the previous "webrev"): http://www.nrubsig.org/people/gisburn/work/solaris/ksh93_integration/ksh93_integration_prototype005_webrev_20070419/non_files/webrev/ * All-in-one unified diff: http://www.nrubsig.org/people/gisburn/work/solaris/ksh93_integration/ksh93_integration_prototype005_webrev_20070419/unified_diff.txt ** Notes: * This review is still labelled "pre-review" (previous rounds were "pre-pre-review") because there are still unresolved issues (see below) and the webrev was created from our Subversion repository and not the final SCCS-based tree. The "pre-review"-label will remain until the currently unsolved issues have been solved. * All issues discussed in the previous review rounds have AFAIK been addressed. * The tree is now based on B61+ast-ksh.2007-04-18 and was compiled and tested on Solaris 11/Nevada B61. This is AFAIK the final source code layout. * All AST sources are in usr/src/lib/lib(ast|pp|dll|cmd|shell)/common/ and usr/src/cmd/ast/ (usr/src/cmd/ksh/ just contains Makefile glue which references sources in usr/src/lib/libshell/ (see below)). All files are 100% identical with the upstream (AT&T) versions. Primary directories are: - usr/src/cmd/ksh/ contains the ksh frontend binary (which is just a 5k wrapper and directly calls into libshell. Note that /usr/bin/(ksh93|rksh93) is a hardlink to /usr/lib/isaexec which selects the matching 32bit or 64bit version of ksh93 stored in usr/bin/$ISA/(ksh93|rksh93)) - usr/src/ast/ contains code which is delivered to /usr/ast/bin/, currently only "msgcc" (used to generate the AST/ksh93 l10n catalogs) is here - usr/src/lib/libshell/ contains the main ksh93 code - usr/src/lib/libcmd/ contains builtin command (such as "cat", "tail", "tee", "wc" etc.) - usr/src/lib/libdll/ contains the code for the dynamic library loader, used to load binary plugins in a platform-independent way - usr/src/lib/libast/ contains a utilty+platform abstraction library, including general purpose function for tree/list/stack/etc. handling, sfio (advanched I/O library), enhanched pattern matching, memory allocators etc. - usr/lib/libpp/ contains a C preprocessor provided as shared library. This is currently only used by "msgcc" * The files in usr/src/lib/lib(ast|pp|dll|cmd|shell)/(${MACH32}|${MACH64})/ except the "Makefile" files are 100% copies from a native AST package build and contain platform-specific configuration information. The source layout is little bit unusal for OS/Net and follows the upstream layout (it cannot be done differently without major surgery in the upstream sources and/or creating a maintaince hazard for future maintainers, for example files with the same name in different subdirs have different functionlity and messing this up will result in painfull debugging sessions with the final conclusion: "Wrong file in wrong place"). * The includes delivered to /usr/include/ast/ are a merge of 32bit and 64bit includes from usr/src/lib/lib(ast|pp|dll|cmd|shell)/(${MACH32}|${MACH64})/include/ast/ created using "/usr/bin/diff -D". This allows the usage of one unified set of includes for 32bit and 64bit binaries instead of shipping two different copies. * usr/src/lib/libast/Makefile.com has the "unusual" (for OS/Net) habit of putting *.o files into subdirs, following the source directory layout in "usr/src/libast/common/". This is just to avoid that the objs/pics subdirs are cramped with *.o files in one single directory (which would drive the maintainers into madness (slowly)). * usr/src/lib/lib(ast|pp|dll|cmd|shell)/Makefile.com list "-erroff=" flags per object file, not globally, resulting in lists which look "huge" because we list each appeance of a suppressed warning explicitly including a comment (and "yes", we're working on reducing the list, step by step). * "mapfile-vers" files are usually placed in usr/src/lib/lib(ast|pp|dll|cmd|shell)/mapfile-vers instead of the common/ subdir since the common/ should be explictly for the upstream sources (with minor exceptions) to make maintaince easier. BTW: There is a small glitch with "usr/src/lib/libcmd/common/mapfile-vers vs. usr/src/lib/libcmd/mapfile-vers" - the file should be moved and not deleted and re-created in it's new location. * usr/src/cmd/ksh/Makefile.ksh93switch contains a master switch to instruct the build whether ksh93 should be build as /usr/bin/ksh or /usr/bin/ksh93 (default). This also affects "usr/src/lib/libc/port/regex/wordexp.c" since the current version depends on a hack in the Sun/Solaris version of /usr/bin/ksh which does not exist in any other version of the korn shell (including ksh93). Toggling the switch in Makefile.ksh93switch will build an alternative version of |libc::wordexp()| which is compatible to ksh93 (the switch and the alternative version of |libc::wordexp()| has been added based on community requests to have such a flag - without such a flag ksh93 cannot be used as /usr/bin/ksh since |libc::wordexp()| will not work without the Sun/Solaris hack in the old /usr/bin/ksh - and that causes SMF to fail (which in turn renderes all the inetd services defunct and/or crashes the SMF service completely, rendering the system more or less unuseable)). * All new Makefiles use SHELL=/usr/bin/ksh (just to avoid that we need more horrible quoting stunts to make the Bourne shell happy&&cooperative (which includes the much-feared "sh: argument list too long"-thing)). This usage was approved by gatekeepers (David.Marker at Sun.COM) in <461EF8AB.1080809 at Sun.COM> * usr/src/cmd/ksh/Makefile.com compiles ksh93 with "-xpagesize_stack=64K" and "-xpagesize_heap=64K" based on benchmarking+profiling with larger datasets * The test suite run in usr/src/cmd/ksh/ is now "off" by default. * http://bugs.grommit.com/show_bug.cgi?id=118 disabled the generation of the AST/ksh93 message catalogs in the default build since the generation depends on ksh93+libpp itself (and the OS/Net rules do not allow using tools which were built within the tree (which would break if libc.so.1 changes)), usr/src/lib/libast/Makefile.libastl10n now contains a build switch ("ON_BUILD_AST_L10N_CATALOGS") which can be used to enable the generation of the l10n catalogs on demand (via setting $ ON_BUILD_AST_L10N_CATALOGS=1 # before building the tree, note that this will use tools from ${ROOT}+LD_LIBRARY_PATH for now). Once all build machines have been updated to a release which contains ksh93, "msgcc" and libpp the AST message catalog generation will be turned-on again (and the Makefile switch will be removed). ** Remaining unresolved issues: * usr/src/tools/findunref/exception_list was populated as discussed in http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2007-March/002447.html &co. (and usr/src/lib/libshell/misc/filelist.txt maintains the list of sources where were removed from the OS/Net tree), however the updates to newer ksh93 versions required additions for the libsum builtins (we don't ship libsum right now, therefore the related builtins in libcmd are currently not compiled-in and the sources were added to the "excepion_list" for now) and http://bugs.grommit.com/show_bug.cgi?id=217 added a few more items (see bug report for a detailed justification). I am reallly praying that the changes are acceptable... * It has been requested to make the |libc::wordexp()|-switch (see the comment about "Makefile.ksh93switch" above) configurable without rebuilding/replacing libc.so.1 each time. My proposal would be to create something like "/etc/default/_libc" (the '_' indicating that this is _not_ an official interface) and add a line like |WORDEXP_USE_KSH93=<boolean>| (where |boolean| may be |0| or |1|). This would replace the complex build infrastructure around "Makefile.ksh93switch" by a simple runtime configuration, greatly simplifying the usage for OpenSolaris distributions. Question is now what is better/worse: A compile-time switch (e.g. "Makefile.ksh93switch") or a runtime switch (e.g. |WORDEXP_USE_KSH93=<boolean>| in "/etc/default/_libc") ... Comments/suggestions/etc. welcome... ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;)
