Re: Improved Build Process
My apologies to all for the suggestion I should not have made. Best Wishes to All.
Re: Improved Build Process
Ugh sorry that I don't read it till end, but even first part looks ugly for me. No, I'm not a programmer, I'm just fuc. newbie in programming because first years I focused on learning and using all types of systems so that I can be good sysadmin (maybe) later. I end with my discoveries on OpenBSD because it's clean, intelligent, stable, secure, has well documentation and its developers really know what they are doing and why they are doing it so. From time of start with OpenBSD I just cry when I use some other systems because they aren't so good even if they are better for some use. OpenBSD has very good filesystem hierarchy http://www.openbsd.org/cgi-bin/man.cgi?query=hier&apropos=0&sektion=0&manpath =OpenBSD+Current&arch=i386&format=html , not something like mmm this can be here in /usr/bin, this will be in /usr/local/bin. this will be in /opt, this will be here and here and here and here and ou I'm a developer so place something here (eg. directly to /) just because I'm developer and I want to reinvent wheel like on other systems. Eg. I like OpenSolaris too, but its hierarchy is quite crazy for me in some cases http://docs.sun.com/app/docs/doc/819-2252/filesystem-5?l=en&a=view it's because of mix of System V and BSD and some other additions sometimes to find something is like adventure game :-) Reagarding CVS... OpenBSD use it's own AnonCVS and I think that it works well for its purpose. You can read in-deep info here http://www.openbsd.org/papers/anoncvs-paper.ps , quite old, but I read it last week and it was very descriptive and useful for me. Style of kernel is described well here http://www.openbsd.org/cgi-bin/man.cgi?query=style&apropos=0&sektion=0&manpat h=OpenBSD+Current&arch=i386&format=html There are manuals how to build ISO either for CD-ROM or USB flash disk so you don't need something special. Just read FAQ and if you want you can make shell scripts for those purposes or write some app with language which is in base and if it will be ok maybe it will end in base system. Eg. http://www.openbsd.org/faq/faq14.html#flashmemLive So in short there are good tools available right now and system is prepared too. In fact it's quite boring in production because it just works and works and works and works so you don't need to fight fight it like with other systems (of course that bugs are even in OpenBSD, but main source of bugs with OpenBSD is between keyboard and chair). Maybe you have good idea in some points, but it's not good idea to have too much multiple tools on one task. I tested DragonflyBSD right now and they have 5(!) tools for packgae installation anyway no one from them was able to install Xorg correctly (maybe problem of VM). I don't think that this mess will help somehow to sysadmin, developer or user. On Sat, Dec 26, 2009 at 9:48 PM, David Shuman wrote: > I am somewhat new to OpenBSD but not systems and/or > systems programming as a whole. B Please excuse any errors > I may have made in my observations and desires. B Please > also feel free to pick and choose between the aspects of > this that appear to be valuable. B (I can provide my versions > of the scripts indicated below if that is of assistance however > many of these may be obvious in their content to experienced > OpenBSD people that follow this group. These scripts may be > missing some desirable error checking code) > > Security is one of the stated objectives of OpenBSD yet the current > build process appears to be difficult to secure largely because the > build directories are numerous and mixed in with directories for > general machine operation. (also difficult to backup/restore if the > user desires to maintain multiple machine configurations using this > process. B While the content of these directories is publicly > available, the specific contents for each configuration is NOT > as are the security requirements to protect the contents for EACH > SPECIFIC configuration. B The directories for many of the operations > appear to be "hard coded" in some of the make scripts to specific > directories. B I even experienced indications that the directory /CVS > seems to be assumed to be a repository that can not be used for > checkout, in my various attempts to do this with the existing code base. > > My suggestion for improvement and simplification include maintaining > a completely separate directory structure(s) for the build process > from normal machine operation. See below for hierarchy and > description: > > /bld B B B B B B B B B B B root directory for build operation > B B B B B B B B B B B B B B B B B (no data this directory/mount only other > B B B B B B B B B B B B B B B B B directories except for the > B B B B B B B B B B B B B B B B B B B B shell scripts and logs) > /bld/sh B B B B B B B B shell scripts to complete various build processes > B B B B B B B B B B B B B B B (see list below) > /bld/sh/log B B B B B
Re: Improved Build Process
If you have so many ideas you should go ahead and start your own
project. Good luck building a successfull project based on 'ideas'.
I'm not even going to read to the end.
> I am somewhat new to OpenBSD but not systems and/or
> systems programming as a whole. Please excuse any errors
> I may have made in my observations and desires. Please
> also feel free to pick and choose between the aspects of
> this that appear to be valuable. (I can provide my versions
> of the scripts indicated below if that is of assistance however
> many of these may be obvious in their content to experienced
> OpenBSD people that follow this group. These scripts may be
> missing some desirable error checking code)
>
> Security is one of the stated objectives of OpenBSD yet the current
> build process appears to be difficult to secure largely because the
> build directories are numerous and mixed in with directories for
> general machine operation. (also difficult to backup/restore if the
> user desires to maintain multiple machine configurations using this
> process. While the content of these directories is publicly
> available, the specific contents for each configuration is NOT
> as are the security requirements to protect the contents for EACH
> SPECIFIC configuration. The directories for many of the operations
> appear to be "hard coded" in some of the make scripts to specific
> directories. I even experienced indications that the directory /CVS
> seems to be assumed to be a repository that can not be used for
> checkout, in my various attempts to do this with the existing code base.
>
> My suggestion for improvement and simplification include maintaining
> a completely separate directory structure(s) for the build process
> from normal machine operation. See below for hierarchy and
> description:
>
> /bld root directory for build operation
>(no data this directory/mount only other
> directories except for the
> shell scripts and logs)
> /bld/sh shell scripts to complete various build processes
> (see list below)
> /bld/sh/loglocation to store logs from build scripts
> from the following basic command
> sh build_script
> >log/build_script.log
> /bld/sh/logrlse? release version of the logs for diff compare
> /bld/sh/logprior? prior version of the logs for diff compare
> /bld/cvs(no data in this directory)
> /bld/cvs/src (src.tar.gz current /usr/src)
> /bld/cvs/src/sys(sys.tar.gz current /usr/src/sys)
> /bld/cvs/ports (ports.tar.gz current /usr/ports)
> /bld/cvs/xenocara (xenocara.tar.gz current /usr/xenocara)
> /bld/obj(no data in this directory)
> /bld/obj/kernel?(kernel obj current ?)
> /bld/obj/userland (userland obj current /usr/obj)
> /bld/obj/xenocara (xenocara obj current /usr/xobj))
> /bld/dest/ (no data in this directory)
> /bld/dest/userland (output current /usr/dest)
> /bld/dest/xenocara (output current /usr/xdest?)
> /bld/rlse (no data in this directory)
> /bld/rlse/src (*.tar.gz for cvs_preload(, etc - /bld/sh?))
> /bld/rlse/{ver}/{arch} release binary files
>
> I imagine in the above directory structure only /bld/rlse/... would be
> optionally required to grant outside read access for tasks like NFS
> or HTTP or FTP, etc. (Perhaps the /bld/rlse directory should be
> named /rlse for that reason alone}.
>
> The remaining directories would be tied to root access only with
> appropriate sudo setup for a user to perform the necessary tasks.
> none of these have been created and I would need to do some
> real research to achieve any of these scripts as defined. The user,
> if specified in the install should be setup with the appropriate sudo
> rights as part of the install.
> ie. it would be nice if you could
> sudo build {script)
> and have the above command execute in one task
> sh /bld/sh/{script} >/bld/sh/log/{script}.log
> and in another task tied to the console window
> tail -f /bld/sh/log/{script}.log
> with this task returning the command prompt
> upon closing of the log file (based on the
> end of the first task).
>
> sudo log {script} {log | logrlse | logprior} {file}
> would be used to copy
>/bld/{log | logrlse | logprior}/{script}.log
>to a user accessible file -- to edit and/or
>submit as documentation for errors
>requiring assistance.
>
>sudo diff {flags} {script} {logrlse | logprior}
> diff {flags} /bld/sh/log/{script}.log \
>
Re: Improved Build Process
Hi David. As a person who has worked with the OpenBSD process for easily 12+ years, and longer with a Variety of other platforms; a Wholesale Change such as you are recommending is going to be negatively looked at. Not that what You are typing about is not possible; In fact having a Slice for the /usr/obj folder makes for faster turnaround when doing sysgens. I my case I have to re-read existing documentation once in a while to keep up-to-date on -even- maintaining -stable or -current... And OpenBSD is pretty simple. But the scope of what you are considering / suggesting is a bit much. One of the firstfold reasons that OpenBSD has become a Unix that I appreciate, is due to the diligence of the development principals in using ASAP rules (As Simple As Possible) since this establishes two things: Clarity and Correctness. As for /usr/ports well, thats because we live in an Open Source and Free World. If something needs improvement, sit-down, code, and pay it forward by sending info to the Principals of the Port in Question. Being new to anything is not an excuse, it means you need to sit down and read the manuals, ask some misc@ questions, and when you feel there is something to contribute -- contribute. Its a Shut-Up and Hack thing, not a How We All can make -this- Unix-Based Operating System look like the SuSE build for s390. -sean > Subject: Improved Build Process > > I am somewhat new to OpenBSD but not systems and/or > systems programming as a whole. Please excuse any errors > > Thanks for reading this. I hope it is considered valuable. > _ Windows Live: Keep your friends up to date with what you do online. http://go.microsoft.com/?linkid=9691815
Re: Improved Build Process
On Sat, Dec 26, 2009 at 03:48:58PM -0500, David Shuman wrote: > Security is one of the stated objectives of OpenBSD yet the current > build process appears to be difficult to secure largely because the > build directories are numerous and mixed in with directories for > general machine operation. (also difficult to backup/restore if the > user desires to maintain multiple machine configurations using this > process. Nope, you've got things wrong. Multiple machine configurations is the security problem there. The more knobs you have, the more tests you need to run to make sure you aren't introducing any stupidity. There is such a thing as making this process TOO configurable. Since you're a complete newbie, you haven't looked at the history of the project, but if you do, you'll notice there's a pattern towards making things ways more reproducible, and removing any useless variation from the base system. For instance, contrary to your average unix distribution, most everyone uses GENERIC (and there has been a lot of time sunk into config to try and make certain it works for everyone).
Improved Build Process
I am somewhat new to OpenBSD but not systems and/or
systems programming as a whole. Please excuse any errors
I may have made in my observations and desires. Please
also feel free to pick and choose between the aspects of
this that appear to be valuable. (I can provide my versions
of the scripts indicated below if that is of assistance however
many of these may be obvious in their content to experienced
OpenBSD people that follow this group. These scripts may be
missing some desirable error checking code)
Security is one of the stated objectives of OpenBSD yet the current
build process appears to be difficult to secure largely because the
build directories are numerous and mixed in with directories for
general machine operation. (also difficult to backup/restore if the
user desires to maintain multiple machine configurations using this
process. While the content of these directories is publicly
available, the specific contents for each configuration is NOT
as are the security requirements to protect the contents for EACH
SPECIFIC configuration. The directories for many of the operations
appear to be "hard coded" in some of the make scripts to specific
directories. I even experienced indications that the directory /CVS
seems to be assumed to be a repository that can not be used for
checkout, in my various attempts to do this with the existing code base.
My suggestion for improvement and simplification include maintaining
a completely separate directory structure(s) for the build process
from normal machine operation. See below for hierarchy and
description:
/bld root directory for build operation
(no data this directory/mount only other
directories except for the
shell scripts and logs)
/bld/sh shell scripts to complete various build processes
(see list below)
/bld/sh/loglocation to store logs from build scripts
from the following basic command
sh build_script
>log/build_script.log
/bld/sh/logrlse? release version of the logs for diff compare
/bld/sh/logprior? prior version of the logs for diff compare
/bld/cvs(no data in this directory)
/bld/cvs/src (src.tar.gz current /usr/src)
/bld/cvs/src/sys(sys.tar.gz current /usr/src/sys)
/bld/cvs/ports (ports.tar.gz current /usr/ports)
/bld/cvs/xenocara (xenocara.tar.gz current /usr/xenocara)
/bld/obj(no data in this directory)
/bld/obj/kernel?(kernel obj current ?)
/bld/obj/userland (userland obj current /usr/obj)
/bld/obj/xenocara (xenocara obj current /usr/xobj))
/bld/dest/ (no data in this directory)
/bld/dest/userland (output current /usr/dest)
/bld/dest/xenocara (output current /usr/xdest?)
/bld/rlse (no data in this directory)
/bld/rlse/src (*.tar.gz for cvs_preload(, etc - /bld/sh?))
/bld/rlse/{ver}/{arch} release binary files
I imagine in the above directory structure only /bld/rlse/... would be
optionally required to grant outside read access for tasks like NFS
or HTTP or FTP, etc. (Perhaps the /bld/rlse directory should be
named /rlse for that reason alone}.
The remaining directories would be tied to root access only with
appropriate sudo setup for a user to perform the necessary tasks.
none of these have been created and I would need to do some
real research to achieve any of these scripts as defined. The user,
if specified in the install should be setup with the appropriate sudo
rights as part of the install.
ie. it would be nice if you could
sudo build {script)
and have the above command execute in one task
sh /bld/sh/{script} >/bld/sh/log/{script}.log
and in another task tied to the console window
tail -f /bld/sh/log/{script}.log
with this task returning the command prompt
upon closing of the log file (based on the
end of the first task).
sudo log {script} {log | logrlse | logprior} {file}
would be used to copy
/bld/{log | logrlse | logprior}/{script}.log
to a user accessible file -- to edit and/or
submit as documentation for errors
requiring assistance.
sudo diff {flags} {script} {logrlse | logprior}
diff {flags} /bld/sh/log/{script}.log \
/bld/sh/{logrlse | logprior}/(script}.log
note based on the above
sh /bld/sh/{script} >/bld/sh/log/(script).log
does not currently place all messages into the log file some
continue to appear on the console -- another item that would
need to be changed. (cvs is one of the offenders here -- I am
not sure if this is reasonably possible to change).
PS a
