Re: More questions on building a release with a read only source tree
On 2/25/08, Don Jackson <[EMAIL PROTECTED]> wrote: > I have a heterogeneous collection of machines on which I run OpenBSD, > both amd64 and i386. > I have separate build machines for each architecture. > I would vastly prefer to download the source once, put it on a local > NAS, and have each build machine build the release it needs. > > In my experience, this doesn't work at all if the build processes > writes into the src tree itself, and historically I have had to keep a All the openbsd releases are built using the same source tree shared via nfs. If it doesn't work, you've done something wrong. The build process has been designed to support multiple builds since forever.
Re: More questions on building a release with a read only source tree
On Sun, Feb 24, 2008 at 11:27:31PM -0800, Don Jackson wrote: > I would like "make release" to use [ a ] "read only source tree" I use lndir(1) to accomplish this. Check your source tree out somewhere else, and use lndir to make a 'copy' in /usr/src. Build from there, no other magic required.
Re: More questions on building a release with a read only source tree
On Mon, Feb 25, 2008 at 7:31 AM, Marco Peereboom <[EMAIL PROTECTED]> wrote: > You want to read lndir(1). This is extremely helpful advice, thank you! I used lndir to create an architecture specific copy of my source tree, and successfully built a release within it. So, this is one way to do what I requested, and is a successful workaround. After I built my release, I checked the arch specific src tree for files that were not symbolic links, using: find . -type f -print All resulting found files were in the ./sys/arch/`machine`/compile directory tree. This leads me to believe that only the compile directory needs to be written to by the make release process. I find it inconsistent and less than optimal that the build of userland pretty much requires the use of a seperate obj directory BSDOBJDIR, the src tree is defined in BSDSRCDIR, and the release and dest directories required by make release are defined as RELEASEDIR and DESTDIR, and all these directories can be defined in distinct separate areas, but that the compile directory used by make release cannot be similarly defined in an alternate location than its default location within BSDSRCDIR. So, I have a gentle request/proposal that the compile directory used by the make release process be specified in some new environment variable (BSDCOMPILEDIR ?), if defined, that location is used as the base for compiling GENERIC, GENERIC.MP, etc, and if undefined, the existing default behavior would be followed. I can imagine that the lndir solution works great (and maybe better) for a certain class of developers/builders/users (maybe people that are constantly building versions of CURRENT?), but I believe that the class of OpenBSD users that follow STABLE and need to support multiple architectures would benefit from this seemingly small and straightforward change to the make release process. The lndir solution works, but is not perfect (just read about some of the caveats in the lndir man page) when things start to diverge between the two subtrees. My proposal above would eliminate the issues created by having link trees back to the virgin source. Best regards, Don Jackson > On Sun, Feb 24, 2008 at 11:27:31PM -0800, Don Jackson wrote: > > The FAQ describes two ways to build the kernel ( > > http://www.openbsd.org/faq/faq5.html#BldKernel ), > > > > # cd /usr/src/sys/arch/i386/conf > > # config GENERIC > > # cd ../compile/GENERIC > > # make clean && make depend && make > > > > or > > Variation on above process: Read-only source tree > > Sometimes, you may wish to ensure your /usr/src/sys directory > > remains untouched. This can be done by using the following process: > >$ cd /somewhere > >$ cp /usr/src/sys/arch/i386/conf/GENERIC . > >$ config -s /usr/src/sys -b . GENERIC > >$ make clean && make depend && make > > > > I would like "make release" to use the "read only source tree" variant > > above, how can I accomplish this? > > > > Right now, I see "make release" do: > > > > cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC > > > > Which is going to attempt to build the GENERIC kernel right there in > > my source tree. > > > > Also, I am having some other weird problem, due to the following logic > > in the Makefile.amd64 which contains: > > > > # source tree is located via $S relative to the compilation directory > > .ifndef S > > S!= cd ../../../..; pwd > > .endif > > AMD64= $S/arch/amd64 > > > > For some reason the above is setting my AMD64 to some weird path that > > is not correct on my system, namely: > > > >cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC > > GENERIC:13: cannot open ../../../../arch/amd64/conf/files.amd64 > > for reading: No such file or directory > > > > *** Error code 1 > > > > Stop in /home/4.2/src/etc (line 11 of etc.amd64/Makefile.inc). > > > > What is the point of the above, and how can I get the path correct for > > this build? > > > > Thanks, > > > > Don
Re: More questions on building a release with a read only source tree
* Don Jackson <[EMAIL PROTECTED]> [2008-02-25 07:24:45]:
> On Mon, Feb 25, 2008 at 5:35 AM, Travers Buda <[EMAIL PROTECTED]> wrote:
>
> > Why on earth are you bothering with this? Please don't tell me
> > it's for security, because that would be inane.
>
> I have a heterogeneous collection of machines on which I run OpenBSD,
> both amd64 and i386.
> I have separate build machines for each architecture.
> I would vastly prefer to download the source once, put it on a local
> NAS, and have each build machine build the release it needs.
>
> In my experience, this doesn't work at all if the build processes
> writes into the src tree itself, and historically I have had to keep a
> "virgin" source tree, and copy to each build machine, which takes a
> long time, and it is really kind of a pain to maintain the consistency
> of 3 copies.
>
> While choosing to avoid the use of the (inflammatory) word "inane", I
> find it curious that in following the proscribed procedure for
> building a release, I have ALREADY built a new kernel for this
> architecture (which is basically the first step before building
> userland,
> and then onto the release itself), (and in my case, I have already
> built both the GENERIC and GENERIC.MP kernels), that the Makefile.inc
> in /usr/src/etc/etc.amd64 goes ahead and does:
>
> # $OpenBSD: Makefile.inc,v 1.7 2006/07/27 02:53:55 deraadt Exp $
>
> .ifdef DESTDIR
> snap_md: bsd bsd.mp bootblocks distrib
> cp ${.CURDIR}/../sys/arch/amd64/compile/GENERIC/bsd \
> ${DESTDIR}/snapshot/bsd
> cp ${.CURDIR}/../sys/arch/amd64/compile/GENERIC.MP/bsd \
> ${DESTDIR}/snapshot/bsd.mp
>
> bsd:
> cd ${.CURDIR}/../sys/arch/amd64/conf && config GENERIC
> cd ${.CURDIR}/../sys/arch/amd64/compile/GENERIC && \
> ${MAKE} clean && ${MAKE} depend && exec ${MAKE}
>
> bsd.mp:
> cd ${.CURDIR}/../sys/arch/amd64/conf && config GENERIC.MP
> cd ${.CURDIR}/../sys/arch/amd64/compile/GENERIC.MP && \
> ${MAKE} clean && ${MAKE} depend && exec ${MAKE}
>
> bootblocks:
> cp ${DESTDIR}/usr/mdec/pxeboot ${DESTDIR}/snapshot
> cp ${DESTDIR}/usr/mdec/cdboot ${DESTDIR}/snapshot
> cp ${DESTDIR}/usr/mdec/cdbr ${DESTDIR}/snapshot
>
> .PHONY: bsd bsd.mp bootblocks
>
> .endif # DESTDIR check
>
> (I discovered this makefile AFTER I had sent my email last night)
>
> Anyway, it looks like one possible solution to my question would be to
> modify this file so that
> the bsd and bsd.mp targets are either no-ops, or perform their make in
> the previously generated kernel build directories,
> and then to change the snap_md target to copy the resulting bsd files
> out of these build directories, and not from the middle
> of the source tree. Of course, I'll have to do this again for the the
> comparable i386 Makefile.inc. It would be preferable if the makefile
> would check an environment variable for the location of where it
> should actually compile things (outside of the src tree!)
> and do it there. If unset, the Makefile could continue to pollute the
> source tree with its builds, if that is what you want.
>
> Questions:
>
> Is there any other way (a better way?) to do what I am looking for?
>
> What other compiles does "make release" perform that involve writing
> into the source tree?
>
> Thanks!
>
> Don
>
>
> > * Don Jackson <[EMAIL PROTECTED]> [2008-02-24 23:27:31]:
> >
> >
> >
> > > The FAQ describes two ways to build the kernel (
> > > http://www.openbsd.org/faq/faq5.html#BldKernel ),
> > >
> > > # cd /usr/src/sys/arch/i386/conf
> > > # config GENERIC
> > > # cd ../compile/GENERIC
> > > # make clean && make depend && make
> > >
> > > or
> > > Variation on above process: Read-only source tree
> > > Sometimes, you may wish to ensure your /usr/src/sys directory
> > > remains untouched. This can be done by using the following process:
> > >$ cd /somewhere
> > >$ cp /usr/src/sys/arch/i386/conf/GENERIC .
> > >$ config -s /usr/src/sys -b . GENERIC
> > >$ make clean && make depend && make
> > >
> > > I would like "make release" to use the "read only source tree" variant
> > > above, how can I accomplish this?
> > >
> > > Right now, I see "make release" do:
> > >
> > > cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC
> > >
> > > Which is going to attempt to build the GENERIC kernel right there in
> > > my source tree.
> > >
> > > Also, I am having some other weird problem, due to the following logic
> > > in the Makefile.amd64 which contains:
> > >
> > > # source tree is located via $S relative to the compilation
> > directory
> > > .ifndef S
> > > S!= cd ../../../..; pwd
> > > .endif
> > > AMD64= $S/arch/amd64
> > >
> > > For some reason the above is setting my AMD64 to some weird path that
> > > is not correct
Re: More questions on building a release with a read only source tree
You want to read lndir(1). On Sun, Feb 24, 2008 at 11:27:31PM -0800, Don Jackson wrote: > The FAQ describes two ways to build the kernel ( > http://www.openbsd.org/faq/faq5.html#BldKernel ), > > # cd /usr/src/sys/arch/i386/conf > # config GENERIC > # cd ../compile/GENERIC > # make clean && make depend && make > > or > Variation on above process: Read-only source tree > Sometimes, you may wish to ensure your /usr/src/sys directory > remains untouched. This can be done by using the following process: >$ cd /somewhere >$ cp /usr/src/sys/arch/i386/conf/GENERIC . >$ config -s /usr/src/sys -b . GENERIC >$ make clean && make depend && make > > I would like "make release" to use the "read only source tree" variant > above, how can I accomplish this? > > Right now, I see "make release" do: > > cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC > > Which is going to attempt to build the GENERIC kernel right there in > my source tree. > > Also, I am having some other weird problem, due to the following logic > in the Makefile.amd64 which contains: > > # source tree is located via $S relative to the compilation directory > .ifndef S > S!= cd ../../../..; pwd > .endif > AMD64= $S/arch/amd64 > > For some reason the above is setting my AMD64 to some weird path that > is not correct on my system, namely: > >cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC > GENERIC:13: cannot open ../../../../arch/amd64/conf/files.amd64 > for reading: No such file or directory > > *** Error code 1 > > Stop in /home/4.2/src/etc (line 11 of etc.amd64/Makefile.inc). > > What is the point of the above, and how can I get the path correct for > this build? > > Thanks, > > Don
Re: More questions on building a release with a read only source tree
On Mon, Feb 25, 2008 at 5:35 AM, Travers Buda <[EMAIL PROTECTED]> wrote:
> Why on earth are you bothering with this? Please don't tell me
> it's for security, because that would be inane.
I have a heterogeneous collection of machines on which I run OpenBSD,
both amd64 and i386.
I have separate build machines for each architecture.
I would vastly prefer to download the source once, put it on a local
NAS, and have each build machine build the release it needs.
In my experience, this doesn't work at all if the build processes
writes into the src tree itself, and historically I have had to keep a
"virgin" source tree, and copy to each build machine, which takes a
long time, and it is really kind of a pain to maintain the consistency
of 3 copies.
While choosing to avoid the use of the (inflammatory) word "inane", I
find it curious that in following the proscribed procedure for
building a release, I have ALREADY built a new kernel for this
architecture (which is basically the first step before building
userland,
and then onto the release itself), (and in my case, I have already
built both the GENERIC and GENERIC.MP kernels), that the Makefile.inc
in /usr/src/etc/etc.amd64 goes ahead and does:
# $OpenBSD: Makefile.inc,v 1.7 2006/07/27 02:53:55 deraadt Exp $
.ifdef DESTDIR
snap_md: bsd bsd.mp bootblocks distrib
cp ${.CURDIR}/../sys/arch/amd64/compile/GENERIC/bsd \
${DESTDIR}/snapshot/bsd
cp ${.CURDIR}/../sys/arch/amd64/compile/GENERIC.MP/bsd \
${DESTDIR}/snapshot/bsd.mp
bsd:
cd ${.CURDIR}/../sys/arch/amd64/conf && config GENERIC
cd ${.CURDIR}/../sys/arch/amd64/compile/GENERIC && \
${MAKE} clean && ${MAKE} depend && exec ${MAKE}
bsd.mp:
cd ${.CURDIR}/../sys/arch/amd64/conf && config GENERIC.MP
cd ${.CURDIR}/../sys/arch/amd64/compile/GENERIC.MP && \
${MAKE} clean && ${MAKE} depend && exec ${MAKE}
bootblocks:
cp ${DESTDIR}/usr/mdec/pxeboot ${DESTDIR}/snapshot
cp ${DESTDIR}/usr/mdec/cdboot ${DESTDIR}/snapshot
cp ${DESTDIR}/usr/mdec/cdbr ${DESTDIR}/snapshot
.PHONY: bsd bsd.mp bootblocks
.endif # DESTDIR check
(I discovered this makefile AFTER I had sent my email last night)
Anyway, it looks like one possible solution to my question would be to
modify this file so that
the bsd and bsd.mp targets are either no-ops, or perform their make in
the previously generated kernel build directories,
and then to change the snap_md target to copy the resulting bsd files
out of these build directories, and not from the middle
of the source tree. Of course, I'll have to do this again for the the
comparable i386 Makefile.inc. It would be preferable if the makefile
would check an environment variable for the location of where it
should actually compile things (outside of the src tree!)
and do it there. If unset, the Makefile could continue to pollute the
source tree with its builds, if that is what you want.
Questions:
Is there any other way (a better way?) to do what I am looking for?
What other compiles does "make release" perform that involve writing
into the source tree?
Thanks!
Don
> * Don Jackson <[EMAIL PROTECTED]> [2008-02-24 23:27:31]:
>
>
>
> > The FAQ describes two ways to build the kernel (
> > http://www.openbsd.org/faq/faq5.html#BldKernel ),
> >
> > # cd /usr/src/sys/arch/i386/conf
> > # config GENERIC
> > # cd ../compile/GENERIC
> > # make clean && make depend && make
> >
> > or
> > Variation on above process: Read-only source tree
> > Sometimes, you may wish to ensure your /usr/src/sys directory
> > remains untouched. This can be done by using the following process:
> >$ cd /somewhere
> >$ cp /usr/src/sys/arch/i386/conf/GENERIC .
> >$ config -s /usr/src/sys -b . GENERIC
> >$ make clean && make depend && make
> >
> > I would like "make release" to use the "read only source tree" variant
> > above, how can I accomplish this?
> >
> > Right now, I see "make release" do:
> >
> > cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC
> >
> > Which is going to attempt to build the GENERIC kernel right there in
> > my source tree.
> >
> > Also, I am having some other weird problem, due to the following logic
> > in the Makefile.amd64 which contains:
> >
> > # source tree is located via $S relative to the compilation directory
> > .ifndef S
> > S!= cd ../../../..; pwd
> > .endif
> > AMD64= $S/arch/amd64
> >
> > For some reason the above is setting my AMD64 to some weird path that
> > is not correct on my system, namely:
> >
> >cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC
> > GENERIC:13: cannot open ../../../../arch/amd64/conf/files.amd64
> > for reading: No such file or directory
> >
> > *** Error code 1
> >
> > Stop in /home/4.2/src/etc (line 11 of etc.amd64/M
Re: More questions on building a release with a read only source tree
* Don Jackson <[EMAIL PROTECTED]> [2008-02-24 23:27:31]: > The FAQ describes two ways to build the kernel ( > http://www.openbsd.org/faq/faq5.html#BldKernel ), > > # cd /usr/src/sys/arch/i386/conf > # config GENERIC > # cd ../compile/GENERIC > # make clean && make depend && make > > or > Variation on above process: Read-only source tree > Sometimes, you may wish to ensure your /usr/src/sys directory > remains untouched. This can be done by using the following process: >$ cd /somewhere >$ cp /usr/src/sys/arch/i386/conf/GENERIC . >$ config -s /usr/src/sys -b . GENERIC >$ make clean && make depend && make > > I would like "make release" to use the "read only source tree" variant > above, how can I accomplish this? > > Right now, I see "make release" do: > > cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC > > Which is going to attempt to build the GENERIC kernel right there in > my source tree. > > Also, I am having some other weird problem, due to the following logic > in the Makefile.amd64 which contains: > > # source tree is located via $S relative to the compilation directory > .ifndef S > S!= cd ../../../..; pwd > .endif > AMD64= $S/arch/amd64 > > For some reason the above is setting my AMD64 to some weird path that > is not correct on my system, namely: > >cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC > GENERIC:13: cannot open ../../../../arch/amd64/conf/files.amd64 > for reading: No such file or directory > > *** Error code 1 > > Stop in /home/4.2/src/etc (line 11 of etc.amd64/Makefile.inc). > > What is the point of the above, and how can I get the path correct for > this build? > > Thanks, > > Don > > Why on earth are you bothering with this? Please don't tell me it's for security, because that would be inane. -- Travers Buda
More questions on building a release with a read only source tree
The FAQ describes two ways to build the kernel ( http://www.openbsd.org/faq/faq5.html#BldKernel ), # cd /usr/src/sys/arch/i386/conf # config GENERIC # cd ../compile/GENERIC # make clean && make depend && make or Variation on above process: Read-only source tree Sometimes, you may wish to ensure your /usr/src/sys directory remains untouched. This can be done by using the following process: $ cd /somewhere $ cp /usr/src/sys/arch/i386/conf/GENERIC . $ config -s /usr/src/sys -b . GENERIC $ make clean && make depend && make I would like "make release" to use the "read only source tree" variant above, how can I accomplish this? Right now, I see "make release" do: cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC Which is going to attempt to build the GENERIC kernel right there in my source tree. Also, I am having some other weird problem, due to the following logic in the Makefile.amd64 which contains: # source tree is located via $S relative to the compilation directory .ifndef S S!= cd ../../../..; pwd .endif AMD64= $S/arch/amd64 For some reason the above is setting my AMD64 to some weird path that is not correct on my system, namely: cd /home/4.2/src/etc/../sys/arch/amd64/conf && config GENERIC GENERIC:13: cannot open ../../../../arch/amd64/conf/files.amd64 for reading: No such file or directory *** Error code 1 Stop in /home/4.2/src/etc (line 11 of etc.amd64/Makefile.inc). What is the point of the above, and how can I get the path correct for this build? Thanks, Don
