Re: CFT: FreeBSD Package Base
On 2019-04-30 17:03, Miroslav Lachman wrote: David Chisnall wrote on 2019/04/30 10:22: On 29/04/2019 21:12, Joe Maloney wrote: With CFT version you chose to build, and package individual components such as sendmail with a port option. That does entirely solve the problem of being able to reinstall sendmail after the fact without a rebuild of the userland (base) port but perhaps base flavors could solve that problem assuming flavors could extend beyond python. This sounds very much like local optimisation. It's now easy to create a custom base image. Great. But how do I express dependencies in ports on a specific base configuration? This is easy if I depend on a specific base package, but how does this work in your model? For example, if I have a package that depends on a library that is an optional part of the base system, how do I express that pkg needs to either refuse to install it, or install a userland pkg that includes that library in place of my existing version as part of the install process? More importantly for the container use case, if I want to take a completely empty jail and do pkg ins nginx (for example), what does the maintainer of the nginx port need to do to express the minimum set of the base system that needs to be installed to allow nginx to work? One of the goals for the pkg base concept was to allow this kind of use case, easily creating a minimal environment required to run a single service. With a monolithic base package set, you're going to need some mechanism other than packages to express the specific base subset package that you need and I think that you need to justify why this mechanism is better than using small individual packages. Will it not be maintainer's nightmare to take care of all the dependencies on the base packages for each port we have in the ports tree? Speaking as a ports maintainer, it will be very annoying. Splitting it into a handful of large ass packages, same as you are presented with during install, would be best. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
The current state of this looks logical: its the bare minimum to represent the .tgz which we recognized from install menu choices, which minimises any real SAT solver dependency load, because these are not cross-dependant in any strong sense. The goal surely would be to prove this works and test. Then, we can move to the state I think I want, and I presume by extension everyone else wants: Sets which are small enough they expose risk of cross dependency, but once a SAT solver reconciles them, you have ONLY the minimum needed to do your job. So a "tiny" FreeBSD could be back under 4MB (hypothetically) because it was /rescue and the kernel, and if you needed /usr/share you loaded a pkg. -G On Wed, May 1, 2019 at 7:33 PM Miroslav Lachman <000.f...@quip.cz> wrote: > > Cy Schubert wrote on 2019/05/01 05:56: > > In message <292eadc6-3662-ec43-1175-53fc25248...@quip.cz>, Miroslav > > Lachman wri > > tes: > >> David Chisnall wrote on 2019/04/30 10:22: > >>> On 29/04/2019 21:12, Joe Maloney wrote: > With CFT version you chose to build, and package individual components > such as sendmail with a port option. That does entirely solve the > problem of being able to reinstall sendmail after the fact without a > rebuild of the userland (base) port but perhaps base flavors could > solve that problem assuming flavors could extend beyond python. > >>> > >>> This sounds very much like local optimisation. It's now easy to create a > >>> custom base image. Great. But how do I express dependencies in ports > >>> on a specific base configuration? This is easy if I depend on a specific > >>> base package, but how does this work in your model? For example, if I > >>> have a package that depends on a library that is an optional part of the > >>> base system, how do I express that pkg needs to either refuse to install > >>> it, or install a userland pkg that includes that library in place of my > >>> existing version as part of the install process? > >>> > >>> More importantly for the container use case, if I want to take a > >>> completely empty jail and do pkg ins nginx (for example), what does the > >>> maintainer of the nginx port need to do to express the minimum set of > >>> the base system that needs to be installed to allow nginx to work? > >>> > >>> One of the goals for the pkg base concept was to allow this kind of use > >>> case, easily creating a minimal environment required to run a single > >>> service. With a monolithic base package set, you're going to need some > >>> mechanism other than packages to express the specific base subset > >>> package that you need and I think that you need to justify why this > >>> mechanism is better than using small individual packages. > >> > >> Will it not be maintainer's nightmare to take care of all the > >> dependencies on the base packages for each port we have in the ports tree? > > > > No more than it is today. Remember, people have been doing this sort of > > thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and > > IBM can do it, I'm sure we can too. The dependency lists will be > > longer. We may require dependency lists that allow the choice of one of > > many prereqs or coreqs. > > They are experts and they are paid for their work. I am not. I am > maintaining a few packages and the reality is I don't know what they > need in base. Till these days I don't care about this kind of > dependency. I am not system developer or programmer and I think there > are more than just me who see this as a kind of problem. > So in this case, pkg base gives me nothing but more work on those packages. > > Miroslav Lachman > ___ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
Cy Schubert wrote on 2019/05/01 05:56: In message <292eadc6-3662-ec43-1175-53fc25248...@quip.cz>, Miroslav Lachman wri tes: David Chisnall wrote on 2019/04/30 10:22: On 29/04/2019 21:12, Joe Maloney wrote: With CFT version you chose to build, and package individual components such as sendmail with a port option. That does entirely solve the problem of being able to reinstall sendmail after the fact without a rebuild of the userland (base) port but perhaps base flavors could solve that problem assuming flavors could extend beyond python. This sounds very much like local optimisation. It's now easy to create a custom base image. Great. But how do I express dependencies in ports on a specific base configuration? This is easy if I depend on a specific base package, but how does this work in your model? For example, if I have a package that depends on a library that is an optional part of the base system, how do I express that pkg needs to either refuse to install it, or install a userland pkg that includes that library in place of my existing version as part of the install process? More importantly for the container use case, if I want to take a completely empty jail and do pkg ins nginx (for example), what does the maintainer of the nginx port need to do to express the minimum set of the base system that needs to be installed to allow nginx to work? One of the goals for the pkg base concept was to allow this kind of use case, easily creating a minimal environment required to run a single service. With a monolithic base package set, you're going to need some mechanism other than packages to express the specific base subset package that you need and I think that you need to justify why this mechanism is better than using small individual packages. Will it not be maintainer's nightmare to take care of all the dependencies on the base packages for each port we have in the ports tree? No more than it is today. Remember, people have been doing this sort of thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and IBM can do it, I'm sure we can too. The dependency lists will be longer. We may require dependency lists that allow the choice of one of many prereqs or coreqs. They are experts and they are paid for their work. I am not. I am maintaining a few packages and the reality is I don't know what they need in base. Till these days I don't care about this kind of dependency. I am not system developer or programmer and I think there are more than just me who see this as a kind of problem. So in this case, pkg base gives me nothing but more work on those packages. Miroslav Lachman ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
In message <292eadc6-3662-ec43-1175-53fc25248...@quip.cz>, Miroslav Lachman wri tes: > David Chisnall wrote on 2019/04/30 10:22: > > On 29/04/2019 21:12, Joe Maloney wrote: > >> With CFT version you chose to build, and package individual components > >> such as sendmail with a port option. That does entirely solve the > >> problem of being able to reinstall sendmail after the fact without a > >> rebuild of the userland (base) port but perhaps base flavors could > >> solve that problem assuming flavors could extend beyond python. > > > > This sounds very much like local optimisation. It's now easy to create a > > custom base image. Great. But how do I express dependencies in ports > > on a specific base configuration? This is easy if I depend on a specific > > base package, but how does this work in your model? For example, if I > > have a package that depends on a library that is an optional part of the > > base system, how do I express that pkg needs to either refuse to install > > it, or install a userland pkg that includes that library in place of my > > existing version as part of the install process? > > > > More importantly for the container use case, if I want to take a > > completely empty jail and do pkg ins nginx (for example), what does the > > maintainer of the nginx port need to do to express the minimum set of > > the base system that needs to be installed to allow nginx to work? > > > > One of the goals for the pkg base concept was to allow this kind of use > > case, easily creating a minimal environment required to run a single > > service. With a monolithic base package set, you're going to need some > > mechanism other than packages to express the specific base subset > > package that you need and I think that you need to justify why this > > mechanism is better than using small individual packages. > > Will it not be maintainer's nightmare to take care of all the > dependencies on the base packages for each port we have in the ports tree? No more than it is today. Remember, people have been doing this sort of thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and IBM can do it, I'm sure we can too. The dependency lists will be longer. We may require dependency lists that allow the choice of one of many prereqs or coreqs. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
David Chisnall wrote on 2019/04/30 10:22: On 29/04/2019 21:12, Joe Maloney wrote: With CFT version you chose to build, and package individual components such as sendmail with a port option. That does entirely solve the problem of being able to reinstall sendmail after the fact without a rebuild of the userland (base) port but perhaps base flavors could solve that problem assuming flavors could extend beyond python. This sounds very much like local optimisation. It's now easy to create a custom base image. Great. But how do I express dependencies in ports on a specific base configuration? This is easy if I depend on a specific base package, but how does this work in your model? For example, if I have a package that depends on a library that is an optional part of the base system, how do I express that pkg needs to either refuse to install it, or install a userland pkg that includes that library in place of my existing version as part of the install process? More importantly for the container use case, if I want to take a completely empty jail and do pkg ins nginx (for example), what does the maintainer of the nginx port need to do to express the minimum set of the base system that needs to be installed to allow nginx to work? One of the goals for the pkg base concept was to allow this kind of use case, easily creating a minimal environment required to run a single service. With a monolithic base package set, you're going to need some mechanism other than packages to express the specific base subset package that you need and I think that you need to justify why this mechanism is better than using small individual packages. Will it not be maintainer's nightmare to take care of all the dependencies on the base packages for each port we have in the ports tree? Miroslav Lachman ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
On 29/04/2019 21:12, Joe Maloney wrote: With CFT version you chose to build, and package individual components such as sendmail with a port option. That does entirely solve the problem of being able to reinstall sendmail after the fact without a rebuild of the userland (base) port but perhaps base flavors could solve that problem assuming flavors could extend beyond python. This sounds very much like local optimisation. It's now easy to create a custom base image. Great. But how do I express dependencies in ports on a specific base configuration? This is easy if I depend on a specific base package, but how does this work in your model? For example, if I have a package that depends on a library that is an optional part of the base system, how do I express that pkg needs to either refuse to install it, or install a userland pkg that includes that library in place of my existing version as part of the install process? More importantly for the container use case, if I want to take a completely empty jail and do pkg ins nginx (for example), what does the maintainer of the nginx port need to do to express the minimum set of the base system that needs to be installed to allow nginx to work? One of the goals for the pkg base concept was to allow this kind of use case, easily creating a minimal environment required to run a single service. With a monolithic base package set, you're going to need some mechanism other than packages to express the specific base subset package that you need and I think that you need to justify why this mechanism is better than using small individual packages. David ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
With CFT version you chose to build, and package individual components such as sendmail with a port option. That does entirely solve the problem of being able to reinstall sendmail after the fact without a rebuild of the userland (base) port but perhaps base flavors could solve that problem assuming flavors could extend beyond python. Joe Maloney Quality Engineering Manager / iXsystems Enterprise Storage & Servers Driven By Open Source > On Apr 29, 2019, at 3:31 PM, Cy Schubert wrote: > > In message <201904291441.x3tefmid072...@gndrsh.dnsmgr.net>, "Rodney W. > Grimes" > writes: >>> On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < >>> freebsd-...@gndrsh.dnsmgr.net> wrote: >>> > > Correct, this is ZFS only. And it's something we're using specific to FreeNAS / TrueOS, which is why I didn't originally mention it as apart of our CFT. Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", calling this FreeBSD pkg base when it is not was wrong, and miss leading. >>> >>> Sorry, I disagree. >> Which is fine. >> >>> This pkg base is independent of the ZFS tool we're using >>> to wrangle boot-environments. Hence why it wasn't mentioned in the CFT. >>> These base packages work the same as existing in-tree pkg base on UFS, no >>> difference. If anything are probably safer due to being able to update all >>> of userland in single extract operation, so you don't have out of order >>> extraction of libc or some such. >> >> You missed the major string change and focused on the edge, >> No comment on calling iXsystems :stuff: FreeBSD instead of FreeNAS/TrueOS? >> >> That was the major point of my statement, your miss leading the user >> community, you yourself said this would never be imported into FreeBSD >> base, so I see no reason that it should be called "FreeBSD package Base", >> as it is not, that is a different project. > > Taking the last comment on this thread to ask a question and maybe > refocus a little. > > The discussion about granularity begs the question, why pkgbase in the > first place? My impression was that it allowed people to select which > components they wanted to either create a lean installation or mix and > match base packages and ports (possibly with flavours to install in > /usr rather than $LOCALBASE) such that maybe person A wanted a stock > install while person B wanted to replace, picking a random example, BSD > tar with GNU tar. Isn't that the real advantage of pkgbase? > > If OTOH it's binary updates V 2.0, what's the point? I'm a little > rhetorical here but you get my point. If I want ipfw instead pf or > ipfilter instead of the others I should have the freedom. Similarly if > I want vim instead of vi I should have the choice to install vim as > /usr/bin/vi. Otherwise all the effort to replace binary updates makes > no sense. > > > -- > Cheers, > Cy Schubert > FreeBSD UNIX: Web: http://www.FreeBSD.org > > The need of the many outweighs the greed of the few. > > > ___ > freebsd-curr...@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
In message <201904291441.x3tefmid072...@gndrsh.dnsmgr.net>, "Rodney W. Grimes" writes: > > On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < > > freebsd-...@gndrsh.dnsmgr.net> wrote: > > > > > > > > > > Correct, this is ZFS only. And it's something we're using specific to > > > FreeNAS / TrueOS, which is why I didn't originally mention it as apart of > > > our CFT. > > > > > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", > > > calling this FreeBSD pkg base when it is not was wrong, > > > and miss leading. > > > > > > > Sorry, I disagree. > Which is fine. > > > This pkg base is independent of the ZFS tool we're using > > to wrangle boot-environments. Hence why it wasn't mentioned in the CFT. > > These base packages work the same as existing in-tree pkg base on UFS, no > > difference. If anything are probably safer due to being able to update all > > of userland in single extract operation, so you don't have out of order > > extraction of libc or some such. > > You missed the major string change and focused on the edge, > No comment on calling iXsystems :stuff: FreeBSD instead of FreeNAS/TrueOS? > > That was the major point of my statement, your miss leading the user > community, you yourself said this would never be imported into FreeBSD > base, so I see no reason that it should be called "FreeBSD package Base", > as it is not, that is a different project. Taking the last comment on this thread to ask a question and maybe refocus a little. The discussion about granularity begs the question, why pkgbase in the first place? My impression was that it allowed people to select which components they wanted to either create a lean installation or mix and match base packages and ports (possibly with flavours to install in /usr rather than $LOCALBASE) such that maybe person A wanted a stock install while person B wanted to replace, picking a random example, BSD tar with GNU tar. Isn't that the real advantage of pkgbase? If OTOH it's binary updates V 2.0, what's the point? I'm a little rhetorical here but you get my point. If I want ipfw instead pf or ipfilter instead of the others I should have the freedom. Similarly if I want vim instead of vi I should have the choice to install vim as /usr/bin/vi. Otherwise all the effort to replace binary updates makes no sense. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
> On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < > freebsd-...@gndrsh.dnsmgr.net> wrote: > > > > > > > Correct, this is ZFS only. And it's something we're using specific to > > FreeNAS / TrueOS, which is why I didn't originally mention it as apart of > > our CFT. > > > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", > > calling this FreeBSD pkg base when it is not was wrong, > > and miss leading. > > > > Sorry, I disagree. Which is fine. > This pkg base is independent of the ZFS tool we're using > to wrangle boot-environments. Hence why it wasn't mentioned in the CFT. > These base packages work the same as existing in-tree pkg base on UFS, no > difference. If anything are probably safer due to being able to update all > of userland in single extract operation, so you don't have out of order > extraction of libc or some such. You missed the major string change and focused on the edge, No comment on calling iXsystems :stuff: FreeBSD instead of FreeNAS/TrueOS? That was the major point of my statement, your miss leading the user community, you yourself said this would never be imported into FreeBSD base, so I see no reason that it should be called "FreeBSD package Base", as it is not, that is a different project. > > > For UFS, there will need to be additional care taken when doing updates. > > > > > > -- > > > Kris Moore > > > Vice President of Engineering > > > iXsystems, Inc > > > Ph: (408) 943-4100 > > > Ph: (408) 943-4101 > > > The Groundbreaking TrueNAS M-Series - > > > Enterprise Storage & Servers Driven By Open Source > > > > > > -Original Message- > > > From: Goran Meki? > > > Sent: Monday, April 29, 2019 9:43 AM > > > To: Kris Moore > > > Cc: Emmanuel Vadot ; FreeBSD Stable < > > freebsd-stable@freebsd.org>; FreeBSD Current ; > > freebsd-pkgb...@freebsd.org; freebsd-...@freebsd.org; > > freebsd-hack...@freebsd.org; freebsd-po...@freebsd.org > > > Subject: Re: CFT: FreeBSD Package Base > > > > > > On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > > > > We've written our own tool "sysutils/sysup" in GO which handles this. > > > > It performs updates using Boot-Environments to ensure that > > > > kernel/world are updated at same time. > > > > > > If I'm right, UFS doesn't support boot environments, so how would it > > work for UFS based installs? > > > > > > I personally feel GO is a bit ackward choice of language for something > > that practically should be part of base. At least I would expect OS > > update/upgrade not to require any external package. > > > > > > Regards, > > > meka > > > > > > ___ > > > freebsd-curr...@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to " > > freebsd-current-unsubscr...@freebsd.org" > > > > > > > > > > -- > > Rod Grimes > > rgri...@freebsd.org > > -- Rod Grimes rgri...@freebsd.org ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
RE: CFT: FreeBSD Package Base
> -Original Message- > From: Matthias Apitz > Sent: Monday, April 29, 2019 10:50 AM > To: Emmanuel Vadot > Cc: Kris Moore ; FreeBSD Stable sta...@freebsd.org>; freebsd-...@freebsd.org; freebsd- > hack...@freebsd.org; FreeBSD Current ; > freebsd-pkgb...@freebsd.org; freebsd-po...@freebsd.org > Subject: Re: CFT: FreeBSD Package Base > > > Why this thread has to go to all these lists? I receive any mail 5 times! > > Matthias Fair point. I'll restrict my replies to the -pkgbase list from here on out, suggest others do the same. Sorry about the noise 😊 > -- > Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176- > 38902045 Public GnuPG key: http://www.unixarea.de/key.pub N € I N zur EU! > "Gegen das EU-Europa der Banken, Konzerne und Kriegstreiber. > Für ein soziales und friedliches Europa der Völker." DKP ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
RE: CFT: FreeBSD Package Base
> -Original Message- > From: Rodney W. Grimes > Sent: Monday, April 29, 2019 10:41 AM > To: Kris Moore > Cc: Rodney W. Grimes ; Goran Mekić > ; Emmanuel Vadot ; FreeBSD > Stable ; FreeBSD Current curr...@freebsd.org>; freebsd-pkgb...@freebsd.org; freebsd- > p...@freebsd.org; freebsd-hack...@freebsd.org; freebsd-po...@freebsd.org > Subject: Re: CFT: FreeBSD Package Base > > > On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < > > freebsd-...@gndrsh.dnsmgr.net> wrote: > > > > > > > > > > Correct, this is ZFS only. And it's something we're using specific > > > > to > > > FreeNAS / TrueOS, which is why I didn't originally mention it as > > > apart of our CFT. > > > > > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", calling > > > this FreeBSD pkg base when it is not was wrong, and miss leading. > > > > > > > Sorry, I disagree. > Which is fine. > > > This pkg base is independent of the ZFS tool we're using > > to wrangle boot-environments. Hence why it wasn't mentioned in the CFT. > > These base packages work the same as existing in-tree pkg base on UFS, no > > difference. If anything are probably safer due to being able to update all > > of userland in single extract operation, so you don't have out of order > > extraction of libc or some such. > > You missed the major string change and focused on the edge, > No comment on calling iXsystems :stuff: FreeBSD instead of > FreeNAS/TrueOS? > > That was the major point of my statement, your miss leading the user > community, you yourself said this would never be imported into FreeBSD > base, so I see no reason that it should be called "FreeBSD package Base", > as it is not, that is a different project. > > -- > Rod Grimes rgri...@freebsd.org I think somehow you've missed the entire point here. This is being brought forth as a FreeBSD CFT in the hopes of upstream adoption. No misleading here whatsoever. The only thing that I wouldn't expect to be imported into base was this external tool we use on FreeNAS/TrueOS to handle our specific use-case of ZFS only. Total strawman here. Seriously, suggest you bother looking at it and reading further to get the full context. If anything this is far less invasive since it doesn't require lots of hacking on base, and can even be used to package old versions of FreeBSD if desired. The only thing I changed to make these images was a patch to bsdinstall to replace dist-file extraction with 'pkg install userland kernel pkg ...'. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
Why this thread has to go to all these lists? I receive any mail 5 times! matthias -- Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub N € I N zur EU! "Gegen das EU-Europa der Banken, Konzerne und Kriegstreiber. Für ein soziales und friedliches Europa der Völker." DKP signature.asc Description: PGP signature
Re: CFT: FreeBSD Package Base
On Mon, 29 Apr 2019 10:05:59 -0400 Kris Moore wrote: > On Mon, Apr 29, 2019 at 9:55 AM Emmanuel Vadot > wrote: > > > On Mon, 29 Apr 2019 09:25:05 -0400 > > Kris Moore wrote: > > > > > On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot > > > wrote: > > > > > > > > > > > Hi Kris, > > > > > > > > On Sun, 28 Apr 2019 15:52:21 -0400 > > > > wrote: > > > > > > > > > FreeBSD Community, > > > > > > > > > > > > > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > > > > 13-current > > > > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > > > > which > > > > > will allow users to perform all updating via the 'pkg' command > > directly. > > > > > Rather than trying to answer all questions in this announcement, > > we've > > > > > created a FAQ page with more details. Please refer to this page, and > > let > > > > us > > > > > know if you have additional questions that we can include on that > > page > > > > going > > > > > forward. > > > > > > > > > > > > > While I appreciate the effort I have some doubt about your > > > > "re-implementation" of pkgbase. I don't see any improvement compared to > > > > what is in base currently, I even see downside of your implementation. > > > > > > > > - How do you plan with the need of updating kernel first, reboot and > > > > updating the rest of the userland after ? (Needed for major and minor > > > > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > > > > -HEAD branch). This is still a problem with the base pkgbase. > > > > > > > > > > We've written our own tool "sysutils/sysup" in GO which handles this. It > > > performs updates using Boot-Environments to ensure that kernel/world are > > > updated at same time. > > > > > > > Which could never be imported into FreeBSD. > > > > Not suggesting it should be. Just information on how we solved that problem > in our own appliance / platforms. For FreeBSD it would need some tooling > still to handle this style of updating, regardless of which pkg base is > used. > > And for what it's worth, FreeBSD is all the poorer for not being able to > bring modern language based tools into the base. Personally I'm hoping the > shift to base-packages makes this a moot point since the idea of 'what is > base' can be diluted to just a manifest of what gets installed out of box. > Just my 2C on the matter though :) > > > > > > > > > > > > > > - This is even worse because you are using the same repository for > > > > base and pkg so if a user pkg update and both kernel and pkg(8) needs > > > > to be updated and pkg use a new syscall or capsicum thing it will be > > > > updated first and couldn't proceed with the rest of the update (this is > > > > a supposition, I haven't personally tested). > > > > > > > > > > See above. > > > > You can selectively update os/kernel and reboot before doing rest. > > > > > > > > > > > > - It seems that multiple kernels isn't supported in your > > > > implementation, this is already supported in pkgbase but still need > > > > some love. This is an important point as it will allow user to choose > > > > easily the kernel that they want to use and will also allow us > > > > developper to push kernels with new features to help testing. > > > > > > > > > > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll > > get > > > the Witness-enabled kernel installed alongside non-debugging one. > > > > Mhm no, the kernel-debug packages only add the debug file > > in /usr/lib/debug/boot/ > > I'm talking about installing multiple kernels in // > > (i.e. /boot/kernel.GENERIC /boot/kernel.MYFEATUREIWANTTOTEST) like > > describe here : > > > > https://wiki.freebsd.org/PkgBase#Project_goals_and_additional_unresolved_issues > > in the "How to handle /boot/kernel and /boot/kernel.$KERNCONF" point. > > > > > Incorrect, os/kernel-debug installs /boot/kernel-debug which is (on > 13-CURRENT) the Witness enabled kernel. os/kernel-debug-symbols are the > /usr/lib/debug bits. I only see kernel-20190420203550_1.txz and kernel-debug-20190420203550.txz in https://pkg.trueos.org/pkg/freebsd-pkgbase/FreeBSD%3A13%3Aamd64/latest/All/ and kernel-debug only contain the debug files. If I'm not looking in the right directory please correct me. > > > > > > > > > > > I think that the only advantage that your solution offers is that if > > > > we remove a componant of base (rcmds for example in 12-CURRENT) those > > > > files would be removed as they are in the userland-base package while > > > > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > > > > will not be deleted in the user computer. > > > > > > > > > > > > > Correct, this is one of the things which prompted us to go this > > direction. > > > Being able to handle crazy mixed WITH/WITHOUT flags was important to us, > > > current pkg base did not handle that so gracefully. > > > > Can you give me more info on this ? What where the WITH/WITHOUT flags > > that causes pro
RE: CFT: FreeBSD Package Base
> > > > > Incorrect, os/kernel-debug installs /boot/kernel-debug which is (on > > 13-CURRENT) the Witness enabled kernel. os/kernel-debug-symbols are > > the /usr/lib/debug bits. > > I only see kernel-20190420203550_1.txz and kernel-debug- > 20190420203550.txz in https://pkg.trueos.org/pkg/freebsd- > pkgbase/FreeBSD%3A13%3Aamd64/latest/All/ > and kernel-debug only contain the debug files. > If I'm not looking in the right directory please correct me. > > > -- > Emmanuel Vadot Ahh, you are correct. I checked and those packages haven't pushed to the mirrors yet, Jenkins is still chewing on a build of them here. I was using the 12-stable packages yesterday which has these changes. They should be synced up to the mirrors in the next 24-48 hours. Sorry about the confusion. -- Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < freebsd-...@gndrsh.dnsmgr.net> wrote: > > > > Correct, this is ZFS only. And it's something we're using specific to > FreeNAS / TrueOS, which is why I didn't originally mention it as apart of > our CFT. > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", > calling this FreeBSD pkg base when it is not was wrong, > and miss leading. > Sorry, I disagree. This pkg base is independent of the ZFS tool we're using to wrangle boot-environments. Hence why it wasn't mentioned in the CFT. These base packages work the same as existing in-tree pkg base on UFS, no difference. If anything are probably safer due to being able to update all of userland in single extract operation, so you don't have out of order extraction of libc or some such. > > > For UFS, there will need to be additional care taken when doing updates. > > > > -- > > Kris Moore > > Vice President of Engineering > > iXsystems, Inc > > Ph: (408) 943-4100 > > Ph: (408) 943-4101 > > The Groundbreaking TrueNAS M-Series - > > Enterprise Storage & Servers Driven By Open Source > > > > -Original Message- > > From: Goran Meki? > > Sent: Monday, April 29, 2019 9:43 AM > > To: Kris Moore > > Cc: Emmanuel Vadot ; FreeBSD Stable < > freebsd-stable@freebsd.org>; FreeBSD Current ; > freebsd-pkgb...@freebsd.org; freebsd-...@freebsd.org; > freebsd-hack...@freebsd.org; freebsd-po...@freebsd.org > > Subject: Re: CFT: FreeBSD Package Base > > > > On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > > > We've written our own tool "sysutils/sysup" in GO which handles this. > > > It performs updates using Boot-Environments to ensure that > > > kernel/world are updated at same time. > > > > If I'm right, UFS doesn't support boot environments, so how would it > work for UFS based installs? > > > > I personally feel GO is a bit ackward choice of language for something > that practically should be part of base. At least I would expect OS > update/upgrade not to require any external package. > > > > Regards, > > meka > > > > ___ > > freebsd-curr...@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscr...@freebsd.org" > > > > > > -- > Rod Grimes > rgri...@freebsd.org > ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
> > Correct, this is ZFS only. And it's something we're using specific to FreeNAS > / TrueOS, which is why I didn't originally mention it as apart of our CFT. Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", calling this FreeBSD pkg base when it is not was wrong, and miss leading. > For UFS, there will need to be additional care taken when doing updates. > > -- > Kris Moore > Vice President of Engineering > iXsystems, Inc > Ph: (408) 943-4100 > Ph: (408) 943-4101 > The Groundbreaking TrueNAS M-Series - > Enterprise Storage & Servers Driven By Open Source > > -Original Message- > From: Goran Meki? > Sent: Monday, April 29, 2019 9:43 AM > To: Kris Moore > Cc: Emmanuel Vadot ; FreeBSD Stable > ; FreeBSD Current ; > freebsd-pkgb...@freebsd.org; freebsd-...@freebsd.org; > freebsd-hack...@freebsd.org; freebsd-po...@freebsd.org > Subject: Re: CFT: FreeBSD Package Base > > On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > > We've written our own tool "sysutils/sysup" in GO which handles this. > > It performs updates using Boot-Environments to ensure that > > kernel/world are updated at same time. > > If I'm right, UFS doesn't support boot environments, so how would it work for > UFS based installs? > > I personally feel GO is a bit ackward choice of language for something that > practically should be part of base. At least I would expect OS update/upgrade > not to require any external package. > > Regards, > meka > > ___ > freebsd-curr...@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > -- Rod Grimes rgri...@freebsd.org ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
On Mon, Apr 29, 2019 at 9:55 AM Emmanuel Vadot wrote: > On Mon, 29 Apr 2019 09:25:05 -0400 > Kris Moore wrote: > > > On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot > > wrote: > > > > > > > > Hi Kris, > > > > > > On Sun, 28 Apr 2019 15:52:21 -0400 > > > wrote: > > > > > > > FreeBSD Community, > > > > > > > > > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > > > 13-current > > > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > > > which > > > > will allow users to perform all updating via the 'pkg' command > directly. > > > > Rather than trying to answer all questions in this announcement, > we've > > > > created a FAQ page with more details. Please refer to this page, and > let > > > us > > > > know if you have additional questions that we can include on that > page > > > going > > > > forward. > > > > > > > > > > While I appreciate the effort I have some doubt about your > > > "re-implementation" of pkgbase. I don't see any improvement compared to > > > what is in base currently, I even see downside of your implementation. > > > > > > - How do you plan with the need of updating kernel first, reboot and > > > updating the rest of the userland after ? (Needed for major and minor > > > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > > > -HEAD branch). This is still a problem with the base pkgbase. > > > > > > > We've written our own tool "sysutils/sysup" in GO which handles this. It > > performs updates using Boot-Environments to ensure that kernel/world are > > updated at same time. > > > > Which could never be imported into FreeBSD. > Not suggesting it should be. Just information on how we solved that problem in our own appliance / platforms. For FreeBSD it would need some tooling still to handle this style of updating, regardless of which pkg base is used. And for what it's worth, FreeBSD is all the poorer for not being able to bring modern language based tools into the base. Personally I'm hoping the shift to base-packages makes this a moot point since the idea of 'what is base' can be diluted to just a manifest of what gets installed out of box. Just my 2C on the matter though :) > > > > > > > > - This is even worse because you are using the same repository for > > > base and pkg so if a user pkg update and both kernel and pkg(8) needs > > > to be updated and pkg use a new syscall or capsicum thing it will be > > > updated first and couldn't proceed with the rest of the update (this is > > > a supposition, I haven't personally tested). > > > > > > > See above. > You can selectively update os/kernel and reboot before doing rest. > > > > > > > - It seems that multiple kernels isn't supported in your > > > implementation, this is already supported in pkgbase but still need > > > some love. This is an important point as it will allow user to choose > > > easily the kernel that they want to use and will also allow us > > > developper to push kernels with new features to help testing. > > > > > > > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll > get > > the Witness-enabled kernel installed alongside non-debugging one. > > Mhm no, the kernel-debug packages only add the debug file > in /usr/lib/debug/boot/ > I'm talking about installing multiple kernels in // > (i.e. /boot/kernel.GENERIC /boot/kernel.MYFEATUREIWANTTOTEST) like > describe here : > > https://wiki.freebsd.org/PkgBase#Project_goals_and_additional_unresolved_issues > in the "How to handle /boot/kernel and /boot/kernel.$KERNCONF" point. > > Incorrect, os/kernel-debug installs /boot/kernel-debug which is (on 13-CURRENT) the Witness enabled kernel. os/kernel-debug-symbols are the /usr/lib/debug bits. > > > > > > > I think that the only advantage that your solution offers is that if > > > we remove a componant of base (rcmds for example in 12-CURRENT) those > > > files would be removed as they are in the userland-base package while > > > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > > > will not be deleted in the user computer. > > > > > > > > > Correct, this is one of the things which prompted us to go this > direction. > > Being able to handle crazy mixed WITH/WITHOUT flags was important to us, > > current pkg base did not handle that so gracefully. > > Can you give me more info on this ? What where the WITH/WITHOUT flags > that causes problems ? > I may have to pick Miwi's brain on this, but I believe some of the issues we saw were when introducing flags such as WITHOUT_RADIUS. Additionally there is a runtime problem to solve. I.E. if you change flags mid-stream, and user updates, there was no clean way on pkg-side to remove those already installed granular packages. Not without external tooling anyway. > > -- > Emmanuel Vadot > ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send
Re: CFT: FreeBSD Package Base
On Mon, 29 Apr 2019 09:25:05 -0400 Kris Moore wrote: > On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot > wrote: > > > > > Hi Kris, > > > > On Sun, 28 Apr 2019 15:52:21 -0400 > > wrote: > > > > > FreeBSD Community, > > > > > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > > 13-current > > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > > which > > > will allow users to perform all updating via the 'pkg' command directly. > > > Rather than trying to answer all questions in this announcement, we've > > > created a FAQ page with more details. Please refer to this page, and let > > us > > > know if you have additional questions that we can include on that page > > going > > > forward. > > > > > > > While I appreciate the effort I have some doubt about your > > "re-implementation" of pkgbase. I don't see any improvement compared to > > what is in base currently, I even see downside of your implementation. > > > > - How do you plan with the need of updating kernel first, reboot and > > updating the rest of the userland after ? (Needed for major and minor > > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > > -HEAD branch). This is still a problem with the base pkgbase. > > > > We've written our own tool "sysutils/sysup" in GO which handles this. It > performs updates using Boot-Environments to ensure that kernel/world are > updated at same time. > Which could never be imported into FreeBSD. > > > > - This is even worse because you are using the same repository for > > base and pkg so if a user pkg update and both kernel and pkg(8) needs > > to be updated and pkg use a new syscall or capsicum thing it will be > > updated first and couldn't proceed with the rest of the update (this is > > a supposition, I haven't personally tested). > > > > See above. > > > > - It seems that multiple kernels isn't supported in your > > implementation, this is already supported in pkgbase but still need > > some love. This is an important point as it will allow user to choose > > easily the kernel that they want to use and will also allow us > > developper to push kernels with new features to help testing. > > > > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll get > the Witness-enabled kernel installed alongside non-debugging one. Mhm no, the kernel-debug packages only add the debug file in /usr/lib/debug/boot/ I'm talking about installing multiple kernels in // (i.e. /boot/kernel.GENERIC /boot/kernel.MYFEATUREIWANTTOTEST) like describe here : https://wiki.freebsd.org/PkgBase#Project_goals_and_additional_unresolved_issues in the "How to handle /boot/kernel and /boot/kernel.$KERNCONF" point. > > > - Since you reduced the granularity on the userland bits it would mean > > that if we use your implementation for -p updates we would download the > > whole userland packages instead of just updating the package that was > > patched. For example with pkgbase, updating from 12.0 to 12.0p1 will > > only update the FreeBSD-runtime package. Yes this package is still big > > to download when you compare to what have changed but until pkg(8) have > > delta pkg supports (and if it will have support, I don't know if > > this is a wish or not) this is the best way to go. > > > > Correct, this is by design. We used the in-tree pkg base for nearly a year, > and found that the granularity didn't really offer any savings from a > download or time perspective. Updating 100+ packages took far longer than a > single one, due to all the meta operations. Additionally in real-world > usage, we found that base packages tended to all get updated at the same > time, which took far longer via pkg, since it had to go and perform 100+ > fetch operations just to download the base system bits. > But you never need to update 100+ packages on a proper pkgbase setup for -p updates. Again on a 12.0 to 12.0-p1 update only one package will be updated. > > > - I see that you are sorting the plist for kernel and userland based > > on the line length [1], why is that ? > > > Whoops! I'll fix :) > > > > > > I think that the only advantage that your solution offers is that if > > we remove a componant of base (rcmds for example in 12-CURRENT) those > > files would be removed as they are in the userland-base package while > > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > > will not be deleted in the user computer. > > > > > Correct, this is one of the things which prompted us to go this direction. > Being able to handle crazy mixed WITH/WITHOUT flags was important to us, > current pkg base did not handle that so gracefully. Can you give me more info on this ? What where the WITH/WITHOUT flags that causes problems ? > Additionally we've > added some additional features, such as being able to 'pkg install os/src' > to get system sources used in exact build, as well as being able to rebuild > your local world
RE: CFT: FreeBSD Package Base
Correct, this is ZFS only. And it's something we're using specific to FreeNAS / TrueOS, which is why I didn't originally mention it as apart of our CFT. For UFS, there will need to be additional care taken when doing updates. -- Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source -Original Message- From: Goran Mekić Sent: Monday, April 29, 2019 9:43 AM To: Kris Moore Cc: Emmanuel Vadot ; FreeBSD Stable ; FreeBSD Current ; freebsd-pkgb...@freebsd.org; freebsd-...@freebsd.org; freebsd-hack...@freebsd.org; freebsd-po...@freebsd.org Subject: Re: CFT: FreeBSD Package Base On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > We've written our own tool "sysutils/sysup" in GO which handles this. > It performs updates using Boot-Environments to ensure that > kernel/world are updated at same time. If I'm right, UFS doesn't support boot environments, so how would it work for UFS based installs? I personally feel GO is a bit ackward choice of language for something that practically should be part of base. At least I would expect OS update/upgrade not to require any external package. Regards, meka ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > We've written our own tool "sysutils/sysup" in GO which handles this. It > performs updates using Boot-Environments to ensure that kernel/world are > updated at same time. If I'm right, UFS doesn't support boot environments, so how would it work for UFS based installs? I personally feel GO is a bit ackward choice of language for something that practically should be part of base. At least I would expect OS update/upgrade not to require any external package. Regards, meka signature.asc Description: PGP signature
Re: CFT: FreeBSD Package Base
On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot wrote: > > Hi Kris, > > On Sun, 28 Apr 2019 15:52:21 -0400 > wrote: > > > FreeBSD Community, > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > 13-current > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > which > > will allow users to perform all updating via the 'pkg' command directly. > > Rather than trying to answer all questions in this announcement, we've > > created a FAQ page with more details. Please refer to this page, and let > us > > know if you have additional questions that we can include on that page > going > > forward. > > > > While I appreciate the effort I have some doubt about your > "re-implementation" of pkgbase. I don't see any improvement compared to > what is in base currently, I even see downside of your implementation. > > - How do you plan with the need of updating kernel first, reboot and > updating the rest of the userland after ? (Needed for major and minor > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > -HEAD branch). This is still a problem with the base pkgbase. > We've written our own tool "sysutils/sysup" in GO which handles this. It performs updates using Boot-Environments to ensure that kernel/world are updated at same time. > - This is even worse because you are using the same repository for > base and pkg so if a user pkg update and both kernel and pkg(8) needs > to be updated and pkg use a new syscall or capsicum thing it will be > updated first and couldn't proceed with the rest of the update (this is > a supposition, I haven't personally tested). > See above. > - It seems that multiple kernels isn't supported in your > implementation, this is already supported in pkgbase but still need > some love. This is an important point as it will allow user to choose > easily the kernel that they want to use and will also allow us > developper to push kernels with new features to help testing. > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll get the Witness-enabled kernel installed alongside non-debugging one. > - Since you reduced the granularity on the userland bits it would mean > that if we use your implementation for -p updates we would download the > whole userland packages instead of just updating the package that was > patched. For example with pkgbase, updating from 12.0 to 12.0p1 will > only update the FreeBSD-runtime package. Yes this package is still big > to download when you compare to what have changed but until pkg(8) have > delta pkg supports (and if it will have support, I don't know if > this is a wish or not) this is the best way to go. > Correct, this is by design. We used the in-tree pkg base for nearly a year, and found that the granularity didn't really offer any savings from a download or time perspective. Updating 100+ packages took far longer than a single one, due to all the meta operations. Additionally in real-world usage, we found that base packages tended to all get updated at the same time, which took far longer via pkg, since it had to go and perform 100+ fetch operations just to download the base system bits. > - I see that you are sorting the plist for kernel and userland based > on the line length [1], why is that ? Whoops! I'll fix :) > > I think that the only advantage that your solution offers is that if > we remove a componant of base (rcmds for example in 12-CURRENT) those > files would be removed as they are in the userland-base package while > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > will not be deleted in the user computer. > Correct, this is one of the things which prompted us to go this direction. Being able to handle crazy mixed WITH/WITHOUT flags was important to us, current pkg base did not handle that so gracefully. Additionally we've added some additional features, such as being able to 'pkg install os/src' to get system sources used in exact build, as well as being able to rebuild your local world / kernel packages using ports "make config" framework is super handy. > > > > > Additionally, I will be hosting a Package Base working group at BSDCan > 2019, > > and welcome user and developer attendance to discuss this and other > ongoing > > package work: > > > > > > > > https://wiki.freebsd.org/DevSummit/201905/PackageBase > > > > I will be present and looking forward to work with you on this. > > Cheers, > > P.S. : FYI I'm working on pkgbase currently and I will have some > patches to commit soon (bsdinstall support, memstick creation that > install a pkgbase aware installaton etc ...). > Great! Looking forward to discussion then! > > [1] : > > https://github.com/trueos/trueos-ports/blob/trueos-master/os/userland-base/Makefile#L35 > > -- > Emmanuel Vadot > ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To
Re: CFT: FreeBSD Package Base
Hi Kris, On Sun, 28 Apr 2019 15:52:21 -0400 wrote: > FreeBSD Community, > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and 13-current > using "TrueOS-inspired" packaged base. These are stock FreeBSD images which > will allow users to perform all updating via the 'pkg' command directly. > Rather than trying to answer all questions in this announcement, we've > created a FAQ page with more details. Please refer to this page, and let us > know if you have additional questions that we can include on that page going > forward. > While I appreciate the effort I have some doubt about your "re-implementation" of pkgbase. I don't see any improvement compared to what is in base currently, I even see downside of your implementation. - How do you plan with the need of updating kernel first, reboot and updating the rest of the userland after ? (Needed for major and minor upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and -HEAD branch). This is still a problem with the base pkgbase. - This is even worse because you are using the same repository for base and pkg so if a user pkg update and both kernel and pkg(8) needs to be updated and pkg use a new syscall or capsicum thing it will be updated first and couldn't proceed with the rest of the update (this is a supposition, I haven't personally tested). - It seems that multiple kernels isn't supported in your implementation, this is already supported in pkgbase but still need some love. This is an important point as it will allow user to choose easily the kernel that they want to use and will also allow us developper to push kernels with new features to help testing. - Since you reduced the granularity on the userland bits it would mean that if we use your implementation for -p updates we would download the whole userland packages instead of just updating the package that was patched. For example with pkgbase, updating from 12.0 to 12.0p1 will only update the FreeBSD-runtime package. Yes this package is still big to download when you compare to what have changed but until pkg(8) have delta pkg supports (and if it will have support, I don't know if this is a wish or not) this is the best way to go. - I see that you are sorting the plist for kernel and userland based on the line length [1], why is that ? I think that the only advantage that your solution offers is that if we remove a componant of base (rcmds for example in 12-CURRENT) those files would be removed as they are in the userland-base package while for pkgbase the FreeBSD-rcmd package will be deleted in the repo and will not be deleted in the user computer. > > Additionally, I will be hosting a Package Base working group at BSDCan 2019, > and welcome user and developer attendance to discuss this and other ongoing > package work: > > > > https://wiki.freebsd.org/DevSummit/201905/PackageBase > I will be present and looking forward to work with you on this. Cheers, P.S. : FYI I'm working on pkgbase currently and I will have some patches to commit soon (bsdinstall support, memstick creation that install a pkgbase aware installaton etc ...). [1] : https://github.com/trueos/trueos-ports/blob/trueos-master/os/userland-base/Makefile#L35 -- Emmanuel Vadot ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
On 29/04/2019 11:52, Christoph Moench-Tegeder wrote: This situation should be improved. Given that etcupdate is in all supported releases, we can even update UPDATING and the Handbook. So, does anyone have a pointer to the official procedure? Basically run: # etcupdate towards the end of your standard update-from-src process. Pretty much all the available arguments to etcupdate have sane defaults, so it's quite likely you can just run the bare command. (Obv. check the man page for details though) It will automatically do a 3-way merge between the latest sources, the sources from the previous time you ran etcupdate and your deployed configuration in /etc Most of the time, that's all you need to do. Occasionally however etcupdate will run into a merge conflict. When that happens etcupdate will display a SVN-like status report showing the conflicting files and refuse to do anything until the conflicts have been resolved. Which you do by: # etcupdate resolve That's an interactive process which runs through all of the conflicts and allows you to examine the differences and select either your own or the upstream version, or to just edit the files and merge the conflicts manually. Should be pretty familiar if you've ever used git or svn... Once everything has been marked as resolved, etcupdate reverts back to the usual behaviour and will automatically merge in any new updates as far as possible. Most of the time using etcupdate just boils down to a single, non-interactive command to keep everything up to date during the update process. Much less faff than using mergemaster. Occasionally you may need to run: # etcupdate -p before you 'make buildworld' -- that's in exactly the same circumstances where you'ld previously have run 'mergemaster -p' and it does the same job: apply any necessary changes to allow your build to succeed. The times you need to do this should be flagged in UPDATING. Cheers, Matthew ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
## Christoph Moench-Tegeder (c...@burggraben.net): > in a few entries without any detail. The Handbook (chapter 25.3) only ^ 23.5, of course. Regards, Christoph -- Spare Space ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
## Lars Engels (lars.eng...@0x20.net): > > What are the plans to get rid of the hellscape known as “mergemaster”? > > Is there anything exciting and new there either in base or any of the > > ixSystems projects? > > There's /usr/sbin/etcupdate in base. Unfortunately, the official documentation on upgrades does not mention etcupdate at all. The release notes point to freebsd-update for binary upgrades, and to /usr/src/UPDATING for source-based upgrades. UPDATING only documents the mergemaster procedure; etcupdate is only mentioned in a few entries without any detail. The Handbook (chapter 25.3) only documents mergemaster. Even after speed-reading etcupdate(8) I'm not sure how the two-step invocation of mergemaster has to be mapped to etcupdate - I feel I'd be inventing my own procedure here (and, of course, if it breaks I get to keep all the pieces). This situation should be improved. Given that etcupdate is in all supported releases, we can even update UPDATING and the Handbook. So, does anyone have a pointer to the official procedure? Regards, Christoph -- Spare Space ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
On Sun, Apr 28, 2019 at 09:29:37PM -0400, Charles Sprickman via freebsd-stable wrote: > > > What are the plans to get rid of the hellscape known as “mergemaster”? > Is there anything exciting and new there either in base or any of the > ixSystems projects? There's /usr/sbin/etcupdate in base. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
RE: CFT: FreeBSD Package Base
Its on the agenda for next months Working Group. That and tooling to help migrating to pkg base and keeping /etc files intact. -- Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source -Original Message- From: Charles Sprickman Sent: Sunday, April 28, 2019 9:30 PM To: Paul Mather Cc: k...@ixsystems.com; FreeBSD Stable Subject: Re: CFT: FreeBSD Package Base > On Apr 28, 2019, at 6:21 PM, Paul Mather wrote: > > On Apr 28, 2019, at 3:52 PM, wrote: > >> FreeBSD Community, >> >> >> >> I'm pleased to announce a CFT for builds of FreeBSD 12-stable and >> 13-current using "TrueOS-inspired" packaged base. These are stock >> FreeBSD images which will allow users to perform all updating via the 'pkg' >> command directly. >> Rather than trying to answer all questions in this announcement, >> we've created a FAQ page with more details. Please refer to this >> page, and let us know if you have additional questions that we can >> include on that page going forward. > > > I currently keep my FreeBSD/arm and FreeBSD/arm64 systems up to date via > PkgBase in FreeBSD 12. It works well for me (crossbuilding and hosting the > PkgBase repository on a FreeBSD/amd64 system). > > What is the difference between the above CFT-created PkgBase and one created > via "make packages" using the native build system > (https://wiki.freebsd.org/PkgBase)? Looking at the FAQ you linked > (https://trueos.github.io/pkgbase-docs/), it seems the above CFT system is > less granular than the one currently produced via the in-tree "make packages" > (which could be a good thing from a simplicity standpoint). Is there > anything else? > > Is the above CFT-produced packages the system that will ultimately > become the way packaged base is produced in FreeBSD 13.0-RELEASE, or > is it just an alternative you want people to try out and evaluate? I > guess I'm not clear what "TrueOS-inspired" packaged base means. :-) What are the plans to get rid of the hellscape known as “mergemaster”? Is there anything exciting and new there either in base or any of the ixSystems projects? Thanks, Charles > > Cheers, > > Paul. > > ___ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
> On Apr 28, 2019, at 6:21 PM, Paul Mather wrote: > > On Apr 28, 2019, at 3:52 PM, wrote: > >> FreeBSD Community, >> >> >> >> I'm pleased to announce a CFT for builds of FreeBSD 12-stable and 13-current >> using "TrueOS-inspired" packaged base. These are stock FreeBSD images which >> will allow users to perform all updating via the 'pkg' command directly. >> Rather than trying to answer all questions in this announcement, we've >> created a FAQ page with more details. Please refer to this page, and let us >> know if you have additional questions that we can include on that page going >> forward. > > > I currently keep my FreeBSD/arm and FreeBSD/arm64 systems up to date via > PkgBase in FreeBSD 12. It works well for me (crossbuilding and hosting the > PkgBase repository on a FreeBSD/amd64 system). > > What is the difference between the above CFT-created PkgBase and one created > via "make packages" using the native build system > (https://wiki.freebsd.org/PkgBase)? Looking at the FAQ you linked > (https://trueos.github.io/pkgbase-docs/), it seems the above CFT system is > less granular than the one currently produced via the in-tree "make packages" > (which could be a good thing from a simplicity standpoint). Is there > anything else? > > Is the above CFT-produced packages the system that will ultimately become the > way packaged base is produced in FreeBSD 13.0-RELEASE, or is it just an > alternative you want people to try out and evaluate? I guess I'm not clear > what "TrueOS-inspired" packaged base means. :-) What are the plans to get rid of the hellscape known as “mergemaster”? Is there anything exciting and new there either in base or any of the ixSystems projects? Thanks, Charles > > Cheers, > > Paul. > > ___ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
RE: CFT: FreeBSD Package Base
Paul, We also started with the current in-base package system, but ultimately abandonded it for a variety of reasons that made unsuitable for us to use in FreeNAS. Some of those were the chaos with having so many hundreds of packages, and the issues when you started combining them with various WITH_*/WITHOUT_* flags. This system doesn't touch the FreeBSD build and instead allows building source / packages directly from ports and poudriere. (Keeping base and ports in sync far easier) "TrueOS-inspired" was just reference to the fact we've been incubating this in TrueOS for about 6 months now to shake out some issues and test our designs. The hope is to envigorate the FreeBSD project to have a discussion about getting real base packages in a release sooner rather than later, especially since current efforts seem somewhat stalled. We're hoping the design we're using here is compelling enough that it can be adopted in FreeBSD. Some cool asides, being able to 'pkg install src' and have /usr/src be kept in sync with the current packages is super handy. 😉 -- Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source -Original Message- From: Paul Mather Sent: Sunday, April 28, 2019 6:22 PM To: k...@ixsystems.com Cc: FreeBSD Stable Subject: Re: CFT: FreeBSD Package Base On Apr 28, 2019, at 3:52 PM, wrote: > FreeBSD Community, > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > 13-current using "TrueOS-inspired" packaged base. These are stock > FreeBSD images which will allow users to perform all updating via the > 'pkg' command directly. > Rather than trying to answer all questions in this announcement, we've > created a FAQ page with more details. Please refer to this page, and > let us know if you have additional questions that we can include on > that page going forward. I currently keep my FreeBSD/arm and FreeBSD/arm64 systems up to date via PkgBase in FreeBSD 12. It works well for me (crossbuilding and hosting the PkgBase repository on a FreeBSD/amd64 system). What is the difference between the above CFT-created PkgBase and one created via "make packages" using the native build system (https://wiki.freebsd.org/PkgBase)? Looking at the FAQ you linked (https://trueos.github.io/pkgbase-docs/), it seems the above CFT system is less granular than the one currently produced via the in-tree "make packages" (which could be a good thing from a simplicity standpoint). Is there anything else? Is the above CFT-produced packages the system that will ultimately become the way packaged base is produced in FreeBSD 13.0-RELEASE, or is it just an alternative you want people to try out and evaluate? I guess I'm not clear what "TrueOS-inspired" packaged base means. :-) Cheers, Paul. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: CFT: FreeBSD Package Base
On Apr 28, 2019, at 3:52 PM, wrote: FreeBSD Community, I'm pleased to announce a CFT for builds of FreeBSD 12-stable and 13-current using "TrueOS-inspired" packaged base. These are stock FreeBSD images which will allow users to perform all updating via the 'pkg' command directly. Rather than trying to answer all questions in this announcement, we've created a FAQ page with more details. Please refer to this page, and let us know if you have additional questions that we can include on that page going forward. I currently keep my FreeBSD/arm and FreeBSD/arm64 systems up to date via PkgBase in FreeBSD 12. It works well for me (crossbuilding and hosting the PkgBase repository on a FreeBSD/amd64 system). What is the difference between the above CFT-created PkgBase and one created via "make packages" using the native build system (https://wiki.freebsd.org/PkgBase)? Looking at the FAQ you linked (https://trueos.github.io/pkgbase-docs/), it seems the above CFT system is less granular than the one currently produced via the in-tree "make packages" (which could be a good thing from a simplicity standpoint). Is there anything else? Is the above CFT-produced packages the system that will ultimately become the way packaged base is produced in FreeBSD 13.0-RELEASE, or is it just an alternative you want people to try out and evaluate? I guess I'm not clear what "TrueOS-inspired" packaged base means. :-) Cheers, Paul. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
