Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On 12/18/2016 4:31 PM, Baptiste Daroussin wrote: > Hi all, > > I have been working for a while on 2 long standing feature request for the > ports > tree: flavors and subpackages. > > For flavors I would like to propose a simple approach first which is more > like a > rework of the slave ports for now: > > Examples available here: > https://reviews.freebsd.org/D8840 (with the implementation) > and > https://reviews.freebsd.org/D8843 > > Design: introduce a 3rd level in the hierarchy and make it work a bit like > slave > ports > > pros: > - all slave ports are self hosted under the same directory: easier for > maintenance > - should work with all existing tools > > cons: > - hackish: it is not really much more than a slave port > - it adds plenty of new Makefiles :( > > I think anyway this is an improvement > > Next step after that is in would be to extend it to allow some dependency on > "I > depend on whatever flavor if port X" > > Subpackages: > Design: > Add a new macro MULTI_PACKAGES > flag plist with an @pkg{suffixofthesubpackage} file > the framework will split the plist into small plist and create all the > packages > All variables like COMMENT can be overridden with a > COMMENT_${suffixofthesubpackage} > > pros: > - simple and working almost now > - allow to simplify lots of ports > - options friendly (_PACKAGE automatically appends a new entry to > MULTI_PACKAGES) > > cons: > - will break the paradigm that certain tools depend on (portmaster/portupgrade > in particular are a huge problem since they are not actively maintained) > > Example of the usage: > https://people.freebsd.org/~bapt/multipackage.diff > > Note that I took the mpg123 as an example because it was a simple one to test > not because it may need subpackages > > As a result you build 3 packages: > mpg123 (the runtime tools) > mpg123-lib: the runtime libraries > mpg123-sndio: the sndio plugin > > LIB_DEPENDS on ports depending on libmpg123.so does not have to be changed, > the > framework already automatically register only the mpg123-lib as a dependency > and > not others. > > Not the example is missing one thing: a dependency between mpeg123-lib and > mpg123 > > The second is not ready yet and would take time to land > > Any comment? > > Best regards, > Bapt > By the way, support for Poudriere on this is progressing. A Pull request was submitted with support for another FLAVORS-type implementation, that pretty much works with your patches as well. I am working to resolve the DEPENDS_ARGS bug first, which also fixes allowing the same ORIGIN into the queue multiple times. Once that is done, then FLAVORS can be supported in Poudriere. -- Regards, Bryan Drewery signature.asc Description: OpenPGP digital signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
2016-12-19 1:31 GMT+01:00 Baptiste Daroussin : > Hi all, > > I have been working for a while on 2 long standing feature request for the > ports > tree: flavors and subpackages. > > For flavors I would like to propose a simple approach first which is more > like a > rework of the slave ports for now: > > Examples available here: > https://reviews.freebsd.org/D8840 (with the implementation) > and > https://reviews.freebsd.org/D8843 > > Design: introduce a 3rd level in the hierarchy and make it work a bit like > slave > ports > > pros: > - all slave ports are self hosted under the same directory: easier for > maintenance > - should work with all existing tools > This is what I really wanted for years especially for ports like spell checker. Some are in dedicated categories such as french/aspell while other are in textproc/-aspell and that's a big mess. OpenBSD ports has something like textproc/aspell/ and that is very nice and clean. If the plan is to do the same, that is definitely a major improvement. > cons: > - hackish: it is not really much more than a slave port > - it adds plenty of new Makefiles :( > > I think anyway this is an improvement > > Next step after that is in would be to extend it to allow some dependency on > "I > depend on whatever flavor if port X" > > Subpackages: > Design: > Add a new macro MULTI_PACKAGES > flag plist with an @pkg{suffixofthesubpackage} file > the framework will split the plist into small plist and create all the > packages > All variables like COMMENT can be overridden with a > COMMENT_${suffixofthesubpackage} > > pros: > - simple and working almost now > - allow to simplify lots of ports > - options friendly (_PACKAGE automatically appends a new entry to > MULTI_PACKAGES) > Also lovely as it will probably remove some packages that duplicate a lot the ports tree (-nox11 for example). If I understood correctly that one. > Any comment? Yes, make it real :) And again, thanks for your amazing and regular good work bapt! -- Demelier David ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On 19/12/2016 07:47, David Demelier wrote: >> I have been working for a while on 2 long standing feature request for the >> ports >> tree: flavors and subpackages. >> >> For flavors I would like to propose a simple approach first which is more >> like a >> rework of the slave ports for now: >> >> Examples available here: >> https://reviews.freebsd.org/D8840 (with the implementation) >> and >> https://reviews.freebsd.org/D8843 >> >> Design: introduce a 3rd level in the hierarchy and make it work a bit like >> slave >> ports >> >> pros: >> - all slave ports are self hosted under the same directory: easier for >> maintenance >> - should work with all existing tools >> > This is what I really wanted for years especially for ports like spell > checker. Some are in dedicated categories such as french/aspell while > other are in textproc/-aspell and that's a big mess. > > OpenBSD ports has something like textproc/aspell/ and that is > very nice and clean. If the plan is to do the same, that is definitely > a major improvement. > I really like this idea, although it's going to add a lot of extra directories and very similar small Makefiles to the ports. Every python port would grow flavours to support two major versions of python just for starters, and those additional Makefiles would be almost identical across the python2 flavour and across the python3 flavour. Is it the intention that -devel or versioned ports are treated as flavours as well? So for example we could end up with lang/python/27 lang/python/33 lang/python/34 and lang/python/35 ? Why can't you have both flavoured and unflavoured variants of the same port -- eg. devel/example as well as devel/example/foo and devel/example/bar ? How well do flavours and sub-packages combine? www/nginx would make a good example there, given it now has both loadable and compiled-in modules plus a couple of slave ports that just build different module load-outs. Matthew signature.asc Description: OpenPGP digital signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Mon, Dec 19, 2016 at 2:45 AM, Matthew Seaman wrote: > Why can't you have both flavoured and unflavoured variants of the same > port -- eg. devel/example as well as devel/example/foo and > devel/example/bar ? It seems like it would make sense to allow devel/example to be a default flavor so that, for instantiated example, editors/vim-lite => editors/vim/lite and editors/vim could potentially be a 'full' flavor or unflavored, if that's your flavor. I personally learn towards default flavor, though, because that gives you a chance to be slightly more descriptive. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
Matthew Seaman wrote on 2016/12/19 09:45: On 19/12/2016 07:47, David Demelier wrote: I have been working for a while on 2 long standing feature request for the ports tree: flavors and subpackages. For flavors I would like to propose a simple approach first which is more like a rework of the slave ports for now: Examples available here: https://reviews.freebsd.org/D8840 (with the implementation) and https://reviews.freebsd.org/D8843 Design: introduce a 3rd level in the hierarchy and make it work a bit like slave ports pros: - all slave ports are self hosted under the same directory: easier for maintenance - should work with all existing tools This is what I really wanted for years especially for ports like spell checker. Some are in dedicated categories such as french/aspell while other are in textproc/-aspell and that's a big mess. OpenBSD ports has something like textproc/aspell/ and that is very nice and clean. If the plan is to do the same, that is definitely a major improvement. I really like this idea, although it's going to add a lot of extra directories and very similar small Makefiles to the ports. Every python port would grow flavours to support two major versions of python just for starters, and those additional Makefiles would be almost identical across the python2 flavour and across the python3 flavour. Can this be processed by some code in Mk/bsd.*.mk? I mean if we can add something to the main Makefile then we don't need to add subdirectories and sub-Makefiles for each Python module port. Miroslav Lachman ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On 12/18/16 19:31, Baptiste Daroussin wrote: > Hi all, > > I have been working for a while on 2 long standing feature request for the > ports > tree: flavors and subpackages. > [...] Off topic, I know, but might this eventually lead to FLAVORS for base? I would be so grateful to have a SCHED_4BSD flavor of base so I didn't have to keep updating my machines from source every time there was a security update. Maybe someone can at least think about it. -- George ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On 12/19/16 19:53, George Mitchell wrote: On 12/18/16 19:31, Baptiste Daroussin wrote: Hi all, I have been working for a while on 2 long standing feature request for the ports tree: flavors and subpackages. [...] Off topic, I know, but might this eventually lead to FLAVORS for base? I would be so grateful to have a SCHED_4BSD flavor of base so I didn't have to keep updating my machines from source every time there was a security update. Maybe someone can at least think about it. -- George While you wait for the improvement you ask, you can use binary updates and just recompile the kernel by hand. Should spare you some time and effort. -- Guido Falsi ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Mon, 19 Dec 2016 01:31:43 +0100 Baptiste Daroussin wrote: > Hi all, > > I have been working for a while on 2 long standing feature request for the > ports > tree: flavors and subpackages. > > For flavors I would like to propose a simple approach first which is more > like a > rework of the slave ports for now: > > Examples available here: > https://reviews.freebsd.org/D8840 (with the implementation) > and > https://reviews.freebsd.org/D8843 > > Design: introduce a 3rd level in the hierarchy and make it work a bit like > slave > ports > > pros: > - all slave ports are self hosted under the same directory: easier for > maintenance > - should work with all existing tools > > cons: > - hackish: it is not really much more than a slave port > - it adds plenty of new Makefiles :( > > I think anyway this is an improvement > > Next step after that is in would be to extend it to allow some dependency on > "I > depend on whatever flavor if port X" > > Subpackages: > Design: > Add a new macro MULTI_PACKAGES > flag plist with an @pkg{suffixofthesubpackage} file > the framework will split the plist into small plist and create all the > packages > All variables like COMMENT can be overridden with a > COMMENT_${suffixofthesubpackage} > > pros: > - simple and working almost now > - allow to simplify lots of ports > - options friendly (_PACKAGE automatically appends a new entry to > MULTI_PACKAGES) > > cons: > - will break the paradigm that certain tools depend on (portmaster/portupgrade > in particular are a huge problem since they are not actively maintained) > > Example of the usage: > https://people.freebsd.org/~bapt/multipackage.diff > > Note that I took the mpg123 as an example because it was a simple one to test > not because it may need subpackages > > As a result you build 3 packages: > mpg123 (the runtime tools) > mpg123-lib: the runtime libraries > mpg123-sndio: the sndio plugin > > LIB_DEPENDS on ports depending on libmpg123.so does not have to be changed, > the > framework already automatically register only the mpg123-lib as a dependency > and > not others. > > Not the example is missing one thing: a dependency between mpeg123-lib and > mpg123 > > The second is not ready yet and would take time to land > > Any comment? > > Best regards, > Bapt Does this approach would manage a file that differ between flavors? Let's say there a libfoo.so file that behave differently wheter an option A is selected or not, but is still present in both cases. On another note, I kinda liked the macports approach to use the "+" separator regarding naming flavors/options, it allows to better distinguish what in the package name and what are the selected options, and handled itself quite well with multiple instances, like "vim+nls+python+x11"... Did you consider something like that? -- Matthieu Volat pgpKyNhOOomNS.pgp Description: OpenPGP digital signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On 19/12/2016 18:53, George Mitchell wrote: > On 12/18/16 19:31, Baptiste Daroussin wrote: >> Hi all, >> >> I have been working for a while on 2 long standing feature request for the >> ports >> tree: flavors and subpackages. >> [...] > > Off topic, I know, but might this eventually lead to FLAVORS for base? > I would be so grateful to have a SCHED_4BSD flavor of base so I didn't > have to keep updating my machines from source every time there was a > security update. Maybe someone can at least think about it. -- George I believe you can already build packages for several different kernel configurations. Cheers, Matthew signature.asc Description: OpenPGP digital signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
Thanks Bapt et al, I use FreeBSD and the ports system extensively, we build everything from source and largely customise approx 25% of the 900 packages we rely upon. I'm more than a little concerned to have changes performed against the ports infrastructure. As our primary sources of (whats coming) "Change" information are the: Quarterly reports and the OS Release Notes; after-the-fact sources are a daily review of https://lists.freebsd.org/pipermail/svn-src-stable-11/2016-December/thread.html for OS impact; and the excellent Freshports. So a few questions: Could you be able to enlighten us (the readers) so we can better understand what will be changed; or share your vision of the benefits and operational impact for operational people that build: from source; and those that only use binary? Is there a transition plan or schedule for the bulk of these changes to occur? Will the flavors/subpackages be developed separately from the existing ports suite? (I'm hoping that the parent ports will be unaffected, and so our existing build procedures continue to build correctly) How will we (the users/admins) track or be informed of changes or better, planned/soon changes? (will changes to ports, particularly parent ports, be co-ordinated through UPDATING or perhaps a new FLAVOURS file if the parent is say a stub and the real decisions are relocated to slaves?) Will there be any guidance regarding how flavours/packages should be created or the criteria for creating sub-packages (secure/insecure; all options on/off; most useable options on; most liked by the maintainer; most likely to be used for a datacentre; most likely to be used for desktops; ...)? Will "The Porter's Handbook" be updated for things like criteria; naming conventions etc? For folks (like me) that build entirely from source and customise options to build the applications, how will flavours/subpackages be of benefit? Will the ability to customise ports, as they exist today, remain? Will I even notice a change? I'd like to plan ahead to make this transition seemless and continue to use FreeBSD and the excellent ports system as we do now. I started with FreeBSD 2.2.8. There were packages available from the FreeBSD website. It was a terrific aid. We also enjoyed the different flavours of jail that were provided by ezjail. However over time, both evolved as did our expertise to customise our ports (~200 custom ports) and Jamie Gratton evolved the jail system to eliminate our need of the excellent ezjail tool. So I can see merit in, what very little I'm guessing of, the next evolution of ports. Aside: we already build different package configurations from existing ports' source. (eg different bind910 with/without kerberos; different samba44's; simultaineous building of dhcp-[server|client|relay] etc) I look forward to being on the same page and to understand where this is going, the likely/potential impact; the naming conventions; etc. Kind regards, Dewayne. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
Hi, > On 19 Dec 2016, at 1:31 AM, Baptiste Daroussin wrote: > > For flavors I would like to propose a simple approach first which is more > like a > rework of the slave ports for now: This progression sure is nice to see! I like "category/portname/flavour" origin a lot, but how is it handled in terms of packages names? Are we going to see something like: # pkg install myport:flavour We shouldn't use "-" or "/" anyway, should we? Please no fancy things like "~" or so. No arbitrary package names... In OpenBSD, installing flavoured packages has been hard to script in the past, offering a prompt whenever the main package is going to be installed. The thing to think about here is that # pkg install myport Should *only* install the default port, especially with -y option. # pkg install myport: This *could* prompt for flavours, then. The nice thing should be the user doesn't have to care about flavours if that is so. Flavours as you showed can be very simple. Why not go the extra mile here: FLAVOURS= sub1 sub2 OPTIONS_sub1= EXPLICIT LIST OF OPTIONS OPTIONS_sub2= ANOTHER LIST OF OPTIONS And keep everything as is. No need for sub-packages? No implied OPTIONS_DEFAULT, no nothing. A single line to grep and change. :) >From this perspective, nothing changes for users of the ports tree, options are defined by the main port and all of its flavours are neatly stored in the Makefile. People can still use all options during rebuild, even the ones only used in flavours. Cheers, Franco ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
> On 20 Dec 2016, at 9:27 AM, Franco Fichtner wrote: > > We shouldn't use "-" or "/" anyway, should we? Please no fancy things > like "~" or so. No arbitrary package names... To emphasise on this: A flavour should act as a full replacement of its unflavoured package, that means the package name must be kept. Only one flavour (or unflavoured) package can be installed at all times. As an example: A weird package "foo" requires "vim", but the user doesn't want to deal with X11, the user should be able to: # pkg install vim:lite foo This should not try to change "vim:lite" to "vim". # pkg install vim This should be perfectly fine afterwards, too. Every "vim" should act as "vim", not revoking the integrity of the package dependency on vim during e.g. pkg upgrade. No forced install should be needed to do this as long as the shared libraries and dependencies are still satisfied. And maybe the moral of the story is that flavours should not be depended on by default, although it could be a possibility for special cases. This is something that is really really needed. An very good example would be Suricata package with Hyperscan right now, where Hyperscan does not work on all amd64 architectures, so we need to have a replacement package. But if that replacement package without Hyperscan needs to be a separate port, any package depending on Suricata (e.g. a distribution or GUI package) will complain about the missing dependency and try to undo a Suricata-No-Hyperscan package[1] as it conflicts and changes back to the defunct package on upgrade. Cheers, Franco [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210490 ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
> On 20 Dec 2016, at 9:42 AM, Franco Fichtner wrote: > > To emphasise on this: And lastly... if we have the automatic "default" flavour that is defined by the OPTIONS_DEFAULT knobs, we could finally avoid pkg upgrading custom builds by knowing that somebody built a "custom" version of their port and that there is no equivalent to upgrade to. This is exciting! Cheers, Franco ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
Hi, I think it's a nice to have and an improvement. It's quite clean, even if the number of Makefile's can really increase. I've some questions: Q1) It seems obvious (at least to me), that DOCS and EXAMPLES should/could become subpackages. How it could be handled by pkg? Are you thinking to add some "magic" to enable or disable the automatic installation of specific subpackages? Q2) are we opening the door the -devel packages like some Linux distros? Q3) Do you think there is a general way to decide what should stay an OPTION and what should/could become a FLAVOR? Q4) Can FLAVORs be in CONFLICT with each others or only conflict-free FLAVOR will be accepted? If ports can depend to FLAVOR, strange CONFLICTS can arise.. Thanks for the great job! I'll keep contributing as much as I can. Best regards, Luca On Tue, Dec 20, 2016 at 10:12 AM, Franco Fichtner wrote: > >> On 20 Dec 2016, at 9:42 AM, Franco Fichtner wrote: >> >> To emphasise on this: > > And lastly... if we have the automatic "default" flavour that is > defined by the OPTIONS_DEFAULT knobs, we could finally avoid pkg > upgrading custom builds by knowing that somebody built a "custom" > version of their port and that there is no equivalent to upgrade > to. > > This is exciting! > > > Cheers, > Franco > ___ > freebsd-ports@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ports > To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org" ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Tue, Dec 20, 2016 at 10:12:04AM +0100, Franco Fichtner wrote: > And lastly... if we have the automatic "default" flavour that is > defined by the OPTIONS_DEFAULT knobs, we could finally avoid pkg > upgrading custom builds by knowing that somebody built a "custom" > version of their port and that there is no equivalent to upgrade > to. > > This is exciting! While it is exciting, I would be sad to see flavours be the solution to pkg not recognizing build OPTIONS_ as part of a package's identity right now[1]. What is not entirely clear to me: Are flavours always a tuple of values for OPTIONS_ defined by their master port? The reason I bring this up: a binary package is identified by the following information: - pkg name (the master's name, unique over ports tree) - version & revision - the artifacts used to build the binary (tarballs, but also build dependencies, ...) - a vector of available options - a vector of values for the available options - (other stuff you could probably find in a talk on reproducible builds) It is obvious that a master port will have *many* binary incarnations. To my understanding, flavours are a comfortable way to write down some commonly used incarnations. Reducing the package manager's job to checking that some incarnation of the package is present is surely better than no support for this. However, I think the logical next step is to have ports declare that they depend on a subset of specific configuration values being used in their dependencies. In this scenario, flavours are no different to pkg than self-built ports with custom-picked non-(flavour|standard)ized options. This, I would very much prefer. Either way: big thanks to bapt and those who contributed so far! -- Christian [1] Please continue reading for what I understand as 'package identity'. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Tue, Dec 20, 2016 at 07:11:11PM +1100, Dewayne Geraghty wrote: > Thanks Bapt et al, > > I use FreeBSD and the ports system extensively, we build everything from > source and largely customise approx 25% of the 900 packages we rely upon. > I'm more than a little concerned to have changes performed against the > ports infrastructure. As our primary sources of (whats coming) "Change" > information are the: Quarterly reports and the OS Release Notes; > after-the-fact sources are a daily review of > https://lists.freebsd.org/pipermail/svn-src-stable-11/2016-December/thread.html > for OS impact; and the excellent Freshports. > > So a few questions: > > Could you be able to enlighten us (the readers) so we can better understand > what will be changed; or share your vision of the benefits and operational > impact for operational people that build: from source; and those that only > use binary? Sure so there are 2 different things that are requested for a long time by lots of users: 1/ flavors This is the avility to say this port should be built with a list of variations by default. There are 2 kind of flavors: a/ the one that are not conflicting: for example being able to have all python ports available for both python 2 and python 3 by default and at the same time. Right now it is not directly possible without a hack b/ the one that makes variations of a given package right now done a hackish way via the (for example) *-nox11, *-static or *-lite. The goal of the change I do propose aims at making both kind of flavors easily maintainable. The difficulty is bringing in that feature without breaking anything for end users. The clean way would be to to just have a new variable in a given port that describes the possible variations. But that would break all existing external tools that deals with the ports tree. Because they all rely on the fact that there is a mapping between a package name and an origin (not that pkg does not rely on that. It means that for poudriere and synth we can easily adapt them, both are very actively maintained and their design should allow "easily" to integrate that But portmaster and portupgrade would be deeply broken as not actively maintained So I decided to go another way: add a third level to the ports tree. So far we have category/port and I do propose to add a third level: category/port/flavor which will keep the paradigm most tools are expected: 1 packagename == 1 origin Maybe some tools would have to be updated a bit, but that would be minor patches With my current patch the only problem I see that the category/port level is unused if there are flavors while I could certainly make it "the default flavor" the drawback of this approach is it will add a lots of new small files and directory. The most important part of the flavors is probably the ability to provide natively support for python2 and python3 at the same time A good side effect of adding a third level is we could now imagine regrouping some ports (the openbsd ports tree does that already) like aspell where we could have: textproc/aspell and textproc/aspell/fr textproc/aspell/en etc which would make things a bit cleaner 2/ sub packages sub packages is the ability from one build to create multiple packages. The goal is to avoid the giant gcc package as a runtime dependency for example where we could provide a gcc-libs package for the runtime libary It would also allow to save a lot of building time for things like php or qt We could end up with on single port (so one single extraction and one single run of the configure script) while keeping the flexbility of the current split which is existing right now in the ports tree with zillion of complex slave ports. The big issue while designing that solution is it cannot be made transparent for the users as it will for sure break the paradigm: 1 pkgname == 1 origin > > Is there a transition plan or schedule for the bulk of these changes to > occur? For flavours it should be transparent if not that would be a bug except if everyone argue I should break the paradigm 1 pkgname == 1 origin and go for the clean implementation > > Will the flavors/subpackages be developed separately from the existing > ports suite? (I'm hoping that the parent ports will be unaffected, and so > our existing build procedures continue to build correctly) I don't see how it can be developped separately, can you elaborate more? > > How will we (the users/admins) track or be informed of changes or better, > planned/soon changes? (will changes to ports, particularly parent ports, > be co-ordinated through UPDATING or perhaps a new FLAVOURS file if the > parent is say a stub and the real decisions are relocated to slaves?) Yes of course UPDATING and MOVED when needed for example shells/bash-static|shells/bash/static| > > Will there be any guidance regarding how flavours/packages should be > created or the criteria for creating sub-packages (secure/insecure; all > op
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Mon, Dec 19, 2016 at 08:45:36AM +, Matthew Seaman wrote: > On 19/12/2016 07:47, David Demelier wrote: > >> I have been working for a while on 2 long standing feature request for the > >> ports > >> tree: flavors and subpackages. > >> > >> For flavors I would like to propose a simple approach first which is more > >> like a > >> rework of the slave ports for now: > >> > >> Examples available here: > >> https://reviews.freebsd.org/D8840 (with the implementation) > >> and > >> https://reviews.freebsd.org/D8843 > >> > >> Design: introduce a 3rd level in the hierarchy and make it work a bit like > >> slave > >> ports > >> > >> pros: > >> - all slave ports are self hosted under the same directory: easier for > >> maintenance > >> - should work with all existing tools > >> > > This is what I really wanted for years especially for ports like spell > > checker. Some are in dedicated categories such as french/aspell while > > other are in textproc/-aspell and that's a big mess. > > > > OpenBSD ports has something like textproc/aspell/ and that is > > very nice and clean. If the plan is to do the same, that is definitely > > a major improvement. > > > > I really like this idea, although it's going to add a lot of extra > directories and very similar small Makefiles to the ports. Every python > port would grow flavours to support two major versions of python just > for starters, and those additional Makefiles would be almost identical > across the python2 flavour and across the python3 flavour. > > Is it the intention that -devel or versioned ports are treated as > flavours as well? So for example we could end up with lang/python/27 > lang/python/33 lang/python/34 and lang/python/35 ? I think that would be a very very bad idea to have that. There should be only one version for python2 and one version for python3 no more > Why can't you have both flavoured and unflavoured variants of the same > port -- eg. devel/example as well as devel/example/foo and > devel/example/bar ? In theory we could I'm working on that direction now. > > How well do flavours and sub-packages combine? www/nginx would make a > good example there, given it now has both loadable and compiled-in > modules plus a couple of slave ports that just build different module > load-outs. > The should fit well. I have an example locally with php I have lang/php56 which is now a single port that create zillions of packages (the same granularity as we have now in ports) and a flavour lang/php56/zts which does the same. Best regards, Bapt signature.asc Description: PGP signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Mon, Dec 19, 2016 at 07:12:02PM +0100, Miroslav Lachman wrote: > Matthew Seaman wrote on 2016/12/19 09:45: > > On 19/12/2016 07:47, David Demelier wrote: > > > > I have been working for a while on 2 long standing feature request for > > > > the ports > > > > tree: flavors and subpackages. > > > > > > > > For flavors I would like to propose a simple approach first which is > > > > more like a > > > > rework of the slave ports for now: > > > > > > > > Examples available here: > > > > https://reviews.freebsd.org/D8840 (with the implementation) > > > > and > > > > https://reviews.freebsd.org/D8843 > > > > > > > > Design: introduce a 3rd level in the hierarchy and make it work a bit > > > > like slave > > > > ports > > > > > > > > pros: > > > > - all slave ports are self hosted under the same directory: easier for > > > >maintenance > > > > - should work with all existing tools > > > > > > > This is what I really wanted for years especially for ports like spell > > > checker. Some are in dedicated categories such as french/aspell while > > > other are in textproc/-aspell and that's a big mess. > > > > > > OpenBSD ports has something like textproc/aspell/ and that is > > > very nice and clean. If the plan is to do the same, that is definitely > > > a major improvement. > > > > > > > I really like this idea, although it's going to add a lot of extra > > directories and very similar small Makefiles to the ports. Every python > > port would grow flavours to support two major versions of python just > > for starters, and those additional Makefiles would be almost identical > > across the python2 flavour and across the python3 flavour. > > Can this be processed by some code in Mk/bsd.*.mk? > I mean if we can add something to the main Makefile then we don't need to > add subdirectories and sub-Makefiles for each Python module port. If we do that we do break the paradigm: 1 package = 1 origin which will break portmaster/portupgrade for example Bapt signature.asc Description: PGP signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Mon, Dec 19, 2016 at 01:53:35PM -0500, George Mitchell wrote: > On 12/18/16 19:31, Baptiste Daroussin wrote: > > Hi all, > > > > I have been working for a while on 2 long standing feature request for the > > ports > > tree: flavors and subpackages. > > [...] > > Off topic, I know, but might this eventually lead to FLAVORS for base? > I would be so grateful to have a SCHED_4BSD flavor of base so I didn't > have to keep updating my machines from source every time there was a > security update. Maybe someone can at least think about it. -- George > Not at all we are only speaking here on changes in the ports tree that require no changes at all in pkg Best regards, Bapt signature.asc Description: PGP signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Mon, Dec 19, 2016 at 08:25:36PM +0100, Matthieu Volat wrote: > On Mon, 19 Dec 2016 01:31:43 +0100 > Baptiste Daroussin wrote: > > > Hi all, > > > > I have been working for a while on 2 long standing feature request for the > > ports > > tree: flavors and subpackages. > > > > For flavors I would like to propose a simple approach first which is more > > like a > > rework of the slave ports for now: > > > > Examples available here: > > https://reviews.freebsd.org/D8840 (with the implementation) > > and > > https://reviews.freebsd.org/D8843 > > > > Design: introduce a 3rd level in the hierarchy and make it work a bit like > > slave > > ports > > > > pros: > > - all slave ports are self hosted under the same directory: easier for > > maintenance > > - should work with all existing tools > > > > cons: > > - hackish: it is not really much more than a slave port > > - it adds plenty of new Makefiles :( > > > > I think anyway this is an improvement > > > > Next step after that is in would be to extend it to allow some dependency > > on "I > > depend on whatever flavor if port X" > > > > Subpackages: > > Design: > > Add a new macro MULTI_PACKAGES > > flag plist with an @pkg{suffixofthesubpackage} file > > the framework will split the plist into small plist and create all the > > packages > > All variables like COMMENT can be overridden with a > > COMMENT_${suffixofthesubpackage} > > > > pros: > > - simple and working almost now > > - allow to simplify lots of ports > > - options friendly (_PACKAGE automatically appends a new entry > > to > > MULTI_PACKAGES) > > > > cons: > > - will break the paradigm that certain tools depend on > > (portmaster/portupgrade > > in particular are a huge problem since they are not actively maintained) > > > > Example of the usage: > > https://people.freebsd.org/~bapt/multipackage.diff > > > > Note that I took the mpg123 as an example because it was a simple one to > > test > > not because it may need subpackages > > > > As a result you build 3 packages: > > mpg123 (the runtime tools) > > mpg123-lib: the runtime libraries > > mpg123-sndio: the sndio plugin > > > > LIB_DEPENDS on ports depending on libmpg123.so does not have to be changed, > > the > > framework already automatically register only the mpg123-lib as a > > dependency and > > not others. > > > > Not the example is missing one thing: a dependency between mpeg123-lib and > > mpg123 > > > > The second is not ready yet and would take time to land > > > > Any comment? > > > > Best regards, > > Bapt > > Does this approach would manage a file that differ between flavors? Let's say > there a libfoo.so file that behave differently wheter an option A is selected > or not, but is still present in both cases. Yes > > On another note, I kinda liked the macports approach to use the "+" separator > regarding naming flavors/options, it allows to better distinguish what in the > package name and what are the selected options, and handled itself quite well > with multiple instances, like "vim+nls+python+x11"... Did you consider > something like that? No because, actually there were some ports doing that in the past. and we removed that because it makes it hard to identify programatically packages Also not that the information about the options used are already stored in the package and pkg can show them to you Best regards, Bapt signature.asc Description: PGP signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
On Tue, Dec 20, 2016 at 05:41:17PM +0100, Luca Pizzamiglio wrote: > Hi, > > I think it's a nice to have and an improvement. > It's quite clean, even if the number of Makefile's can really increase. > > I've some questions: > > Q1) It seems obvious (at least to me), that DOCS and EXAMPLES > should/could become subpackages. > How it could be handled by pkg? Are you thinking to add some "magic" > to enable or disable the automatic installation of specific > subpackages? It depends having a subpackage for a bunch of README files is useless while having a subpackage for ports with loads of html/pdf files would deserve it yes. Note that there is "magic" planned in pkg at all :) > > Q2) are we opening the door the -devel packages like some Linux distros? That is an entirely different direction and this would be a policy discussion. In short: 1/ technically, yes it would allow that 2/ is that what I am aiming at: not at all > > Q3) Do you think there is a general way to decide what should stay an > OPTION and what should/could become a FLAVOR? Nope that would be probably mostly driven by common sense > > Q4) Can FLAVORs be in CONFLICT with each others or only conflict-free > FLAVOR will be accepted? > If ports can depend to FLAVOR, strange CONFLICTS can arise.. yes flavor can conflict but pkg will detect the conflict and propose to replace the previous flavor if needed > Best regards, Bapt signature.asc Description: PGP signature
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
> On 22 Dec, 2016, at 13:04, Baptiste Daroussin wrote: > > The clean way would be to to just have a new variable in a given port that > describes the possible variations. But that would break all existing external > tools that deals with the ports tree. Because they all rely on the fact that > there is a mapping between a package name and an origin (not that pkg does not > rely on that. It's more than just cleaner; it improves the development workflow dramatically. Variable-based flavours can be added, modified, and removed easily. c/p/f may necessitate recopies and potentially tricky quarterly backports. Flavours and subpackages are a big deal. I'd prefer that aging, non-actively-developed not drive design decisions. I feel like the flavour and subpackage omelettes are worth cracking those eggs for. > So I decided to go another way: add a third level to the ports tree. So far we > have category/port and I do propose to add a third level: category/port/flavor > which will keep the paradigm most tools are expected: 1 packagename == 1 > origin They're not necessarily redundant: variable-based flavours provide for combinations of options, and 3rd-level ports provide a meaningful way to categorize nearly-identical ports (like textproc/aspell/{en,fr}). Personally I'd love to see both those things happen. # Adam -- Adam Weinberger ad...@adamw.org https://www.adamw.org ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
Baptiste Daroussin wrote on 2016/12/22 21:08: On Mon, Dec 19, 2016 at 07:12:02PM +0100, Miroslav Lachman wrote: Matthew Seaman wrote on 2016/12/19 09:45: On 19/12/2016 07:47, David Demelier wrote: I have been working for a while on 2 long standing feature request for the ports tree: flavors and subpackages. For flavors I would like to propose a simple approach first which is more like a rework of the slave ports for now: Examples available here: https://reviews.freebsd.org/D8840 (with the implementation) and https://reviews.freebsd.org/D8843 Design: introduce a 3rd level in the hierarchy and make it work a bit like slave ports pros: - all slave ports are self hosted under the same directory: easier for maintenance - should work with all existing tools This is what I really wanted for years especially for ports like spell checker. Some are in dedicated categories such as french/aspell while other are in textproc/-aspell and that's a big mess. OpenBSD ports has something like textproc/aspell/ and that is very nice and clean. If the plan is to do the same, that is definitely a major improvement. I really like this idea, although it's going to add a lot of extra directories and very similar small Makefiles to the ports. Every python port would grow flavours to support two major versions of python just for starters, and those additional Makefiles would be almost identical across the python2 flavour and across the python3 flavour. Can this be processed by some code in Mk/bsd.*.mk? I mean if we can add something to the main Makefile then we don't need to add subdirectories and sub-Makefiles for each Python module port. If we do that we do break the paradigm: 1 package = 1 origin which will break portmaster/portupgrade for example But we don't have that now. For example dns/py-dnspython can create py27-dnspython, py33-dnspython, py34-dnspython, py35-dnspython - four different packages from one origin, one Makefile. OK, I noticed now that slave port for py3 was added "recently" but even this slave port can create 3 different packages according to what default version of python 3 is installed. Another thing is that portmaster and portupgrade are not well maintained and I think ports / ports framework evolution should not be slowed down because of "3rd party" utilities. Or we will end up with adding tons of directories and small files just to create endless pile of slave ports. Miroslav Lachman ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: HEADSUP: FLAVORS (initial version) and subpackages proposals
> On 23 Dec 2016, at 10:34 AM, Miroslav Lachman <000.f...@quip.cz> wrote: > > But we don't have that now. For example dns/py-dnspython can create > py27-dnspython, py33-dnspython, py34-dnspython, py35-dnspython - four > different packages from one origin, one Makefile. Noticed that too. This could be easily solved with slave ports that set this for e.g. py27-dnspython: USES= python:2.7 and in the master port USES?= python USES+= other-uses-stuff In fact, py-dnspython master port already does it this way. But most Python ports do not. Cheers, Franco ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"