Re: Starting with poudriere
Hi! > On 15 Feb 2020, at 21:43, Dan McGrath wrote: > > You would run "poudriere bulk", then sit back sipping coffee while it > > churns through all of the packages. > > Hang on a second, so the intended use for poudriere is to build ALL packages? Only those you want, and automatically those, that are required to build those. For example, I have a file 'server', which is used to build all the ports I need for servers. --- archivers/arc archivers/arj archivers/cabextract archivers/deb2targz archivers/gtar archivers/lha archivers/libmspack archivers/libzip [...] --- I use a small script: --- #!/usr/local/bin/bash poudriere ports -u stmp=`date +%s` cd ~/pkg cat php74 server > all poudriere bulk -T -f ~/pkg/all -j cur > ~/logs/cur-all-$stmp --- to build those. My build host runs current, and it can build for current and older versions: poudriere jail -l lists my build systems: JAILNAME VERSION ARCH METHOD TIMESTAMP PATH 112 11.2-RELEASE amd64 ftp 2018-06-28 07:36:14 /pou/jails/112 113 11.3-RELEASE amd64 ftp 2019-07-11 06:42:40 /pou/jails/113 120 12.0-RELEASE-p9 amd64 ftp 2019-08-10 15:06:39 /pou/jails/120 121 12.1-RELEASE amd64 ftp 2019-11-04 20:37:14 /pou/jails/121 12i 12.1-RELEASE i386 ftp 2019-11-04 21:46:18 /pou/jails/12i arm 13.0-CURRENT 1300013 r344710 arm64.aarch64 svn+ssh 2019-03-02 14:35:50 /pou/jails/arm cur 13.0-CURRENT 1300054 r354278 amd64 svn+ssh 2019-11-03 13:17:47 /pou/jails/cur curi 13.0-CURRENT 1300054 r354278 i386 svn+ssh 2019-11-03 12:12:11 /pou/jails/curi > Right now I have two jails setup, one for 12.1 amd64 and one for 11.3 i386. > That seems like a *LOT* of compiling/building. Yes, it's a lot of compiling. I only build our selection of ports (approx. 2000 ports), and all it requirements, in sum it's approx. 3800 ports. To be fair, I could probably strip a awful lot of the 2000 ports, that number is that high because I did not find the time to prune that list. And: I do not build it every day, only one every week or so. And: If I want to build a new port, I have a different script, to build exactly that and its requirements: --- #!/usr/local/bin/bash echo $1 > ~/pkg/one poudriere ports -u poudriere bulk -T -f ~/pkg/one -j cur --- Called like this: bulkcurone shells/bash it just builds bash, and updates bash and its requirements. If that means to rebuild gmake, it will also rebuild gmake. So after such a 'bulkone' action, if I update the target system using pkg upgrade it might upgrade only bash, or a few other packages, or a lot, if I did not upgrade that target system for a long time. Which is fine, because, for almost all cases, nothing breaks if all that stuff is updated. I can even test complex dependencies. I do this using jail-specific make.conf files, for example: /usr/local/etc/poudriere.d/121-default-make.conf WARNING_WAIT=0 WITH_MPM=event DEFAULT_VERSIONS= perl5=5.30 python=3.6 python3=3.6 ruby=2.6 pgsql=12 php=7.3 mysql=10.4m gcc=9 samba=4.10 If I want to experiment with php 7.4, I change that field, rebuild all (bulk121), wait a bit and everything is set. > And once I've build, say, ImageMagick or > postfix/dovecot/mariadb/apache/etc how do I then deploy them to the > 11.3 server (as in, a different machine)? I have a webserver running on the box, https://repo.opsec.eu/ (not public) or on the company build host, https://repo.nepustil.net/ (public). There, we have the ready-to-use package repos available, just as symlinks into the poudriere directory tree. On my target systems, I use: mkdir -p /usr/local/etc/pkg/repos/ cat > nepustil.conf
Re: x11/x3270
El día sábado, febrero 15, 2020 a las 02:55:21p. m. +0100, Kurt Jaeger escribió: > Hi! > > > Hey, there's an opportunity for you to step up and update that port! > > (And perhaps ping the maintainer if he's still around and interested > > in that port, not having done anything in about five years is an > > indicator of sorts. This port can be yours, at the cost of a patch!) > > This can be yours for the cost of some run-tests now 8-) > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244138 > > has a patch for an update. Thanks, I can't test this at the moment because I'm still building ports on head with poudriere. Will come back to this later. matthias -- Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub signature.asc Description: PGP signature
Re: Starting with poudriere
On 16 Feb 2020, at 02:02, Kurt Jaeger wrote: > /usr/local/etc/poudriere.d/121-default-make.conf And this conf file applies when you use the jail named exactly “121-default”? So for my jail named 121x64 the conf file would be "121x64-make.conf" And for 113x86 "113x86-make.conf" Ok, this is starting to make some sense. Now I just need to get a list of the ports pon the 11.3 system currently in the form / so I can at least set that up with the current ports. Not sure I quite get how the Webserver lets your other machines get the packages in such a way that they can be dropped in place, but I don’t need to do that right now anyway (that will be a task for when I have migrated that machine to 12.1 amd64 anyway), hopefully in the next month. -- "He has no enemies, but is intensely disliked by his friends.." Oscar Wilde ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Starting with poudriere
Hi! > On 16 Feb 2020, at 02:02, Kurt Jaeger wrote: > > /usr/local/etc/poudriere.d/121-default-make.conf > > And this conf file applies when you use the jail named exactly '121-default' The 121-default is the name of the jail and the name of the ports tree used. You can have several ports trees in parallel: $ poudriere ports -l PORTSTREE METHOD TIMESTAMP PATH default portsnap 2020-02-16 10:02:15 /pou/ports/default snmp portsnap 2019-11-01 16:21:32 /pou/ports/snmp I have one to test net/net-snmp 5.8 as well. > So for my jail named 121x64 the conf file would be > > "121x64-make.conf" > > And for 113x86 > > "113x86-make.conf" Not quite, see above. > Ok, this is starting to make some sense. Now I just need to get > a list of the ports on the 11.3 system currently in the form > / so I can at least set that up with the current > ports. The man page for pkg-query shows many options to ask for all this. pkg query '%o' gives you this list. > Not sure I quite get how the Webserver lets your other machines > get the packages in such a way that they can be dropped in place, > but I don't need to do that right now anyway (that will be a task > for when I have migrated that machine to 12.1 amd64 anyway), hopefully > in the next month. pkg accesses the repo, downloads some meta data (like meta.txz), and compares that data with the packages you already have. -- p...@opsec.eu+49 171 3101372Now what ? ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Starting with poudriere
On 16/02/2020 00:14, Dan McGrath wrote: Hi, Just a bit of a heads up that poudriere will require you to be on the new version of FreeBSD before you can build for it on the current system. For example, if you are running 12.1, and you upgrade poudriere's jail to 13.0, it will complain that you have to be running that version on the host you are using it on. Ideally, poudriere should be running on it's own dedicated system, not the one you intend to deploy to. Just a note that this is not a strict requirement. I have been upgrading from FreeBSD 9 to 12 currently and was always building on the same system that I am deploying to. Yes, poudriere will complain that the jail is newer than the base system, but that did not create any major practical problem for me yet. I think only on one occasion I got a build error due to missing symbols. Then the solution for me was to upgrade the base system. This of course broke the applications that were installed for the older base, but thanks to the FreeBSD's separation of base from ports, it's still possible to start FreeBSD with just the command line. Then I finished building the ports and reinstalled them. Not that I encourage this approach, it might create additional issues to solve, but it is possible/manageable and shouldn't be held against using poudriere. GrzegorzJ ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Starting with poudriere
Hi, On Sun, Feb 16, 2020 at 10:51 AM Grzegorz Junka wrote: > Just a note that this is not a strict requirement. I have been upgrading > from FreeBSD 9 to 12 currently and was always building on the same > system that I am deploying to. Yes, poudriere will complain that the > jail is newer than the base system, but that did not create any major > practical problem for me yet. > > I think only on one occasion I got a build error due to missing symbols. > Then the solution for me was to upgrade the base system. This of course > broke the applications that were installed for the older base, but > thanks to the FreeBSD's separation of base from ports, it's still > possible to start FreeBSD with just the command line. Then I finished > building the ports and reinstalled them. > > Not that I encourage this approach, it might create additional issues to > solve, but it is possible/manageable and shouldn't be held against using > poudriere. > Ah, good to know. It's been a long time now since I ran into that, so I was a little hazy on the details of the error. I love the separation of FreeBSD from ports, and indeed, being able to recover from broken userland is nice, although I hate the days where I remove libs first, then try run sudo, and have to go in via BMC. Worse is if you have a critical website down. Especially if you have to stop and spend a bunch of time compiling ports before you go live! :D Also, tex and llvm, wtf is with those build times?! heh Anyway, thanks for clarifying! Cheers, Dan McGrath ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Starting with poudriere
@lbutlr writes: > On 15 Feb 2020, at 21:43, Dan McGrath wrote: > > You would run "poudriere bulk", then sit back sipping coffee while it > > churns through all of the packages. > > Hang on a second, so the intended use for poudriere is to build ALL > packages? That's not a poudriere constraint, it's a packages vs. ports thing. And it's not ALL PACKAGES, it's all the packages you intend to use. You can run a system using pkgs, because the packages are built from a single consistent ports tree. If you're going to build and install things from ports, you should build and install *all the things* from ports so that you can guarantee that everything's using consistent [versions/configurations of] dependencies. If/when you update the ports tree, you should rebuild *all the things* [that have changed]. You can sometimes get away with installing one or two things from ports, but eventually, sadness happens. Populating your own package repository with pkgs that you build from ports gives you the best of both worlds. You can manage your system using the `pkg` tool and/but you get to choose the versions and configuration of the ports you want to use. > Right now I have two jails setup, one for 12.1 amd64 and one for > 11.3 i386. That seems like a *LOT* of compiling/building. I used to be able to build everything I needed to build a postfix/dovecot/etc... mailserver (loosely this: https://www.c0ffee.net/blog/mail-server-guide/) in under an hour on a 2-core 4GB server at ARP networks. Sadly I learned to love ripgrep, so now I need to build rust, which takes ages and ages. > And once I’ve build, say, ImageMagick or > postfix/doveco/.=mariadb/apache/etc how do I then deploy them to > the 11.3 server (as in, a different machine)? The digital ocean tutorial I pointed you at earlier explains this nicely, in the "Configuring Package Clients" section. Rather than bother with nginx I just use: s3cmd sync -F --delete-removed /usr/local/poudriere/data/packages/12_1-ports/.latest/ s3:///pkg-builder/packages/FreeBSD:12:amd64-ports/ and then perhaps an `s3cmd setacl ...` to set permissions as required. g. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Starting with poudriere
John Kennedy writes:
> On Sat, Feb 15, 2020 at 09:02:39PM -0700, @lbutlr wrote:
> > [...]
> > Am I writing a config file for this every port I want to build?
>
> Personally, I have a single, custom make.conf that I maintain and shove into
> /usr/local/etc/poudriere.d (default location I believe).
>
> Inside the make.conf, you can bracket non-default options like this:
>
> .if ${.CURDIR:M*/ftp/curl}
> OPTIONS_FILE_UNSET += TLS_SRP
> .endif
>
I managed these bits via ansible, using
https://github.com/reallyenglish/ansible-role-poudriere
g.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Starting with poudriere
Dan McGrath writes: > [...] I am not sure about repo priorities, or how you would deal > with conflicts with build options that pull in common ports. It is > something I have been meaning to look into, sorry! Perhaps someone else > here can give some advice? > One way to solve this is via "portshaker", which can layer a "thin" ports tree on top of the standard tree. Here's a [perhaps not entirely graceful, but It Works For Me] example where I layer a couple of ports onto the standard tree. https://github.com/hartzell/freebsd-ports I use the resulting tree for poudriere builds and populate jails with e.g. my LMS audio system. g. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Starting with poudriere
@lbutlr writes: > [...] > Not sure I quite get how the Webserver lets your other machines get > the packages in such a way that they can be dropped in place, [...] Nicely described in the DigitalOcean Tutorial[do] But basically, you put your tree full of built packages somewhere where they're accessible via http[s] and the you configure the pkg system on the client machines to use that url instead of pkg.freebsd.org (or whatever the standard is). Other things are possible, you could put them on a file server, mount them via NFS onto the client and configure the pkg system to use file:// URLS g. [do]: https://www.digitalocean.com/community/tutorials/how-to-set-up-a-poudriere-build-system-to-create-packages-for-your-freebsd-servers ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
FreeBSD ports you maintain which are out of date
Dear port maintainer, The portscout new distfile checker has detected that one or more of your ports appears to be out of date. Please take the opportunity to check each of the ports listed below, and if possible and appropriate, submit/commit an update. If any ports have already been updated, you can safely ignore the entry. You will not be e-mailed again for any of the port/version combinations below. Full details can be found at the following URL: http://portscout.freebsd.org/po...@freebsd.org.html Port| Current version | New version +-+ multimedia/webvfx-qt5 | 0.4.4-20160823 | 1.2.0 +-+ If any of the above results are invalid, please check the following page for details on how to improve portscout's detection and selection of distfiles on a per-port basis: http://portscout.freebsd.org/info/portscout-portconfig.txt Reported by:portscout! ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
