Re: Overhaul package handling in puppet 8
Hi, On Wednesday, April 10, 2024 11:05 CEST, "Sebastian Reitenbach" wrote: > On Wednesday, April 10, 2024 08:26 CEST, Giovanni Bechis > wrote: > > > On 4/10/24 07:10, Sebastian Reitenbach wrote: > > > Hi, > > > On Tuesday, April 09, 2024 22:43 CEST, Giovanni Bechis > > > wrote: > > > > > >> On Tue, Apr 09, 2024 at 10:17:30PM +0200, Sebastian Reitenbach wrote: > > >>> Since we now have a recent Puppet in ports, I started looking at how > > >>> packages are handled with Puppet. > > >>> My current trouble is that it wasn't really possible to install banches > > >>> properly: i.e. can't properly install gimp, > > >>> or auto* based on branch. Or esp, when want to install multiple of > > >>> them, it was just not possible. > > >>> For ports where branches conflict, i.e. postfix, this was working, but > > >>> had to specify exact version, and on every > > >>> upgrade bump the version _very_ annoying. > > >>> > > >>> Currently Puppet allows to install packages of a given version (ensure > > >>> => "X.Y.Z"), or to follow updates (ensure => "latest"). > > >>> > > >> [...] > > >>> This is just for Puppet 8. Anyone still on Puppet 7? It should be > > >>> easily ported to Puppet 7 as well. > > >>> > > >> I am on Puppet 7 and I do not have time to upgrade to Puppet 8 soon; I > > > > > > My upgrade from Puppet 7 to 8 took quite a long time. They removed a lot > > > of backward compat > > > shim, and if you have many old and partially hand made or patched modules > > > like I do, it took > > > quite a while to update all of them > > > > > upgrading is on my queue and I have lot of hand made modules, other then > > that, can a Puppet 7 client connect to a Puppet 8 server ? > > Yes, it worked for me, it's also recommended to upgrade master (and db) > first, then update the clients. > Depending on your modules, you may already see some fallout when just > upgrading the server, > and then even more, when upgrading the agents ;) > This is now in, but just for Puppet8, I left Puppet 7 alone for now. For anyone using Puppet8, for Puppet managed packages that have branches, you now have to use the package name including the branch, i.e. puppet%8, openldap-server%openldap, gimp%stable, jdk%1.8 etc. as you would using pkg_add cheers, Sebastian
Re: Overhaul package handling in puppet 8
On Wednesday, April 10, 2024 08:26 CEST, Giovanni Bechis wrote: > On 4/10/24 07:10, Sebastian Reitenbach wrote: > > Hi, > > On Tuesday, April 09, 2024 22:43 CEST, Giovanni Bechis > > wrote: > > > >> On Tue, Apr 09, 2024 at 10:17:30PM +0200, Sebastian Reitenbach wrote: > >>> Since we now have a recent Puppet in ports, I started looking at how > >>> packages are handled with Puppet. > >>> My current trouble is that it wasn't really possible to install banches > >>> properly: i.e. can't properly install gimp, > >>> or auto* based on branch. Or esp, when want to install multiple of them, > >>> it was just not possible. > >>> For ports where branches conflict, i.e. postfix, this was working, but > >>> had to specify exact version, and on every > >>> upgrade bump the version _very_ annoying. > >>> > >>> Currently Puppet allows to install packages of a given version (ensure => > >>> "X.Y.Z"), or to follow updates (ensure => "latest"). > >>> > >> [...] > >>> This is just for Puppet 8. Anyone still on Puppet 7? It should be easily > >>> ported to Puppet 7 as well. > >>> > >> I am on Puppet 7 and I do not have time to upgrade to Puppet 8 soon; I > > > > My upgrade from Puppet 7 to 8 took quite a long time. They removed a lot of > > backward compat > > shim, and if you have many old and partially hand made or patched modules > > like I do, it took > > quite a while to update all of them > > > upgrading is on my queue and I have lot of hand made modules, other then > that, can a Puppet 7 client connect to a Puppet 8 server ? Yes, it worked for me, it's also recommended to upgrade master (and db) first, then update the clients. Depending on your modules, you may already see some fallout when just upgrading the server, and then even more, when upgrading the agents ;) > > > Attached an untested version of the same changes for the Puppet 7 port. > > > >> am also using "ensure => latest" syntax but I can change my code to get > >> rid of it OpenBSD will no more support it. > > > > What's your use-case to using "ensure => latest"? If there's really a "good > > one", > > I can look into re-adding the feature. > > My use-case up to now for sparingly using "latest" was to force using > > "snapshot" > > branch of some packages. > > > my OpenBSD Puppet server manages Linux and Windows servers as well, on those > OS I want to update automatically only some trusted packages, not all the > packages the distro wants to update. > Anyway I can change that part of my code without big issues. Ah, OK, I only use it to run my home network, and that's purely OpenBSD machines that are managed (: > Cheers > Giovanni > > >> > >> Cheers > >>Giovanni > >> > >> > >>> cheers, > >>> Sebastian > >>> > >>> > >>> Index: Makefile > >>> === > >>> RCS file: /cvs/ports/sysutils/ruby-puppet/8/Makefile,v > >>> diff -u -r1.2 Makefile > >>> --- Makefile 20 Mar 2024 21:21:14 - 1.2 > >>> +++ Makefile 9 Apr 2024 19:51:40 - > >>> @@ -1,6 +1,7 @@ > >>> PORTROACH= limit:^7 > >>> > >>> VERSION=8.5.1 > >>> +REVISION=0 > >>> > >>> RUN_DEPENDS+= > >>> converters/ruby-multi_json,${MODRUBY_FLAVOR}>=1.13,<2 \ > >>> > >>> devel/ruby-concurrent-ruby,${MODRUBY_FLAVOR}>=1,<2 \ > >>> Index: patches/patch-lib_puppet_provider_package_openbsd_rb > >>> === > >>> RCS file: > >>> /cvs/ports/sysutils/ruby-puppet/8/patches/patch-lib_puppet_provider_package_openbsd_rb,v > >>> diff -u -r1.2 patch-lib_puppet_provider_package_openbsd_rb > >>> --- patches/patch-lib_puppet_provider_package_openbsd_rb 20 Mar 2024 > >>> 21:21:14 - 1.2 > >>> +++ patches/patch-lib_puppet_provider_package_openbsd_rb 9 Apr 2024 > >>> 19:51:40 - > >>> @@ -1,42 +1,89 @@ > >>> -- Handle errors from pkg_add > >>> -- Handle uninstall_options being 'nil' by default > >>> -- If no flavor speficied, force the empty flavor with '--' > >>> - but skipping the % un-ambiguity pkg names > >>> -- Bail out on shortform PKG_PATH (i.e. 'ftp.openbsd.org') > >>> -- pkg.conf is gone > >>> -- properly handle packages with multiple versions and flavors, > >>> - i.e. postfix-XXX-flavor > >>> - > >>> +- get rid of versionable (no ensure => "version X.X.X") > >>> +- get rid of upgradeable (ensure => latest) > >>> +- properly support branches > >>> > >>> Index: lib/puppet/provider/package/openbsd.rb > >>> --- lib/puppet/provider/package/openbsd.rb.orig > >>> +++ lib/puppet/provider/package/openbsd.rb > >>> -@@ -24,6 +24,8 @@ Puppet::Type.type(:package).provide :openbsd, :parent > >>> - has_feature :upgradeable > >>> +@@ -6,10 +6,14 @@ require_relative '../../../puppet/provider/package' > >>> + Puppet::Type.type(:package).provide :openbsd, :parent => > >>> Puppet::Provider::Package do > >>> + desc "OpenBSD's f
Re: Overhaul package handling in puppet 8
On 4/10/24 07:10, Sebastian Reitenbach wrote: Hi, On Tuesday, April 09, 2024 22:43 CEST, Giovanni Bechis wrote: On Tue, Apr 09, 2024 at 10:17:30PM +0200, Sebastian Reitenbach wrote: Since we now have a recent Puppet in ports, I started looking at how packages are handled with Puppet. My current trouble is that it wasn't really possible to install banches properly: i.e. can't properly install gimp, or auto* based on branch. Or esp, when want to install multiple of them, it was just not possible. For ports where branches conflict, i.e. postfix, this was working, but had to specify exact version, and on every upgrade bump the version _very_ annoying. Currently Puppet allows to install packages of a given version (ensure => "X.Y.Z"), or to follow updates (ensure => "latest"). [...] This is just for Puppet 8. Anyone still on Puppet 7? It should be easily ported to Puppet 7 as well. I am on Puppet 7 and I do not have time to upgrade to Puppet 8 soon; I My upgrade from Puppet 7 to 8 took quite a long time. They removed a lot of backward compat shim, and if you have many old and partially hand made or patched modules like I do, it took quite a while to update all of them upgrading is on my queue and I have lot of hand made modules, other then that, can a Puppet 7 client connect to a Puppet 8 server ? Attached an untested version of the same changes for the Puppet 7 port. am also using "ensure => latest" syntax but I can change my code to get rid of it OpenBSD will no more support it. What's your use-case to using "ensure => latest"? If there's really a "good one", I can look into re-adding the feature. My use-case up to now for sparingly using "latest" was to force using "snapshot" branch of some packages. my OpenBSD Puppet server manages Linux and Windows servers as well, on those OS I want to update automatically only some trusted packages, not all the packages the distro wants to update. Anyway I can change that part of my code without big issues. Cheers Giovanni Cheers Giovanni cheers, Sebastian Index: Makefile === RCS file: /cvs/ports/sysutils/ruby-puppet/8/Makefile,v diff -u -r1.2 Makefile --- Makefile20 Mar 2024 21:21:14 - 1.2 +++ Makefile9 Apr 2024 19:51:40 - @@ -1,6 +1,7 @@ PORTROACH=limit:^7 VERSION= 8.5.1 +REVISION= 0 RUN_DEPENDS+= converters/ruby-multi_json,${MODRUBY_FLAVOR}>=1.13,<2 \ devel/ruby-concurrent-ruby,${MODRUBY_FLAVOR}>=1,<2 \ Index: patches/patch-lib_puppet_provider_package_openbsd_rb === RCS file: /cvs/ports/sysutils/ruby-puppet/8/patches/patch-lib_puppet_provider_package_openbsd_rb,v diff -u -r1.2 patch-lib_puppet_provider_package_openbsd_rb --- patches/patch-lib_puppet_provider_package_openbsd_rb20 Mar 2024 21:21:14 - 1.2 +++ patches/patch-lib_puppet_provider_package_openbsd_rb9 Apr 2024 19:51:40 - @@ -1,42 +1,89 @@ -- Handle errors from pkg_add -- Handle uninstall_options being 'nil' by default -- If no flavor speficied, force the empty flavor with '--' - but skipping the % un-ambiguity pkg names -- Bail out on shortform PKG_PATH (i.e. 'ftp.openbsd.org') -- pkg.conf is gone -- properly handle packages with multiple versions and flavors, - i.e. postfix-XXX-flavor - +- get rid of versionable (no ensure => "version X.X.X") +- get rid of upgradeable (ensure => latest) +- properly support branches Index: lib/puppet/provider/package/openbsd.rb --- lib/puppet/provider/package/openbsd.rb.orig +++ lib/puppet/provider/package/openbsd.rb -@@ -24,6 +24,8 @@ Puppet::Type.type(:package).provide :openbsd, :parent - has_feature :upgradeable +@@ -6,10 +6,14 @@ require_relative '../../../puppet/provider/package' + Puppet::Type.type(:package).provide :openbsd, :parent => Puppet::Provider::Package do + desc "OpenBSD's form of `pkg_add` support. + ++OpenBSD has the concept of package branches, providing multiple versions of the ++same package, i.e. `stable` vs. `snapshot`. To select a specific branch, ++suffix the package name with % sign follwed by the branch name, i.e. `gimp%stable`. ++ + This provider supports the `install_options` and `uninstall_options` + attributes, which allow command-line flags to be passed to pkg_add and pkg_delete. + These options should be specified as an array where each element is either a +- string or a hash." ++string or a hash." + + commands :pkginfo => "pkg_info", +:pkgadd => "pkg_add", +@@ -18,220 +22,94 @@ Puppet::Type.type(:package).provide :openbsd, :parent + defaultfor 'os.name' => :openbsd + confine 'os.name' => :openbsd + +- has_feature :versionable + has_feature :install_options + has_feature :uninstall_options +- has_feature :upgradeable has_feature :supports_flavors -+ mk_res
Re: Overhaul package handling in puppet 8
Hi, On Tuesday, April 09, 2024 22:43 CEST, Giovanni Bechis wrote: > On Tue, Apr 09, 2024 at 10:17:30PM +0200, Sebastian Reitenbach wrote: > > Since we now have a recent Puppet in ports, I started looking at how > > packages are handled with Puppet. > > My current trouble is that it wasn't really possible to install banches > > properly: i.e. can't properly install gimp, > > or auto* based on branch. Or esp, when want to install multiple of them, it > > was just not possible. > > For ports where branches conflict, i.e. postfix, this was working, but had > > to specify exact version, and on every > > upgrade bump the version _very_ annoying. > > > > Currently Puppet allows to install packages of a given version (ensure => > > "X.Y.Z"), or to follow updates (ensure => "latest"). > > > [...] > > This is just for Puppet 8. Anyone still on Puppet 7? It should be easily > > ported to Puppet 7 as well. > > > I am on Puppet 7 and I do not have time to upgrade to Puppet 8 soon; I My upgrade from Puppet 7 to 8 took quite a long time. They removed a lot of backward compat shim, and if you have many old and partially hand made or patched modules like I do, it took quite a while to update all of them Attached an untested version of the same changes for the Puppet 7 port. > am also using "ensure => latest" syntax but I can change my code to get > rid of it OpenBSD will no more support it. What's your use-case to using "ensure => latest"? If there's really a "good one", I can look into re-adding the feature. My use-case up to now for sparingly using "latest" was to force using "snapshot" branch of some packages. > > Cheers > Giovanni > > > > cheers, > > Sebastian > > > > > > Index: Makefile > > === > > RCS file: /cvs/ports/sysutils/ruby-puppet/8/Makefile,v > > diff -u -r1.2 Makefile > > --- Makefile20 Mar 2024 21:21:14 - 1.2 > > +++ Makefile9 Apr 2024 19:51:40 - > > @@ -1,6 +1,7 @@ > > PORTROACH= limit:^7 > > > > VERSION= 8.5.1 > > +REVISION= 0 > > > > RUN_DEPENDS+= > > converters/ruby-multi_json,${MODRUBY_FLAVOR}>=1.13,<2 \ > > devel/ruby-concurrent-ruby,${MODRUBY_FLAVOR}>=1,<2 \ > > Index: patches/patch-lib_puppet_provider_package_openbsd_rb > > === > > RCS file: > > /cvs/ports/sysutils/ruby-puppet/8/patches/patch-lib_puppet_provider_package_openbsd_rb,v > > diff -u -r1.2 patch-lib_puppet_provider_package_openbsd_rb > > --- patches/patch-lib_puppet_provider_package_openbsd_rb20 Mar 2024 > > 21:21:14 - 1.2 > > +++ patches/patch-lib_puppet_provider_package_openbsd_rb9 Apr 2024 > > 19:51:40 - > > @@ -1,42 +1,89 @@ > > -- Handle errors from pkg_add > > -- Handle uninstall_options being 'nil' by default > > -- If no flavor speficied, force the empty flavor with '--' > > - but skipping the % un-ambiguity pkg names > > -- Bail out on shortform PKG_PATH (i.e. 'ftp.openbsd.org') > > -- pkg.conf is gone > > -- properly handle packages with multiple versions and flavors, > > - i.e. postfix-XXX-flavor > > - > > +- get rid of versionable (no ensure => "version X.X.X") > > +- get rid of upgradeable (ensure => latest) > > +- properly support branches > > > > Index: lib/puppet/provider/package/openbsd.rb > > --- lib/puppet/provider/package/openbsd.rb.orig > > +++ lib/puppet/provider/package/openbsd.rb > > -@@ -24,6 +24,8 @@ Puppet::Type.type(:package).provide :openbsd, :parent > > - has_feature :upgradeable > > +@@ -6,10 +6,14 @@ require_relative '../../../puppet/provider/package' > > + Puppet::Type.type(:package).provide :openbsd, :parent => > > Puppet::Provider::Package do > > + desc "OpenBSD's form of `pkg_add` support. > > + > > ++OpenBSD has the concept of package branches, providing multiple > > versions of the > > ++same package, i.e. `stable` vs. `snapshot`. To select a specific > > branch, > > ++suffix the package name with % sign follwed by the branch name, i.e. > > `gimp%stable`. > > ++ > > + This provider supports the `install_options` and `uninstall_options` > > + attributes, which allow command-line flags to be passed to pkg_add > > and pkg_delete. > > + These options should be specified as an array where each element is > > either a > > +- string or a hash." > > ++string or a hash." > > + > > + commands :pkginfo => "pkg_info", > > +:pkgadd => "pkg_add", > > +@@ -18,220 +22,94 @@ Puppet::Type.type(:package).provide :openbsd, :parent > > + defaultfor 'os.name' => :openbsd > > + confine 'os.name' => :openbsd > > + > > +- has_feature :versionable > > + has_feature :install_options > > + has_feature :uninstall_options > > +- has_feature :upgradeable > > has_feature :supports_flavors > > > > -+ mk_resource_methods > > -+ > > def self.instances > > -
Re: Overhaul package handling in puppet 8
On Tue, Apr 09, 2024 at 10:17:30PM +0200, Sebastian Reitenbach wrote: > Since we now have a recent Puppet in ports, I started looking at how packages > are handled with Puppet. > My current trouble is that it wasn't really possible to install banches > properly: i.e. can't properly install gimp, > or auto* based on branch. Or esp, when want to install multiple of them, it > was just not possible. > For ports where branches conflict, i.e. postfix, this was working, but had to > specify exact version, and on every > upgrade bump the version _very_ annoying. > > Currently Puppet allows to install packages of a given version (ensure => > "X.Y.Z"), or to follow updates (ensure => "latest"). > [...] > This is just for Puppet 8. Anyone still on Puppet 7? It should be easily > ported to Puppet 7 as well. > I am on Puppet 7 and I do not have time to upgrade to Puppet 8 soon; I am also using "ensure => latest" syntax but I can change my code to get rid of it OpenBSD will no more support it. Cheers Giovanni > cheers, > Sebastian > > > Index: Makefile > === > RCS file: /cvs/ports/sysutils/ruby-puppet/8/Makefile,v > diff -u -r1.2 Makefile > --- Makefile 20 Mar 2024 21:21:14 - 1.2 > +++ Makefile 9 Apr 2024 19:51:40 - > @@ -1,6 +1,7 @@ > PORTROACH= limit:^7 > > VERSION= 8.5.1 > +REVISION=0 > > RUN_DEPENDS+= > converters/ruby-multi_json,${MODRUBY_FLAVOR}>=1.13,<2 \ > devel/ruby-concurrent-ruby,${MODRUBY_FLAVOR}>=1,<2 \ > Index: patches/patch-lib_puppet_provider_package_openbsd_rb > === > RCS file: > /cvs/ports/sysutils/ruby-puppet/8/patches/patch-lib_puppet_provider_package_openbsd_rb,v > diff -u -r1.2 patch-lib_puppet_provider_package_openbsd_rb > --- patches/patch-lib_puppet_provider_package_openbsd_rb 20 Mar 2024 > 21:21:14 - 1.2 > +++ patches/patch-lib_puppet_provider_package_openbsd_rb 9 Apr 2024 > 19:51:40 - > @@ -1,42 +1,89 @@ > -- Handle errors from pkg_add > -- Handle uninstall_options being 'nil' by default > -- If no flavor speficied, force the empty flavor with '--' > - but skipping the % un-ambiguity pkg names > -- Bail out on shortform PKG_PATH (i.e. 'ftp.openbsd.org') > -- pkg.conf is gone > -- properly handle packages with multiple versions and flavors, > - i.e. postfix-XXX-flavor > - > +- get rid of versionable (no ensure => "version X.X.X") > +- get rid of upgradeable (ensure => latest) > +- properly support branches > > Index: lib/puppet/provider/package/openbsd.rb > --- lib/puppet/provider/package/openbsd.rb.orig > +++ lib/puppet/provider/package/openbsd.rb > -@@ -24,6 +24,8 @@ Puppet::Type.type(:package).provide :openbsd, :parent > - has_feature :upgradeable > +@@ -6,10 +6,14 @@ require_relative '../../../puppet/provider/package' > + Puppet::Type.type(:package).provide :openbsd, :parent => > Puppet::Provider::Package do > + desc "OpenBSD's form of `pkg_add` support. > + > ++OpenBSD has the concept of package branches, providing multiple > versions of the > ++same package, i.e. `stable` vs. `snapshot`. To select a specific branch, > ++suffix the package name with % sign follwed by the branch name, i.e. > `gimp%stable`. > ++ > + This provider supports the `install_options` and `uninstall_options` > + attributes, which allow command-line flags to be passed to pkg_add and > pkg_delete. > + These options should be specified as an array where each element is > either a > +- string or a hash." > ++string or a hash." > + > + commands :pkginfo => "pkg_info", > +:pkgadd => "pkg_add", > +@@ -18,220 +22,94 @@ Puppet::Type.type(:package).provide :openbsd, :parent > + defaultfor 'os.name' => :openbsd > + confine 'os.name' => :openbsd > + > +- has_feature :versionable > + has_feature :install_options > + has_feature :uninstall_options > +- has_feature :upgradeable > has_feature :supports_flavors > > -+ mk_resource_methods > -+ > def self.instances > - packages = [] > - > -@@ -46,12 +48,6 @@ Puppet::Type.type(:package).provide :openbsd, :parent > - > - packages << new(hash) > - hash = {} > +-packages = [] > +- > ++final = [] > + begin > +- execpipe(listcmd) do |process| > +-# our regex for matching pkg_info output > +-regex = /^(.*)-(\d[^-]*)[-]?([\w-]*)(.*)$/ > +-fields = [:name, :ensure, :flavor] > +-hash = {} > ++ packages = listcmd > ++ packages.each { |package, value| > ++if !package.empty?() > ++ value[:provider] = self.name > ++ final << new(value) > ++end > ++ } > ++ return final > + > +-# now turn each returned line into a package object > +-process.each_line { |line| > +- match = regex.match(line