Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[micah]

intrigeri  writes:

> Hi,
>
> micah:
>> Apollon Oikonomopoulos  writes:
 On the master, I see nothing in the puppet logs, but I do see in the
 apache logs:
 
 newpuppetmaster:8140 0.0.0.0 - - [03/Feb/2017:08:41:30 -0800] "GET 
 /production/certificate/puppetdb? HTTP/1.1" 404 5361 "-" "Ruby"
 
 but nothing else. The puppetmaster has no certs pending to be signed and
 only has one cert signed (the puppetmaster itself). There is nothing in
 /var/lib/puppet/ssl on the master besides the puppetmaster cert bits.
 
 I'm wondering if this works for others, or if maybe this part of the
 puppet3 compatibility was missed?
>>>
>>> From the looks of it, you're using a Webrick puppetmaster. You should 
>>> switch to puppet-master-passenger instead :)
>
>> Hmmm, I've got that installed:
>
>> ii  puppet-master-passenger  4.8.2-1 all 
>>  configuration management system, scalable master service
>
> [...]
>
>> am I missing something here?
>
> Micah, has this been fixed since then somehow? If not, may you please
> report it as a separate bug, since it already affects Stretch and is
> somewhat off-topic on a bug report that's about "Enable a Puppet
> master to connect to PuppetDB"?

Yes, sorry that the result wasn't fed back here - it was a mistake on my
part and unrelated to this package.

micah

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[intrigeri]

Hi,

micah:
> Apollon Oikonomopoulos  writes:
>>> On the master, I see nothing in the puppet logs, but I do see in the
>>> apache logs:
>>> 
>>> newpuppetmaster:8140 0.0.0.0 - - [03/Feb/2017:08:41:30 -0800] "GET 
>>> /production/certificate/puppetdb? HTTP/1.1" 404 5361 "-" "Ruby"
>>> 
>>> but nothing else. The puppetmaster has no certs pending to be signed and
>>> only has one cert signed (the puppetmaster itself). There is nothing in
>>> /var/lib/puppet/ssl on the master besides the puppetmaster cert bits.
>>> 
>>> I'm wondering if this works for others, or if maybe this part of the
>>> puppet3 compatibility was missed?
>>
>> From the looks of it, you're using a Webrick puppetmaster. You should 
>> switch to puppet-master-passenger instead :)

> Hmmm, I've got that installed:

> ii  puppet-master-passenger  4.8.2-1 all  
> configuration management system, scalable master service

[...]

> am I missing something here?

Micah, has this been fixed since then somehow? If not, may you please
report it as a separate bug, since it already affects Stretch and is
somewhat off-topic on a bug report that's about "Enable a Puppet
master to connect to PuppetDB"?

Cheers,
-- 
intrigeri

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[intrigeri]

Hi!

Apollon Oikonomopoulos (Thu, 2 Feb 2017):
> Following up, here's a more detailed course of action:

[...]

>  - As soon as 4.8.2-1 enters testing, I intend to upload 4.8.2-2, with 
>the following changes:
> * Restore the vim-puppet and puppet-el binary packages, which were 
>   removed in 4.4.2-1.
> * Import the PuppetDB terminus from PuppetDB 4.3.0.
> * Ship the PuppetDB terminus in puppet-terminus-puppetdb. (Closes: 
>   #826551)

>It will go through NEW for puppet-terminus-puppetdb, vim-puppet and 
>puppet-el, but there should be no problems here.

>  - When 4.8.2-2 hits unstable and if all is well, I will file for an 
>unblock request to have 4.8.2-2 migrate to testing. At that point the 
>decision will be up to the release team.

>  - If and when 4.8.2-2 is accepted in testing, I will file for a 
>jessie-pu to update 3.7.2-4 in Jessie to also include the PuppetDB 
>terminus (from PuppetDB 2.3.8). Again, the terminus will be provided 
>in a new binary package, puppet-terminus-puppetdb.

> If all goes well, we should end up with PuppetDB-enabled versions in 
> both Jessie and Stretch. Also, for the puppet-terminus-puppetdb package 
> I'm using PuppetDB's version numbers (and not the puppet source 
> version), so that the package can be taken over by the puppetdb source 
> when the latter is ready for Debian.

I really like this plan. Is it still up-to-date?

Cheers,
-- 
intrigeri

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[micah]

Apollon Oikonomopoulos  writes:

>> On the master, I see nothing in the puppet logs, but I do see in the
>> apache logs:
>> 
>> newpuppetmaster:8140 0.0.0.0 - - [03/Feb/2017:08:41:30 -0800] "GET 
>> /production/certificate/puppetdb? HTTP/1.1" 404 5361 "-" "Ruby"
>> 
>> but nothing else. The puppetmaster has no certs pending to be signed and
>> only has one cert signed (the puppetmaster itself). There is nothing in
>> /var/lib/puppet/ssl on the master besides the puppetmaster cert bits.
>> 
>> I'm wondering if this works for others, or if maybe this part of the
>> puppet3 compatibility was missed?
>
> From the looks of it, you're using a Webrick puppetmaster. You should 
> switch to puppet-master-passenger instead :)

Hmmm, I've got that installed:

ii  puppet-master-passenger  4.8.2-1 all
  configuration management system, scalable master service

and apache is configured to use it:

lrwxrwxrwx 1 root root 37 Feb  3 09:45 puppet-master.conf -> 
../sites-available/puppet-master.conf
root@newpuppetmaster:/etc/apache2/sites-enabled# 

and it is running:

root 10853  0.0  0.0 105732  7860 ?Ss   09:46   0:00 
/usr/sbin/apache2 -k start
root 10854  0.0  0.1 474520  9792 ?Ssl  09:46   0:00 Passenger 
watchdog
root 10857  0.0  0.1 1174672 11580 ?   Sl   09:46   0:00 Passenger core
nobody   10862  0.0  0.1 483104 11400 ?Sl   09:46   0:00 Passenger 
ust-router
www-data 10880  0.4  0.0 523892  6680 ?Sl   09:46   0:00 
/usr/sbin/apache2 -k start
www-data 10881  0.6  0.0 589428  6672 ?Sl   09:46   0:00 
/usr/sbin/apache2 -k start
puppet   10945  7.1  0.6 157664 56976 ?Sl   09:46   0:01 Passenger 
AppPreloader: /usr/share/puppet/rack/puppet-master
puppet   10967  0.2  0.6 292752 53796 ?Sl   09:46   0:00 Passenger 
RubyApp: /usr/share/puppet/rack/puppet-master

am I missing something here?

micah

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[Apollon Oikonomopoulos]

On 11:55 Fri 03 Feb , micah wrote:
> 
> Hi,
> 
> Apollon Oikonomopoulos  writes:
> >  - puppet 4.8.2-1 will (hopefully) migrate to testing tomorrow, 3 days 
> >before the Freeze. This will be the first version in Stretch 
> >supporting Puppet 3 clients.
> 
> This has migrated. I've upgraded my Stretch puppet4 server to 4.8.2-1
> and am testing it.
> 
> Unfortunately, I've already found a problem. If I have a new puppet3
> node and I do:
> 
> root@puppetdb:~# puppet agent -t
> Exiting; no certificate found and waitforcert is disabled
> root@puppetdb:~#
> 
> It doesn't generate a CSR, there is no /var/lib/puppet/ssl
> directory. Yes, this is puppet3 that is failing here, but I suspect it
> is because it is not getting the right response from the master.
> 
> On the master, I see nothing in the puppet logs, but I do see in the
> apache logs:
> 
> newpuppetmaster:8140 0.0.0.0 - - [03/Feb/2017:08:41:30 -0800] "GET 
> /production/certificate/puppetdb? HTTP/1.1" 404 5361 "-" "Ruby"
> 
> but nothing else. The puppetmaster has no certs pending to be signed and
> only has one cert signed (the puppetmaster itself). There is nothing in
> /var/lib/puppet/ssl on the master besides the puppetmaster cert bits.
> 
> I'm wondering if this works for others, or if maybe this part of the
> puppet3 compatibility was missed?

>From the looks of it, you're using a Webrick puppetmaster. You should 
switch to puppet-master-passenger instead :)

Apollon

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[micah]

Hi,

Apollon Oikonomopoulos  writes:
>  - puppet 4.8.2-1 will (hopefully) migrate to testing tomorrow, 3 days 
>before the Freeze. This will be the first version in Stretch 
>supporting Puppet 3 clients.

This has migrated. I've upgraded my Stretch puppet4 server to 4.8.2-1
and am testing it.

Unfortunately, I've already found a problem. If I have a new puppet3
node and I do:

root@puppetdb:~# puppet agent -t
Exiting; no certificate found and waitforcert is disabled
root@puppetdb:~#

It doesn't generate a CSR, there is no /var/lib/puppet/ssl
directory. Yes, this is puppet3 that is failing here, but I suspect it
is because it is not getting the right response from the master.

On the master, I see nothing in the puppet logs, but I do see in the
apache logs:

newpuppetmaster:8140 0.0.0.0 - - [03/Feb/2017:08:41:30 -0800] "GET 
/production/certificate/puppetdb? HTTP/1.1" 404 5361 "-" "Ruby"

but nothing else. The puppetmaster has no certs pending to be signed and
only has one cert signed (the puppetmaster itself). There is nothing in
/var/lib/puppet/ssl on the master besides the puppetmaster cert bits.

I'm wondering if this works for others, or if maybe this part of the
puppet3 compatibility was missed?

micah


signature.asc
Description: PGP signature

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[micah anderson]

Apollon Oikonomopoulos  writes:

> On 09:29 Thu 02 Feb , micah wrote:
>> Apollon Oikonomopoulos  writes:
>> 
>> ...
>> >  - As soon as 4.8.2-1 enters testing, I intend to upload 4.8.2-2, with 
>> >the following changes:
>> ...
>> >It will go through NEW for puppet-terminus-puppetdb, vim-puppet and 
>> >puppet-el, but there should be no problems here.
>> 
>> According to the release page[0]:
>> 
>> [2017-Jan-05] Soft freeze (no new packages, no re-entry, 10-day migrations)
>> 
>> Doesn't that mean that this package wont be able to make it through NEW?
>
> It will go through NEW (unstable is not affected by the freeze). Also 
> the "no new packages" stuff actually means "no new source packages in 
> testing".

Ah, I didn't realize this subtle difference.

> I know it's a bit late (but not too late), but I think we should have a 
> shot with the release team. The change is not that big and it will not 
> break existing systems anyway.

I think so too - especially considering DSA uses puppet with
storedconfigs.

micah

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[Apollon Oikonomopoulos]

On 09:29 Thu 02 Feb , micah wrote:
> Apollon Oikonomopoulos  writes:
> 
> ...
> >  - As soon as 4.8.2-1 enters testing, I intend to upload 4.8.2-2, with 
> >the following changes:
> ...
> >It will go through NEW for puppet-terminus-puppetdb, vim-puppet and 
> >puppet-el, but there should be no problems here.
> 
> According to the release page[0]:
> 
> [2017-Jan-05] Soft freeze (no new packages, no re-entry, 10-day migrations)
> 
> Doesn't that mean that this package wont be able to make it through NEW?

It will go through NEW (unstable is not affected by the freeze). Also 
the "no new packages" stuff actually means "no new source packages in 
testing".

I know it's a bit late (but not too late), but I think we should have a 
shot with the release team. The change is not that big and it will not 
break existing systems anyway.

Cheers,
Apollon

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[micah]

Apollon Oikonomopoulos  writes:

...
>  - As soon as 4.8.2-1 enters testing, I intend to upload 4.8.2-2, with 
>the following changes:
...
>It will go through NEW for puppet-terminus-puppetdb, vim-puppet and 
>puppet-el, but there should be no problems here.

According to the release page[0]:

[2017-Jan-05] Soft freeze (no new packages, no re-entry, 10-day migrations)

Doesn't that mean that this package wont be able to make it through NEW?

micah

0. https://release.debian.org/

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[Apollon Oikonomopoulos]

Hi all,

On 22:21 Wed 01 Feb , Apollon Oikonomopoulos wrote:
> Well, for puppet 4 we could add a puppet-terminus-puppetdb binary 
> package to the puppet source. I don't think it makes sense to upload 
> the PuppetDB source just for a couple of Ruby files which are not a 
> direct part of PuppetDB anyway.

Following up, here's a more detailed course of action:

 - puppet 4.8.2-1 will (hopefully) migrate to testing tomorrow, 3 days 
   before the Freeze. This will be the first version in Stretch 
   supporting Puppet 3 clients.

 - As soon as 4.8.2-1 enters testing, I intend to upload 4.8.2-2, with 
   the following changes:
* Restore the vim-puppet and puppet-el binary packages, which were 
  removed in 4.4.2-1.
* Import the PuppetDB terminus from PuppetDB 4.3.0.
* Ship the PuppetDB terminus in puppet-terminus-puppetdb. (Closes: 
  #826551)

   It will go through NEW for puppet-terminus-puppetdb, vim-puppet and 
   puppet-el, but there should be no problems here.

 - When 4.8.2-2 hits unstable and if all is well, I will file for an 
   unblock request to have 4.8.2-2 migrate to testing. At that point the 
   decision will be up to the release team.

 - If and when 4.8.2-2 is accepted in testing, I will file for a 
   jessie-pu to update 3.7.2-4 in Jessie to also include the PuppetDB 
   terminus (from PuppetDB 2.3.8). Again, the terminus will be provided 
   in a new binary package, puppet-terminus-puppetdb.

If all goes well, we should end up with PuppetDB-enabled versions in 
both Jessie and Stretch. Also, for the puppet-terminus-puppetdb package 
I'm using PuppetDB's version numbers (and not the puppet source 
version), so that the package can be taken over by the puppetdb source 
when the latter is ready for Debian.

Regards,
Apollon

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[Apollon Oikonomopoulos]

Hi Georg,

On 20:04 Wed 01 Feb , Georg Faerber wrote:
> On 17-02-01 20:27:54, Apollon Oikonomopoulos wrote:
> > I have already prepared an update (3.7.2-4+deb8u1, available on [0]) 
> > and will file for a jessie-pu to get the SRM's opinion on this.
> 
> We've just spoken for a while in IRC how to proceed with this and had
> "invented" a plan of action, which was a bit different, but anyway, your
> proposal is much better. Thanks for this!
> 
> > I think we should deal with Puppet 4 supporting PuppetDB the same way 
> > (i.e. ship the termini directly in puppet) via an unblock request after 
> > 4.8.2-1 has migrated to testing.
> 
> So, in this case, there is no need for an extra puppet-termini package,
> right? 

Well, for puppet 4 we could add a puppet-terminus-puppetdb binary 
package to the puppet source. I don't think it makes sense to upload the 
PuppetDB source just for a couple of Ruby files which are not a direct 
part of PuppetDB anyway.

> 
> > Opinions/ideas?
> 
> All in all, I really like your proposal.
> 
> Two (minor) notes:
> 
>   - I'm wondering if d/NEWS should be extended giving an example how to
> do the export and import.

I'll probably add a new README file for that.

> 
>   - This should be documented in the stretch release / upgrade notes.

Sure, I'll make a note for that. We also need to document a couple of 
other issues, including the fact that PuppetDB is not yet packaged etc.

Cheers,
Apollon

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[Georg Faerber]

Hi Apollon, all,

On 17-02-01 20:27:54, Apollon Oikonomopoulos wrote:
> Glad to see this work!
> 
> As it seems, we have two issues here:
> 
>  1. The Puppet 3.7 master in Jessie does not support `puppet 
> storeconfigs export'
>  1. The Puppet 4.8 master in Stretch does not have the PuppetDB terminus 
> readily available
> 
> I'm cloning this as a new bug for the export support missing in puppet 
> 3.7. I think the best course of action is to update puppet in Jessie via 
> a stable update to include the following files from the PuppetDB 2.3.8 
> source:
> 
>  ext/master/lib/face/storeconfigs.rb (patched)
>  ext/master/lib/application/storeconfigs.rb
>  ext/master/lib/util/puppetdb/*
> 
> I have already prepared an update (3.7.2-4+deb8u1, available on [0]) and 
> will file for a jessie-pu to get the SRM's opinion on this.

We've just spoken for a while in IRC how to proceed with this and had
"invented" a plan of action, which was a bit different, but anyway, your
proposal is much better. Thanks for this!

> I think we should deal with Puppet 4 supporting PuppetDB the same way 
> (i.e. ship the termini directly in puppet) via an unblock request after 
> 4.8.2-1 has migrated to testing.

So, in this case, there is no need for an extra puppet-termini package,
right? 

> Opinions/ideas?

All in all, I really like your proposal.

Two (minor) notes:

  - I'm wondering if d/NEWS should be extended giving an example how to
do the export and import.

  - This should be documented in the stretch release / upgrade notes.

Thanks again,
cheers,
Georg


signature.asc
Description: Digital signature

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[Apollon Oikonomopoulos]

Control: clone -1 -2
Control: retitle -2 puppet: does not support exporting storeconfigs to PuppetDB
Control: reassign -2 puppet
Control: found -2 puppet/3.7.2-4

Hi micah, all,

On 12:07 Wed 01 Feb , micah wrote:
> micah  writes:
> 
> > I agree that it doesn't look hard to add the terminus package, but I was
> > hoping we could provide some kind of upgrade path for people to keep
> > their storedconfig database, but I can't seem to figure out what is
> > going on here.
> 
> Ok, I got it working:
> 
> 1. wget http://downloads.puppetlabs.com/puppetdb/puppetdb-2.3.8.tar.gz
> 2. verify and uncompress it
> 3. cp -avp puppetdb-2.3.8/ext/master/lib/puppet/* 
> /usr/lib/ruby/vendor_ruby/puppet/
> 4. copy active record dbadapter details to [main] section of puppet.conf
> 5. apply the attached patch[0] to
> /usr/lib/ruby/vendor_ruby/puppet/face/storeconfigs.rb

> --- storeconfigs.orig.rb  2016-08-24 09:04:48.428728886 +
> +++ storeconfigs.rb   2016-08-24 09:51:34.658495419 +
> @@ -35,16 +35,15 @@
>  begin
>Puppet::Rails.connect
>  
> -  # Fetch all nodes, including exported resources and their params
> -  nodes = Puppet::Rails::Host.all(:include => {:resources => 
> [:param_values, :puppet_tags]},
> -  :conditions => {:resources => 
> {:exported => true}})
> -
> -  catalogs = nodes.map { |node| node_to_catalog_hash(node) }
> -
>catalog_dir = File.join(workdir, 'catalogs')
>FileUtils.mkdir(catalog_dir)
> -
> -  catalogs.each do |catalog|
> + 
> +  nodes = []
> +  # Fetch all nodes, including exported resources and their params
> +  Puppet::Rails::Host.find_each(:include => {:resources => 
> [:param_values, :puppet_tags]},
> +:conditions => {:resources => 
> {:exported => true}}, batch_size: 1) do |node|
> +catalog = node_to_catalog_hash(node)
> + nodes << node[:name]
>  filename = File.join(catalog_dir, 
> "#{catalog[:data][:name]}.json")
>  
>  File.open(filename, 'w') do |file|
> @@ -52,7 +51,7 @@
>  end
>end
>  
> -  node_names = nodes.map(&:name).sort
> +  node_names = nodes.sort
>  
>timestamp = Time.now

> 6. puppet storeconfigs export
> 7. copy the exported file to the server running puppetdb software and
> import the data with:
> puppetdb import --infile ./storeconfigs-2017XX.tar.gz
> 
> This wouldn't be too hard to make work in a debian package, so people
> can actually upgrade, but we need the termini package first.
> 
> micah
> 
> 0. https://tickets.puppetlabs.com/browse/PDB-165

Glad to see this work!

As it seems, we have two issues here:

 1. The Puppet 3.7 master in Jessie does not support `puppet 
storeconfigs export'
 1. The Puppet 4.8 master in Stretch does not have the PuppetDB terminus 
readily available

I'm cloning this as a new bug for the export support missing in puppet 
3.7. I think the best course of action is to update puppet in Jessie via 
a stable update to include the following files from the PuppetDB 2.3.8 
source:

 ext/master/lib/face/storeconfigs.rb (patched)
 ext/master/lib/application/storeconfigs.rb
 ext/master/lib/util/puppetdb/*

I have already prepared an update (3.7.2-4+deb8u1, available on [0]) and 
will file for a jessie-pu to get the SRM's opinion on this.

I think we should deal with Puppet 4 supporting PuppetDB the same way 
(i.e. ship the termini directly in puppet) via an unblock request after 
4.8.2-1 has migrated to testing.

Opinions/ideas?

Regards,
Apollon

[0] 
https://anonscm.debian.org/cgit/pkg-puppet/puppet.git/log/?h=storeconfigs-export

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[micah]

micah  writes:

> I agree that it doesn't look hard to add the terminus package, but I was
> hoping we could provide some kind of upgrade path for people to keep
> their storedconfig database, but I can't seem to figure out what is
> going on here.

Ok, I got it working:

1. wget http://downloads.puppetlabs.com/puppetdb/puppetdb-2.3.8.tar.gz
2. verify and uncompress it
3. cp -avp puppetdb-2.3.8/ext/master/lib/puppet/* 
/usr/lib/ruby/vendor_ruby/puppet/
4. copy active record dbadapter details to [main] section of puppet.conf
5. apply the attached patch[0] to
/usr/lib/ruby/vendor_ruby/puppet/face/storeconfigs.rb
--- storeconfigs.orig.rb	2016-08-24 09:04:48.428728886 +
+++ storeconfigs.rb	2016-08-24 09:51:34.658495419 +
@@ -35,16 +35,15 @@
 begin
   Puppet::Rails.connect
 
-  # Fetch all nodes, including exported resources and their params
-  nodes = Puppet::Rails::Host.all(:include => {:resources => [:param_values, :puppet_tags]},
-  :conditions => {:resources => {:exported => true}})
-
-  catalogs = nodes.map { |node| node_to_catalog_hash(node) }
-
   catalog_dir = File.join(workdir, 'catalogs')
   FileUtils.mkdir(catalog_dir)
-
-  catalogs.each do |catalog|
+ 
+  nodes = []
+  # Fetch all nodes, including exported resources and their params
+  Puppet::Rails::Host.find_each(:include => {:resources => [:param_values, :puppet_tags]},
+:conditions => {:resources => {:exported => true}}, batch_size: 1) do |node|
+catalog = node_to_catalog_hash(node)
+	nodes << node[:name]
 filename = File.join(catalog_dir, "#{catalog[:data][:name]}.json")
 
 File.open(filename, 'w') do |file|
@@ -52,7 +51,7 @@
 end
   end
 
-  node_names = nodes.map(&:name).sort
+  node_names = nodes.sort
 
   timestamp = Time.now
6. puppet storeconfigs export
7. copy the exported file to the server running puppetdb software and
import the data with:
puppetdb import --infile ./storeconfigs-2017XX.tar.gz

This wouldn't be too hard to make work in a debian package, so people
can actually upgrade, but we need the termini package first.

micah

0. https://tickets.puppetlabs.com/browse/PDB-165

Bug#826551: [Pkg-puppet-devel] RFP: puppetdb-termini -- Enable a Puppet master to connect to PuppetDB

[micah]

Hi,

Georg Faerber  writes:

> I think the following might be of interest:
>
> I've tested the proposed way of intrigeri, which is described at [1]:
>
>> puppetdb-termini has no dependencies except puppet-agent. It just
>> ships 16 .rb files, that live in the upstream Puppet Git repository,
>> and are distributed in PuppetDB upstream tarballs.
>
> This is described at [2] as well.
>
> I've set up a puppetmaster out of j-bp, copied the .rb files into
> '/usr/lib/ruby/vendor_ruby/puppet', set up upstream puppetdb in another
> machine, and configured the puppetmaster to talk to the puppetdb. This
> works as expected, and creating a puppet-termini package doesn't seem to
> be hard. As the puppet packages are team maintained, I could join the
> team and create such an initial package, if that's the way to go.

I just tried to do this too and can confirm what you found.

However, I was trying to follow the directions[0] to migrate my
storedconfigs database to puppetdb. In order to do that, you need to do
a 'puppet storeconfigs export'.

That only works in puppet3, so I grabbed the puppetdb-termini package[1]
version 3.2.4[2] as this was the latest version I could find that wasn't
version 4, and copied those .rb files as you did, and then was able to
run 'puppet storeconfigs export', but weirdly it doesn't actually
generate the tarball that I expect:

root@newpuppetmaster:~# puppet storeconfigs --verbose export
/usr/lib/ruby/vendor_ruby/puppet/defaults.rb:465: warning: key :queue_type is 
duplicated and overwritten on line 466
Info: Connecting to mysql database: puppet
root@newpuppetmaster:~#

that is the right database and right database connector, but I dont see
it doing anything, even when I strace.

I agree that it doesn't look hard to add the terminus package, but I was
hoping we could provide some kind of upgrade path for people to keep
their storedconfig database, but I can't seem to figure out what is
going on here.

any ideas?

micah

0. https://docs.puppet.com/puppetdb/3.0/migrate.html
1. later it was renamed terminus
2. 
https://apt.puppetlabs.com/pool/jessie/PC1/p/puppetdb/puppetdb-termini_3.2.4-1puppetlabs1_all.deb


signature.asc
Description: PGP signature