Re: [Puppet Users] PuppetDB: manually import reports

[Wyatt Alt]

On 4/19/18 4:46 AM, Thomas Müller wrote:

HI

I've got some prod puppetserver/puppetdb and some dev 
puppetserver/puppetdb. But to have the complete overview over all 
nodes with the prod puppetdb I'd like to import the reports from the 
dev puppetserver (stored by reports=store config) into the prod puppetdb.


is there some hidden tool to do so? I wasn't able to find anything in 
that direction.


Reading 
https://github.com/puppetlabs/puppetdb/blob/master/puppet/lib/puppet/reports/puppetdb.rb 
this could maybe adapted to read a yaml file and then send it to puppetdb.


If I'm understanding you right, you could normally use the import/export 
tools for this:


https://puppet.com/docs/puppetdb/5.0/anonymization.html#using-the-export-command

There's a corresponding "admin" API on PuppetDB you can search for. The 
process would be to do an export, extract the resulting tarball and 
remove everything but reports (if desired), then tar it up again and run 
it through the import tool. Unfortunately though, this is broken for me 
on current PDB due to PDB-3796. If you're on an older version it may be 
worth a try -- it worked at some point. If you've got the bug it'll 
cause your dev server to OOM and restart.


Assuming that's broken for you too, I think the most tractable way to do 
what you're asking is basically what you're suggesting -- either parse 
the yaml reports into the report wire format 
(https://puppet.com/docs/puppetdb/5.1/api/wire_format/report_format_v8.html) 
and post to your prod PDB's commands endpoint 
(https://puppet.com/docs/puppetdb/5.1/api/command/v1/commands.html) or 
get the json reports out of your dev PuppetDB, in batches to work around 
the bug, and do the equivalent parsing/posting. The wire formats change 
from time to time so take care to use whatever version of the docs 
aligns with your PDB version.


Wyatt




- Thomas
--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/d5a3b811-655f-4497-84de-a5693954d08e%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/ae2bf29b-c3e5-c0cf-8bff-a75258cad299%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Multiple Upgrades required for PuppetDB 2.3.x -> 4.4.1 and 5.x requirement?

[Wyatt Alt]

On 12/05/2017 11:25 PM, Dietrich, Stefan wrote:

My main concern were the database migrations run by PuppetDB during the 
upgrade, which seems not to be an issue (despite the ancient version).
Yes, there shouldn't be a problem there (though they may take some time 
depending on your data).


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/1e997efe-e697-c23f-9611-fd1b2972c242%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Multiple Upgrades required for PuppetDB 2.3.x -> 4.4.1 and 5.x requirement?

[Wyatt Alt]

On 12/05/2017 01:57 PM, Dietrich, Stefan wrote:

Hello,

in order to finish our Puppet 4.x migration (and than Puppet 5.x, yay), 
PuppetDB has to be upgraded as well.

If I read the upgrade docs correctly, a direct upgrade from PuppetDB 2.3.x is 
not supported [1].
So in order to do this, an upgrade to the latest version for each major release 
is required:

2.3.x -> 3.2.4 -> 4.4.1 (-> 5.1.3 once we go to Puppet 5)

Is this correct?
It's a bit more complicated than that, unfortunately. The docs aren't 
completely correct on this, and the best source of info is probably this 
ticket: https://tickets.puppetlabs.com/browse/DOCUMENT-719


In the comments I've outlined the upgrade path, which I just tested. The 
key thing to watch out for is to make sure you upgrade puppetdb before 
the puppetdb terminus, and ensure everything is working before moving 
the terminus and puppet server to version 5. Note also that there's a 
postgres bump, so it may make sense to get that out of the way first.


Additionally, the release notes of PuppetDB 5 mention a hard dependency on 
Puppet Agent 5.x [2]
So for PuppetDB 5 _all_ agents have to be at 5.x first?
Or is it sufficient to have Puppet Agent 5.x on Puppetserver/PuppetDB hosts?
I had no issues running a 4.x agent against a 5.x puppetserver/puppetdb. 
The puppetdb upgrade will pull in a 5.x agent on the puppetdb host itself.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/56ef91e1-b757-dcaa-d734-fb7089be2b88%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Re: order by value on fact_contents

[Wyatt Alt]

Hi Bryan,
This is a bug -- right now fact_contents doesn't support ordering on value, 
but the docs as you say, as well as this ticket: 
https://tickets.puppetlabs.com/browse/PDB-1649 suggest that's a regression. 
I've created https://tickets.puppetlabs.com/browse/PDB-3687 to track the 
fix. In the meantime I've updated the 4.2.3.x docs to reflect reality, and 
we'll roll that update forward to master. Thanks for bringing this up and 
sorry for the news.

Wyatt

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/2165accd-ddf2-4907-99a6-bf0d78d3b577%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB Upgrade Question from 2.1.1-1 to 2.7.2-1

[Wyatt Alt]

Hey Peter,

PuppetDB never released a 2.7 series (highest 2.x went was 2.3.x), so 
it's tough to answer your question specifically. In general, the size of 
your data (for which number of nodes is a proxy) can definitely have a 
large effect on migration times, but it largely depends on the specific 
upgrade. Structured facts landed sometime in 2.x after 2.1 and was a 
relatively heavy migration, but there are many other examples.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/ef1f076e-c43d-a34b-41c4-0993766b5c20%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Problem in PuppetDB

[Wyatt Alt]

On 07/20/2017 11:29 AM, Alexandre Monteiro wrote:



We are unable to create the recommended pg_trgm indexes due to
the extension not being installed correctly.  Run the command:

CREATE EXTENSION pg_trgm;

as the database super user on the PuppetDB database to correct
this, then restart PuppetDB.

Can this impact the viewing of my PuppetDB?
Sure -- creating the extension and restarting PuppetDB will cause an 
index to be created that will make certain types of regular expression 
queries against facts more performant. I recommend creating it, but it 
isn't required for the service to operate.


Wyatt






Em quinta-feira, 20 de julho de 2017 15:13:38 UTC-3, Wyatt Alt escreveu:

That's the behavior I'd expect -- I believe the issue is you're
just not
hitting an endpoint. If you're trying to verify that the service
is up
and running, the way we typically do that is with

curl http://localhost:8080/pdb/meta/v1/version
<http://localhost:8080/pdb/meta/v1/version>

which should return a blob with a version number.

Wyatt


On 07/20/2017 08:28 AM, Alexandre Monteiro wrote:
> Hi folks,
>
> I'm installing PuppetDB on my desktop. All configuration of
> puppetserver, postgre is OK .. but occurring return 302 in my curl
> test. I read many forums and it still did not work ..
> Here is the error:
>
> # curl -v http://localhost:8080
> * About to connect() to localhost port 8080 (#0)
> *   Trying 127.0.0.1...
> * Connected to localhost (127.0.0.1) port 8080 (#0)
> > GET / HTTP/1.1
> > User-Agent: curl/7.29.0
> > Host: localhost:8080
> > Accept: */*
> >
> < HTTP/1.1 302 Found
> < Date: Thu, 20 Jul 2017 15:27:18 GMT
> < Location: /pdb/dashboard/index.html
> < Content-Length: 0
> < Server: Jetty(9.2.z-SNAPSHOT)
> <
> * Connection #0 to host localhost left intact
>
> Has anyone seen this yet?
>
> Tks!!
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from
it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
>

https://groups.google.com/d/msgid/puppet-users/809eaf77-52b8-4451-be17-5556b8416943%40googlegroups.com

<https://groups.google.com/d/msgid/puppet-users/809eaf77-52b8-4451-be17-5556b8416943%40googlegroups.com>

>

<https://groups.google.com/d/msgid/puppet-users/809eaf77-52b8-4451-be17-5556b8416943%40googlegroups.com?utm_medium=email_source=footer

<https://groups.google.com/d/msgid/puppet-users/809eaf77-52b8-4451-be17-5556b8416943%40googlegroups.com?utm_medium=email_source=footer>>.

> For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
<mailto:puppet-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/70798eb8-e204-483d-8f3d-ce24c6e4d082%40googlegroups.com 
<https://groups.google.com/d/msgid/puppet-users/70798eb8-e204-483d-8f3d-ce24c6e4d082%40googlegroups.com?utm_medium=email_source=footer>.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/1d77350d-87aa-644a-7cd3-f11298607a82%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Problem in PuppetDB

[Wyatt Alt]

That's the behavior I'd expect -- I believe the issue is you're just not 
hitting an endpoint. If you're trying to verify that the service is up 
and running, the way we typically do that is with


curl http://localhost:8080/pdb/meta/v1/version

which should return a blob with a version number.

Wyatt


On 07/20/2017 08:28 AM, Alexandre Monteiro wrote:

Hi folks,

I'm installing PuppetDB on my desktop. All configuration of 
puppetserver, postgre is OK .. but occurring return 302 in my curl 
test. I read many forums and it still did not work ..

Here is the error:

# curl -v http://localhost:8080
* About to connect() to localhost port 8080 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Thu, 20 Jul 2017 15:27:18 GMT
< Location: /pdb/dashboard/index.html
< Content-Length: 0
< Server: Jetty(9.2.z-SNAPSHOT)
<
* Connection #0 to host localhost left intact

Has anyone seen this yet?

Tks!!

--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/809eaf77-52b8-4451-be17-5556b8416943%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/0140200f-c5fb-73ca-66b8-0be1344c8bac%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: PuppetDB - High CPU Large number of KahaDB files and very little work going to postgresql

[Wyatt Alt]

The userlist is most likely your your issue. This seems analogous to 
FACT-1345 and PDB-2631 for context -- we usually see this with the os, 
mountpoints, and partitions facts. What's generating the userlist and 
what are the numbers it's valued with? Do you have code that generates 
the fact? There may be ways you can restructure it to make it easier on 
the database.


Wyatt

On 06/30/2017 06:11 AM, Peter Krawetzky wrote:
So if I'm reading this correctly, the userlist#~(number) represents 
the value of the userlist fact?  If that is the case, the size of the 
userlist fact is 228k each and every time puppet agent runs with 
approximately 3300 nodes.


On Wednesday, June 28, 2017 at 12:25:57 PM UTC-4, Peter Krawetzky wrote:

Last Sunday we hit a wall on our 3.0.2 puppetdb server.  The cpu
spiked and the KahaDB logs started to grow eventually almost
filling a filesystem.  I stopped the service, removed the mq
directory per a troubleshooting guide, and restarted.  After
several minutes the same symptoms began again and I have not been
able to come up with a puppetdb or postgresql config to fix this.

We tried turning off storeconfig in the puppet.conf file on our
puppet master servers but that doesn't appear to have resolved the
problem.  I also can't find a good explanation as to what this
parameter really does or does not do even in the puppet server
documentation.  Anyone have a better insight into this?

Also is there a way to just turn off puppetdb?

I've attached a file that is a snapshot of the puppetdb dashboard.

Anyone experience anything like this?

--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/814883b8-9077-48b8-a94a-42865881f94d%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/d26d012c-5a5b-50c3-4646-7b2de752f8e0%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: PuppetDB - High CPU Large number of KahaDB files and very little work going to postgresql

[Wyatt Alt]

Peter,

You said no work is going to postgresql but how did you determine that? Are
you able to see if PuppetDB (java) or postgres is consuming the CPU? How is
disk activity?

Slow command processing is often correlated with long database queries --
it would be useful to turn on slow query logging for postgres
(log_min_duration_statement) and watch for anything taking longer than a
second or two. The guide here:
https://docs.puppet.com/puppetdb/4.0/pdb_support_guide.html#puppetdb-4.0:-support-and-troubleshooting-guide
has some additional information that is mostly relevant to your version in
case it's useful.

If you see some slow queries, report back and we can go from there.

Some other questions:
* Was the setup working fine up to when you posted this? How long has it
been running?
* Have you changed anything in your infrastructure that can be correlated
to this? Sudden onset of this kind of thing is often due to introduction of
e.g inlined file content if that helps
* what does your PDB setup look like (how many machines, what kinds of
machines, how much RAM, etc).

Wyatt

On Wed, Jun 28, 2017 at 12:54 PM, Mike Sharpton  wrote:

> Hmm, I had thought these were one in the same.  I don't have a 3
> environment to look at anymore.  Good luck.
>
> On Wednesday, June 28, 2017 at 2:37:35 PM UTC-5, Peter Krawetzky wrote:
>>
>> I looked at both documents and the second one references the scheduler
>> log files filling up.  Mine are actually in the KahaDB directory.
>>
>> On Wednesday, June 28, 2017 at 12:25:57 PM UTC-4, Peter Krawetzky wrote:
>>>
>>> Last Sunday we hit a wall on our 3.0.2 puppetdb server.  The cpu spiked
>>> and the KahaDB logs started to grow eventually almost filling a
>>> filesystem.  I stopped the service, removed the mq directory per a
>>> troubleshooting guide, and restarted.  After several minutes the same
>>> symptoms began again and I have not been able to come up with a puppetdb or
>>> postgresql config to fix this.
>>>
>>> We tried turning off storeconfig in the puppet.conf file on our puppet
>>> master servers but that doesn't appear to have resolved the problem.  I
>>> also can't find a good explanation as to what this parameter really does or
>>> does not do even in the puppet server documentation.  Anyone have a better
>>> insight into this?
>>>
>>> Also is there a way to just turn off puppetdb?
>>>
>>> I've attached a file that is a snapshot of the puppetdb dashboard.
>>>
>>> Anyone experience anything like this?
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/puppet-users/06d2078d-120e-46b7-8f60-006582a39d47%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3F0U-uwdmkmNCA0VYUL%3Dc8ab0rm%2Bv_K20JMTUcoDUKzrA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] resource_title breaks the 40 character limit

[Wyatt Alt]

On 04/22/2017 11:54 PM, Bill Sirinek wrote:


A side effect of this change is that my PuppetDB command queue depth 
went from just over 100 to 0, and has stayed there for a couple hours 
now, maybe occasionally poking up to 2-3 briefly. At least I think it 
was related to this change. :)
You're probably right about that. Since you run Puppet on a regular 
interval, I expect you'd have had a relatively constant number of 
reports in retries at any given time, which would keep your queue 
nonempty. Anyway, glad to help and that it's resolved.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/18afa1b5-67c2-3235-0f99-e869d0bd201b%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] resource_title breaks the 40 character limit

[Wyatt Alt]

On 04/22/2017 10:35 AM, Bill Sirinek wrote:


2017-04-22 13:30:25.912 EDT 
[db:pe-puppetdb,sess:58fb8caf.8cbb,pid:36027,vtid:29/23277,tid:396758] 
STATEMENT:  INSERT INTO resource_events ( new_value, 
corrective_change, property, file, report_id, old_value, 
containing_class, certname_id, line, resource_type, status, 
resource_title, timestamp, containment_path, message ) VALUES ( $1, 
$2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15 ) RETURNING *
2017-04-22 13:30:25.978 EDT 
[db:pe-puppetdb,sess:58fb8caf.8cbb,pid:36027,vtid:29/23281,tid:396760] 
ERROR:  value too long for type character varying(40)


This is probably being caused by the property names rather than resource 
titles. The only varchar(40) columns in that table are the property name 
and the event status, and I'm assuming you're not doing anything custom 
with the statuses (which typically come from Puppet).


There is no supported workaround for this, but I put up a PR here 
https://github.com/puppetlabs/puppetdb/pull/2268 to resolve it.


An unsupported workaround would be to shut down PuppetDB, connect to 
postgres via psql and do this:


\c pe-puppetdb
alter table resource_events alter column property type text;

This could take anywhere from seconds to 30+ minutes depending on how 
much data you have, so if that's a concern you can get in touch with 
support and coordinate with them. Doing this kind of thing is usually a 
really bad idea, but in this case it won't hurt because a future 
migration to change the old varchar column to text will simply be a noop.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/7765e54e-6bdf-3fc3-b301-84c5679b4ec7%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] [PuppetDB] records not being expired from puppetdb?

[Wyatt Alt]

Hey Christopher,

This is the default behavior of PuppetDB -- my guess is you can address 
it by tuning your node-ttl and node-purge-ttl parameters, which are 
described here:


https://docs.puppet.com/puppetdb/latest/configure.html#node-ttl

PuppetDB won't expire or delete node data unless those parameters are 
set, and they aren't by default. The reports are deleted after 14 days 
by default (report-ttl setting), which would explain why you can see 
node data but no reports.


Wyatt


On 02/21/2017 11:05 AM, Christopher Wood wrote:

Our security department raised that point that some nodes present in puppetdb 
are not for current or recently decommissioned servers.

Does anybody have a spare hint as to why these nodes haven't become expired 
over the last few months of not being servers, or where I can look for more 
information? (PuppetDB 3.2.4.)

More details:

On further investigation, I can retrieve old catalogs for these nodes. The catalogs are weeks or 
months old, and I thought the nodes themselves might have been expired by now. Sure enough, there 
is nothing in the "deactivated" or "expired" fields in the certnames table in 
PostgreSQL for these nodes. The hosts are definitely gone as servers.

curl --data-urlencode 'query=["=", "certname", "myhost.mydomain.com"]' -v -s -S -X 
GET --cacert $ca --cert $cert --key $key https://puppetdb2.mydomain.com:8081/pdb/query/v4/catalogs 
>/tmp/myhost.json

I am unable to retrieve reports for these nodes (200 response from puppetdb but 
no actual report, '[]'). Likewise they do not appear in Puppet Explorer as 
nodes. (Same as above but /reports not /catalogs.)

When I deactivated one of these nodes (puppet node deactivate) I was still able to 
retrieve the same old catalog that I was able to before, but this time the 
"deactivated" field in the certnames table was filled in.

puppetdb=# select * from certnames where certname = 'myhost.mydomain.com';
-[ RECORD 1 ]+---
id   | 2035
certname | myhost.mydomain.com
latest_report_id |
deactivated  | 2017-02-21 12:28:25.495-05
expired  |

We're on a slightly older version of PuppetDB (3.2.4). That said, Puppetdb has 
been ticking along just fine for months and this is the first problem I can 
remember.

bash-4.1$ rpm -q postgresql95-server
postgresql95-server-9.5.0-2PGDG.rhel6.x86_64

bash-4.1$ rpm -q puppetdb
puppetdb-3.2.4-1.el6.noarch

(Also, I can't find any reference to this issue with google searches or looking 
on tickets.puppetlabs.com, and this is as far as I can figure out this issue.)



--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/709a1e9b-1a10-ef9c-2144-5728bf2527d5%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Re: Puppet DB prune / vacuum

[Wyatt Alt]

Hey Anfield,

prune_upto was retired before 2016.2, along with the console database 
itself (the data is in PuppetDB now). The way to control report deletion is 
via the node_ttl, node_purge_ttl, and report_ttl PuppetDB configuration 
settings, which are documented here:
https://docs.puppet.com/puppetdb/latest/configure.html#node-ttl

The vacuum operations on the page you linked may be necessary if you start 
to run into performance issues resulting from database bloat, but they'll 
only clear dead tuples in the database; they won't delete data. My guess is 
you'll be fine with just the config settings, but you can read more about 
vacuum here if needed:

https://www.postgresql.org/docs/9.4/static/sql-vacuum.html

Wyatt


On Thursday, February 16, 2017 at 1:41:11 PM UTC-8, Anfield wrote:
>
> For cleaning up report data is the parameter prune_upto still valid? - 
> related to version 2016.2
>
> Or has the vacuum parameter replaced it? Do they both essentially do the 
> same thing?
>
> https://docs.puppet.com/pe/2016.2/maintain_console-db.html
>
> Thanks
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/494fb6ec-6cc5-44ba-a0df-3e07f28fb17b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Re: puppetdb 4.3 on PG 9.4 fresh install fails to start with DB UTF8 encoding error

[Wyatt Alt]

Hi,

This seems like a bug to me, but I can't reproduce it on postgres 9.4 + 
centos 7.3 and the combination of PDB 4.3 and postgres 9.4 is widely 
used/tested.

Can you give a bit more detail about your environment? I'd like to know 
your OS and version and the method you used to install postgres. It would 
also be good to see the output of

'\l+'

executed in psql, which will give you a table like this:
https://gist.github.com/wkalt/bbffafe68ec4018f7bcbc5b62961dc24

Thanks,
Wyatt

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/f4cf6ce7-c8fb-41ce-af40-acb4384993d3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Could not retrieve catalog from remote server: Error 400 Unsupported query parameter 'command'

[Wyatt Alt]

On 01/16/2017 07:05 PM, alohaaa...@gmail.com wrote:

Hello,
I just added postgresql 9.6 to my puppet server and receive this error.
Error: Could not retrieve catalog from remote server: Error 400 on 
SERVER: [400 ] Unsupported query parameter 'command'


I also looked at the debug info and installed the gem msgpack just in 
case, which seemed like just a warning and an optional installation.


Thanks for the help.

#Installation info.
* Both puppetdb and puppetserver are on the same host.
* I can telnet to port 8081 from the puppet client
* I signed the certificate on the puppet master after running puppet 
agent -t on the client

* Versions are:
puppet server: 2.3.0
puppetdb: 3.2.4
puppet client: 4.4.1
What version of the PuppetDB terminus are you using on the master? The 
package should be "puppetdb-termini" and the version should match 
PuppetDB's.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/43cb4f99-6d03-d7a9-2f71-6db779f6b077%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Urgent Help Required | Puppet run is dead slow

[Wyatt Alt]

On 12/12/2016 10:09 AM, Harish Kothuri wrote:

I see that following errors are popping up from puppetdb. There are quite a
few of these and also would like to know for which machines it is coming
from

2016-12-12 10:08:16,040 ERROR [c.p.p.command]
[afe425be-8413-49a8-8086-79ebaa00927a]
[store report] Retrying after attempt 13, due to:
org.postgresql.util.PSQLException:
ERROR: duplicate key value violates unique constraint "reports_pkey"
org.postgresql.util.PSQLException: ERROR: duplicate key value violates
unique constraint "reports_pkey"


This error occurs when a report is submitted to PuppetDB twice -- the first
time it will be stored, and the second time this error will occur. It's
conflicting on the hash of a report, which includes certname and creation
time, so these errors will pretty much only arise in the case a two
submissions.

We've seen these caused by puppetdb imports on nonempty databases, or
sometimes sporadically due to bug in shutdown coordination between PuppetDB
and ActiveMQ (these have gotten much better since your version; in fact,
AMQ is gone). It's not typically a cause for concern and is IMO unlikely to
be the root of your issue, though it could be a symptom of PDB restarts
like Matthaus said.

At this point if I were you I'd look in the PuppetDB logs to try and
correlate a restart with the connection failure you mentioned. In
particular, keep an eye out for places where a startup is logged but the
preceding shutdown is not, which generally indicates that an OOM occurred.
Depending on how your service is configured you may have better ways of
detecting this, such as a puppetdb-daemon.log file in the puppetdb log
directory.

As for knowing which machines the errors originated from, the messages that
prompt the failures will become available in your dead letter office after
16 retries, and will contain the certnames. You can read more about that
here:
https://docs.puppet.com/puppetdb/latest/maintain_and_tune.html#clean-up-the-dead-letter-office

Wyatt

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3FU3UEfCZm-TtrMBQ_y%3DHPqYQAdFx2C%3DhvO3RMWMxS_KQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 4.2.4 is now available!

[Wyatt Alt]

PuppetDB 4.2.4 - October 27, 2016

PuppetDB 4.2.4 Downloads



Available in native package format as part of Puppet Collection 1 (PC1).
More information on the PC1 repositories is available here:
http://bit.ly/1HQJDNb

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site by submitting a
ticket with an “Affects Version/s” field of “PDB 4.2.4”:
https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/4.2/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 4.2.4 Release Notes



PuppetDB 4.2.4 is a backward-compatible bugfix release. It includes several
fixes and improvements for the PE-only “corrective_change” feature and
removes some inappropriate caching that could cause OOMs in installations
with many very large resource parameters, as well as a number of other
minor fixes.

More information on the specifics of the release can be found in the
official release notes: https://docs.puppetlabs.com/
puppetdb/4.2/release_notes.html

Contributors

---
Andrew Roetker, Jeremy Barlow, Karel Březina, Ken Barber, Kylo Ginsberg,
Molly Waggett, Rob Browning, Ryan Senior, Wyatt Alt, and Tiffany Longworth

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3EFsKwkuyXqdyfg3Li65tOXOJcHHT16MxGUXDBOG5P%3D7A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] puppetdb 4.x filling up /opt/puppetlabs/server/data/puppetdb/mq

[Wyatt Alt]

Hey Ryan,

At first glance this sounds like 
https://tickets.puppetlabs.com/browse/PDB-2390, but that was fixed in 
4.0.0 so it shouldn't be the issue. The fact that you have 2 gigs under 
discarded means you are sending a significant number of commands that 
PuppetDB is unable to accept. There are numerous reasons this can happen 
-- you'll probably find more info in your logs. Given that you have so 
many discards, it could be that the bloat in your scheduler directory is 
actually legitimate (malformed commands get retried, and while being 
retried they go to scheduler.


My suggestion would be to look at your PuppetDB logs and/or the files 
under discard (these will have errors in addition to the actual message) 
to figure out the problem. Post back with the errors if need be and we 
can trouble shoot from there.


Wyatt

On 9/21/16 5:28 AM, Ryan Anderson wrote:
I'm using puppetdb-4.0.0-1.el7 (open source) and over time the 
ActiveMQ part of it has been saving a bunch of logs 
under /opt/puppetlabs/server/data/puppetdb/mq, and there does not 
appear to be any built-in log rotation, at least in the version I am 
using. After using this puppetdb a couple months, this is the sizing I 
observe:


  * 6.5GB = /opt/puppetlabs/server/data/puppetdb/mq/localhost/scheduler
  * 2GB = /opt/puppetlabs/server/data/puppetdb/mq/discard


The only solution I found online is 
this: https://docs.puppet.com/puppetdb/4.2/trouble_kahadb_corruption.html. 
The workaround there is to rename the mq directory and restart 
puppetdb, but it is intended as a troubleshoot, and I am seeking to 
save disk space and avoid regular puppetdb restarts.


What files are safe to delete? Is this still a problem if I upgrade to 
puppetdb 4.2?

--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/ec2b2314-cd8f-41c4-a6d6-16936ec157c9%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/64e20ffd-5802-32a6-e03b-0398574fbd94%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 4.2.2 is released!

[Wyatt Alt]

PuppetDB 4.2.2 - August 25, 2016

PuppetDB 4.2.2 Downloads



Available in native package format as part of Puppet Collection 1 (PC1).
More information on the PC1 repositories is available here:
http://bit.ly/1HQJDNb

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site by submitting a
ticket with an "Affects Version" field of "PDB 4.2.2":
https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/4.2/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 4.2.2 Release Notes



PuppetDB 4.2.2 is a backward-compatible bugfix release that fixes an issue
with a migration up to 4.2.0 that could cause an out of memory error and
reduces CPU usage associated with our internationalization library.


Consult the release notes for more information: https://docs.puppetlabs.com/
puppetdb/4.2/release_notes.html.


-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3GFsZ_i85EKXA1oMZQGKERV7QG04gT5VtsafVLXSe%3DBrQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: Puppetdb has not logged any new input in past 7 hours

[Wyatt Alt]

Hey Bret, sorry for the gap in communication. Try making these changes:

* to the [master] section,
storeconfigs=true
storeconfigs_backend=puppetdb
reports=puppetdb

* to the [agent] section for each node, change "reports=puppetdb" to 
"report=true".


I think that should straighten it out.

Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/fee444eb-5c18-1b96-00b0-cd9e14542353%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 4.2.1 is released!

[Wyatt Alt]

PuppetDB 4.2.1 - August 16, 2016

PuppetDB 4.2.1 Downloads



Available in native package format as part of Puppet Collection 1 (PC1).
More information on the PC1 repositories is available here:
http://bit.ly/1HQJDNb

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site by submitting a
ticket with an "Affects Version" field of "PDB 4.2.1":
https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/4.2/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 4.2.1 Release Notes



PuppetDB 4.2.1 is a backward-compatible bugfix release that fixes a
regression with single-argument usage of "and" and "or" clauses in the
query language, and also a bug that prevents the "noop" reports field from
being set when used with older versions of puppet-agent.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3HbB%3DiO5LzJOZiTP%2B14PkN80XK1BFq0OV_KxGCjoEkVwA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: Need to move puppetdb database -- best approach?

[Wyatt Alt]

On 8/17/16 6:26 AM, Bret Wortman wrote:


On Wednesday, August 17, 2016 at 9:19:34 AM UTC-4, Bret Wortman wrote:

I'll confess that I know next to nothing about Postgres databases,
so I'm going to ask before I royally mess anything up: what's the
best way to relocate my database from one partition to another? I
need to move my files from their current location (which seems to
be /opt/puppetlabs/server/data/puppetdb/mq/localhost to somewhere
under /data. I know I'll need to shut puppetdb and postgresql down
to do the move, but how do I ensure everything is available &
ready when I need to bring things back up again?

The reason is that I'm getting messages in my puppetdb log files
like this:

Store limit is 102400 mb (current store usage is 10 mb). The data
directory:
/opt/puppetlabs/server/data/puppetdb/mq/localhost/KahaDB only ahs
20671 mb of usable space - resetting to maximum available disk
space: 20682.



Before going down the road of moving postgres, note that this error 
isn't actually talking about postgres, but rather PuppetDB's vardir, 
which is used by activemq. Vardir can be changed with this parameter: 
https://docs.puppet.com/puppetdb/latest/configure.html#vardir.


The warning is saying you only have 20 gigs of space in your vardir, and 
so the store limit will be reset to 20 gigs instead of the default of 
100. You can make the warning go away by setting this parameter to 20 
gigs: 
https://docs.puppet.com/puppetdb/latest/configure.html#store-usage, or 
you can change the vardir parameter to point somewhere on /data, though 
the queue will not generally get to 20 gigs unless something is very 
wrong (note that you currently use 10mb). The warning is common and not 
generally a big deal.


If it happens that postgres is also on that 20 gig partition and you're 
still worried about that, the process for moving the cluster would be


* stop puppetdb and postgres
* copy the postgres data directory (can be located with "show 
data_directory;" run in psql) to the /data partition using cp -a, to 
keep permissions
* update the PGDATA variable in your postgres systemd unit file to point 
to the new location

* restart postgres + puppetdb

If all goes well things will start up, and you can delete the original, 
but you may have to work through some permissions issues to ensure that 
the postgres user really can access the new location. I'd definitely 
recommend testing the process first and somehow ensuring you're able to 
wind it back easily.


Wyatt



The /data partition is much larger and I won't have to worry about
filling up my root partition this way.

Thanks,


Bret

--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/a04a8c97-2047-4877-aa85-e259796d80cd%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/a78bf5a3-c81c-3435-38da-05e56af18222%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: Puppetdb has not logged any new input in past 7 hours

[Wyatt Alt]

Can you post a gist of your master's full puppet.conf?

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3GVqS3rND_%2Bbwfyn14waXkwHy8WTydNZM8E8Q1pQptp%3DA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Re: Puppetdb has not logged any new input in past 7 hours

[Wyatt Alt]

I'd check the timestamps in the output of

curl -X GET http://localhost:8080/pdb/query/v4/nodes/fs1603.our.net

(with a real certname if that one is fake). These will tell you when the
last report/facts/catalog for the node were stored.

Nothing about the log output you've posted indicates an issue storing data,
so I'm wondering if the problem is with what puppet explorer/puppetboard
are considering an "update". For example maybe reports got switched off and
only reports are considered in that determination (I have no idea if this
is true).

Wyatt

On Wed, Aug 17, 2016 at 9:39 AM, Bret Wortman  wrote:

> Versions:
>
>
> # rpm -qa | grep puppet
> puppetserver-2.4.0-1.el7.noarch
> puppetdb-4.2.0-1.el7.noarch
> puppetexplorer-2.0.0-1.noarch
> puppetdb-termini-4.2.0-1.el7.noarch
> puppet-agent-1.6.0-1.el7.x86_64
> # rpm -qa | grep postgres
> postgresql95-libs-9.5.4-1PGDG.rhel7.x86_64
> postgresql95-9.5.4-IPGDG.rhel7.x86_64
> postgresql95-server-9.5.4-IPGDG.rhel7.x86_64
> postgresql95-contrib-9.5.4-IPGDG.rhel7.x86_64
> #
>
>
> On Wednesday, August 17, 2016 at 12:37:12 PM UTC-4, Bret Wortman wrote:
>>
>> My puppetdb instance is up and running but hasn't stored any updates of
>> any kind in the past 7 hours, according to both Puppet Explorer and
>> Puppetboard.
>>
>> The process is running and so is postgres. Puppet configs haven't changed
>> in that time. /var/log/puppetlabs/puppetdb/puppetdb.log shows plenty of
>> updates coming in:
>>
>> 2016-08-17 16:31:48,887 INFO  [p.p.command] [-UUID-] [replace facts]
>> branfile1.our.net
>> 2016-08-17 16:31:59,136 INFO  [p.p.command] [-UUID-] [replace facts]
>> zs311.our.net
>> 2016-08-17 16:32:11,347 INFO  [p.p.command] [-UUID-] [replace facts]
>> gs1205.our.net
>> 2016-08-17 16:32:12,982 WARN  [p.p.q.engine] The event-counts entity is
>> experimental and may be altered or removed in the future.
>> 2016-08-17 16:32:15,237 INFO  [p.p.command] [-UUID-] [replace facts]
>> fs1603.our.net
>>
>>
>> and so on.
>>
>> There's nothing obviously wrong when Puppetdb starts up in the logfile,
>> either. What should I be looking at to see why it's accepting these updates
>> but (apparently) not storing them? Does anyone know postgres better than I
>> do who could give me some way to directly query the database to see if one
>> of the above updates (or any other, for that matter) exists?
>>
>> Thanks!
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/puppet-users/8a76426c-1da9-4d24-9ffe-ffc9d8cae59b%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3FB9JcJRTXRUpjO1_NdVwV0THs0krUVu5agyi29a8Ficw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] puppetdb instllation form source

[Wyatt Alt]

Hi Joaquin,

If you're able to install with the module that's the best option by far, 
since it will handle all the configuration for you.


If something is requiring you to install from source, the instructions 
are here:

https://docs.puppet.com/puppetdb/4.1/install_from_source.html#step-2-option-a-install-from-source

Wyatt

On 7/26/16 9:39 AM, Joaquin Henriquez wrote:

Hi Guys

Trying to install puppetdb form source.
I downloaded it from git b but then what?

If I modify the puppet.conf to enable puppetdb it will lok into file 
confdir/puppetdb.conf which doesn't exist under the git version.


Doing the config.ini rename to that puppetdb.conf will need to add the 
# to few lines to comment them out.


Jetty for the puppetdb.conf (under /etc/puppetlabs/puppet) only allows 
Jetty port 8080 which even if I use "puppet master --verbose 
--no-daemonize" will not come up.
If adding to the puppetdb.conf the ssl-host and ssl-port will not work 
either as it give a parse error.


Error: Could not retrieve facts for testbedocg: Failed to find facts 
from PuppetDB at puppet:8140: undefined method `url_path' for 
Puppet::Util::Puppetdb:Module

Error: undefined method `puppet3compat?' for Puppet::Util::Puppetdb:Module


Should I install puppetdb from module or form git source?


--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/b994a0ce-0736-40e9-8277-bfe5f3986284%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/ababb8f3-a03a-0933-2cf5-579445f4e5e3%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PQL Questions

[Wyatt Alt]

On 07/10/2016 01:32 PM, R.I.Pienaar wrote:

hey,

Been playing with the PQL language and it's quite nice, have 2 questions.

I want to do a regex case insensitive match, docs mention you can do whatever
Postgres supports but I can't figure out how to do case insensitive matches
with PQL?

Postgres supports ~ and ~* operators but PQL only supports ~. So I suppose
I could match [aA][bB][cC] to get case insensitivity, but thats a tad ugly

Hey R.I,

I agree -- the docs are incorrect here and if you link the page you're 
looking at I'll change them. PuppetDB currently only supports postgres's 
~ operator, but I think adding support for ~* would be a good feature 
and technically easy, so I've created 
https://tickets.puppetlabs.com/browse/PDB-2861 to address it. For now 
your hack is probably the best you can do.


Next I have a fact like:

   foo => { bar => ["a", "b", "c"]}

How do I search into that where bar contains "b"? There's an "in" operator
but I can't seem figure out both how to search inside the sub hash and also
how to match inside a array like that?
Querying structured facts in PQL is still pretty inconvenient in some 
cases, but you could do this with something like


fact_contents{path ~> ["foo", "bar", "\\d"] and value = "b"}

This can be inserted as a subquery to get the certnames associated with 
such a value, e.g


nodes{certname in fact_contents[certname]{path ~> ["foo", "bar", "\\d"] 
and value = "b"}}


Incidentally, we have two tickets in flight (PDB-2632 and PDB-2633) that 
will enable the nicer syntax


inventory{facts.foo.bar.match("\\d") = "b"}

to do the same thing (subject to tweaking still but that's the gist of 
it). These should be included in the next feature release, assuming no 
issues pop up. Having some kind of explicit handling for array 
containment seems like an interesting idea -- we'll have to give it some 
thought.


Wyatt


---
R.I.Pienaar




--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/2a4505ee-0615-517c-a2c0-de0036202f3c%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] BROKEN PUPPETDB

[Wyatt Alt]

Virat,

Please describe what is going wrong with puppetdb. If there is an error 
in the puppetdb log, share it in a gist.


Wyatt

On 06/30/2016 02:45 PM, Virat wrote:

Hello All,

I have configured opensource puppet master 4.5.2 & puppetdb 4.1.2. 
Then started connecting master and db servers as below.


puppet master
/etc/puppetlabs/puppet/puppet.conf
[master]
vardir = /opt/puppetlabs/server/data/puppetserver
logdir = /var/log/puppetlabs/puppetserver
rundir = /var/run/puppetlabs/puppetserver
pidfile = /var/run/puppetlabs/puppetserver/puppetserver.pid
codedir = /etc/puppetlabs/code
server = puppetmaster.com
dns_alt_names = puppetmaster.com, puppetmaster
reports = store,puppetdb
storeconfigs_backend = puppetdb
storeconfigs = true
environment_timeout = unlimited

[main]
certname = puppetmaster.com
server = puppetmaster.com
environment = production
runinterval = 1h
strict_variables = true
#pluginsync = true
#trusted

/etc/puppetlabs/puppet/puppetdb.conf
[main]
server_urls = https://puppetdb.com:8081

/etc/puppetlabs/puppet/routes.yaml
---
master:
  facts:
terminus: puppetdb
cache: yaml
/etc/puppetlabs/puppetdb/conf.d/jetty.ini
[jetty]
# IP address or hostname to listen for clear-text HTTP. To avoid 
resolution

# issues, IP addresses are recommended over hostnames.
# Default is `localhost`.
# host = 
host = puppetmaster ip

# Port to listen on for clear-text HTTP.
port = 8080

# The following are SSL specific settings. They can be configured
# automatically with the tool `puppetdb ssl-setup`, which is normally
# ran during package installation.

# IP address to listen on for HTTPS connections. Hostnames can also be 
used

# but are not recommended to avoid DNS resolution issues. To listen on all
# interfaces, use `0.0.0.0`.
ssl-host = 0.0.0.0

# The port to listen on for HTTPS connections
ssl-port = 8081

# Private key path
ssl-key = /etc/puppetlabs/puppetdb/ssl/private.pem

# Public certificate path
ssl-cert = /etc/puppetlabs/puppetdb/ssl/public.pem

# Certificate authority path
ssl-ca-cert = /etc/puppetlabs/puppetdb/ssl/ca.pem

# Access logging configuration path. To turn off access logging
# comment out the line with `access-log-config=...`
access-log-config = /etc/puppetlabs/puppetdb/request-logging.xml



Is there any wrong in configuration ?


Thanks !

--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/3ef56b7f-4195-4df5-bf75-d072d675b148%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/1bccc831-3eb7-47a6-01ee-d2af98aa09d6%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Installing PuppetDB on Debian 8 with Puppet 3.7

[Wyatt Alt]

On 06/29/2016 10:30 PM, John Naggets wrote:

Hi Wyatt,

Any ideas where I can find PuppetDB v2.x? In the APT repository of 
PuppetLabs the oldest I can find is version 3.2.
Those are in the non-PC1 repos at apt.puppetlabs.com, but is no 2.3.x 
package for Jessie there. The wheezy ones appear to work on jessie but I 
haven't tested it thoroughly.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/6535d91b-d881-15ec-b36c-9ff93a35d6a4%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Installing PuppetDB on Debian 8 with Puppet 3.7

[Wyatt Alt]

John,

You'll want a PuppetDB version in the 2.x series -- 3+ requires Puppet 4.

On the offchance that you're just getting started, note that you can 
install the latest puppetserver and puppet-agent packages from the same 
PC1 repo, and that the latest PuppetDB will work with those.


Wyatt

On 6/29/16 1:54 PM, John Naggets wrote:
Thanks Nick for the pointer, I have now installed the 
puppetlabs-release-pc1-jessie.deb package from apt.puppetlabs.com and 
I am trying to install PuppetDB with the following command:


sudo puppet resource package puppetdb ensure=latest

Unfortunately I get the following a dependency error message which I 
have copied below. I suppose I have to install a specific version of 
PuppetDB in order to work with Debian's version 3.7 of Puppet, but 
which one?


Regards
J.


Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o 
DPkg::Options::=--force-confold install puppetdb' returned 100: 
Reading package lists...

Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 puppetdb : Depends: puppet (>= 3.8.1-1puppetlabs1) or
 puppet-agent but it is not going to be installed
Depends: puppet (< 5.0.0-1puppetlabs1) or
 puppet-agent but it is not going to be installed
E: Unable to correct problems, you have held broken packages.




On Wednesday, June 29, 2016 at 6:47:46 PM UTC+2, Nick Miller wrote:

John,

Puppet provides a repo for Debian Jessie. You can install it with
the rpm found here: https://apt.puppetlabs.com/

Best,
Nick

On Wed, Jun 29, 2016 at 4:46 AM, John Naggets
 wrote:

Hello,

I am using Debian 8 with the puppetmaster package from the
official Debian 8 repository. Now as Debian does not provide a
puppetdb package I wanted to know from which source I should
install PuppetDB? and also which version of PuppetDB to
install? Will PuppetDB version 4.1 work with Puppet 3.7?

Cheers,
John
-- 
You received this message because you are subscribed to the

Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/puppet-users/db41d374-f796-466a-bbe6-32bd85907642%40googlegroups.com

.
For more options, visit https://groups.google.com/d/optout
.




-- 


OnyxPoint-logo-symbol-primary.png



Nicholas Miller
Consultant | Onyx Point, Inc.

7050 Hi Tech Drive, Suite 102

Hanover, MD. 21076
e: nick@onyxpoint.com
w: 443-655-3675

copmany.pngcareers.pngproduct.pngmeetups.pngblog.png


--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/83c2c1d4-ce54-40c4-ba7a-0d4029196424%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/0ba944a3-dd50-b773-4748-8b802818dca2%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Broken puppetdb

[Wyatt Alt]

On 06/12/2016 11:29 AM, Wyatt Alt wrote:


This is the configuration file for the puppetdb terminus, and puppet 
server uses it to find puppetdb. Without the file, the hostname 
"puppetdb" is assumed, hence the error you see. You need to configure 
it like this:


https://docs.puppet.com/puppetdb/latest/connect_puppet_master.html#edit-puppetdbconf
Sorry, just realized you're trying to disable puppetdb. I'd recommend 
removing the storeconfigs option as Trevor says.


Wyatt


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/dc320060-cd27-7505-3ab0-e4be2a83cffa%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Broken puppetdb

[Wyatt Alt]

On 06/11/2016 07:35 PM, Zeke Dehnert wrote:


There was a puppetdb.conf file in the /etc/puppetlabs/puppet 
directory, but I moved it out, restarted puppet server and I’m still 
getting errors:


Error: Could not retrieve catalog from remote server: Error 400 on 
SERVER: Failed to execute 
'/pdb/cmd/v1?checksum=2e834ba5e4cdeae4ab27b04da26fa8c52cfbcfca=4=it-lnx-01.enphaseenergy.com=replace_facts' 
on at least 1 of the following 'server_urls': https://puppetdb:8081


This is the configuration file for the puppetdb terminus, and puppet 
server uses it to find puppetdb. Without the file, the hostname 
"puppetdb" is assumed, hence the error you see. You need to configure it 
like this:


https://docs.puppet.com/puppetdb/latest/connect_puppet_master.html#edit-puppetdbconf

Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/94771c47-e936-2c36-c746-4a14dfd6437a%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB Query based on facts

[Wyatt Alt]

Mike,

Where I was going with that is you might get results with

curl -X GET http://localhost:8080/pdb/query/v4/nodes -d 'query=["and",
["=", ["fact", "operatingsystemmajrelease"], "7"], ["=", ["fact", "facta"],
"true"]]'

or

curl -X GET http://localhost:8080/pdb/query/v4/nodes -d 'query=["and",
["=", ["fact", "operatingsystemmajrelease"], "7"], ["=", ["fact", "facta"],
"True"]]'

Let me know if one of those does it.

Wyatt


On Fri, Jun 3, 2016 at 10:37 AM, Mike Sharpton <sharpt...@gmail.com> wrote:

> Yep, the value of my fact is a string.  I will keep googling around to see
> what I can find.  Thanks,
>
> Mike
>
>
> On Friday, June 3, 2016 at 11:31:28 AM UTC-5, Wyatt Alt wrote:
>>
>> Hey Mike,
>>
>> I think operatingsystemmajrelease might be a string instead of an int,
>> based on https://tickets.puppetlabs.com/browse/FACT-962. You also might
>> verify that facta is valued with a real boolean instead of a stringified
>> bool (the reference to casing made me wonder.)
>>
>> Wyatt
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/08f94fe4-c922-4ead-aced-400212c7a058%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/08f94fe4-c922-4ead-aced-400212c7a058%40googlegroups.com?utm_medium=email_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3FAaY40bD_ZYgSP0QNY3znz83Wto7rdD2dZk7y2erYCCg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB Query based on facts

[Wyatt Alt]

Hey Mike,
I'm thinking you want something like this:

curl -X GET http://localhost:8080/pdb/query/v4/nodes -d 'query=["and",
["=", ["fact", "operatingsystemmajrelease"], 7], ["=", ["fact", "facta"],
true]]'

Wyatt



On Fri, Jun 3, 2016 at 6:40 AM, Mike Sharpton  wrote:

> Hey all,
>
> I am trying to do what should be a simple thing.  I need to query PuppetDB
> to gather a list of machines based on arbitrary facta being equal to true
> and the operatingsystemmajrelease fact being 7.  I have searched around and
> found a few examples, but can't get them to work properly.  I saw and
> example from Wyatt on here, but I must be doing something wrong as I can't
> even get that to run.  I have PuppetDB 3.2.0.
>
> I'm not sure if I should be running my query against this endpoint
> /pdb/query/v4/facts/, or drill into the facta uri and run a in/and against
> the operatingsystemmajrelease fact to get what I need.  All I need back is
> the certname of the machines in which facta is true and
> operatingsystemmajrelease is 7.  If someone has a quick one liner, it would
> be appreciated, otherwise I can bust out the data and filter it with
> Excel.  Thanks,
>
> Mike
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/5a1fec0e-1d02-442f-9465-9660a6087a8f%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3HK%3DihNjuz8PGWcKM_XsQsHcQ46DCqBw-o8etY8uG-JpQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB resources are not available

[Wyatt Alt]

Harish,

The reason for that is the resources returned by puppet resource come from
the RAL, rather than PuppetDB. If you were to declare puppet resources with
the content included in your paste, then they would show up in PuppetDB.

PuppetDB will only include version information on package resources if the
package is managed by puppet and the version is specified in the resource
declaration. The most straightforward way to get information on all
packages installed (e.g not managed by Puppet) into PuppetDB is to create a
custom fact. I saw a module posted to the forge to do this the other day:
https://forge.puppet.com/rmueller/pkg_inventory

Looks like it currently supports only redhat but if you're on something
else you should be able to get an idea of how to adapt it based on the fact
defined here:
https://github.com/roman-mueller/rmueller-pkg_inventory/blob/master/lib/facter/pkg_inventory.rb
.

Given the fact you could get the data by querying
localhost:8080/v4/facts/packages, assuming the name of the fact is
"packages".

Hope that helps.

Wyatt

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3FrcyX9-BV9jSOocbi3V_4NAfBg9zNJsCTz5F%2BBSrD7rw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB resources are not available

[Wyatt Alt]

Hey Harish,

Your issue could be that trailing backslash. PuppetDB is likely 
interpretting it as a query for all resources with type "" (i.e empty 
string).


You should be able to get all package resources with 
/v3/resources/Package, and you can restrict that to a given node with


curl -X GET http://:8080/v3/resources/Package -d 
'query=["=","certname","example.com"]'


Wyatt



On 05/18/2016 11:08 AM, Harish Kothuri wrote:

Hi Ken,

Thanks for your reply.

I got the resource and catalog endpoints working after changing the 
confdir ownership with " sudo chown -R puppet:puppet `sudo puppet 
config print confdir`".


Now, i just need to get the resource list for all machines or for a 
specific machine. I can't seem to find the resource/package list from 
" http://sdin-swt-ctf-01:8080/v3/resources/;


Plz help.

Thanks

On Wednesday, May 18, 2016 at 6:47:17 PM UTC+5:30, Ken Barber wrote:

So at a high level 

The resources are populated from successful Puppet runs that submit
their catalogs to PuppetDB. So step 1, run puppet agent -t or
whatever, if you aren't seeing something like this in your
puppetdb.log:

2016-05-18 14:12:28,855 INFO  [command-proc-3055] [p.p.command]
[898d1f2d-f96b-450f-a627-7fb90eb7d491] [replace catalog]
macbook-pro-9.lan

Then the connection between puppet master & puppetdb is not
configured
correctly, these instructions cover that:

https://docs.puppet.com/puppetdb/4.0/connect_puppet_master.html


(adjust for your version of PuppetDB, which looks like its a 2.x
to me)

You should also check your puppet master logs to see if there are any
errors while submitting the catalog.

ken.

On Wed, May 18, 2016 at 7:21 AM, Harish Kothuri
 wrote:
> Hi,
>
> I am trying to access resources from PuppetDB API as follows,
API makes a
> successful call but empty response.
>
> API call:
> curl -X GET 'http://sdin-swt-ctf-01:8080/v3/resources/
'
>
> Ouput:
> [ ]
>
> Is there anything that i need to enable to populate the same?
>
>
> Thanks
>
> --
> You received this message because you are subscribed to the
Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from
it, send an
> email to puppet-users...@googlegroups.com .
> To view this discussion on the web visit
>

https://groups.google.com/d/msgid/puppet-users/ca08dcc7-f58b-462e-b9ab-6c6dd5610fe8%40googlegroups.com

.

> For more options, visit https://groups.google.com/d/optout
.

--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/69a5f06d-374d-4229-9cab-9bbd98e1c4f5%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/ef1cf576-d3f1-dfa6-6c6f-8d9a8a33ece6%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Problem with puppetdb encoding

[Wyatt Alt]

Hey Angel,

This sounds like a bug to me but I haven't had time to reproduce it. Would
you mind filing a ticket at tickets.puppet.com?

Wyatt



On Wed, May 4, 2016 at 1:37 AM, Angel L. Mateo  wrote:

> Hello,
>
> I'm having a problem with puppetdb. I have deploy a new
> puppetserver (running in a ubuntu 14.04 server and installed from AIO
> package) that uses an already existing puppetdb (2.3.8, I know there are
> newer versions, but this puppetdb is also used by a puppet 3.8 master).
>
> With my current puppet 3.8 server I don't have any problem, but in
> the new puppetserver I'm having problem with the encoding of the stored
> resources.
>
> For example, I have a resource like this:
>
> @@omd_nagios_service {"${::nagios_hostname}_Trafico_eth0":
>   ensure  => hiera('profile::nagios::ensure', 'present'),
>   host_name   => $::nagios_hostname,
>   service_description => 'Tráfico eth0',
>   contact_groups  => hiera('profile::nagios::contact_groups',
> undef),
>   use => 'Servicio_Generico_Telematica_Grafica',
>   check_command   => 'check_interface_traffic!eth0 90 95 public
> 1000',
>   tag => 'omd-nagios-telematica',
>   site=> 'telematica',
> }
>
> This resource is in catalogs applid in puppet 3.8 master and
> puppetserver. For clients using the puppet 3.8 master the information
> stored in the puppetdb database are like:
>
>  resource |name |
>value
>
> --+-+--
>  5aaad4db237176d84701792d35fd9c04ce0b26ae | check_command   |
> "check_interface_traffic!eth0 90 95 public 1000"
>  5aaad4db237176d84701792d35fd9c04ce0b26ae | contact_groups  | ""
>  5aaad4db237176d84701792d35fd9c04ce0b26ae | site|
> "telematica"
>  5aaad4db237176d84701792d35fd9c04ce0b26ae | use |
> "Servicio_Generico_Telematica_Grafica"
>  5aaad4db237176d84701792d35fd9c04ce0b26ae | host_name   |
> "tlm_xenon21"
>  5aaad4db237176d84701792d35fd9c04ce0b26ae | ensure  | "present"
>  5aaad4db237176d84701792d35fd9c04ce0b26ae | tag |
> "omd-nagios-telematica"
>  5aaad4db237176d84701792d35fd9c04ce0b26ae | service_description | "Tráfico
> eth0"
>
> but for clients in the new puppetserver are like:
>
>  resource |name |
>value
>
> --+-+--
>  ff19d1eaf82d344bc406f6a4ba5b5c21d48fc10d | ensure  | "present"
>  ff19d1eaf82d344bc406f6a4ba5b5c21d48fc10d | host_name   |
> "tlm_mus31"
>  ff19d1eaf82d344bc406f6a4ba5b5c21d48fc10d | service_description |
> "Tr��fico eth0"
>  ff19d1eaf82d344bc406f6a4ba5b5c21d48fc10d | use |
> "Servicio_Generico_Telematica_Grafica"
>  ff19d1eaf82d344bc406f6a4ba5b5c21d48fc10d | check_command   |
> "check_interface_traffic!eth0 90 95 public 1000"
>  ff19d1eaf82d344bc406f6a4ba5b5c21d48fc10d | tag |
> "omd-nagios-telematica"
>  ff19d1eaf82d344bc406f6a4ba5b5c21d48fc10d | site|
> "telematica"
>
> where the service_description attribute that should be 'Tráfico
> eth0' is 'Tr��fico eth0'.
>
> I have checked that encoding in the database is correct:
>
> ~$ sudo -u postgres psql -P pager=off -l
>  List of databases
>  Name |  Owner   | Encoding |   Collate   |Ctype| Access
> privileges
>
> --+--+--+-+-+---
> ...
>  puppetdb | postgres | UTF8 | es_ES.UTF-8 | es_ES.UTF-8 |
> =T/postgres  +
>   |  |  | | |
> postgres=CTc/postgres+
>
> The only different I have found is that puppet 3.8 master (run
> with passenger) has LANG=C, but puppetserver java process has
> LANG=es_ES.UTF-8.
>
> Any help? Thank you.
>
>
> --
> Angel L. Mateo Martínez
> Sección de Telemática
> Área de Tecnologías de la Información
> y las Comunicaciones Aplicadas (ATICA)
> http://www.um.es/atica
> Tfo: 868887590
> Fax: 86337
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/5729B4CC.9010202%40um.es.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an 

Re: [Puppet Users] PuppetDB Issue with large array-valued fact

[Wyatt Alt]

Mike,

If you have no issue dropping and recreating the full database, that's 
totally a workaround here (you know your requirements better than I, so 
please don't take this as an endorsement of the approach :-) ).


To do this just stop PuppetDB, drop the puppetdb database, and recreate 
the database with the options you want (the usual ones are described 
here, but some people use different: 
https://docs.puppet.com/puppetdb/latest/configure.html#using-postgresql), 
and restart PuppetDB.


PuppetDB won't recreate the database for you but once that's in place 
it'll create the tables/indices on startup. You may also consider 
setting gc-interval to something less than 1 week, unless there's a good 
reason for it being so long. A one-week interval will allow a lot of 
time for bloat to build up.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/7fdc0ed6-b08b-cecf-a182-079ff854ae02%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB Issue with large array-valued fact

[Wyatt Alt]

Hey Mike, give this a shot (in a psql session):

begin;
delete from facts where fact_path_id in (select id from fact_paths where 
name=any('{"disks", "partitions", "mountpoints"}'));

delete from fact_paths where id not in (select fact_path_id from facts);
delete from fact_values where id not in (select fact_value_id from facts);
commit;


If you hit a transaction rollback you may need to run it with PDB 
stopped. Those last two deletes may take some time since your 
gc-interval is long, so you should probably run it in tmux/screen or 
something.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/419e97c3-edc9-cebf-bfd7-cd4fb3d45b18%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB Issue with large array-valued fact

[Wyatt Alt]

On 04/19/2016 10:39 AM, Mike Sharpton wrote:
 Again,*_thank you_* very much.  If I could buy you a beer, I would. 
 The machines in question are a mix of RHEL5/6/7.
Hah, you're very welcome. Thanks for confirming the OS; that means this 
isn't just a Solaris issue like that facter ticket suggests.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5716FC3A.407%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB Issue with large array-valued fact

[Wyatt Alt]

Hey Mike,

The unsatisfying answer is that PuppetDB handles giant facts 
(particularly array-valued facts) pretty badly right now, and facter's 
disks, partitions, and mountpoints facts can all get pretty huge in 
cases such as SANs and similar. Can you try and see if the bulk of those 
fact paths are coming from a small set of your nodes? I expect this 
query might help:


https://gist.github.com/wkalt/4a58b9a97c79eee31971e5fc04dec0e4

You can mask the facts on a per-node basis by creating a custom fact 
with value nil and weight 100 as described here:


https://docs.puppet.com/facter/3.1/custom_facts.html#fact-precedence

(this assumes you aren't using these facts for anything, but that sounds 
like the case.)


Longer term, this is something we need to fix on our end. I created 
https://tickets.puppetlabs.com/browse/PDB-2631 to track the issue. 
https://tickets.puppetlabs.com/browse/FACT-1345 may also be related.


If you get those nodes tracked down, would you mind telling us the 
operating system?


Wyatt


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5716546A.9020400%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] puppetdb unique facts list

[Wyatt Alt]

Hey jamese,

This is exactly what the fact-names endpoint does. It's documented here:
https://docs.puppet.com/puppetdb/4.0/api/query/v4/fact-names.html

does that suit your needs?

Wyatt

On Sun, Apr 10, 2016 at 11:08 PM, jamese  wrote:

> Hi,
>
> I'm looking for a way to retrieve a list of fact names from puppetdb.
> I don't want to look for facts for a particular host, I'm trying to gather
> a complete list of unique fact names that exist against any host.
> Does anyone know if this is possible?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/9b15dd68-cd98-477e-9d24-2c32534de4f5%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3G%2Bbda-moVHHZdAcbFDsyrU%2BT_x%3DUm1td1-kDLpa6VyNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB: Input to insert-facts-pv-pairs! does not match schema

[Wyatt Alt]

On 03/14/2016 06:34 AM, Akos Hencz wrote:

Hello,

We have a few nodes that fail to submit facts to PuppetDB, and the 
replace facts commands end up in the DLO. The exception is: 
clojure.lang.ExceptionInfo: Input to insert-facts-pv-pairs! does not 
match schema.
I suspect it is a problem with one of the facts, but I cannot 
determine which one. Looking at the output of 'facter -p' on any of 
the nodes seem to be fine.


I would like to ask for some advice on how to investigate the cause of 
this problem. :)


We run Puppet 3.8.4, PuppetDB 2.3.4, and PuppetServer 1.1.3. You can 
see the full exception text below.


Cheers,
Akos


clojure.lang.ExceptionInfo: Input to insert-facts-pv-pairs! does
not match schema: [nil (named (not (some (check %
a-clojure.lang.LazySeq) schemas)) pairs)] {:error [nil (named (not
(some (check % a-clojure.lang.LazySeq) schemas)) pairs)], :value
[44289 ([17 954] [90 58] [100 207518370] [147 137] [72 86] [96
274005961] [49 207518365] [31 57830745] [152 1307] [48 928] [104
295874761] [81 21735] [167 302087295] [162 167699] [68 25] [1 826]
[57 23] [119 291018089] [12 302087311] [7 912] [139 193] [134 539]
[205 28] [137 1299] [34 49] [18 200] [4 7644696] [14 7] [35
59021633] [1526 302087296] [58 73] [120 830] [163 65] [178 1293]
[80 870] [1527 302087294] [95 274005961] [40 157758900] [77
302087312] [116 101387066] [111 112] [41 57830745] [269911
302087303] [269910 193] [269912 57830745] [269909 57830745]
[269908 57830745] [269914 57830745] [269913 302087292] [195 65]
[1248 207518367] [2472 800] [2487 136] [2489 765] [2491 488] [2467
207518365] [2478 830] [2485 800] [2475 207518370] [2496 793] [2501
207518365] [2486 765] [2466 488] [93 302087302] [126 136] [23 105]
[15 50105231] [115 110] [136 299] [121 800] [129 57830729] [56
232889320] [56434 8543] [276 939] [70 75] [106 302087301] [98 229]
[110 57830742] [45 67] [24 883] [107 904] [124 182796497] [99 89]
[47985 302087314] [47983 81057077] [47982 223071] [47989 294247]
[47988 492094] [47991 82005831] [47987 228539865] [47986 81057081]
[47990 136333789] [47984 137976183] [49100 563] [49098 82071940]
[49093 136059519] [49095 108941256] [49116 82065378] [49114
82063765] [49118 82063767] [49117 82063764] [49121 82063916]
[49120 16973] [49115 63121] [49119 88397486] [49150 82064260] [66
302087301] [5 302087302] [63 31] [37 287849745] [51 25] [28
302087305] [73 3927] [485 28] [59 125] [13 302087310] [91 780] [54
94] [127 299] [11 765] [1528 488] [156 65] [215 28] [239
302087304] [21 295871169] [38 302087315] [25 882] [114 17] [101
57830743] [86 57830733] [39 302087308] [297092 195213941] [131
302087293] [130 802] [22 923] [1246 302087313] [2494 302087300]
[2462 68508392] [2473 57830743] [2457 3927] [67 1326] [133
302087305] [71 136778] [65 207518365] [2 169693143] [103 167718]
[19 763] [1529 302087289] [2461 28] [2453 302087302] [2500 200]
[2458 21735] [2493 302087302] [2477 295871174] [2464 200] [2492
539] [2459 8224991] [2476 167718] [2483 38] [2495 302087291] [2451
136778] [2460 91] [2488 17] [2454 67] [2469 112] [2463 302087305]
[2470 302087302] [141 1317] [88 302087297] [52 929] [1250 8224986]
[53 302087300] [123 46] [75 302087307] [135 57830744] [2246
302087298] [2471 302087309] [2455 539] [2452 302087302] [2465
188476291] [2484 188476291] [nil 302087288] [2497 1326] [2590
302087290] [2498 302087311] [33 25] [83 302087305] [79 87] [89 38]
[188 140] [69 169693143] [8 302087306] [122 13] [140 1283] [84 94]
[1249 295871170] [2479 295871169] [2525 295871169] [2474 59021633]
[2468 59021633] [164 65] [92 295871169] [456928 36] [6 1608] [248
302087299] [16 57830733] [1015 104292798] [174 236] [128
302087302] [20 68508392] [102 765] [113 793] [125 80] [26 765]
[118 101387069] [9 296645235] [30 290] [108 800] [76 302087302]
[183 44875635] [158 296658966] [1530 8224991] [278 938] [29
295871174] [10 91] [105 302087291] [27 302087302] [142 181]
[263862 302087287] [50 67] [196 167697] [1531 268074168] [2482
266778936] [2490 57830732] [2499 116471] [2481 929] [109 928] [43
2780] [146 65] [1532 31] [97 59021633] [36 67] [254 28] [3 1671]
[87 200] [62 802] [181 912] [191 1299] [74 904] [60 57830733] [44
302087297] [94 296930720] [85 59021638])], :schema
[#schema.core.One{:schema Int, :optional? false, :name factset-id}
#schema.core.One{:schema (either [[(one Int "path-id") (one Int
"value-id")]] #{[(one Int "path-id") (one Int "value-id")]}),
:optional? false, :name pairs}], :type :schema.core/error}


--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web 

Re: [Puppet Users] Updating facts for a client in PuppetDB independently from the master

[Wyatt Alt]

On 02/17/2016 07:48 AM, Trevor Vaughan wrote:



I took a look at the API and it seems like I might have to use 
'replace_facts' but I'd need to fetch the last fact set, update it, 
and then resubmit it with the updated information which isn't ideal 
but would be doable.



This is your best bet unfortunately. There's no support for 
nonoverwriting facts submission.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/56C4E939.4040407%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 3.2.3 is now available!

[Wyatt Alt]

PuppetDB 3.2.3 - January 11, 2016

PuppetDB 3.2.3 Downloads



Available in native package format as part of Puppet Collection 1 (PC1).
More information on the PC1 repositories is available here:
http://bit.ly/1HQJDNb

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 3.2.3: https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/3.2/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 3.2.3 Release Notes



PuppetDB 3.2.3 is a backward-compatible bugfix release that addresses an
issue affecting new installs on non-english PostgreSQL installations, as
well as submission failures that can occur on catalogs containing large
amounts of binary data. More information on the specifics of the release
can be found in the official release notes:
https://docs.puppetlabs.com/puppetdb/3.2/release_notes.html

Contributors
---
Andrew Roetker, Ken Barber, Rob Browning, Rob Nelson, Russell Mull, Ryan
Senior, and Wyatt Alt

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3Hmtu9dAgOmkBxezspkCD7bzGmz3kW_EN%3DemaE1KVmp8g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB: Some resources not showing up in reports

[Wyatt Alt]

On 11/19/2015 10:46 PM, Dirk Heinrichs wrote:

Am 19.11.2015 um 23:59 schrieb Wyatt Alt:


do you have a shareable manifest that will reproduce the issue?


Unfortunately not. It installs our inhouse SW... What about a simple 
manifest which exec's a script that just prints 1 or 2 thousand lines 
to stdout.
That's what I tried the other day, but the resource showed up fine. Just 
confirmed it again. If you'd like to trouble shoot feel free to open a 
ticket up. Seems like there's something strange going on, I'm just not 
sure we've got the cause right.


--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/564F4ADB.40307%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB: Some resources not showing up in reports

[Wyatt Alt]

On 11/19/15 6:25 AM, Dirk Heinrichs wrote:

Hi,

using Puppet4 together with a PostgreSQL-backed PuppetDB to store 
reports. We have some exec resources that don't show up in the reports 
stored in PuppetDB (but in the "puppet agent" output as well as 
last_run_report.yaml).


These particular exec resources emit lots of console output. Could 
this be the reason for them being omitted from the reports stored in 
PuppetDB?
No, that shouldn't make a difference. I just tried to replicate this and 
my resource showed up in the reports output -- do you have a shareable 
manifest that will reproduce the issue?


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/564E5449.1070907%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Announce: PuppetDB 2.3.8 has been released!

[Wyatt Alt]

On 10/20/15 1:34 PM, Daniel Urist wrote:
Is there any update on when puppetdb will be available for Debian 8 
(jessie)? Puppetserver is now available according to the announcement 
for v.2.1.2.



On Wed, Oct 14, 2015 at 4:34 PM, Wyatt Alt <wy...@puppetlabs.com 
<mailto:wy...@puppetlabs.com>> wrote:


PuppetDB 2.3.8  October 14, 2015


PuppetDB 2.3.8 Downloads




Available in native package format in the release repositories at:

http://yum.puppetlabs.comandhttp://apt.puppetlabs.com


For information on how to enable the Puppet Labs repos, see:


http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories


Binary tarball:http://downloads.puppetlabs.com/puppetdb/


Source:http://github.com/puppetlabs/puppetdb


Please report feedback via the Puppet Labs tickets site, using an
affected PuppetDB version of
2.3.8:https://tickets.puppetlabs.com/browse/PDB


Documentation:http://docs.puppetlabs.com/puppetdb/2.3/


Puppet module:http://forge.puppetlabs.com/puppetlabs/puppetdb


PuppetDB 2.3.8 Release Notes




PuppetDB 2.3.8 is a backwards-compatible bugfix release that fixes
an issue where simultaneous updates to large numbers of fact
values could produce a prepared statement that exceeded the max
allowed by the Postgres JDBC driver. See
https://tickets.puppetlabs.com/browse/PDB-2003for details.


For more information and upgrade advice, consult the detailed
release notes here:

https://docs.puppetlabs.com/puppetdb/2.3/release_notes.html


Contributors



Ken Barber, Ryan Senior, Wyatt Alt
-- 
You received this message because you are subscribed to the Google

Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to puppet-users+unsubscr...@googlegroups.com
<mailto:puppet-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/puppet-users/CAJDiH3EXsYgy_Nm5ZWkVbV%2B3Lvd1k%3DFa5H6ZwN3VryVMXe1buQ%40mail.gmail.com

<https://groups.google.com/d/msgid/puppet-users/CAJDiH3EXsYgy_Nm5ZWkVbV%2B3Lvd1k%3DFa5H6ZwN3VryVMXe1buQ%40mail.gmail.com?utm_medium=email_source=footer>.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
<mailto:puppet-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAEo6%3DKZsJLXBkj1fgkPTAqod6qNnNk%3D%2BzGPW8tr2LEPdJ7O7jw%40mail.gmail.com 
<https://groups.google.com/d/msgid/puppet-users/CAEo6%3DKZsJLXBkj1fgkPTAqod6qNnNk%3D%2BzGPW8tr2LEPdJ7O7jw%40mail.gmail.com?utm_medium=email_source=footer>.

For more options, visit https://groups.google.com/d/optout.

Hey Daniel,

I'm working on getting that set up right now. Shooting for the next week 
or two if all goes well.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5626A975.1000907%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 2.3.8 has been released!

[Wyatt Alt]

PuppetDB 2.3.8  October 14, 2015

PuppetDB 2.3.8 Downloads



Available in native package format in the release repositories at:

http://yum.puppetlabs.com and http://apt.puppetlabs.com

For information on how to enable the Puppet Labs repos, see:

http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 2.3.8: https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/2.3/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 2.3.8 Release Notes



PuppetDB 2.3.8 is a backwards-compatible bugfix release that fixes an issue
where simultaneous updates to large numbers of fact values could produce a
prepared statement that exceeded the max allowed by the Postgres JDBC
driver. See https://tickets.puppetlabs.com/browse/PDB-2003 for details.

For more information and upgrade advice, consult the detailed release notes
here:

https://docs.puppetlabs.com/puppetdb/2.3/release_notes.html

Contributors


Ken Barber, Ryan Senior, Wyatt Alt

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3EXsYgy_Nm5ZWkVbV%2B3Lvd1k%3DFa5H6ZwN3VryVMXe1buQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB type casting wierdness

[Wyatt Alt]

You're encountering PDB-170 (https://tickets.puppetlabs.com/browse/PDB-170.)

It's a known bug, and last time I looked at it wasn't clear whether the 
issue was Puppet or PDB, but I just tried it on Puppet 4.2.2/PuppetDB 
3.1.0 and I can't reproduce:


https://gist.github.com/wkalt/83d8d8170d9f20aad331

so I'm thinking it may have been fixed on the Puppet side. What versions 
are you running?


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/561BCF13.2000501%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB Service Won't Install/Start

[Wyatt Alt]

Hey Dan,

I see the bottom of a java stacktrace in your log snippet there -- could 
you get the full stacktrace from journalctl and stick it in a gist?


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/561707FB.9070502%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB 2.3 shows ? for "Collection Queries"

[Wyatt Alt]

PuppetDB 3.x and PuppetDB 2.x both work with Puppet 4. PuppetDB 3.x does
not work with Puppet 3.

Wyatt

On Fri, Oct 2, 2015 at 7:39 AM, Lori Cho  wrote:

> I was reading release notes. PuppetDB 3.x ou works with Puppet 4?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3E715kZWogN6kGF1i0JpkxjOGBiKmLRs8cHSzGuTNBg%2BQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB 2.3 shows ? for "Collection Queries"

[Wyatt Alt]

On 9/30/15 8:05 PM, Lori Cho wrote:

How come the ticket is still open if fixed?? I came across that when googling 
earlier but with no responses to the ticket it was unclear what to make of it.
It was fixed incidentally in unrelated work. I've closed the ticket -- 
thanks for pointing that out.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/560D61CB.2030003%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB 2.3 shows ? for "Collection Queries"

[Wyatt Alt]

On 9/30/15 3:33 PM, Wyatt Alt wrote:

On 9/30/15 2:25 PM, Lori Cho wrote:

We are using puppetdb 2.3.4 with postgres.

When looking at the dashboard, the following only shows '?', 
"Collection Queries", "Enqueuing", "DLO Compression", "DLO Size on 
Disk", "Discarded Messages".


Can someone advise as to why this might be?


--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, 
send an email to puppet-users+unsubscr...@googlegroups.com 
<mailto:puppet-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/0bf238ac-7b43-4c40-b4b8-6b680feb9417%40googlegroups.com 
<https://groups.google.com/d/msgid/puppet-users/0bf238ac-7b43-4c40-b4b8-6b680feb9417%40googlegroups.com?utm_medium=email_source=footer>. 


For more options, visit https://groups.google.com/d/optout.


Hi Lori,
The issue with "Collection Queries" and "Enqueuing" is a known bug 
that's described here:

https://tickets.puppetlabs.com/browse/PDB-1113

it's been fixed on PDB 3.x. Under working conditions those represent 
how long it takes to query PDB's resources endpoint, and to post 
commands to PDB.


The DLO-related metrics relate to failed commands, and they are 
appearing as ? because none of your commands have failed, and thus the 
dead letter office (DLO) does not yet exist. If you want to verify 
that it works, you can cause a command failure by posting a report to 
the commands endpoint twice (this will fail because of a constraint 
violation, and cause some log noise as the command is retried).

https://gist.github.com/40b4acfb4ea4c6776478

Wyatt
By the way, doing that would also put some useless data in your database 
when it succeeds on the first submission, so I wouldn't actually try it 
unless you're on a test system.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/560C63CC.9040100%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB 2.3 shows ? for "Collection Queries"

[Wyatt Alt]

On 9/30/15 2:25 PM, Lori Cho wrote:

We are using puppetdb 2.3.4 with postgres.

When looking at the dashboard, the following only shows '?', 
"Collection Queries", "Enqueuing", "DLO Compression", "DLO Size on 
Disk", "Discarded Messages".


Can someone advise as to why this might be?


--
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/0bf238ac-7b43-4c40-b4b8-6b680feb9417%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


Hi Lori,
The issue with "Collection Queries" and "Enqueuing" is a known bug 
that's described here:

https://tickets.puppetlabs.com/browse/PDB-1113

it's been fixed on PDB 3.x. Under working conditions those represent how 
long it takes to query PDB's resources endpoint, and to post commands to 
PDB.


The DLO-related metrics relate to failed commands, and they are 
appearing as ? because none of your commands have failed, and thus the 
dead letter office (DLO) does not yet exist. If you want to verify that 
it works, you can cause a command failure by posting a report to the 
commands endpoint twice (this will fail because of a constraint 
violation, and cause some log noise as the command is retried).

https://gist.github.com/40b4acfb4ea4c6776478

Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/560C6328.8070905%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Puppetdb garbage collection failing

[Wyatt Alt]

On 9/29/15 12:20 AM, Matt Jarvis wrote:


 count | name

---+-

 1 | macaddress_qvb34470225_cd

 1 | mtu_qbr2fb476b3_ff

 1 | speed_qvbfa2ec4e3_15

 1 | macaddress_qvo547572f9_14

 1 | speed_qvo2e200191_c0

 1 | mtu_qbr5eaffca5_fb

 1 | macaddress_qbr0d4ed278_e3

 1 | mtu_qvb8166a899_d1

 1 | speed_qvb4e0d1069_13

 1 | speed_qvbb2d99f31_86

 1 | mtu_qbr65afa39a_9a

 1 | speed_qvb336884d1_12

 1 | speed_qvbf81c2831_4f

 1 | mtu_qbr6d9cbcfc_82

 1 | mtu_qbr441a8d9c_9e

 1 | macaddress_qbrb400a4cf_a3

 1 | mtu_qbr0bdbfadc_6a

 1 | macaddress_qbrf9e0c7d4_7b

 1 | macaddress_qbr3fe74368_2f

 1 | macaddress_qvoc943cbcd_c3

 1 | macaddress_qvb7e04f0db_2b

 1 | mtu_qbrb42e4516_13

 1 | macaddress_qvbefdec85e_5b

 1 | mtu_qbr4575c981_84

 1 | speed_qvbb771b00f_b4

 1 | speed_qvo04f9f59c_d2

 1 | macaddress_qbre4308db4_12

 1 | speed_qvb997d8a21_72

 1 | mtu_qvo699d2518_05

 1 | mtu_qvbc5dcb18f_8b

 1 | mtu_qvb766c608d_7a

 1 | speed_qvo137786a3_ce

 1 | speed_qvo02ec32fd_28

 1 | macaddress_qbr3b6455da_f1

 1 | mtu_qvb993a2dfb_5e

 1 | macaddress_qvo14369bd5_d3


Is that enough of that query result ? We're an OpenStack public cloud 
provider, so in our cluster we have many network interfaces changing a 
lot when new virtual networks and machines are created - those are all 
related to virtual interfaces. Looks like the majority of that table 
is full of them.




It's enough to shoot down my theory about structured facts. Assuming the 
"desc" was included in the order by, that result indicates that you 
aren't storing any structured facts at all.


The long parameter list in the query you've identified represents the 
fact paths (equivalent to fact names when there are no structured facts) 
that become invalidated when a node updates its set of facts in 
PuppetDB. In the case of a structured fact, this could happen if you 
inserted an element at the beginning of a large array, but with flat 
facts like you appear to have I think this would have to mean that a) 
the node has 26k+ facts associated with it and b) 26k facts are being 
renamed or removed between the last successful puppet run and the run 
that's failing.


The final parameter ($26355 in your case) represents the name of the 
node that's failing, and you can get the associated certname with the 
query by getting the value of that parameter from your postgres logs and 
issuing


select certname from factsets where id=;

from psql.

Can you give me answers to the following:
- has PuppetDB been running fine prior to this issue or have you 
recently adopted it?

- does it seem possible that you have no structured facts in your database?
- can you give me the first 10 rows of this query?
select count(*),factset_id from facts group by factset_id order by count 
desc;

- can you get the certname of the failing node using
select certname from factsets where id=;
and send me the output of
curl -X GET http://localhost:8080/v4/factsets -d 
'query=["=","certname",""]'
- once you have the certname, is there anything special about that node 
that you're aware of?
- can you send me the compressed contents of the failed replace-facts 
commands in your dead letter directory? These will be located at

/opt/puppetlabs/server/data/puppetdb/mq/discarded/replace-facts
if you're on PC1 and
/var/lib/puppetdb/mq/discarded/replace-facts
if you aren't, assuming you're using the default pathing.

Additionally, this is probably going to require some back and forth 
between us -- if you want to chime in on the ticket at 
https://tickets.puppetlabs.com/browse/PDB-2003 
<https://tickets.puppetlabs.com/browse/PDB-2003> we can continue the 
discussion there, and if you're on IRC I'm available in #puppet on 
freenode as wkalt, mostly during work hours on US pacific time.


Thanks,
Wyatt


On Monday, September 28, 2015 at 6:45:49 PM UTC+1, Wyatt Alt wrote:

On 09/28/2015 10:39 AM, Wyatt Alt wrote:

On 09/28/2015 05:40 AM, Matt Jarvis wrote:

We seem to have hit a bit of an issue with puppetdb garbage
collection. Initial symptoms were exceptions in the puppetdb logs :

Retrying after attempt 6, due to:
org.postgresql.util.PSQLException: This connection has been closed.


And on the postgres side :


LOG:  incomplete message from client


Having turned up the logging on postgres, it appears that the query


DELETE FROM fact_paths fp

  WHERE fp.id <http://fp.id> in ( $some_ids )  AND NOT
EXISTS (SELECT 1 FROM facts f

  WHERE f.fact_path_id in ( $some_more_ids ) AND f.fact_path_id
= fp.id <http://fp.id>

AND f.factset_id <> $26355)


is the cuplrit. This query is absolutely massive, with over
26000 id's specified as parameters - as soon as the query 

Re: [Puppet Users] Puppetdb garbage collection failing

[Wyatt Alt]

On 09/28/2015 05:40 AM, Matt Jarvis wrote:
We seem to have hit a bit of an issue with puppetdb garbage 
collection. Initial symptoms were exceptions in the puppetdb logs :


Retrying after attempt 6, due to: org.postgresql.util.PSQLException: 
This connection has been closed.



And on the postgres side :


LOG:  incomplete message from client


Having turned up the logging on postgres, it appears that the query


DELETE FROM fact_paths fp

  WHERE fp.id in ( $some_ids )  AND NOT EXISTS (SELECT 1 FROM 
facts f


  WHERE f.fact_path_id in ( $some_more_ids 
) AND f.fact_path_id = fp.id


AND f.factset_id <> $26355)


is the cuplrit. This query is absolutely massive, with over 26000 id's 
specified as parameters - as soon as the query is executed, postgres 
returns incomplete message from client and drops the connection.



puppetdb is 2.3.7-1puppetlabs1

postgres is 9.3


Does anyone have any clues what's going on here ?


Thanks


Matt


DataCentred Limited registered in England and Wales no. 05611763 --
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5fe3bad3-71a7-4348-a9ff-24d8a0284a1c%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.

Hey Matt,

I can reproduce this by inserting a value at the beginning of an 
extremely large array-valued structured fact, but we'll need to know 
more about your particular data to confirm whether that's your 
particular issue. This could be some large custom fact you're creating 
or something generated by a module.


I've created a ticket here around this issue here
https://tickets.puppetlabs.com/browse/PDB-2003

can you connect to the database via psql and share (either here or in 
the ticket) the output of


select count(*),name from fact_paths group by name order by count desc;

?

My hope is that that will identify one or more large structured facts 
associated with a lot of leaf values, and then we'll need to figure out 
where they're coming from.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/56097B44.6030700%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Puppetdb garbage collection failing

[Wyatt Alt]

On 09/28/2015 10:39 AM, Wyatt Alt wrote:

On 09/28/2015 05:40 AM, Matt Jarvis wrote:
We seem to have hit a bit of an issue with puppetdb garbage 
collection. Initial symptoms were exceptions in the puppetdb logs :


Retrying after attempt 6, due to: org.postgresql.util.PSQLException: 
This connection has been closed.



And on the postgres side :


LOG:  incomplete message from client


Having turned up the logging on postgres, it appears that the query


DELETE FROM fact_paths fp

  WHERE fp.id in ( $some_ids )  AND NOT EXISTS (SELECT 1 FROM 
facts f


  WHERE f.fact_path_id in ( 
$some_more_ids ) AND f.fact_path_id = fp.id


AND f.factset_id <> $26355)


is the cuplrit. This query is absolutely massive, with over 26000 
id's specified as parameters - as soon as the query is executed, 
postgres returns incomplete message from client and drops the 
connection.



puppetdb is 2.3.7-1puppetlabs1

postgres is 9.3


Does anyone have any clues what's going on here ?


Thanks


Matt


DataCentred Limited registered in England and Wales no. 05611763 --
You received this message because you are subscribed to the Google 
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, 
send an email to puppet-users+unsubscr...@googlegroups.com 
<mailto:puppet-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5fe3bad3-71a7-4348-a9ff-24d8a0284a1c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Hey Matt,

I can reproduce this by inserting a value at the beginning of an 
extremely large array-valued structured fact, but we'll need to know 
more about your particular data to confirm whether that's your 
particular issue. This could be some large custom fact you're creating 
or something generated by a module.


I've created a ticket here around this issue here
https://tickets.puppetlabs.com/browse/PDB-2003

can you connect to the database via psql and share (either here or in 
the ticket) the output of


select count(*),name from fact_paths group by name order by count desc;

?

My hope is that that will identify one or more large structured facts 
associated with a lot of leaf values, and then we'll need to figure 
out where they're coming from.


Wyatt



Just to clarify, I think the top few rows of that result should be 
enough to illustrate -- no need to include the whole thing.


Wyatt

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/56097C44.6030602%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] how were your puppet 3-4 upgrades?

[Wyatt Alt]

On 8/24/15 12:59 PM, Christopher Wood wrote:

(Although I may dump/restore the data via puppetdb since that's the actual api 
to the data, we do not log in via PostgreSQL.)

Others can speak to the rest of the plan, but just to clarify in case 
I'm understanding you right: the PuppetDB import/export tools are not 
intended to work across different versions of PDB, so an export of 2.x 
data will not be importable on 3.x. I can see that this is a hole in our 
documentation, which I'll fix up in a minute.


Instead, the necessary data migrations will be handled by the upgrade 
itself on the first startup of PuppetDB 3.0. Taking a pg_dump or 
PuppetDB export for a backup prior to the upgrade is still a prudent 
thing to do in case a downgrade is required, but that will only allow 
you to restore from the version you came from (possibly earlier ones too 
in the export case, but only by happenstance). For your use case I'd 
recommend pg_dump over PuppetDB export since it'll run a lot faster, but 
either one will get the job done.


Wyatt

--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/55DB8009.6080300%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Re: PuppetDB and Debian Jessie

[Wyatt Alt]

Jessie support in PuppetDB is blocked on support in Puppet Server. As 
Martin has said, there is no established delivery date for that yet, but 
it's currently supported in the nightly repos (for puppetserver, not PDB) 
and my understanding is that it will be coming to PC1 in the next 
puppetserver release.

Wyatt

On Friday, August 14, 2015 at 1:19:49 AM UTC-7, oschad wrote:

 Hi everybody, 

 does somebody know when the puppetdb packages will be available to 
 build a complete puppet master/server based on Debian Jessie? 

 Best Regards 
 Oli 


-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/1dfb5a33-a036-4597-bb40-b24065c20af2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 2.3.7 is available

[Wyatt Alt]

PuppetDB 2.3.7 August 13, 2015

PuppetDB 2.3.7 Downloads



Available in native package format in the release repositories at:

http://yum.puppetlabs.com and http://apt.puppetlabs.com

For information on how to enable the Puppet Labs repos, see:

http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 2.3.7: https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/2.3/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 2.3.7 Release Notes



PuppetDB 2.3.7 is a backwards-compatible bugfix release that applies the
default max-frame-size setting to affect ActiveMQ consumer threads in
addition to producers (which were already affected). This could cause
issues with submission of extremely large commands.

The max-frame-size setting is currently not configurable due to a separate
bug (https://tickets.puppetlabs.com/browse/PDB-1909), but the default
should be sufficient in all cases but the most extreme.

For more information and upgrade advice, consult the detailed release notes
here:

https://docs.puppetlabs.com/puppetdb/2.3/release_notes.html

Contributors


Andrew Roetker, Russell Mull

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3GWXKOZ3xNcSuJ_8za5tLHEdK9R5euKrSSG_-6NRsea8g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Advice on Puppet update to 4

[Wyatt Alt]

Fabrice,

Correct. The differences are listed here: 
http://docs.puppetlabs.com/puppetdb/3.0/api/query/v4/upgrading-from-v3.html 
. Anything missing from that list is a docs bug that we'll be glad to fix.


Wyatt

--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/55B41DB6.40908%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 2.3.6 is now available!

[Wyatt Alt]

PuppetDB 2.3.6 July 16, 2015

PuppetDB 2.3.6 Downloads



Available in native package format in the release repositories at:

http://yum.puppetlabs.com and http://apt.puppetlabs.com

For information on how to enable the Puppet Labs repos, see:

http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 2.3.6: https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/2.3/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 2.3.6 Release Notes



PuppetDB 2.3.6 is a backwards-compatible bugfix release that improves the
downgrade process from 3.0, adds deprecation warnings for PostgreSQL
versions 9.3 and below, and fixes an issue with relationships on resource
aliases in the puppetdb terminus.

For more information and upgrade advice, consult the detailed release notes
here:

https://docs.puppetlabs.com/puppetdb/2.3/release_notes.html

Contributors



Andrew Roetker, Ken Barber, Nick Fagerlund, Rob Browning, Russell Mull,
Wyatt
Alt

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3HBUCeeUOSqMGVPLQzCTR3c5Hs8MujmVmPKpdHW22hkqQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Re: Failed to upgrade a PuppetDB that runs into a non-standard directories

[Wyatt Alt]

Hey Benjamin,

This is a config validation error. You have a webapps setting somewhere 
in your config directory. That's not a setting PuppetDB will use, and PDB 
will fail to start if it's present. Remove that setting and things should 
work fine.

Wyatt

On Thursday, July 9, 2015 at 8:50:15 AM UTC-7, Benjamin Parmentier wrote:

 Hi,


 I actually have a PuppetDB 2.2.2 that runs without any problems into a non 
 standard directories :

 /SERVICES/puppetdb/{conf,logs,tmp,var}

 I also use a custom script to start/stop PuppetDB.
 I extract the command to start puppetdb from the init-script :

 /usr/bin/java -Xmx192m -XX:OnOutOfMemoryError=kill -9 %p -XX:+
 HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/SERVICES/puppetdb/logs/
 puppetdb-oom.hprof -Djava.security.egd=file:/dev/urandom -cp /usr/share/
 puppetdb/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core 
 services -cp /SERVICES/puppetdb/conf



 I tried to upgrade to the version 2.3.5 and puppetdb won't start 
 anymore... I have this stracktrace in log file :

 2015-07-09 17:29:48,677 INFO  [o.e.j.u.log] Logging initialized @41541ms
 2015-07-09 17:29:50,738 INFO  [p.t.s.w.jetty9-service] Initializing web 
 server(s).
 2015-07-09 17:29:50,752 INFO  [p.t.s.w.jetty9-service] Starting web 
 server(s).
 2015-07-09 17:29:51,028 INFO  [p.t.s.w.jetty9-core] Starting web server.
 2015-07-09 17:29:51,039 INFO  [o.e.j.s.Server] jetty-9.2.z-SNAPSHOT
 2015-07-09 17:29:51,147 INFO  [o.e.j.s.ServerConnector] Started 
 ServerConnector@5442accb{HTTP/1.1}{0.0.0.0:8082}
 2015-07-09 17:29:51,282 INFO  [o.e.j.s.ServerConnector] Started 
 ServerConnector@5d63d496{SSL-HTTP/1.1}{0.0.0.0:8081}
 2015-07-09 17:29:51,282 INFO  [o.e.j.s.Server] Started @44151ms
 2015-07-09 17:29:51,284 WARN  [c.p.p.config] The configuration item 
 `url-prefix` in the [global] section is deprecated. It will be removed in 
 the future.
 2015-07-09 17:29:51,334 ERROR [p.t.internal] Error during service start!!!
 clojure.lang.ExceptionInfo: Value does not match schema: {:webapps 
 disallowed-key}
 at schema.core$validate.invoke(core.clj:161) ~[na:na]
 at 
 com.puppetlabs.puppetdb.config$configure_puppetdb.invoke(config.clj:217) 
 ~[na:na]
 at 
 com.puppetlabs.puppetdb.config$convert_config.invoke(config.clj:224) 
 ~[na:na]
 at 
 com.puppetlabs.puppetdb.config$process_config_BANG_.invoke(config.clj:364) 
 ~[na:na]
 at 
 com.puppetlabs.puppetdb.cli.services$start_puppetdb.invoke(services.clj:261) 
 ~[na:na]
 at 
 com.puppetlabs.puppetdb.cli.services$reify__21278$service_fnk__17647__auto___positional$reify__21289.start(services.clj:366)
  
 ~[na:na]
 at 
 puppetlabs.trapperkeeper.services$eval17483$fn__17497$G__17473__17500.invoke(services.clj:8)
  
 ~[na:na]
 at 
 puppetlabs.trapperkeeper.services$eval17483$fn__17497$G__17472__17504.invoke(services.clj:8)
  
 ~[na:na]
 at 
 puppetlabs.trapperkeeper.internal$run_lifecycle_fn_BANG_.invoke(internal.clj:152)
  
 ~[na:na]
 at 
 puppetlabs.trapperkeeper.internal$run_lifecycle_fns.invoke(internal.clj:180) 
 ~[na:na]
 at 
 puppetlabs.trapperkeeper.internal$build_app_STAR_$reify__19027.start(internal.clj:447)
  
 [na:na]
 at 
 puppetlabs.trapperkeeper.internal$boot_services_STAR_$fn__19039.invoke(internal.clj:471)
  
 [na:na]
 at 
 puppetlabs.trapperkeeper.internal$boot_services_STAR_.invoke(internal.clj:469)
  
 [na:na]
 at 
 puppetlabs.trapperkeeper.core$boot_with_cli_data.invoke(core.clj:113) 
 [na:na]
 at puppetlabs.trapperkeeper.core$run.invoke(core.clj:144) [na:na]
 at puppetlabs.trapperkeeper.core$main.doInvoke(core.clj:159) 
 [na:na]
 at clojure.lang.RestFn.applyTo(RestFn.java:137) [puppetdb.jar:na]
 at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na]
 at 
 com.puppetlabs.puppetdb.cli.services$_main.doInvoke(services.clj:373) 
 [na:na]
 at clojure.lang.RestFn.invoke(RestFn.java:421) [puppetdb.jar:na]
 at clojure.lang.Var.invoke(Var.java:383) [puppetdb.jar:na]
 at clojure.lang.AFn.applyToHelper(AFn.java:156) [puppetdb.jar:na]
 at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na]
 at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na]
 at com.puppetlabs.puppetdb.core$run_command.invoke(core.clj:87) 
 [na:na]
 at com.puppetlabs.puppetdb.core$_main.doInvoke(core.clj:95) [na:na]
 at clojure.lang.RestFn.invoke(RestFn.java:436) [puppetdb.jar:na]
 at clojure.lang.Var.invoke(Var.java:388) [puppetdb.jar:na]
 at clojure.lang.AFn.applyToHelper(AFn.java:160) [puppetdb.jar:na]
 at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na]
 at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na]
 at clojure.main$main_opt.invoke(main.clj:315) [puppetdb.jar:na]
 at clojure.main$main.doInvoke(main.clj:420) [puppetdb.jar:na]
 at clojure.lang.RestFn.invoke(RestFn.java:482) 

[Puppet Users] Announce: PuppetDB 3.0 has been released!

[Wyatt Alt]

PuppetDB 3.0.0 - July 9, 2015

PuppetDB 3.0.0 Downloads



Available in native package format as part of Puppet Collection 1 (PC1).
More information on the PC1 repositories is available here:
http://bit.ly/1HQJDNb

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 3.0.0: https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/3.0/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 3.0.0 Release Notes



PuppetDB 3.0.0 is a major, backward-incompatible release containing
powerful new features and packaging changes to conform with the All-In-One
packaging standards adopted by Puppet 4. It is important to note that
PuppetDB 3.0 depends on Puppet 4.

Users should consult the release notes here:

https://docs.puppetlabs.com/puppetdb/3.0/release_notes.html

and the specific notes on new and backward-incompatible API features here:

https://docs.puppetlabs.com/puppetdb/3.0/api/query/v4/upgrading-from-v3.html

for detailed information and upgrading advice.

Some changes we’re excited about are:

   -

   retirement of the v2 and v3 query APIs, and promotion of v4 as the
   stable API
   -

   support for automatic failover between multiple instances of PuppetDB
   -

   group_by query operator and foundational support for aggregate functions
   in queries (currently only count)
   -

   reports now include logs, metrics, and events from the associated Puppet
   run, and express whether or not the run used the --noop flag
   -

   extract can be used as a top-level query operator, to limit the fields a
   query returns
   -

   retirement of support for versions of PostgreSQL prior to 9.4
   -

   deprecation of support for HSQLDB (with retirement slated for the next
   major release)



Contributors


Andrew Roetker, Andrii Nikitiuk, Chris Price, deadpoint, Eric Sorenson,
Jean Bond, John Duarte, Jorie Tappa, Ken Barber, Brian LaMetterey, Lars
Windolf, Mathieu Parent, Matthaus Owens, Melissa Stone, Nick Fagerlund, Rob
Braden, Rob Browning, Roger Ignazio, Russell Mull, Ryan Senior, Wayne
Warren, Wyatt Alt

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3HT4hfj3c3k%2Bnf5sjLhhMvqtsG%2Bjq1iqMg88azBht5NKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Not all host reports in PuppetDB

[Wyatt Alt]

Hey Jim,

Do you see anything in your puppetserver log you can correlate to this?
Also curious if your Broke and no longer processing comment is in
relation to this issue or preexisting.

One interesting, possibly unrelated thing to know is that the report
processors in reports attribute in your config file are parsed and
applied in order, so if there were intermittent problems with your
reporturl that somehow caused the run to bail uncompleted, that could
prevent reports from making it to PuppetDB and would match with what you're
seeing. Just a theory though -- I don't know what would cause that and I've
been unsuccessful demonstrating it locally.

Wyatt

On Mon, Jul 6, 2015 at 5:51 AM, Jim Miller stl...@gmail.com wrote:

 Hello everyone,

 I hadn't noticed until recently but a large % of host reports are not
 getting into PuppetDB and I looking for ways to determine what the cause
 is.  I could see an all or nothing but not this.  I'm not worried about the
 data for those hosts but I'm un-sure what steps to take.  I sure hope this
 is an easy fix.

 For example in the puppetdb.log file, I'm seeing for some hosts where
 reports are getting stored and facts and catalogs are getting replaced.
 For other hosts just replacing facts and catalogs are getting replaced.

 I don't understand enough of the internals to know what to do here and I'm
 not sure how to peer into the database.  I did upgrade from an pretty old
 version of PuppetDB (1.x something to 2.3.3) and I can't help but feel that
 could be the cause but I'm just guessing.

 We did use Puppetboard but now use Puppetboard.

 Here's the relevant config values for puppet.conf -- just the master
 section
 [master]
 certname = puppet.ourdomain.com
 report = true
 # reports log, store for non-dashboard or puppetdb
 # reports = log,store
 reports = http,puppetdb,rrdgraph
 reporturl = http://puppet.centric.com:3000/reports/upload  # Broke
 and no longer processing
# removed by Jim 5/8/2015
#  ### CONFIGS FOR PUPPET-DASHBOARD
#  node_terminus = exec
#  external_nodes = /usr/bin/env PUPPET_DASHBOARD_URL=
 http://puppet.centric.com:3000
 /usr/share/puppet-dashboard/bin/external_node
 ### CONFIGS FOR PUPPETDB
 storeconfigs = true
 storeconfigs_backend = puppetdb
 ssldir = $vardir/ssl

 We're running
 Puppet Server 3.7.5
 PuppetDB and PuppetDB-Terminus 2.3.3





  --
 You received this message because you are subscribed to the Google Groups
 Puppet Users group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to puppet-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/puppet-users/b9581bf2-e372-46e9-b06d-c75e193554cc%40googlegroups.com
 https://groups.google.com/d/msgid/puppet-users/b9581bf2-e372-46e9-b06d-c75e193554cc%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3Fx3E12tJg4MWtDYUt_H%3D6KRXA-EN-M-XiR2GvbX8rWuQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 2.3.5 is now available!

[Wyatt Alt]

PuppetDB 2.3.5 - June 4, 2015

PuppetDB 2.3.5 Downloads



Available in native package format in the release repositories at:

http://yum.puppetlabs.com and http://apt.puppetlabs.com

For information on how to enable the Puppet Labs repos, see:

http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 2.3.5: https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/2.3/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 2.3.5 Release Notes



PuppetDB 2.3.5 is a backwards-compatible bugfix release that removes a
dependency on rubygem-json that prevented installation alongside the
all-in-one agent on RHEL 5 and 6, fixes a bug that caused us to reject
resource relationship titles containing newlines, and lessens database
contention during command processing. Users that upgrade will see fewer
commands retried due to rollbacks of concurrent transactions in PostgresSQL
(signified in PostgreSQL logs by “ERROR:  could not serialize access due to
concurrent update”)

For more information and upgrade advice, consult the detailed release notes
here:

https://docs.puppetlabs.com/puppetdb/2.3/release_notes.html

Contributors



John Duarte, Ken Barber, Melissa Stone, Rob Browning, Russell Mull, Wyatt
Alt

Changelog

-

John Duarte (2):

 ef409a4 (PDB-1300) Get tests working with Puppet 4 AIO packaging

 1c5859a (PDB-1300) Set conditional beaker options for AIO

Ken Barber (3):

 766608a (PDB-1469) Remove rubygem-json dependency

 93df2aa (PDB-1300) Use manage_firewall = false instead of flushing
iptables later manually

 721a040 (PDB-1300) Wrap aio options parts in conditional

Melissa Stone (1):

 cf2e086 (maint) Do not build stable or testing

Rob Browning (2):

 621d0c0 (PDB-1263) Use beaker EC2 subnet rotation

 d853fd9 (PDB-1454) Handle IOException while starting MQ

Russell Mull (1):

 af26e29 (PDB-1529) Allow newlines in resource names

Wyatt Alt (3):

 3be0e9c Update nodes.markdown

 706df77 (maint) don't activate the node when it's already active
 ddf2e91 (PDB-1571) update release notes for 2.3.5 release

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3HVsL_8Be7Gy1LwMAveLZjv_1Kk9Cp4qb%2BzDEoH%2BLCXoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 2.3.4 is now available!

[Wyatt Alt]

PuppetDB 2.3.4 - May 7, 2015

PuppetDB 2.3.4 Downloads



Available in native package format in the release repositories at:

http://yum.puppetlabs.com and http://apt.puppetlabs.com

For information on how to enable the Puppet Labs repos, see:

http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 2.3.4: https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/2.3/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 2.3.4 Release Notes



PuppetDB 2.3.4 is a backwards-compatible bugfix release that addresses an
issue

that would prevent fact storage for nodes under certain circumstances.
Users experiencing recurring failures of the “replace facts” command due to
constraint violations on fact_value_id are encouraged to upgrade.

For more information and upgrade advice, consult the detailed release notes
here:

https://docs.puppetlabs.com/puppetdb/2.3/release_notes.html

Contributors



Andrew Roetker, Deepak Giridharagopal, Ken Barber, Melissa Stone, Rob
Browning,

Wyatt Alt

Changelog

-

Andrew Roetker (3):

 a8eede4 (PDB-1412) Update rspec dependency from 2.x to 3.x

 71875d5 Revert (PDB-1412) Update rspec dependency from 2.x to 3.x

 7d43528 (PDB-1485) Deprecate url-prefix setting

Deepak Giridharagopal (1):

 c3370dc (PDB-1428) Suppress version-check INFO logging

Ken Barber (3):

 6b322eb (maint) Bump to latest pristmatic/schema

 131ccbf (maint) Bump to latest trapperkeeper/jetty9/kitchensink

 209d31c (docs) Fix path to puppet terminus for source based installs

Melissa Stone (2):

 5f0fbf0 Remove Fedora 19 from build targets

 858fd9d (maint) Remove lucid from build targets

Rob Browning (3):

 df69e5a (PDB-1431) Remove PE packages from el-7 AMI

 797cff7 (PDB-1448) Test swapping path values in a replace

 cbc6048 (PDB-1448) GC values that swap paths correctly

Wyatt Alt (6):

 74e4c7a (maint) add upgrading from v3 document to stable branch

 350ba80 Update configure.markdown

 8e7b5c3 Update facts.markdown

 3199533 (maint) make facts response streaming

 2621541 (PDB-1494) Update release notes for 2.3.4.
 f33b28e (PDB-1479) deprecate event counts and aggregate event counts

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3Fm43L0uEuAssiR3Yq4jkWAacLDOVOanp%3D1sukMNjv5NQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 2.3.2 is now available!

[Wyatt Alt]

PuppetDB 2.3.2 - April 1, 2015

PuppetDB 2.3.2 Downloads


Available in native package format in the release repositories at:
http://yum.puppetlabs.com and http://apt.puppetlabs.com

For information on how to enable the Puppet Labs repos, see:
http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 2.3.2:
https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/2.3/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 2.3.2 Release Notes


PuppetDB 2.3.2 is a backwards-compatible bugfix release that corrects two
problems which could prevent migration to the database format introduced in
2.3.1. Users that attempted to upgrade and got a foreign key constraint
violation like this:

update or delete on table fact_paths violates foreign key constraint
fact_values_path_id_fk on table fact_values

or users who experienced issues because they use do not use the PUBLIC
schema will need to upgrade to 2.3.2. Users who upgraded successfully will
be unaffected by the changes. Neither issue presented a risk of data loss.

For more details, consult the release notes here:
https://docs.puppetlabs.com/puppetdb/2.3/release_notes.html

Contributors

Rob Browning

Changelog
-
Rob Browning (2):
  1e173c3 (PDB-1362) Drop values constraint earlier in fact paths
migration
  34675b5 (PDB-1363) Revert (maint) applied-migrations: don't ignore
exceptions

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3EW6v72DvLEt46n6zWprRADmRv_KRbTA-hrw2AZw3k9cQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: PuppetDB 2.3.1 is now available!

[Wyatt Alt]

PuppetDB 2.3.1 - March 31, 2015

PuppetDB 2.3.1 Downloads



Available in native package format in the release repositories at:

http://yum.puppetlabs.com and http://apt.puppetlabs.com

For information on how to enable the Puppet Labs repos, see:

http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 2.3.1: https://tickets.puppetlabs.com/browse/PDB

https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/2.3/

Puppet module: http://forge.puppetlabs.com/puppetlabs/puppetdb

PuppetDB 2.3.1 Release Notes



PuppetDB 2.3.1 is a backward-compatible bugfix release primarily focused on
the storage of facts. Users experiencing constraint violation errors or
noticing heavy database loads on PuppetDB 2.2.0+ are encouraged to upgrade.

For more information and upgrade advice, consult the detailed release notes
here:

https://docs.puppetlabs.com/puppetdb/2.3/release_notes.html

Contributors



Andrew Roetker, Ben S, John Duarte, Ken Barber, Matthaus Owens, Nick
Fagerlund, Rob Browning, Russell Mull, Ryan Senior, Wyatt Alt

Changelog

-

Andrew Roetker (1):



 79fab2f (PDB-1328) Add contributors to and edit release notes





Ben S (1):



 6804e75 (docs) Replace Checkin with Check In





John Duarte (2):



 21d13e2 (PDB-1305) Update install from source docs for AIO


 8ffe8ff (PDB-1305) Change all-in-one refs to Puppet 4





Ken Barber (1):



 113206f (docs) Add .html suffixes to 2.3.0 release notes





Matthaus Owens (1):



 ea4e651 (maint) Remove facter requirement from rpm specfile





Nick Fagerlund (1):



 7739f6d (docs) Add docs sidebar nav (TOC) file to PuppetDB repo, as
_puppetdb_nav.html




Rob Browning (4):


 53210f6 (maint) applied-migrations: don't ignore exceptions


 d5cbad2 (PDB-1031) Accommodate move of fact paths to facts


 dd21714 (PDB-1031) Verify gc behavior in update race test


 a97ecf2 (PDB-1328) Add preliminary 2.3.1 release notes





Russell Mull (2):


 9c59032 (PDB-1307) Update to tk-jetty 1.3.0


 f7604f0 (PDB-1342) Fix trusted facts fallback





Ryan Senior (3):



 dbb9f71 (PDB-1031) Fix query engine, v3 endpoints for fact_path move


 7531aa5 (PDB-1031) Speed migration with unique facts table


 0b4e92e (PDB-1031) Reproduce the fact paths constraint issue





Wyatt Alt (1):


 93c8f54 (PDB-1286) backport resource timestamp normalization to stable

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3E15vUytNhTrsTES7nvqLWqK%2Bx_yzAcFxZ_ZJ-%2BGxBjQw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Upcoming changes in the PuppetDB 3.0 query API

[Wyatt Alt]

With the 3.0 major release of PuppetDB in the next few months, we will 
be retiring the v2 and v3 PuppetDB query APIs and adopting v4 as the new 
stable.  At the same time, we will release the final batch of new 
features for the v4 API, which means that v4 in 3.0 will be different 
than the current experimental v4 endpoint in 2.x.



Moving to the v4 API has permitted the unification of our HTTP endpoints 
under a single central query engine, which has paid great dividends in 
consistency between endpoints and the ease and speed of new feature 
development. Nonetheless, we understand that the simultaneous promotion 
of v4 and retirement of v3 will cause some inconvenience, so we want to 
make sure users are aware of the changes.



For information on the upcoming features and differences, please see the 
draft API migration guide:


http://docs.puppetlabs.com/puppetdb/master/api/query/v4/upgrading-from-v3.html


Note that the API is still under development and the document will be 
updated as new changes are merged.



To test the latest changes, see our nightly snapshots:

http://nightlies.puppetlabs.com/puppetdb/


Note that this is still unreleased code, and not suitable for production 
use.


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/550733CD.3080203%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] puppetdb console and docs

[Wyatt Alt]

Hey James,

The MQ depth is labeled Command Queue depth in the dashboard. I just
updated the docs to make that connection explicit, so your link should be
updated shortly.

That page isn't intended to document the dashboard specifically, so not all
the fields are described. We don't really document the dashboard in a
centralized place, but I like the idea and I made this ticket to create one
https://tickets.puppetlabs.com/browse/PDB-1285

Catalog Duplication specifically is dupes / (dupes + new), where dupes and
new are both counters that are respectively incremented when a catalog is
stored with the same content as one that has already been stored, and when
a catalog is stored with original content.

Both counters start at zero at startup and only apply relative to catalogs
processed since startup.

Wyatt

On Wed, Mar 11, 2015 at 4:17 AM, James Green james.mk.gr...@gmail.com
wrote:

 The documentation at
 https://docs.puppetlabs.com/puppetdb/latest/maintain_and_tune.html talks
 about checking the MQ depth.

 I'm not seeing such a thing on our dashboard but plenty of others not
 documented such as Catalogue Duplication.

 Am I looking at outdated documentation?

 James

  --
 You received this message because you are subscribed to the Google Groups
 Puppet Users group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to puppet-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/puppet-users/CAMH6%2BaxTxqy5yC7AX%2BstW0379khse0dAXBAf3UqbKv0QtOB27A%40mail.gmail.com
 https://groups.google.com/d/msgid/puppet-users/CAMH6%2BaxTxqy5yC7AX%2BstW0379khse0dAXBAf3UqbKv0QtOB27A%40mail.gmail.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3EeMpCKgoCZ1qxM55p3v%3De1-W5b%2B0%3D8oF2OU9z4yN74VQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB 1.6.3 Garbage collection fails abruptly

[Wyatt Alt]

Hey Andreas,

 Can you give some more information about how this cropped up?

- Have you recently upgraded PuppetDB, or changed any other part of your 
Puppet/PuppetDB installation?
- Was PuppetDB running fine before this started occurring, or are you 
still getting it set up?

- What version of PuppetDB are you running?
- What version of Puppet are you running?
- What version of Postgres are you running?
- Can you post the full stacktrace, perhaps in a gist?
- What is the operating system running PuppetDB?
- When the new commands are failing, they should be showing up in 
/var/lib/puppetdb/mq/discarded. If so, can you find one of the failed 
commands submitted after this error and gist that as well?


That should help get the ball rolling -- to my knowledge we haven't seen 
this before.


Thanks,
Wyatt


On 01/02/2015 06:44 AM, Andreas Paul wrote:

Hi there,

the PuppetDB garbage collection fails sometimes after 5 minutes and 
sometimes after 2-3 hours with the following error message:


2014-12-29 12:12:57,312 INFO  [pool-3-thread-1] [cli.services] 
Starting database garbage collection
2014-12-29 16:31:39,106 ERROR [pool-3-thread-1] [cli.services] Error 
during garbage collection



2015-01-02 08:03:18,464 INFO  [pool-3-thread-3] [cli.services] 
Starting database garbage collection

[...]
2015-01-02 08:08:38,255 ERROR [pool-3-thread-3] [cli.services] Error 
during garbage collection
java.sql.BatchUpdateException: Batch entry 0 DELETE FROM 
resource_params_cache WHERE NOT EXISTS (SELECT * FROM 
catalog_resources cr WHERE cr.resource=resource_params_cache.resource) 
was aborted.  Call getNextException to see the cause.
at 
org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2746)
at 
org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:457)
at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1887)
at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2893)

[...]
2015-01-02 08:08:38,256 INFO  [pool-3-thread-3] [cli.services] 
Starting sweep of stale nodes (threshold: 35 days)
2015-01-02 08:08:38,500 INFO  [pool-3-thread-3] [cli.services] 
Finished sweep of stale nodes (threshold: 35 days)




In this time no command can be processed by any PuppetDB server so the 
queue starts to increase very quickly.



Here are some information about our PuppetDB data sets involved in the 
garbage collection query:


# Explain output for select
explain  select * FROM resource_params_cache WHERE NOT EXISTS (SELECT * FROM 
catalog_resources cr WHERE cr.resource=resource_params_cache.resource);
 QUERY PLAN
--
  Nested Loop Anti Join  (cost=0.00..2089716.08 rows=19861941 width=314)
-  Seq Scan on resource_params_cache  (cost=0.00..1267703.93 rows=19876193 
width=314)
-  Index Only Scan using idx_catalog_resources_resource on 
catalog_resources cr  (cost=0.00..11.15 rows=270 width=41)
  Index Cond: (resource = (resource_params_cache.resource)::text

Tables:
  Schema |  Name   | Type  |Owner|  Size   | Description
+-+---+-+-+-
  public | catalog_resources   | table | puppet_prod | 1696 MB |
  public | catalogs| table | puppet_prod | 11 MB   |
  public | certname_facts  | table | puppet_prod | 588 MB  |
  public | certname_facts_metadata | table | puppet_prod | 6128 kB |
  public | certnames   | table | puppet_prod | 14 MB   |
  public | edges   | table | puppet_prod | 1227 MB |
  public | latest_reports  | table | puppet_prod | 9296 kB |
  public | reports | table | puppet_prod | 942 MB  |
  public | resource_events | table | puppet_prod | 11 GB   |
  public | resource_params | table | puppet_prod | 98 GB   |
  public | resource_params_cache   | table | puppet_prod | 91 GB   |
  public | schema_migrations   | table | puppet_prod | 40 kB   |

# Indexes
  Schema | Name | Type  |Owner| 
 Table  |  Size   | Description
+--+---+-+-+-+-
  public | catalog_resources_pkey   | index | puppet_prod | 
catalog_resources   | 536 MB  |
  public | catalogs_certname_key| index | puppet_prod | 
catalogs| 4680 kB |
  public | catalogs_hash_key| index | puppet_prod | 
catalogs| 6632 kB |
  public | catalogs_pkey

Re: [Puppet Users] puppetdb can't delete reports in the future

[Wyatt Alt]

Hey Ryan,

That makes sense, though I'd guess that new reports are being stored and 
are available through the API, they just aren't reflected as latest 
since that's determined by the end_time stamp.  report-ttl will be 
ineffective without resetting to the future, and then you'd be deleting 
reports you may want to keep.


Best bet would be to clear them manually in psql:

delete from reports where end_time  current_timestamp;

Wyatt


On 12/30/2014 09:37 AM, Ryan Anderson wrote:
I have some systems that were deliberately changed to a time in the 
future for testing, then changed back to normal when done. They work 
with puppet fine now, but their puppetdb reports have timestamps in 
the future and new reports will not be added. They show up in 
puppetboard with the inaccurate status of their 'last' report (which 
is/was in the future), and I am unable to get rid of the reports with 
'puppet node deactivate'. How can I get rid of these reports? 
Normally, one would toy around with the report-ttl setting, but that 
is not working for me. Will this require a postgresql command?

--
You received this message because you are subscribed to the Google 
Groups Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
mailto:puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/2c8fd749-b5f4-4736-9f32-2c8cd712ca51%40googlegroups.com 
https://groups.google.com/d/msgid/puppet-users/2c8fd749-b5f4-4736-9f32-2c8cd712ca51%40googlegroups.com?utm_medium=emailutm_source=footer.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/54A3A598.8000307%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] puppetdb org.postgresql.util.PSQLException

[Wyatt Alt]

Hey Fabio,

Got a few questions:

Did you have PuppetDB running properly before you started seeing this 
issue or are you still setting it up?


Does it persist after postgres and PuppetDB are restarted?

When you restart PDB and use it, does it immediately produce this error 
or does it work for some time first? If it works for some time, is there 
some event can can correlate the error to?


Is there anything in your postgres log that indicates why the connection 
is being closed?


Finally, this could be your db connections timing out. Any chance you 
see some improvement by setting conn-keep-alive to a small number like 1?


https://docs.puppetlabs.com/puppetdb/master/configure.html#conn-keep-alive

If none of that sheds light you can send me your postgres and PDB logs 
and we can troubleshoot from there.


Wyatt



On 12/19/2014 02:15 AM, Fabio Sangiovanni wrote:

Hi everybody, I'm incurring in an issue with puppetdb.
I keep on seeing this in /var/log/puppetdb/puppetdb.log:

2014-12-19 10:45:55,957 WARN  [c.p.jdbc] Caught exception. Last 
attempt, throwing exception.
2014-12-19 10:45:55,961 ERROR [c.p.p.command] 
[f74061b8-1350-4b9e-9b77-b52f6d919ef9] [replace facts] Retrying after 
attempt 9, due to: org.postgresql.util.PSQLException: This connection 
has been closed.

org.postgresql.util.PSQLException: This connection has been closed.
at 
org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:822) 
~[puppetdb.jar:na]
at 
org.postgresql.jdbc2.AbstractJdbc2Connection.setAutoCommit(AbstractJdbc2Connection.java:769) 
~[puppetdb.jar:na]
at 
com.jolbox.bonecp.ConnectionHandle.setAutoCommit(ConnectionHandle.java:1063) 
~[puppetdb.jar:na]
at 
clojure.java.jdbc.internal$transaction_STAR_.invoke(internal.clj:222) 
~[na:na]
at 
com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761$fn__6762.invoke(jdbc.clj:290) 
~[na:na]
at 
clojure.java.jdbc.internal$with_connection_STAR_.invoke(internal.clj:186) 
~[na:na]
at 
com.puppetlabs.jdbc$with_transacted_connection_fn$fn__6761.invoke(jdbc.clj:287) 
~[na:na]
at 
com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740$fn__6741$fn__6742.invoke(jdbc.clj:259) 
~[na:na]
at 
com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740$fn__6741.invoke(jdbc.clj:258) 
~[na:na]
at 
com.puppetlabs.jdbc$eval6739$retry_sql_STAR___6740.invoke(jdbc.clj:250) ~[na:na]
at 
com.puppetlabs.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:286) 
~[na:na]
at 
com.puppetlabs.puppetdb.command$eval11543$fn__11546.invoke(command.clj:379) 
~[na:na]

at clojure.lang.MultiFn.invoke(MultiFn.java:231) ~[puppetdb.jar:na]
at 
com.puppetlabs.puppetdb.command$produce_message_handler$fn__11715.invoke(command.clj:647) 
~[na:na]
at 
com.puppetlabs.puppetdb.command$wrap_with_discard$fn__11664$fn__11668.invoke(command.clj:554) 
~[na:na]
at 
com.puppetlabs.puppetdb.command.proxy$java.lang.Object$Callable$7da976d4.call(Unknown 
Source) ~[na:na]
at com.yammer.metrics.core.Timer.time(Timer.java:91) 
~[puppetdb.jar:na]
at 
com.puppetlabs.puppetdb.command$wrap_with_discard$fn__11664.invoke(command.clj:553) 
~[na:na]
at 
com.puppetlabs.puppetdb.command$wrap_with_exception_handling$fn__11649$fn__11650.invoke(command.clj:507) 
~[na:na]
at 
com.puppetlabs.puppetdb.command.proxy$java.lang.Object$Callable$7da976d4.call(Unknown 
Source) ~[na:na]
at com.yammer.metrics.core.Timer.time(Timer.java:91) 
~[puppetdb.jar:na]
at 
com.puppetlabs.puppetdb.command$wrap_with_exception_handling$fn__11649.invoke(command.clj:506) 
~[na:na]
at 
com.puppetlabs.puppetdb.command$wrap_with_command_parser$fn__11659.invoke(command.clj:529) 
[na:na]
at 
com.puppetlabs.puppetdb.command$wrap_with_meter$fn__11639.invoke(command.clj:467) 
[na:na]
at 
com.puppetlabs.puppetdb.command$wrap_with_thread_name$fn__11673.invoke(command.clj:569) 
[na:na]
at 
com.puppetlabs.mq$create_message_listener$reify__10820.onMessage(mq.clj:270) 
[na:na]
at 
org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:560) 
[puppetdb.jar:na]
at 
org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:498) 
[puppetdb.jar:na]
at 
org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467) 
[puppetdb.jar:na]
at 
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:325) 
[puppetdb.jar:na]
at 
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263) 
[puppetdb.jar:na]
at 
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058) 
[puppetdb.jar:na]
at 

Re: [Puppet Users] Client self-deregistration from PuppetDB

[Wyatt Alt]

To follow up on what others have said, the method described in the 
command example


https://docs.puppetlabs.com/puppetdb/2.2/api/commands.html#examples-using-curl

will deactivate your node, which is different than what we call 
purging, whether or not you intended.


Deactivating a node will not purge it from the database, it will simply 
mark it deactivated and cause it to be filtered from the output of most 
queries.  The node is actually purged from the database when it has been 
deactivated for the length of time specified by the setting 
node-purge-ttl, which is documented here:


https://docs.puppetlabs.com/puppetdb/latest/configure.html#node-purge-ttl

If you set this to 0m, nodes will be purged on the next garbage 
collection (configurable by the gc-interval setting, 60 minutes by default.)


Wyatt
On 12/11/14 1:04 AM, Martin Alfke wrote:

Hi Matt,
On 09 Dec 2014, at 19:58, Matt Wise m...@nextdoor.com wrote:


We boot up/shut-down 50-100 hosts a day on average... we're exploring PuppetDB, 
but I'm concerned about the model of just 'waiting' for hosts to be purged 
based on some checkin time. Is there any way to have our hosts send a signal 
through the puppet-masters (or directly to puppetdb?) to purge themselves when 
they're being terminated?

You can use the puppetdb rest api:
https://docs.puppetlabs.com/puppetdb/2.2/api/index.html

In my actual project we disable hosts via VM management system using this API.
Works like a charm.

hth,

Martin



--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/548A2797.10100%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Querying PuppetDB for nodes that match certain facts

[Wyatt Alt]

Abhi,

Try this:
curl -X GET http://localhost:8080/v4/facts/ipaddress --data-urlencode 
'query=[in,certname,[extract,certname,[select-nodes,[=,[fact,kernel],Linux


Wyatt

On 12/11/14 12:14 PM, a...@littlewiki.in wrote:

Hello.

I want to get the ip-addresses of all Linux machines. Here's my query 
using curl


query=[=, name, ipaddress],[in, certname,[extract, 
certname, [select-nodes,[and,[=, name, kernel],[=, 
value, Linux]



This returns me the ipaddresses of all the machines. If I add an and 
condition to the two blocks, it returns a query malformed error. 
Essentially, I want to get nodes that match factA=B and factC=D and 
return factX. and factY of each matched node.



How do I go about doing this?

--
Abhi
--
You received this message because you are subscribed to the Google 
Groups Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
mailto:puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/8fd9d60b-4e25-4e25-8c98-7559bec45ba4%40googlegroups.com 
https://groups.google.com/d/msgid/puppet-users/8fd9d60b-4e25-4e25-8c98-7559bec45ba4%40googlegroups.com?utm_medium=emailutm_source=footer.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/548A3198.8030901%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB errors

[Wyatt Alt]

Paul,

Thanks for that. It looks like 22128 has not been cleaned up properly 
for some reason, and it's likely being skipped over in the gc process 
because the facts cleanup identifies values to delete from that column 
in the facts table.


Are there failed commands in your /var/lib/puppetdb/mq/discarded 
directory you can tie this to? I'd like to take a look at that if you 
can send me a tarball.


This issue would likely disappear if you dropped and recreated the 
puppetdb database and allowed the nodes to repopulate it, but that could 
take some maneuvering depending on your situation/data needs.


Wyatt

On 11/20/14 12:29 AM, Paul Seymour wrote:



On Wednesday, 19 November 2014 17:56:23 UTC, Wyatt Alt wrote:

Hey Paul,

That's some kind of DB corruption. Shouldn't be happening. Do you
have
any sense of when this started or whether it was tied to a recent
upgrade? Is there a stacktrace in your logs that you could gist?
Does it
happen on every agent run or only occasionally? Is it always the same
path_id like in the other ticket?

Also would you mind reporting the output of this?

select * from facts where fact_value_id in (50319,22128);


Thanks Wyatt,

I cannot be sure if this is tied to an upgrade specifically but I just 
happened to be keeping an eye
on the logs a little more lately (working on an ELK thing to 
monitoring them).


A stack trace is available at 
https://gist.github.com/PsychoSid/b1a479d7d6fec7477263


The output from the select is:-
$ psql
psql (9.3.5)
Type help for help.

postgres=# \c puppetdb
You are now connected to database puppetdb as user postgres.
puppetdb=# select * from facts where fact_value_id in (50319,22128);
 factset_id | fact_value_id
+---
 22 | 50319
(1 row)

Hope this helps.

Cheers
Paul
--
You received this message because you are subscribed to the Google 
Groups Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
mailto:puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/7baf1653-d0b4-4fcf-8bcb-039b43af99a5%40googlegroups.com 
https://groups.google.com/d/msgid/puppet-users/7baf1653-d0b4-4fcf-8bcb-039b43af99a5%40googlegroups.com?utm_medium=emailutm_source=footer.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/546F6EF0.6060402%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB errors

[Wyatt Alt]

Manually deleting that row (id=22128) in fact_values might also do the
trick if it's only one affected node/value, but wouldn't tell much about
how you got to this state so could be leaving something broken.  It would
be interesting to get the discarded tarball in any case.

On Fri, Nov 21, 2014 at 8:57 AM, Wyatt Alt wy...@puppetlabs.com wrote:

  Paul,

 Thanks for that. It looks like 22128 has not been cleaned up properly for
 some reason, and it's likely being skipped over in the gc process because
 the facts cleanup identifies values to delete from that column in the facts
 table.

 Are there failed commands in your /var/lib/puppetdb/mq/discarded directory
 you can tie this to? I'd like to take a look at that if you can send me a
 tarball.

 This issue would likely disappear if you dropped and recreated the
 puppetdb database and allowed the nodes to repopulate it, but that could
 take some maneuvering depending on your situation/data needs.

 Wyatt


 On 11/20/14 12:29 AM, Paul Seymour wrote:



 On Wednesday, 19 November 2014 17:56:23 UTC, Wyatt Alt wrote:

 Hey Paul,

 That's some kind of DB corruption. Shouldn't be happening. Do you have
 any sense of when this started or whether it was tied to a recent
 upgrade? Is there a stacktrace in your logs that you could gist? Does it
 happen on every agent run or only occasionally? Is it always the same
 path_id like in the other ticket?

 Also would you mind reporting the output of this?

 select * from facts where fact_value_id in (50319,22128);


  Thanks Wyatt,

  I cannot be sure if this is tied to an upgrade specifically but I just
 happened to be keeping an eye
 on the logs a little more lately (working on an ELK thing to monitoring
 them).

  A stack trace is available at
 https://gist.github.com/PsychoSid/b1a479d7d6fec7477263

  The output from the select is:-
 $ psql
 psql (9.3.5)
 Type help for help.

  postgres=# \c puppetdb
 You are now connected to database puppetdb as user postgres.
 puppetdb=# select * from facts where fact_value_id in (50319,22128);
  factset_id | fact_value_id
 +---
  22 | 50319
 (1 row)

  Hope this helps.

  Cheers
 Paul
  --
 You received this message because you are subscribed to the Google Groups
 Puppet Users group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to puppet-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/puppet-users/7baf1653-d0b4-4fcf-8bcb-039b43af99a5%40googlegroups.com
 https://groups.google.com/d/msgid/puppet-users/7baf1653-d0b4-4fcf-8bcb-039b43af99a5%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3EtXEdC-5MvuC-aOZxAYAvk9ja3ZOShtd6p5J6nPKoT_w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB service not running

[Wyatt Alt]

These errors are red herrings as far as PDB not running is concerned. 
You can make them go away by changing the temp-usage and store-usage in 
the command-processing section of your config file to limit ActiveMQ to 
a space less than the size of your disk.


https://docs.puppetlabs.com/puppetdb/latest/configure.html#store-usage

As far as PuppetDB not running, maybe you can provide more information? 
Have you tried restarting the service?


Wyatt

On 11/18/14 6:54 AM, mike wrote:

Hello Eveyone
I've Puppet Server 3.7 running on Centos 6, i try install Puppetdb but 
when upload service inside the log i have the next error:


[..]
2014-11-18 11:33:56,676 INFO  [o.a.k.j.Journal] ignoring zero length, 
partially initialised journal data file: db-1.log number = 1 , length = 0
2014-11-18 11:33:56,779 WARN  [o.a.a.b.BrokerService] Store limit is 
10 mb, whilst the data directory: 
/var/lib/puppetdb/mq/localhost/KahaDB only has 31266 mb of usable space
2014-11-18 11:33:56,779 ERROR [o.a.a.b.BrokerService] Temporary Store 
limit is 5 mb, whilst the temporary data directory: 
/var/lib/puppetdb/mq/localhost/tmp_storage only has 31266 mb of usable 
space
2014-11-18 11:33:56,780 INFO  [c.p.p.c.services] Starting 1 command 
processor threads

[..]

And the puppetdb port (8081) isn't running

Thanks.
--
You received this message because you are subscribed to the Google 
Groups Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
mailto:puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/25d53d3e-960e-4c8d-a5a3-51898be7b72c%40googlegroups.com 
https://groups.google.com/d/msgid/puppet-users/25d53d3e-960e-4c8d-a5a3-51898be7b72c%40googlegroups.com?utm_medium=emailutm_source=footer.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/546CCA4A.9090600%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB errors

[Wyatt Alt]

Hey Paul,

That's some kind of DB corruption. Shouldn't be happening. Do you have 
any sense of when this started or whether it was tied to a recent 
upgrade? Is there a stacktrace in your logs that you could gist? Does it 
happen on every agent run or only occasionally? Is it always the same 
path_id like in the other ticket?


Also would you mind reporting the output of this?

select * from facts where fact_value_id in (50319,22128);

Wyatt
On 11/19/14 12:15 AM, Paul Seymour wrote:

Hello,

Getting a couple of errors thrown with PuppetDB

2014-11-19 08:03:49,120 ERROR [c.p.p.command] 
[22329b48-7c20-4ec4-beb2-15cd02e11bff] [replace facts] Retrying after 
attempt 8, due to: org.postgresql.util.PSQLException: ERROR: update or 
delete on table fact_paths violates foreign key constraint 
fact_values_path_id_fk on table fact_values

  Detail: Key (id)=(512) is still referenced from table fact_values.
org.postgresql.util.PSQLException: ERROR: update or delete on table 
fact_paths violates foreign key constraint fact_values_path_id_fk 
on table fact_values

  Detail: Key (id)=(512) is still referenced from table fact_values.


Which is similar to PDB-1031 albeit with a different fact (ours is a 
custom one not _timestamp).


puppetdb=# select * from fact_paths where id = 512;
 id  | value_type_id | depth |  name  |   
   path

-+---+---++
 512 | 0 | 0 | XX_ethernet_eth0_intralink_unit_id 
| XX_ethernet_eth0_intralink_unit_id

(1 row)

puppetdb=# select * from fact_values where path_id = 512;
  id   | path_id | value_type_id |  value_hash| 
value_integer | value_float | value_string | value_boolean | value_json

---+-+---+--+---+-+--+---+
 50319 | 512 | 0 | 
80e54cb487aed67b98d7dc92feaac988bf95bcc7 |   |   | 
107827   |   |
 22128 | 512 | 0 | 
47e3876c8d334accbeb827775e728643581bbb83 |   |   | 
107292   |   |


Is it just noise or something to be concerned about ? This is with 
v2.2.1 and PostGres 9.3.5


Thanks
Paul
--
You received this message because you are subscribed to the Google 
Groups Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to puppet-users+unsubscr...@googlegroups.com 
mailto:puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/08bf4cb9-6c80-489d-a0ef-26627f395f5e%40googlegroups.com 
https://groups.google.com/d/msgid/puppet-users/08bf4cb9-6c80-489d-a0ef-26627f395f5e%40googlegroups.com?utm_medium=emailutm_source=footer.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/546CD9C0.2050308%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Puppet -- Problem accessing /v3/commands

[Wyatt Alt]

Hey Stacey,

This error can be generated by queries made during the PuppetDB startup 
sequence and also if PuppetDB is upgraded but not restarted. Can your 
report your PuppetDB version with


curl -G http://localhost:8080/v4/version
(for the appropriate hostname)

and if you've just upgraded, ensure that you've restarted the service?

Wyatt

On Sat Oct 18 08:46:38 2014, Stella wrote:

Hi guys,

I am us puppet 3.6.2 and dashboard. When I run puppet agent --test,
got this error:

Error: Could not retrieve catalog from remote server: Error 400 on
SERVER: Failed to submit 'replace facts' command for example.com to
PuppetDB at example.com:8081: [404 Not Found] htmlheadmeta
http-equiv=Content-Type content=text/html;/titleError 404
/title/headbodyh2HTTP ERROR: 404/h2pProblem accessing
/v3/commands. Reason:preNot Found/pre/phr
/ismallPowered by Jetty:///small/i/body/html

What could be the problem?

Thanks,
Stacey



--
You received this message because you are subscribed to the Google
Groups Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send
an email to puppet-users+unsubscr...@googlegroups.com
mailto:puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/6f50b072-138e-454e-a97a-e90bf49d52fd%40googlegroups.com
https://groups.google.com/d/msgid/puppet-users/6f50b072-138e-454e-a97a-e90bf49d52fd%40googlegroups.com?utm_medium=emailutm_source=footer.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5442B091.3020901%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce PuppetDB 2.2.1 is now available

[Wyatt Alt]

PuppetDB 2.2.1 - October 14, 2014

PuppetDB 2.2.1 Downloads


Available in native package format in the release repositories at:
http://yum.puppetlabs.com and http://apt.puppetlabs.com

For information on how to enable the Puppet Labs repos, see:
http://docs.puppetlabs.com/guides/puppetlabs_package_repositories.html#open-source-repositories

Binary tarball: http://downloads.puppetlabs.com/puppetdb/

Source: http://github.com/puppetlabs/puppetdb

Please report feedback via the Puppet Labs tickets site, using an affected
PuppetDB version of 2.2.1:
https://tickets.puppetlabs.com/browse/PDB

Documentation: http://docs.puppetlabs.com/puppetdb/2.2/

Puppet module:
http://forge.puppetlabs.com/puppetlabs/puppetdb

Release notes: https://docs.puppetlabs.com/puppetdb/2.2/release_notes.html

PuppetDB 2.2.1 Release Notes


This is a backwards-compatible bugfix release following the 2.2.0 feature
release that introduced structured facts.  For more information and upgrade
advice, consult the detailed release notes here:

https://docs.puppetlabs.com/puppetdb/2.2/release_notes.html

Contributors


Justin Holguin, Ken Barber, Kylo Ginsberg, Russel Sim, Ryan Senior and
Wyatt Alt

Changelog
-

Justin Holguin (1):
 24835c9 (DOCUMENT-18) Mention DLO cleanup in docs

Ken Barber (14):
 d48be30 (PDB-707) Fix bonecp 57P01 handling
 9b53a30 Allow beaker rake task to accept proper preserve-hosts options
 3715db6 (testing) Remove open_postgres_port usage from puppetdb class
 2a0624a (PDB-868) Add exponential back offs for db retry logic
 2d79c18 Switch to using c3.large testing instances
 3ebbfa2 Mention that a -contrib package is required when install
pg_trgm
 278a32a Missing check for fact_values_string_trgm
 7f66fbd (PDB-905) Fix containment-path for skipped events
 b8584a3 Try reverting to c1.medium
 178f673 Drain queues where appropriate to avoid race conditions
 38a4b11 Revert Try reverting to c1.medium
 b07e5fc More clarification around using an IP address for ssl-host and
host
 3057823 Fix the dependency for puppet for the PuppetDB package for the
correct version
 af9843b (PDB-904) Switch fact_values.value_string trgm index to be GIN
not GIST

Kylo Ginsberg (1):
 bb82c4e (docs) Fix some spelling mistakes and one word choice.

Russell Sim (1):
 4af2462 (docs) Fixed contributor link

Ryan Senior (5):
 72a1588 Changed our acceptance test setup to pin on facter 2.1.0
 ee6be59 Revert Changed our acceptance test setup to pin on facter
2.1.0
 021e864 (PDB-900) Make performance improvements to fact_values GC
process
 484d39d Fix for acceptance test issue related to fact path - fact
value constraint violation
 7d0c662 Fixed bug when updating the values for an existing certname's
factset

Wyatt Alt (7):
 e9e4cbb (PDB-847) fix PE compatibility bug
 403073f (maint/stable) fix documented behavior of =
 24d655d (maint) fix commands API examples
 6b83789 (PDB-865) PDB 2.2 migration fails when nodes have no facts
 6b81b4b (PDB-13) Add wheezy to PDB acceptance tests
 b405a48 (PDB-653) metrics update on PDB start
 bb8f1bc Update release notes for PDB 2.2.1 release.

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3Fp%2ByyZ1_J9oQr9vzjjVNaVDKgu%2B7vkw5FOBt1haGXn2g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] puppetdb export - WTF?!

[Wyatt Alt]

Hey Jon, you can inform us how many reports you have without rerunning the
export.  That would be useful information:

curl -D - -G http://localhost:8080/v4/reports --data-urlencode
'include-total=true' -o /dev/null

obviously substituting your own host if it's different.  Also note the dash
after D. What does it give you under X-Records?

Wyatt


On Sun, Oct 12, 2014 at 6:19 PM, JonY ethrbu...@gmail.com wrote:

 I pulled the plug on it after  48 hours. The file is 14Mb at this point.

 I may start the Postgres server and try the export again. Perhaps if
 puppetdb isn't running this will be more efficient.

 On Friday, October 10, 2014 6:13:04 PM UTC-7, Wyatt Alt wrote:

 Hey Jon,

 Thanks for the update and the ticket:
 https://tickets.puppetlabs.com/browse/PDB-947

 We've been trying to reproduce this today and have had relatively
 little luck. As you alluded earlier, it seems like a possible
 contributor might be the number of reports per node.  Would you mind
 giving us the sizes of your exported tarball and contained folders, as
 well as the average number of reports per node?

 I'm currently running a 200 node hsql export with 745 reports per node
 (149k total), which is beyond the scale we would expect to be
 performant on hsql, and at current pace it will still be done in seven
 hours or so.

 To the extent that you're still having trouble a stack dump may be
 helpful in diagnosing the issue.

 Thanks,
 Wyatt


 On Thu Oct  9 03:50:13 2014, JonY wrote:
  Update: 45 hours - ~75% complete
 
  On Tuesday, October 7, 2014 1:32:02 PM UTC-7, JonY wrote:
 
  Running for 7 hours now. Has exported ~15-20% of the data.
 
  I'm intrigued to see what I end up with.
 
 
  On Tuesday, October 7, 2014 1:19:04 PM UTC-7, Wyatt Alt wrote:
 
  Thanks Jony.
 
  I've loaded up an embedded database to comparable capacity and
  while export isn't quick, it's not nearly as slow as you're
  experiencing.
 
  Fro your process list the PDB appears to be running with a max
  heap size (Xmx) of 1024m.  Perhaps increasing this could make
  a difference?
 
  The export process appears to be using 192m, but I think that
  should be using the same JAVA_ARGS as PuppetDB itself and so
  could be a bug.
 
  Wyatt
 
 
  On Tue, Oct 7, 2014 at 11:03 AM, JonY ethr...@gmail.com
 wrote:
 
  https://gist.github.com/ce60c590a0531c0b09cd.git
  https://gist.github.com/ce60c590a0531c0b09cd.git
 
 
  # rpm -qa | grep puppet
  puppet-server-3.7.1-1.el6.noarch
  puppetlabs-release-6-11.noarch
  puppetdb-terminus-2.2.0-1.el6.noarch
  puppet-3.7.1-1.el6.noarch
  vim-puppet-2.7.20-1.el6.rf.noarch
  puppetdb-2.2.0-1.el6.noarch
 
  Am storing 30 days of data. Yes - it's a fair bit more
  then the default.
 
 
 
 
  On Tuesday, October 7, 2014 9:00:17 AM UTC-7, Wyatt Alt
 wrote:
 
  Hey JonY,
 
  Sounds interesting.  What version of PuppetDB are you
  using?  Do you have reports, facts, and catalogs, or
  only some of those? Can you paste your config.ini?
 
  Also can you give the output of
 
  ps aux |grep java
  top -n1
  free
 
  in a gist maybe?
 
  Wyatt
 
 
 
 
 
  On Tue, Oct 7, 2014 at 8:30 AM, JonY
  ethr...@gmail.com wrote:
 
  (ok - terrribly unprofessional title.. I get it).
 
  I'm trying to 'do the right thing' and move from
  the embedded DB to Postgres. Following the
  instructions I figured I would dump out the
  contents of the embedded DB and import this into
  Postgres.
 
  So I start 'puppetdb export --outfile someplace'.
 
  Pour some coffee.
 
  Watch it dump one node.
 
  Have some more coffee.
 
  Wait.
 
  One more node.
 
  Hours pass. Glaciers melt.
 
  3+ hours into the process and it has dumped 10
  nodes (of  100). At this rate I'm looking at
  about 1.5 days to get this done. Really?
 
  System has 16 cores, 32Gb of RAM.. barely running
  at idle. Tell me I missed some critical parameter.
 
 
  --
  You received this message because you are
  subscribed to the Google Groups Puppet Users
 group.
  To unsubscribe from this group and stop receiving
  emails from it, send an email to
  puppet-users

Re: [Puppet Users] puppetdb export - WTF?!

[Wyatt Alt]

Hey Jon,

Thanks for the update and the ticket: 
https://tickets.puppetlabs.com/browse/PDB-947


We've been trying to reproduce this today and have had relatively 
little luck. As you alluded earlier, it seems like a possible 
contributor might be the number of reports per node.  Would you mind 
giving us the sizes of your exported tarball and contained folders, as 
well as the average number of reports per node?


I'm currently running a 200 node hsql export with 745 reports per node 
(149k total), which is beyond the scale we would expect to be 
performant on hsql, and at current pace it will still be done in seven 
hours or so.


To the extent that you're still having trouble a stack dump may be 
helpful in diagnosing the issue.


Thanks,
Wyatt


On Thu Oct  9 03:50:13 2014, JonY wrote:

Update: 45 hours - ~75% complete

On Tuesday, October 7, 2014 1:32:02 PM UTC-7, JonY wrote:

Running for 7 hours now. Has exported ~15-20% of the data.

I'm intrigued to see what I end up with.


On Tuesday, October 7, 2014 1:19:04 PM UTC-7, Wyatt Alt wrote:

Thanks Jony.

I've loaded up an embedded database to comparable capacity and
while export isn't quick, it's not nearly as slow as you're
experiencing.

Fro your process list the PDB appears to be running with a max
heap size (Xmx) of 1024m.  Perhaps increasing this could make
a difference?

The export process appears to be using 192m, but I think that
should be using the same JAVA_ARGS as PuppetDB itself and so
could be a bug.

Wyatt


On Tue, Oct 7, 2014 at 11:03 AM, JonY ethr...@gmail.com wrote:

https://gist.github.com/ce60c590a0531c0b09cd.git
https://gist.github.com/ce60c590a0531c0b09cd.git


# rpm -qa | grep puppet
puppet-server-3.7.1-1.el6.noarch
puppetlabs-release-6-11.noarch
puppetdb-terminus-2.2.0-1.el6.noarch
puppet-3.7.1-1.el6.noarch
vim-puppet-2.7.20-1.el6.rf.noarch
puppetdb-2.2.0-1.el6.noarch

Am storing 30 days of data. Yes - it's a fair bit more
then the default.




On Tuesday, October 7, 2014 9:00:17 AM UTC-7, Wyatt Alt wrote:

Hey JonY,

Sounds interesting.  What version of PuppetDB are you
using?  Do you have reports, facts, and catalogs, or
only some of those? Can you paste your config.ini?

Also can you give the output of

ps aux |grep java
top -n1
free

in a gist maybe?

Wyatt





On Tue, Oct 7, 2014 at 8:30 AM, JonY
ethr...@gmail.com wrote:

(ok - terrribly unprofessional title.. I get it).

I'm trying to 'do the right thing' and move from
the embedded DB to Postgres. Following the
instructions I figured I would dump out the
contents of the embedded DB and import this into
Postgres.

So I start 'puppetdb export --outfile someplace'.

Pour some coffee.

Watch it dump one node.

Have some more coffee.

Wait.

One more node.

Hours pass. Glaciers melt.

3+ hours into the process and it has dumped 10
nodes (of  100). At this rate I'm looking at
about 1.5 days to get this done. Really?

System has 16 cores, 32Gb of RAM.. barely running
at idle. Tell me I missed some critical parameter.


--
You received this message because you are
subscribed to the Google Groups Puppet Users group.
To unsubscribe from this group and stop receiving
emails from it, send an email to
puppet-users...@__googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/__msgid/puppet-users/1fbe251a-__2c25-42ea-9529-d01bbf561980%__40googlegroups.com

https://groups.google.com/d/msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980%40googlegroups.com?utm_medium=emailutm_source=footer.
For more options, visit
https://groups.google.com/d/__optout
https://groups.google.com/d/optout.


--
You received this message because you are subscribed to
the Google Groups Puppet Users group.
To unsubscribe from this group and stop receiving emails
from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit

https

Re: [Puppet Users] puppetdb export - WTF?!

[Wyatt Alt]

Hey JonY,

Sounds interesting.  What version of PuppetDB are you using?  Do you have
reports, facts, and catalogs, or only some of those? Can you paste your
config.ini?

Also can you give the output of

ps aux |grep java
top -n1
free

in a gist maybe?

Wyatt





On Tue, Oct 7, 2014 at 8:30 AM, JonY ethrbu...@gmail.com wrote:

 (ok - terrribly unprofessional title.. I get it).

 I'm trying to 'do the right thing' and move from the embedded DB to
 Postgres. Following the instructions I figured I would dump out the
 contents of the embedded DB and import this into Postgres.

 So I start 'puppetdb export --outfile someplace'.

 Pour some coffee.

 Watch it dump one node.

 Have some more coffee.

 Wait.

 One more node.

 Hours pass. Glaciers melt.

 3+ hours into the process and it has dumped 10 nodes (of  100). At this
 rate I'm looking at about 1.5 days to get this done. Really?

 System has 16 cores, 32Gb of RAM.. barely running at idle. Tell me I
 missed some critical parameter.


  --
 You received this message because you are subscribed to the Google Groups
 Puppet Users group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to puppet-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980%40googlegroups.com
 https://groups.google.com/d/msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3Ex25NJ6e2sLa%2BVzp6GFCJhzv2Jmo1%3D8GgdLQZ4ueheng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] PuppetDB throwing tons of exceptions

[Wyatt Alt]

Hi Guillaume,

Can you post the output of

rpm -qa | grep puppet

?

It looks as if your puppetdb-terminus version could be out of sync with
that of puppetdb.

Wyatt


On Wed, Sep 3, 2014 at 7:17 AM, Guillaume Blairon g...@yom.be wrote:

 Hi,

 We're using puppetdb 2.1.0 and puppet 3.6.2 on CentOS. PuppetDB has been
 working fine for a few weeks.

 Yesterday we added a new puppet master to handle our ~400 servers. When we
 added the new master in the SRV records, PuppetDB began to show an erratic
 behaviour :
 - the number of iops on /var has raised from 50/sec to 500+/sec
 - the /var/lib/puppetdb/mq/localhost/scheduler is being filled with
 db-*.log files until filesystem gets full
 - the logs are filled with this type of messages (~1 per second) :
 ---
 2014-09-03 15:48:51,006 ERROR [c.p.p.command]
 [24506b0f-6fcd-4009-9761-bfe3652ee1bd] [replace facts] Retrying after
 attempt 5, due to: java.lang.IllegalArgumentException: No method in
 multimethod 'process-command!' for dispatch value: [replace facts 3]
 java.lang.IllegalArgumentException: No method in multimethod
 'process-command!' for dispatch value: [replace facts 3]
 at clojure.lang.MultiFn.getFn(MultiFn.java:160) ~[puppetdb.jar:na]
 at clojure.lang.MultiFn.invoke(MultiFn.java:231) ~[puppetdb.jar:na]
 at
 com.puppetlabs.puppetdb.command$produce_message_handler$fn__10782.invoke(command.clj:630)
 ~[na:na]
 at
 com.puppetlabs.puppetdb.command$wrap_with_discard$fn__10731$fn__10735.invoke(command.clj:537)
 ~[na:na]
 at
 com.puppetlabs.puppetdb.command.proxy$java.lang.Object$Callable$7da976d4.call(Unknown
 Source) ~[na:na]
 at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na]
 at
 com.puppetlabs.puppetdb.command$wrap_with_discard$fn__10731.invoke(command.clj:536)
 ~[na:na]
 at
 com.puppetlabs.puppetdb.command$wrap_with_exception_handling$fn__10716$fn__10717.invoke(command.clj:490)
 ~[na:na]
 at
 com.puppetlabs.puppetdb.command.proxy$java.lang.Object$Callable$7da976d4.call(Unknown
 Source) ~[na:na]
 at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na]
 at
 com.puppetlabs.puppetdb.command$wrap_with_exception_handling$fn__10716.invoke(command.clj:489)
 ~[na:na]
 at
 com.puppetlabs.puppetdb.command$wrap_with_command_parser$fn__10726.invoke(command.clj:512)
 [na:na]
 at
 com.puppetlabs.puppetdb.command$wrap_with_meter$fn__10706.invoke(command.clj:450)
 [na:na]
 at
 com.puppetlabs.puppetdb.command$wrap_with_thread_name$fn__10740.invoke(command.clj:552)
 [na:na]
 at com.puppetlabs.mq$create_message_listener$reify__9909.onMessage(mq.clj:270)
 [na:na]
 at
 org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:560)
 [puppetdb.jar:na]
 at
 org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:498)
 [puppetdb.jar:na]
 at
 org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:467)
 [puppetdb.jar:na]
 at
 org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:325)
 [puppetdb.jar:na]
 at
 org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
 [puppetdb.jar:na]
 at
 org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
 [puppetdb.jar:na]
 at
 org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050)
 [puppetdb.jar:na]
 at
 org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947)
 [puppetdb.jar:na]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
 ---

 Until now, the only solution we have found is to stop puppetdb,
 remove /var/lib/puppetdb/mq/localhost and restart puppetdb. But it's only
 buying us a few hours before the filesystem gets full again.

 I'm not actually sure that the problem has a relation with the new master,
 but that's more or less when the problem started. Do you have an idea of
 what's happening here ? Is there something else I could check ?

 Cheers,
 --
 Guillaume

  --
 You received this message because you are subscribed to the Google Groups
 Puppet Users group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to puppet-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/puppet-users/85fb831e-4bf4-4d3d-acd3-a9d8892839e4%40googlegroups.com
 https://groups.google.com/d/msgid/puppet-users/85fb831e-4bf4-4d3d-acd3-a9d8892839e4%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because 

Re: [Puppet Users] Could not find terminus puppetdb for indirection facts

[Wyatt Alt]

Hi Spriya,

That error will occur if you don't have puppetdb-terminus installed.  
Have you installed the terminus as described here?


https://docs.puppetlabs.com/puppetdb/2.2/connect_puppet_master.html

Your routes.yaml file also needs three dashes at the top, not two.

If that doesn't help please reply with your operating system.

Wyatt

On Wed Sep  3 12:20:44 2014, Spriya wrote:


Hi ,

I am having an issue when trying to configure puppetdb. Here is my error:
*Error: Could not retrieve catalog from remote server: Error 400 on
SERVER: Could not find terminus puppetdb for indirection facts*

*cat routes.yaml
--
 master:
   facts:
 terminus: puppetdb
 cache: yaml *
*

cat puppetdb.conf
[main]
server = hostname
port = 8081*


I dont why the issue is comming still.

Please help me.

--
You received this message because you are subscribed to the Google
Groups Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send
an email to puppet-users+unsubscr...@googlegroups.com
mailto:puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/b9f82808-1baa-4288-9725-1277a80eed79%40googlegroups.com
https://groups.google.com/d/msgid/puppet-users/b9f82808-1baa-4288-9725-1277a80eed79%40googlegroups.com?utm_medium=emailutm_source=footer.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5407713F.1000808%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] havina an issue regarding puppet agent run

[Wyatt Alt]

Hi Spriya,

Can you try accessing your database server directly at the host and port
specified in /etc/puppetdb/conf.d/jetty.ini and see if you get the same
error?

e.g curl http://localhost:8080/v4/version

Wyatt




On Fri, Aug 29, 2014 at 11:30 AM, Spriya supriya.uppalap...@gmail.com
wrote:

 Hi,

 I installed puppet server using opensource.when i run puppet agent -t i am
 having this issue:


 *Error: Could not retrieve catalog from remote server: Error 400 on
 SERVER: Failed to submit 'replace facts' command for example.com
 http://example.com to PuppetDB at example.com:8081
 http://example.com:8081: [404 Not Found] htmlheadmeta
 http-equiv=Content-Type
 content=text/html;charset=ISO-8859-1/titleError 404
 /title/headbodyh2HTTP ERROR: 404/h2pProblem accessing
 /v3/commands. Reason:preNot Found/pre/phr /ismallPowered by
 Jetty:///small/i/body/htmlWarning: Not using cache on failed
 catalogError: Could not retrieve catalog; skipping run*


 Here are my version information:








 *rpm -qa | grep
 puppetpuppetdb-2.2.0-1.el6.noarchpuppetlabs-release-6-6.noarchpuppet-3.6.2-1.el6.noarchpuppet-dashboard-1.2.23-1.el6.noarchpuppet-server-3.6.2-1.el6.noarchrubygem-puppet-lint-0.3.2-1.el6.noarchpuppetdb-terminus-2.2.0-1.el6.noarchpuppetlabs-release-6-10.noarch*


 Please  help me

 --
 You received this message because you are subscribed to the Google Groups
 Puppet Users group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to puppet-users+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/puppet-users/1166ffe0-eed5-449d-93c9-8d981997ff90%40googlegroups.com
 https://groups.google.com/d/msgid/puppet-users/1166ffe0-eed5-449d-93c9-8d981997ff90%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAJDiH3E%2BV5ACB%3DrQpinnOEg2ooA4WzG94vJLkyhFnP7XaC5o8g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.