Jira (PDB-5282) Empty certname in PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Charlie Sharpsteen (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Charlie Sharpsteen updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  Empty certname in PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Charlie Sharpsteen  
 
 
Summary: 
 Empty certname in PuppetDB command data results in NullPointerException during startup  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.136030.1632267300144%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Charlie Sharpsteen (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Charlie Sharpsteen updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Charlie Sharpsteen  
 
 
Priority: 
 Normal Major  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.136027.163226766%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Charlie Sharpsteen (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Charlie Sharpsteen updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Charlie Sharpsteen  
 

  
 
 
 
 

 
 PuppetDB "commands" are operations to update the data associated with puppet nodes. These commands are identified by certname, and odd things can happen if an empty string or other null value is provided as the certname. One consequence is a {{java.lang.NullPointerException}} during startup.  This   When PE DR is enabled, this  exception prevents the PuppetDB service from exiting maintenance mode and responding to read and write requests.h2. Reproduction Case - Install PE 2019.8.8 on a CentOS 7 node  and add a DR replica   - Deactivate an empty certname: {{puppet node deactivate ''}}  - Re-start the {{pe-puppetdb}} serviceh3. OutcomeThe deactivate command results in the following error message being recorded to the PuppetDB log:{noformat}2021-09- 21T22 21T23 : 20 21 : 47 07 . 271Z 566Z  ERROR [p.p.threadpool] Reporting unexpected error from thread cmd-proc-thread- 13 2  to stderr and logjava.lang.NullPointerException: null at metrics.meters$mark_BANG_.invokeStatic(meters.clj:76) at metrics.meters$mark_BANG_.invoke(meters.clj:72) at metrics.meters$mark_BANG_.invokeStatic(meters.clj:74) at metrics.meters$mark_BANG_.invoke(meters.clj:72) at puppetlabs.puppetdb.command$mark_both_metrics_BANG_.invokeStatic(command.clj:244) at puppetlabs.puppetdb.command$mark_both_metrics_BANG_.invoke(command.clj:240) at puppetlabs.puppetdb.command$process_message$retry__37262.invoke(command.clj:755) at puppetlabs.puppetdb.command$process_message.invokeStatic(command.clj:810) at puppetlabs.puppetdb.command$process_message.invoke(command.clj:742) at puppetlabs.puppetdb.command$message_handler$fn__37274.invoke(command.clj:820) at puppetlabs.puppetdb.threadpool$dochan$fn__36851$fn__36852.invoke(threadpool.clj:116) at puppetlabs.puppetdb.threadpool$gated_execute$fn__36813.invoke(threadpool.clj:69) at clojure.lang.AFn.run(AFn.java:22) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829){noformat}This also leaves behind a command int the PuppetDB processing queue with an empty certname:{noformat}# find /opt/puppetlabs/server/data/puppetdb/stockpile/cmd/q -type f -print -exec cat {} \;/opt/puppetlabs/server/data/puppetdb/stockpile/cmd/q/ 50 20 - 1632262847254 1632266467556 - 115_rm 103_rm -node_3_.json{"certname":"","producer_timestamp":"2021-09- 21T22 21T23 : 20 21 : 47 07 . 139 453 +00:00"}{noformat}Upon restart, the PuppetDB services fails to initialize the command processing pool and never exits maintenance mode:{noformat}2021-09- 21T22 21T23 : 33 22 : 58 20 . 072Z 499Z  ERROR [p.p.threadpool] Reporting unexpected error from thread cmd-proc-thread-1 to stderr and logjava.lang.NullPointerException: nullat metrics.meters$mark_BANG_.invokeStatic(meters.clj:76)

Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Blatt (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Blatt commented on  PDB-5282  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
 When this command is submitted to a PuppetDB set up to sync, startup will hang before the sync phase. I wonder if it is attempting to process the command in memory but failing to handle the error thrown or DLO/retry logic correctly because it is not using a command processing thread?  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.135974.1632265320209%40Atlassian.JIRA.


Jira (PUP-11163) Prevent thundering herd via deterministic run scheduling

2021-09-21 Thread Reid Vandewiele (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Reid Vandewiele updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Puppet /  PUP-11163  
 
 
  Prevent thundering herd via deterministic run scheduling   
 

  
 
 
 
 

 
Change By: 
 Reid Vandewiele  
 

  
 
 
 
 

 
 h2. ProblemOne of the most-viewed articles in Open Source Puppet Assist between September of 2020 and March of 2021 was [determine a thundering herd condition in Puppet|https://ospassist.puppet.com/hc/en-us/articles/360040327753-determine-a-thundering-herd-condition-in-puppet-] (OSPA-21).Thundering herds have been an out-of-box problem in Puppet for more than a decade, and for users "in the know", there is a clear, bulletproof solution: use splayed, deterministic run start times to evenly distribute agent checkins.Problematically, this doesn't work with our daemonized agent. Customers must instead disable the agent, and use OS schedulers such as Cron (Posix) and Scheduled Tasks (Windows). See [this module|https://forge.puppet.com/modules/reidmv/puppet_run_scheduler] for full automation of the configuration.h2. Desired SolutionThe Puppet agent should provide a deterministic run scheduling mode which evenly distributes agent runs automatically, and eliminates the possibility of thundering herds developing. This mode should become the new default mode.Characteristics: * Scheduled run start times should be based on computed wall clock times. Not on variable-time events such as previous run end times. * An agent's individual run start time should include a random offset element, seeded on the agent's certname setting _+ a random seed.†_ * The agent should schedule and perform a first-start run quickly (immediately, or splayed if splay is configured) on daemon startup, after which subsequent run starts should adhere to the computed wall-clock scheduleOn startup, an agent might schedule runs thusly. # Schedule a startup run for {{now() + $splay}} # Schedule the first regular daemon run for {{now() + ((($run_interval - (now() % $run_interval)) + fqdn_rand($certname)) % $run_interval)}} # Every subsequent run should be scheduled exactly {{$run_interval}} after of the previously scheduled run * Optionally, a provision can be made to skip a scheduled run if the previous scheduled run completed too soon within a configured proximity. This may happen after the special startup run, for example, or if a previously scheduled run entered into a backoff/retry loop based on server 503 responses and so did not start/complete for much longer than expected  We advise our largest customers to achieve this scheduling algorithm today in order to better distribute load, and to prevent thundering herds. We should eliminate the onerous workaround by incorporating it into our product, and in so doing deliver the same benefits to all Puppet users, as well as eliminate one of our most common troubleshooting exasperations.† It is not important for a given agent to always select the same wall-clock start times; only for agents to have different, evenly spread start times, and for subsequent runs to be +consistently+ spaced. Including a random seed element permits the

Jira (PUP-11163) Prevent thundering herd via deterministic run scheduling

2021-09-21 Thread Reid Vandewiele (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Reid Vandewiele updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Puppet /  PUP-11163  
 
 
  Prevent thundering herd via deterministic run scheduling   
 

  
 
 
 
 

 
Change By: 
 Reid Vandewiele  
 

  
 
 
 
 

 
 h2. ProblemOne of the most-viewed articles in Open Source Puppet Assist between September of 2020 and March of 2021 was [determine a thundering herd condition in Puppet|https://ospassist.puppet.com/hc/en-us/articles/360040327753-determine-a-thundering-herd-condition-in-puppet-] (OSPA-21).Thundering herds have been an out-of-box problem in Puppet for more than a decade, and for users "in the know", there is a clear, bulletproof solution: use splayed, deterministic run start times to evenly distribute agent checkins.Problematically, this doesn't work with our daemonized agent. Customers must instead disable the agent, and use OS schedulers such as Cron (Posix) and Scheduled Tasks (Windows). See [this module|https://forge.puppet.com/modules/reidmv/puppet_run_scheduler] for full automation of the configuration.h2. Desired SolutionThe Puppet agent should provide a deterministic run scheduling mode which evenly distributes agent runs automatically, and eliminates the possibility of thundering herds developing. This mode should become the new default mode.Characteristics: * Scheduled run start times should be based on computed wall clock times. Not on variable-time events such as previous run end times. * An agent's individual run start time should include a random offset element, seeded on the agent's certname setting _+ a random seed.†_ * The agent should schedule and perform a first-start run quickly (immediately, or splayed if splay is configured) on daemon startup, after which subsequent run starts should adhere to the computed wall-clock scheduleOn startup, an agent might schedule runs thusly. # Schedule a startup run for {{now() + $splay}} # Schedule the first regular daemon run for {{now() + ((($run_interval - (now() % $run_interval)) + fqdn_rand($certname)) % $run_interval)}} # Every subsequent run should be scheduled exactly {{$run_interval}} after of the previously scheduled run * Optionally, a provision can be made to skip a scheduled run if the previous scheduled run completed too soon within a configured proximity. This may happen after the special startup run, for example, or if a previously scheduled run entered into a backoff/retry loop based on server 503 responses and so did not start/complete for much longer than expected  We advise our largest customers to achieve this scheduling algorithm today in order to better distribute load, and to prevent thundering herds. We should eliminate the onerous workaround by incorporating it into our product, and in so doing deliver the same benefits to all Puppet users, as well as eliminate one of our most common troubleshooting exasperations.† It is not important for a given agent to always select the same wall-clock start times; only for agents to have different, evenly spread start times, and for subsequent runs to be +consistently+ spaced.  Including a random seed element permits th

Jira (PUP-11163) Prevent thundering herd via deterministic run scheduling

2021-09-21 Thread Reid Vandewiele (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Reid Vandewiele updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Puppet /  PUP-11163  
 
 
  Prevent thundering herd via deterministic run scheduling   
 

  
 
 
 
 

 
Change By: 
 Reid Vandewiele  
 

  
 
 
 
 

 
 h2. ProblemOne of the most-viewed articles in Open Source Puppet Assist between September of 2020 and March of 2021 was [determine a thundering herd condition in Puppet|https://ospassist.puppet.com/hc/en-us/articles/360040327753-determine-a-thundering-herd-condition-in-puppet-] (OSPA-21).Thundering herds have been an out-of-box problem in Puppet for more than a decade, and for users "in the know", there is a clear, bulletproof solution: use splayed, deterministic run start times to evenly distribute agent checkins.Problematically, this doesn't work with our daemonized agent. Customers must instead disable the agent, and use OS schedulers such as Cron (Posix) and Scheduled Tasks (Windows). See [this module|https://forge.puppet.com/modules/reidmv/puppet_run_scheduler] for full automation of the configuration.h2. Desired SolutionThe Puppet agent should provide a deterministic run scheduling mode which evenly distributes agent runs automatically, and eliminates the possibility of thundering herds developing. This mode should become the new default mode.Characteristics: * Scheduled run start times should be based on computed wall clock times. Not on variable-time events such as previous run end times. * An agent's individual run start time should include a random offset element, seeded on the agent's certname setting  _+ a random seed.†_  * The agent should schedule and perform a first-start run quickly (immediately, or splayed if splay is configured) on daemon startup, after which subsequent run starts should adhere to the computed wall-clock scheduleOn startup, an agent might schedule runs thusly. # Schedule a startup run for {{now() + $splay}} # Schedule the first regular daemon run for {{now() + ((($run_interval - (now() % $run_interval)) + fqdn_rand($certname)) % $run_interval)}} # Every subsequent run should be scheduled exactly {{$run_interval}} after of the previously scheduled run * Optionally, a provision can be made to skip a scheduled run if the previous scheduled run completed too soon within a configured proximity. This may happen after the special startup run, for example, or if a previously scheduled run entered into a backoff/retry loop based on server 503 responses and so did not start/complete for much longer than expected  We advise our largest customers to achieve this scheduling algorithm today in order to better distribute load, and to prevent thundering herds. We should eliminate the onerous workaround by incorporating it into our product, and in so doing deliver the same benefits to all Puppet users, as well as eliminate one of our most common troubleshooting exasperations. † It is not important for a given agent to always select the same wall-clock start times; only for agents to have different, evenly spread start times, and for subsequent runs to be +consistently+ spaced.  
  

Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Charlie Sharpsteen (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Charlie Sharpsteen updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Charlie Sharpsteen  
 

  
 
 
 
 

 
 PuppetDB "commands" are operations to update the data associated with puppet nodes. These commands are identified by certname, and odd things can happen if an empty string or other null value is provided as the certname. One consequence is a {{java.lang.NullPointerException}} during startup. This exception prevents the PuppetDB service from exiting maintenance mode and responding to read and write requests.h2. Reproduction Case - Install PE 2019.8.8 on a CentOS 7 node  - Deactivate an empty certname: {{puppet node deactivate ''}}  - Re-start the {{pe-puppetdb}} serviceh3 .  OutcomeThe deactivate command results in the following error message being recorded to the PuppetDB log:{noformat}2021-09-21T22:20:47.271Z ERROR [p.p.threadpool] Reporting unexpected error from thread cmd-proc-thread-13 to stderr and logjava.lang.NullPointerException: null at metrics.meters$mark_BANG_.invokeStatic(meters.clj:76) at metrics.meters$mark_BANG_.invoke(meters.clj:72) at metrics.meters$mark_BANG_.invokeStatic(meters.clj:74) at metrics.meters$mark_BANG_.invoke(meters.clj:72) at puppetlabs.puppetdb.command$mark_both_metrics_BANG_.invokeStatic(command.clj:244) at puppetlabs.puppetdb.command$mark_both_metrics_BANG_.invoke(command.clj:240) at puppetlabs.puppetdb.command$process_message$retry__37262.invoke(command.clj:755) at puppetlabs.puppetdb.command$process_message.invokeStatic(command.clj:810) at puppetlabs.puppetdb.command$process_message.invoke(command.clj:742) at puppetlabs.puppetdb.command$message_handler$fn__37274.invoke(command.clj:820) at puppetlabs.puppetdb.threadpool$dochan$fn__36851$fn__36852.invoke(threadpool.clj:116) at puppetlabs.puppetdb.threadpool$gated_execute$fn__36813.invoke(threadpool.clj:69) at clojure.lang.AFn.run(AFn.java:22) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829){noformat}This also leaves behind a command int the PuppetDB processing queue with an empty certname:{noformat}# find /opt/puppetlabs/server/data/puppetdb/stockpile/cmd/q -type f -print -exec cat {} \;/opt/puppetlabs/server/data/puppetdb/stockpile/cmd/q/50-1632262847254-115_rm-node_3_.json{"certname":"","producer_timestamp":"2021-09-21T22:20:47.139+00:00"}{noformat}Upon restart, the PuppetDB services fails to initialize the command processing pool and never exits maintenance mode:{noformat}2021-09-21T22:33:58.072Z ERROR [p.p.threadpool] Reporting unexpected error from thread cmd-proc-thread-1 to stderr and logjava.lang.NullPointerException: nullat metrics.meters$mark_BANG_.invokeStatic(meters.clj:76)at metrics.meters$mark_BANG_.invoke(meters.clj:72)at metrics.meters$mark_BANG_.invokeStatic(meters.clj:74)at metrics.meters$mark_BANG_.in

Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Charlie Sharpsteen (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Charlie Sharpsteen updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Charlie Sharpsteen  
 

  
 
 
 
 

 
 Under certain unknown circumstances, data stored in the  PuppetDB  command queue  "commands" are operations to update the data associated with puppet nodes. These commands are identified by certname, and odd things  can  cause  happen if an empty string or other null value is provided as the certname. One consequence is  a {{java.lang.NullPointerException}} during startup. This exception prevents the PuppetDB service from exiting maintenance mode and responding to read and write requests. h2. Reproduction Case - Install PE 2019.8.8 on a CentOS 7 node  - Deactivate an empty certname: {{puppet node deactivate ''}}  - Re-start the {{pe-puppetdb}} serviceh3 Outcome The  most relevant errors printed  deactivate command results in the following error message being recorded  to the  logs are  PuppetDB log :{noformat}2021-09- 21T17 21T22 : 43 20 :47. 616Z INFO  [c.z.h.HikariDataSource] PDBWritePool - Start completed.2021-09-21T17:43:47.623Z 271Z  ERROR [p.p. c.services threadpool ] Reporting unexpected error  from thread cmd-proc-thread-13  to stderr and logjava.lang.NullPointerException: nullat  clojure  metrics . core meters $ name mark_BANG_ .invokeStatic( core meters .clj: 1595 76 )at  clojure  metrics . core meters $ name mark_BANG_ .invoke( core meters .clj: 1589 72 )at  clojure.core$mapv$fn__8445.invoke(core.clj:6912)at clojure.lang.PersistentVector.reduce(PersistentVector.java:343)at clojure.core$reduce.invokeStatic(core.clj:6827)at clojure.core$mapv.invokeStatic(core.clj:6903)at clojure.core$mapv.invoke(core.clj:6903)at puppetlabs.puppetdb. metrics. core meters $ keyword__GT_metric_name mark_BANG_ .invokeStatic( core meters .clj: 26 74 )at  puppetlabs.puppetdb. metrics. core meters $ keyword__GT_metric_name mark_BANG_ .invoke( core meters .clj: 20 72 )at puppetlabs.puppetdb.command$ create_metrics$to_metric_name_fn__36929 mark_both_metrics_BANG_ . invoke(command.clj:169)at puppetlabs.puppetdb.command$create_metrics. invokeStatic(command.clj: 172 244 )at puppetlabs.puppetdb.command$ create_metrics mark_both_metrics_BANG_ .invoke(command.clj: 168 240 )at puppetlabs.puppetdb.command$ create_metrics_for_command_BANG_ process_message $ fn__36935 retry__37262 .invoke(command.clj: 230 755 )at  clojure.lang.Atom.swap(Atom.java:37)at clojure.core$swap_BANG_.invokeStatic(core.clj:2352)at clojure.core$swap_BANG_.invoke(core.clj:2345)at  puppetlabs.puppetdb.command$ create_metrics_for_command_BANG_ process_message .invokeStatic(command.clj: 222 810 )at puppetlabs.puppetdb.command$ create_metrics_for_command_BANG_ process_message .invoke(command.clj: 215 742 )at puppetlabs.puppetdb.command$ inc_cmd_depth.invokeStatic(command.clj:237)at puppetlabs.puppetdb.command message_handler $ inc_cmd_depth fn__37274 .invoke(command.clj: 233 820 )at puppetlabs.p

Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Blatt (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Blatt updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Austin Blatt  
 
 
Acceptance Criteria: 
 * When using the "new" command syntax, the HTTP endpoint should reject a command that has the empty string for a certname ie. &certname=& in the URI* When using the "old" command syntax,  if we read load  the command  into memory to read the certname in order to store it in stockpile we  should  be rejected  reject it then. Otherwise the command processor should reject it  and  DLOed  put it in the DLO  without any retries .  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.135910.1632263580181%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Blatt (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Blatt updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Austin Blatt  
 
 
Acceptance Criteria: 
 * When using the "new" command syntax, the HTTP endpoint should reject a command that has the empty string for a certname ie. &certname=& in the URI* When using the "old" command syntax, if we  read  load the command into memory to read the certname in order to store it in stockpile we should reject it then. Otherwise the command processor should reject it and put it in the DLO without any retries.  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.135911.1632263580225%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Blatt (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Blatt updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Austin Blatt  
 
 
Acceptance Criteria: 
 * When using the "new" command syntax, the HTTP endpoint should reject a command that has the empty string for a certname ie. &certname=& in the URI* When using the "old" command syntax, the command should be rejected and DLOed without any retries  
 
 
Summary: 
 Using an empty string as a PuppetDB  command  certname  data  results in  a  NullPointerException during startup  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.135893.1632263340530%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Blatt (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Blatt commented on  PDB-5282  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
 One more reproduction case using the old command API  
 
 
 
 
 curl -X POST http://localhost:8080/pdb/cmd/v1 \  
 
 
   -H 'Content-Type:application/json' \  
 
 
   -d '{"command": "deactivate node", "version": 3, "payload": { "certname": "", "producer_timestamp": "2020-01-15T15:43:28-08:00" } }'
  
 
 
 
   
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, s

Jira (PDB-5282) Using an empty string as a command certname results in a NullPointerException during startup

2021-09-21 Thread Charlie Sharpsteen (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Charlie Sharpsteen updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  Using an empty string as a command certname results in a NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Charlie Sharpsteen  
 
 
Summary: 
 PuppetDB Using an empty string as a  command  data  certname  results in  a  NullPointerException during startup  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.135883.1632263220052%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Blatt (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Blatt commented on  PDB-5282  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
 Also sounds like it was reproducible in PE with  
 
 
 
 
 puppet node deactivate ''
  
 
 
 
   
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.135881.1632263100035%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Blatt (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Blatt commented on  PDB-5282  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
 Reproduction case using curl  
 
 
 
 
 curl -X POST 'http://localhost:8080/pdb/cmd/v1?command=deactivate%20node&version=3&certname=&producer-timestamp=2019-07-16T09:17:25-0700' \  
 
 
   -H 'Content-Type:application/json' \  
 
 
   -d '{ "certname": null, "producer_timestamp": "2020-01-15T15:43:28-08:00" }'
  
 
 
 
  errors with  
 
 
 
 
 2021-09-21 15:19:48,093 ERROR [cmd-proc-thread-6] [p.p.threadpool] Reporting unexpected error from thread cmd-proc-thread-6 to stderr and log  
 
 
 java.lang.NullPointerException: null  
 
 
 at metrics.meters$mark_BANG_.invokeStatic(meters.clj:76)  
 
 
 at metrics.meters$mark_BANG_.invoke(meters.clj:72)  
 
 
 at metrics.meters$mark_BANG_.invokeStatic(meters.clj:74)  
 
 
 at metrics.meters$mark_BANG_.invoke(meters.clj:72)  
 
 
 at puppetlabs.puppetdb.command$mark_both_metrics_BANG_.invokeStatic(command.clj:244)  
 
  

Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Boyd (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Boyd updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Austin Boyd  
 
 
Labels: 
 jira_escalated  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.135792.1632261300111%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Austin Boyd (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Austin Boyd updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Change By: 
 Austin Boyd  
 
 
Zendesk Ticket Count: 
 1  
 
 
Zendesk Ticket IDs: 
 45879  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.416719.1632261204000.135793.1632261300157%40Atlassian.JIRA.


Jira (PDB-5282) PuppetDB command data results in NullPointerException during startup

2021-09-21 Thread Charlie Sharpsteen (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Charlie Sharpsteen created an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 PuppetDB /  PDB-5282  
 
 
  PuppetDB command data results in NullPointerException during startup   
 

  
 
 
 
 

 
Issue Type: 
  Bug  
 
 
Affects Versions: 
 PDB 6.17.0  
 
 
Assignee: 
 Unassigned  
 
 
Components: 
 PuppetDB  
 
 
Created: 
 2021/09/21 2:53 PM  
 
 
Priority: 
  Normal  
 
 
Reporter: 
 Charlie Sharpsteen  
 

  
 
 
 
 

 
 Under certain unknown circumstances, data stored in the PuppetDB command queue can cause a java.lang.NullPointerException during startup. This exception prevents the PuppetDB service from exiting maintenance mode and responding to read and write requests. The most relevant errors printed to the logs are:  
 
 
 
 
 2021-09-21T17:43:47.616Z INFO  [c.z.h.HikariDataSource] PDBWritePool - Start completed.  
 
 
 2021-09-21T17:43:47.623Z ERROR [p.p.c.services] Reporting unexpected error to stderr and log  
 
 
 java.lang.NullPointerException: null  
 
 
   

Jira (PDB-5258) Investigate where does the `null?` operator work

2021-09-21 Thread Oana Tanasoiu (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Oana Tanasoiu commented on  PDB-5258  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Investigate where does the `null?` operator work   
 

  
 
 
 
 

 
 I tested with 5.1.6, 5.1.0, 4.1.4 (this version doesn't have the inventory endpoint implemented) and the query doesn't work.  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.414804.1631089159000.135365.1632243600222%40Atlassian.JIRA.


Jira (FACT-3071) macosx_productversion_major returns wrong value for Big Sur

2021-09-21 Thread Ciprian Badescu (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Ciprian Badescu updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Facter /  FACT-3071  
 
 
  macosx_productversion_major returns wrong value for Big Sur   
 

  
 
 
 
 

 
Change By: 
 Ciprian Badescu  
 
 
Team: 
 Night's Watch  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.414066.1630534919000.134952.1632233220032%40Atlassian.JIRA.


Jira (PUP-11058) Detailed exitcode for 503s

2021-09-21 Thread Ciprian Badescu (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Ciprian Badescu updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Puppet /  PUP-11058  
 
 
  Detailed exitcode for 503s   
 

  
 
 
 
 

 
Change By: 
 Ciprian Badescu  
 
 
Issue Type: 
 Bug Improvement  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.398736.1620652163000.134759.1632219900088%40Atlassian.JIRA.


Jira (PUP-11167) Provider redhat must have features 'maskable' to set 'enable' to 'mask' during rspec-puppet testing

2021-09-21 Thread Ciprian Badescu (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Ciprian Badescu commented on  PUP-11167  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Provider redhat must have features 'maskable' to set 'enable' to 'mask' during rspec-puppet testing   
 

  
 
 
 
 

 
 /cc: Ben Ford  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.406490.1625845962000.134749.1632219720035%40Atlassian.JIRA.


Jira (PUP-11189) Add functions to iterate over files/templates in the current module

2021-09-21 Thread Ciprian Badescu (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Ciprian Badescu commented on  PUP-11189  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Add functions to iterate over files/templates in the current module   
 

  
 
 
 
 

 
 We agree it is likely an improvement for the core functionality, but due to other issues demanding precedence, we don’t anticipate being able to address this any time soon. As such we are closing this as “Won’t Fix.” We may revisit it at a later time, and if so will re-open this ticket  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.408658.1627373801000.134716.1632218400095%40Atlassian.JIRA.


Jira (PUP-11213) Puppet/Puppetserver can be taken out by bad regex

2021-09-21 Thread Ciprian Badescu (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Ciprian Badescu commented on  PUP-11213  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Puppet/Puppetserver can be taken out by bad regex   
 

  
 
 
 
 

 
 Thank you for filing this issue. While we agree this is an improvement, addressing this issue would require a substantial architecture change that we do not anticipate being able to undertake due to other issues taking precedence. As such, this ticket will be closed as “Won’t Fix”. We may revisit this at a later time, and if so, will re-open this ticket.  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.412818.1629461682000.134712.1632218220144%40Atlassian.JIRA.


Jira (PUP-11229) Update puppet `instance?` methods to use `instance_of?` instead of `is_a?`

2021-09-21 Thread Ciprian Badescu (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Ciprian Badescu updated an issue  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
 Puppet /  PUP-11229  
 
 
  Update puppet `instance?` methods to use `instance_of?` instead of `is_a?`   
 

  
 
 
 
 

 
Change By: 
 Ciprian Badescu  
 
 
Epic Link: 
 PUP-5773  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.413157.1629790983000.134673.1632215820190%40Atlassian.JIRA.


Jira (PDB-5257) Example code for PuppetDB query is not correct

2021-09-21 Thread Henry Wang (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Henry Wang commented on  PDB-5257  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Example code for PuppetDB query is not correct   
 

  
 
 
 
 

 
 Hi, I see the doc is still wrong without update: https://puppet.com/docs/puppetdb/7/api/query/examples-pql.html#fact-report-status-filtering-with-dot-notation Please update the doc. Thank you.  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.414481.1630996799000.134573.1632208080026%40Atlassian.JIRA.


Jira (PUP-10491) Allow adding descriptive or administrative information to resources

2021-09-21 Thread Ciprian Badescu (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Ciprian Badescu commented on  PUP-10491  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Allow adding descriptive or administrative information to resources   
 

  
 
 
 
 

 
 Moved this to Open so we can discuss it on triage  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)  
 
 

 
   
 

  
 

  
 

   





-- 
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.357687.1588699805000.134564.1632207780032%40Atlassian.JIRA.