Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title John Duarte updated an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: John Duarte QA Status: Reviewed Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Wyatt Alt updated an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Wyatt Alt Sprint: PuppetDB2015-02-25,PuppetDB2015-03-11,PuppetDB2015-03-25 ,PuppetDB2015-04-08 Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Wyatt Alt commented on PDB-1227 Re: Test new table structure and fact path/value GC at scale This patch looks great based on my performance testing. On stable we can reproduce the fact_paths problem with 30,000 simulated nodes checking in every 30 minutes with a random percentage of 10. The problem does not occur on stable at 20,000 nodes, so the breaking point is somewhere between 20k and 30k. The CPUs on the postgres server are pegged on deletes more or less as soon as commands switch from being processed with add-facts! to update-facts!, i.e once the database stores 30k certnames. The gc issues at this point will cause the queue size to explode and all kinds of problems to occur. This patch on the other hand can handle 30k and 50k nodes without breaking a sweat – in fact, using the -i and -n settings we have not been able to submit factsets fast enough to back up the queue. When we set -n to 10, only 17 commands per second were being submitted, which means the benchmark tool was underperforming (or our metrics are wrong). Queue size remains at or around 0 during both additions and updates for any of those numbers. The -n and -N flags of the benchmark tool allow you to submit commands unthrottled. Doing this will back up the queue both here and on stable, but is not a realistic scenario. Migration of 30,000 nodes of data from stable to this patch took two and a half minutes, which is fine. All in all this seems to completely solve our problem. Add Comment This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Wyatt Alt assigned an issue to Wyatt Alt PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Wyatt Alt Assignee: KennethBarber WyattAlt Add Comment This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Steve Barlow updated an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Steve Barlow Story Points: 1 2 Add Comment This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Steve Barlow updated an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Steve Barlow Sprint: PuppetDB2015-02-25,PuppetDB2015-03-11,PuppetDB2015-03-25 ,PuppetDB2015-04-08 Add Comment This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Rob Browning updated an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Rob Browning Fix Version/s: PDB2.3.0 Fix Version/s: PDB2.3.x Add Comment This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Steve Barlow updated an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Steve Barlow Sprint: PuppetDB2015-02-25,PuppetDB2015-03-11 ,PuppetDB2015-03-25 Add Comment This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Steve Barlow updated an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Steve Barlow Sprint: PuppetDB2015-02-25 ,PuppetDB2015-03-11 Add Comment This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Kenneth Barber updated an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Kenneth Barber Fix Version/s: PDB2.3.0 Add Comment This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Kenneth Barber assigned an issue to Kenneth Barber PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Change By: Kenneth Barber Assignee: KennethBarber Add Comment This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups Puppet Bugs group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (PDB-1227) Test new table structure and fact path/value GC at scale
Title: Message Title Ryan Senior created an issue PuppetDB / PDB-1227 Test new table structure and fact path/value GC at scale Issue Type: Bug Assignee: Unassigned Created: 2015/02/10 12:58 PM Priority: Normal Reporter: Ryan Senior Once we have PDB-1224 and PDB-1225 in place and know what the query will look like for PDB-1226 (even when the background process isn't complete), we should test the new structure and queries at a heavy load level. We can run the disassociated value as a separate cronjob like process for now, just to simulate the effect of the query on the running system. We should be able to use Kenneth Barber's setup for this Add Comment This message was sent by
