Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Rob Browning commented on PDB-4937 Re: Add a default statement timeout to user queries In the end we implemented timeouts both on the pdb and pg sides, though the pg side timeouts aren't as precise because pg's statement_timeouts appear to apply to each underlying fetch batch for a top-level select, rather than the select as a whole. After some discussion, including CS, we also settled on a default statement timeout of 10m, which we expect to be long enough to avoid any significant disruption of existing environments (that aren't already likely in related trouble). Ideally, it might be even lower, something we can revisit later. Assuming everything works right, that timeout should also substantially mitigate (or outright avoid) the causes of a number of serious escalations we've seen in the past, since no queries will be allowed to remain open for (sometimes) hours, unintentionally. Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.7579.1682107800027%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Rob Browning updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Rob Browning Epic Link: PDB PE - 4969 35436 Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.7568.1682106840025%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Rob Browning updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Rob Browning Release Notes: Enhancement Release Notes Summary: PuppetDB now supports query timeouts for queries to the `query/` endpoint via an [optional query parameter]. A [default] and a [maximum] can also be specified in the configuration. The current default is ten minutes. Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.7567.1682106780090%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Cas Donoghue updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Cas Donoghue Sprint: Skeletor 03/15/2023, Skeletor 03/29/2023, Skeletor 04/12/2023 , Skeletor 04/26/2024 Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.4913.1681323121754%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Fix Version/s: PDB 8.0.0 Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.4877.1681323060197%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Nirupama Mantha updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Nirupama Mantha Labels: jira_escalated Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.2511.1680537240148%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Nirupama Mantha updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Nirupama Mantha Labels: jira_escalated Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.2510.1680537240103%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Cas Donoghue updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Cas Donoghue Sprint: Skeletor 03/15/2023, Skeletor 03/29/2023 , Skeletor 04/12/2023 Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.1741.168011465%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Cas Donoghue updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Cas Donoghue Sprint: Skeletor 03/15/2023 , Skeletor 03/29/2023 Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.963.1678904580038%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Rob Browning assigned an issue to Rob Browning PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Rob Browning Sprint: Skeletor 03/15/2023 Story Points: 1 Assignee: Rob Browning Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.3947.1678307580029%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title
Austin Blatt updated an issue
PuppetDB / PDB-4937
Add a default statement timeout to user queries
Change By:
Austin Blatt
Remaining questions* What is the default statement timeout for a query? 1min, 5min, 10min?JDBC does _not_ respect Thread interrupts so we could implement this by tracking running jdbc queries and calling JDBC's Statement.cancel() method.Alternatively, and likely preferably, we can use statement timeouts to abort a query after a set time. Either by using setQueryTimeout (or setQueryTimeoutMs), and allow pgjdbc to handle things from there (see https://github.com/pgjdbc/pgjdbc/blob/5c78edb1412467636f996416ad37118f651db86e/pgjdbc/src/test/java/org/postgresql/test/jdbc2/StatementTest.java#L737-L762 for an example).Or, we can set explicit statement timeouts in the SQL if the above does not work{code} puppetdb-> SET statement_timeout TO 1 I puppetdb-> ;SET I puppetdb=> select pg_sleep(2);ERROR: canceling statement due to statement timeout{code}
Add Comment
This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title
Austin Blatt updated an issue
PuppetDB / PDB-4937
Add a default statement timeout to user queries
Change By:
Austin Blatt
Remaining questions* What is the default statement timeout for a query? 1min, 5min, 10min? JDBC does _not_ respect Thread interrupts so we could implement this by tracking running jdbc queries and calling JDBC's Statement.cancel() method.Alternatively, and likely preferably, we can use statement timeouts to abort a query after a set time. Either by using setQueryTimeout (or setQueryTimeoutMs), and allow pgjdbc to handle things from there (see https://github.com/pgjdbc/pgjdbc/blob/5c78edb1412467636f996416ad37118f651db86e/pgjdbc/src/test/java/org/postgresql/test/jdbc2/StatementTest.java#L737-L762 for an example).Or, we can set explicit statement timeouts in the SQL if the above does not work{code}SET statement_timeout TO 1 I puppetdb-> ;SET I puppetdb=> select pg_sleep(2);ERROR: canceling statement due to statement timeout{code}
Add Comment
This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt assigned an issue to Unassigned PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Assignee: Rob Browning Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.55573.1667498880109%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Priority: Low Normal Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.55572.1667498880055%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Team: HA Skeletor Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.55571.1667498820027%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Cas Donoghue commented on PDB-4937 Re: Add a default statement timeout to user queries Then engineering work for this is pretty straight forward. We need to work with product (David Piekny ) and CS to come up with reasonable defaults and nail down which stream we target the work for. Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.50378.1666291260042%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Charlie Sharpsteen commented on PDB-4937 Re: Add a default statement timeout to user queries Re-opening this one. We've got a ticket where a user is reporting queries that persist for 1 day or longer and end up blocking other PuppetDB activities by holding locks. The root cause of this behavior is likely some other issue, but a default timeout for queries is a first line of defense that would help to blunt the impact. Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.45217.1665516900284%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Charlie Sharpsteen updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Charlie Sharpsteen Zendesk Ticket Count: 1 Zendesk Ticket IDs: 49787 Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.45212.1665516780225%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Alvin Rodis updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Alvin Rodis Labels: jira_escalated Add Comment This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.45211.1665516780050%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title David McTavish updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: David McTavish Priority: Normal Low Add Comment This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.13339.1644504240157%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title David McTavish updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: David McTavish Story Points: 5 Add Comment This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.13335.1644504060035%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Zachary Kent updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Zachary Kent Epic Link: PDB-4969 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.82327.1605729060028%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Mihai Buzgau updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Mihai Buzgau Labels: platform_7 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.75855.1605026280095%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Rob Browning commented on PDB-4937 Re: Add a default statement timeout to user queries We're setting this work aside until/unless we come up with a more sophisticated approach. That is, with streaming responses, we can't return a custom status (say 503) to indicate a timeout once we've started streaming, and we might also want to provide a finer distinction than just 503. In order to pursue this further, we might want to investigate chunked transfer encoding "trailing headers", where we could report a timeout. Alternately, we could change our output format in order to allow us to send an indication of the timeout in the result stream itself that the client could then detect and handle. The rest of the work went fairly smoothly, so were we able to come up with a palatable solution to this question, then we should be able to resume the effort. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.73170.1604601900100%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Fix Version/s: PDB 6.14.0 Fix Version/s: PDB 7.0.0 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.71527.1604511900113%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt assigned an issue to Rob Browning PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Assignee: Rob Browning Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.60646.1603312080092%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt assigned an issue to Unassigned PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Assignee: Austin Blatt Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.60534.1603304880039%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Acceptance Criteria: * Query parameter to increase/decrease statement timeout for that query* Config/env parameter to increase/decrease statement timeout for _all_ queries * Disabled by a default value of 0 in PuppetDB 6.Y.0 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.60487.1603302840026%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Fix Version/s: PDB 6.14.0 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.60486.1603302780033%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt updated an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Change By: Austin Blatt Story Points: 5 Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.375530.1603302604000.60485.1603302720081%40Atlassian.JIRA.
Jira (PDB-4937) Add a default statement timeout to user queries
Title: Message Title Austin Blatt created an issue PuppetDB / PDB-4937 Add a default statement timeout to user queries Issue Type: Improvement Assignee: Austin Blatt Created: 2020/10/21 10:50 AM Fix Versions: PDB 7.0.0 Labels: platform_7 Priority: Normal Reporter: Austin Blatt Remaining questions What is the default statement timeout for a query? 1min, 5min, 10min? Add Comment
