[jira] [Commented] (SOLR-16982) Trip a Circuit Breaker only for external requests
[ https://issues.apache.org/jira/browse/SOLR-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18014827#comment-18014827 ] Jan Høydahl commented on SOLR-16982: This Jira is shipped and closed, so not a perfect place for disussing your suggestion. Better open a dev-list tread on the subject, or a new JIRA. > Trip a Circuit Breaker only for external requests > - > > Key: SOLR-16982 > URL: https://issues.apache.org/jira/browse/SOLR-16982 > Project: Solr > Issue Type: Improvement > Components: Circuit Breakers >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Fix For: 9.4 > > Attachments: CircuitBreakerSearchComponent.java > > Time Spent: 1h > Remaining Estimate: 0h > > In a sharded system, Circuit Breakers may now break a shard-request, failing > the entire query request and in worst-case cause data loss (?) in an update > request. > We should see whether we can skip CB checking for the internal requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (SOLR-16982) Trip a Circuit Breaker only for external requests
[ https://issues.apache.org/jira/browse/SOLR-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18012675#comment-18012675 ] Russell Black commented on SOLR-16982: -- I've attached a file showing an alternate approach to this problem, which implements the circuit breaker as a search component, and truly protects servers against high CPU, even in sharded environments. The file is called CircuitBreakerSearchComponent.java. Attaching it here in case it helps anyone else. /** * Circuit Breaker Search Component ** This component monitors CPU usage and can trip a circuit breaker * to prevent further processing if the CPU usage exceeds a specified threshold. * It is designed to be used in Solr to protect against high CPU usage * that could lead to performance degradation or system instability. * It can be configured in the solrconfig.xml file. * * It is intended to be configured as the first search component * * Example configuration: * * * * 95 *. * * * ... * * circuitBreaker * * * * * This component checks the CPU usage before processing a request. * If the CPU usage exceeds the configured threshold, it will throw an exception * to prevent further processing. * * The exception thrown is ExitableDirectoryReader.ExitingReaderException, * which is the same exception used by Solr when the timeAllowed parameter is exceeded. */ > Trip a Circuit Breaker only for external requests > - > > Key: SOLR-16982 > URL: https://issues.apache.org/jira/browse/SOLR-16982 > Project: Solr > Issue Type: Improvement > Components: Circuit Breakers >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Fix For: 9.4 > > Attachments: CircuitBreakerSearchComponent.java > > Time Spent: 1h > Remaining Estimate: 0h > > In a sharded system, Circuit Breakers may now break a shard-request, failing > the entire query request and in worst-case cause data loss (?) in an update > request. > We should see whether we can skip CB checking for the internal requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (SOLR-16982) Trip a Circuit Breaker only for external requests
[ https://issues.apache.org/jira/browse/SOLR-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18012270#comment-18012270 ] Russell Black commented on SOLR-16982: -- This ticket makes circuit breakers useless for me. For a sharded system, the searches are performed by the internal requests to the individual shards, which places additional load on them, even if their CPU is pegged and would otherwise break the circuit. The problem of a circuit-broken shard in an internal request could be handled by simply recognizing the returned 429 for what it is, and handling it gracefully (setting response to partial results). > Trip a Circuit Breaker only for external requests > - > > Key: SOLR-16982 > URL: https://issues.apache.org/jira/browse/SOLR-16982 > Project: Solr > Issue Type: Improvement > Components: Circuit Breakers >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Fix For: 9.4 > > Time Spent: 1h > Remaining Estimate: 0h > > In a sharded system, Circuit Breakers may now break a shard-request, failing > the entire query request and in worst-case cause data loss (?) in an update > request. > We should see whether we can skip CB checking for the internal requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (SOLR-16982) Trip a Circuit Breaker only for external requests
[ https://issues.apache.org/jira/browse/SOLR-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828354#comment-17828354 ] Jan Høydahl commented on SOLR-16982: We forgot to update Ref.Guide for this change. I'll prepare a PR as a hotfix for ref-guide and tag it with this JIRA. > Trip a Circuit Breaker only for external requests > - > > Key: SOLR-16982 > URL: https://issues.apache.org/jira/browse/SOLR-16982 > Project: Solr > Issue Type: Improvement > Components: Circuit Breakers >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Fix For: 9.4 > > Time Spent: 40m > Remaining Estimate: 0h > > In a sharded system, Circuit Breakers may now break a shard-request, failing > the entire query request and in worst-case cause data loss (?) in an update > request. > We should see whether we can skip CB checking for the internal requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (SOLR-16982) Trip a Circuit Breaker only for external requests
[ https://issues.apache.org/jira/browse/SOLR-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767884#comment-17767884 ] ASF subversion and git services commented on SOLR-16982: Commit cc778c74676a340debbaab2ce3b599089a0e46af in solr's branch refs/heads/branch_9x from Jan Høydahl [ https://gitbox.apache.org/repos/asf?p=solr.git;h=cc778c74676 ] SOLR-16982: Trip a Circuit Breaker only for external requests (#1930) Co-authored-by: Christine Poerschke (cherry picked from commit d9c65fb1a64f6e85d5b869519ff2dbc1f529cb65) > Trip a Circuit Breaker only for external requests > - > > Key: SOLR-16982 > URL: https://issues.apache.org/jira/browse/SOLR-16982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Circuit Breakers >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > In a sharded system, Circuit Breakers may now break a shard-request, failing > the entire query request and in worst-case cause data loss (?) in an update > request. > We should see whether we can skip CB checking for the internal requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
[jira] [Commented] (SOLR-16982) Trip a Circuit Breaker only for external requests
[ https://issues.apache.org/jira/browse/SOLR-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17767877#comment-17767877 ] ASF subversion and git services commented on SOLR-16982: Commit d9c65fb1a64f6e85d5b869519ff2dbc1f529cb65 in solr's branch refs/heads/main from Jan Høydahl [ https://gitbox.apache.org/repos/asf?p=solr.git;h=d9c65fb1a64 ] SOLR-16982: Trip a Circuit Breaker only for external requests (#1930) Co-authored-by: Christine Poerschke > Trip a Circuit Breaker only for external requests > - > > Key: SOLR-16982 > URL: https://issues.apache.org/jira/browse/SOLR-16982 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Circuit Breakers >Reporter: Jan Høydahl >Assignee: Jan Høydahl >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > In a sharded system, Circuit Breakers may now break a shard-request, failing > the entire query request and in worst-case cause data loss (?) in an update > request. > We should see whether we can skip CB checking for the internal requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
