There was no feedback on the topic of global Circuit Breaker vs per-core. So I opened https://issues.apache.org/jira/browse/SOLR-16974 to propose supporting Global CB. This is still on design level, please weigh in with your thoughts.
Jan > 30. aug. 2023 kl. 22:02 skrev Jan Høydahl <jan....@cominvent.com>: > > Hi > > For those interested in this topic, here is the JIRA, which has a Pull > Request for review: https://issues.apache.org/jira/browse/SOLR-16954 > > Jan Høydahl > >> 3. jul. 2023 kl. 22:28 skrev Pierre Salagnac <pierre.salag...@gmail.com>: >> >> Hi Jan, >> >> As far as I know, Solr only supports circuit breaks for queries at the >> moment. >> >> We have a custom integration of circuit breakers for indexing (in Solr 8, >> so that's not fully aligned with what in solr 9) with a custom >> UpdateRequestProcessor. Basically, a new instance of every update processor >> is created by calling the corresponding factory. We then check system stats >> (CPU...) at the first handled doc. >> Maybe there are other ways of integrating circuit breakers, but at least >> doing so in an update processor was not intrusive in the code. >> >> Probably it is worth it to have something standard in Solr. A "symetric" >> class of SearchHandler to add CB integration could be UpdateRequestHandler. >> >> Pierre >> >> >> Le mar. 27 juin 2023 à 13:32, Jan Høydahl <jan....@cominvent.com> a écrit : >> >>> Hi, >>> >>> Solr has CPU and Memory circuit breakers that will terminate Search >>> Requests only. >>> See >>> https://solr.apache.org/guide/solr/latest/deployment-guide/circuit-breakers.html >>> for docs. >>> >>> Example: >>> >>> <circuitBreaker class="solr.CircuitBreakerManager" enabled="true"> >>> <str name="memEnabled">true</str> >>> <str name="memThreshold">75</str> >>> <str name="cpuEnabled">true</str> >>> <str name="cpuThreshold">75</str> >>> </circuitBreaker> >>> >>> >>> A Solr node typically gets overloaded by the combined update and query >>> traffic, and >>> I'm looking into enabling ciricuit breakers for update requests. For many >>> workloads, pausing >>> update traffic would resolve the situation, with the benefit of users not >>> being affected by aborted >>> queries. >>> >>> So ideally I'd want to be able to choose to enable/disable CB on >>> update/query individually. >>> Or better, to kill update requests on e.g. 80% threshold and search >>> requests on 90% threshold. >>> >>> The current breaker impl for search < >>> https://github.com/apache/solr/blob/branch_9_2/solr/core/src/java/org/apache/solr/handler/component/SearchHandler.java#L372:380> >>> [1] is hardcoded into SearchHandler in such a way that if ANY >>> of the configured breakers trips, search requests are aborted. Also the >>> breakers seem to be generic >>> in nature, named CPUCircuitBreaker < >>> https://github.com/apache/solr/blob/branch_9_2/solr/core/src/java/org/apache/solr/util/circuitbreaker/CPUCircuitBreaker.java> >>> and MemoryCircuitBreaker < >>> https://github.com/apache/solr/blob/branch_9_2/solr/core/src/java/org/apache/solr/util/circuitbreaker/MemoryCircuitBreaker.java>, >>> so adding new UpdateCPUCircuitBreaker >>> does not seem to be the intention here. >>> >>> I'm also unclear on the usefulness of having CB on the core level and not >>> the node level. If you have >>> 10 cores from 10 collections on a node, and only some have CBs while >>> others do not, the node will >>> still be overloaded unless the admin convinces every collection owner to >>> implement the same CBs? >>> >>> So my question becomes - how do we enable CB for update requests into this >>> mix in a clean way? >>> >>> Jan --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org For additional commands, e-mail: dev-h...@solr.apache.org
