[ https://issues.apache.org/jira/browse/CASSANDRA-11320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tyler Hobbs updated CASSANDRA-11320: ------------------------------------ Status: Open (was: Patch Available) > Improve backoff policy for cqlsh COPY FROM > ------------------------------------------ > > Key: CASSANDRA-11320 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11320 > Project: Cassandra > Issue Type: Improvement > Components: Tools > Reporter: Stefania > Assignee: Stefania > Labels: doc-impacting > Fix For: 3.0.x, 3.x > > > Currently we have an exponential back-off policy in COPY FROM that kicks in > when timeouts are received. However there are two limitations: > * it does not cover new requests and therefore we may not back-off > sufficiently to give time to an overloaded server to recover > * the pause is performed in the receiving thread and therefore we may not > process server messages quickly enough > There is a static throttling mechanism in rows per second from feeder to > worker processes (the INGESTRATE) but the feeder has no idea of the load of > each worker process. However it's easy to keep track of how many chunks a > worker process has yet to read by introducing a bounded semaphore. > The idea is to move the back-off pauses to the worker processes main thread > so as to include all messages, new and retries, not just the retries that > timed out. The worker process will not read new chunks during the back-off > pauses, and the feeder process can then look at the number of pending chunks > before sending new chunks to a worker process. > [~aholmber], [~aweisberg] what do you think? -- This message was sent by Atlassian JIRA (v6.3.4#6332)