[
https://issues.apache.org/jira/browse/CASSANDRA-19225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Strawderman updated CASSANDRA-19225:
--------------------------------------------
Description:
Executing a CQL statement asynchronously and then subsequently cancelling it
can result in the RequestThrottler becoming "full", as its never informed of
the cancellation.
CqlRequestHandler does have a callback that handles cancellation, however it
doesn't inform the throttler. This is particularly bad when using a throttler
such as ConcurrencyLimitingRequestThrottler which tracks the number of
in-flight requests as well as queues requests beyond a certain threshold,
neither of which gets properly updated when a request is cancelled.
To reproduce, one can simply do the following:
{code:java}
AsyncCqlSession session;
Statement statement;
CompletableFuture<AsyncResultSet> future =
session.executeAsync(statement).toCompletableFuture();
future.cancel(true);
{code}
Doing this repeatedly with a throttler such as
ConcurrencyLimitingRequestThrottler you can observe that the throttler is never
informed of the cancellation, and eventually it will become saturated with
cancelled requests and reject the request. The issue likely applies to any of
the methods in AsyncCqlSession, as they all use CqlRequestHandler under the
hood.
We fixed the issue in our application by installing a custom CqlRequestHandler
that calls `signalTimeout` on the throttler, and verified that it resolves the
problem; adding that logic to the existing callback seems like the most
straight forward solution.
was:
Executing a CQL statement asynchronously and then subsequently cancelling it
can result in the RequestThrottler becoming "full", as its never informed of
the cancellation.
CqlRequestHandler does have a callback that handles cancellation, however it
doesn't inform the throttler. This is particularly bad when using a throttler
such as ConcurrencyLimitingRequestThrottler which tracks the number of
in-flight requests as well as queues requests beyond a certain threshold,
neither of which gets properly updated when a request is cancelled.
To reproduce, one can simply do the following:
```java
AsyncCqlSession session;
Statement statement;
CompletableFuture<AsyncResultSet> future =
session.executeAsync(statement).toCompletableFuture();
future.cancel(true);
```
Doing this repeatedly with a throttler such as
ConcurrencyLimitingRequestThrottler you can observe that the throttler is never
informed of the cancellation, and eventually it will become saturated with
cancelled requests and reject the request. The issue likely applies to any of
the methods in AsyncCqlSession, as they all use CqlRequestHandler under the
hood.
We fixed the issue in our application by installing a custom CqlRequestHandler
that calls `signalTimeout` on the throttler, and verified that it resolves the
problem; adding that logic to the existing callback seems like the most
straight forward solution.
> Async Query Cancellation Not Propagated To RequestThrottler
> -----------------------------------------------------------
>
> Key: CASSANDRA-19225
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19225
> Project: Cassandra
> Issue Type: Bug
> Reporter: Patrick Strawderman
> Priority: Normal
>
> Executing a CQL statement asynchronously and then subsequently cancelling it
> can result in the RequestThrottler becoming "full", as its never informed of
> the cancellation.
> CqlRequestHandler does have a callback that handles cancellation, however it
> doesn't inform the throttler. This is particularly bad when using a throttler
> such as ConcurrencyLimitingRequestThrottler which tracks the number of
> in-flight requests as well as queues requests beyond a certain threshold,
> neither of which gets properly updated when a request is cancelled.
>
> To reproduce, one can simply do the following:
> {code:java}
> AsyncCqlSession session;
> Statement statement;
> CompletableFuture<AsyncResultSet> future =
> session.executeAsync(statement).toCompletableFuture();
> future.cancel(true);
> {code}
>
> Doing this repeatedly with a throttler such as
> ConcurrencyLimitingRequestThrottler you can observe that the throttler is
> never informed of the cancellation, and eventually it will become saturated
> with cancelled requests and reject the request. The issue likely applies to
> any of the methods in AsyncCqlSession, as they all use CqlRequestHandler
> under the hood.
>
> We fixed the issue in our application by installing a custom
> CqlRequestHandler that calls `signalTimeout` on the throttler, and verified
> that it resolves the problem; adding that logic to the existing callback
> seems like the most straight forward solution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
