Hi Attlila,
We did have larger partitions which are now below 100MB threshold after we
ran nodetool repair. And now we do see most of the time, query runs are
running successfully but there is a small percentage of query runs which
are still failing.
Regarding your comment ```considered with your fetchSize together (driver
setting on the query level)```, can you elaborate more on it? Are you
suggesting to reduce the fetchSize (right now fetchSize is 5000) for this
query?
Also, we are trying to use prefetch feature as well but it is also not
helping. Following is the code:
Iterator<Row> iter = resultSet.iterator();
while (iter.hasNext()) {
if (resultSet.getAvailableWithoutFetching() <= fetchSize &&
!resultSet.isFullyFetched()) {
resultSet.fetchMoreResults();
}
Row row = iter.next();
.....
}
Thanks,
Deepak
On Sat, Sep 19, 2020 at 6:56 PM Deepak Sharma <sharma.dee...@salesforce.com>
wrote:
> Thanks Attila and Aaron for the response. These are great insights. I will
> check and get back to you in case I have any questions.
>
> Best,
> Deepak
>
> On Tue, Sep 15, 2020 at 4:33 AM Attila Wind <attilaw@swf.technology>
> wrote:
>
>> Hi Deepak,
>>
>> Aaron has right - in order being able to help (better) you need to share
>> those details
>>
>> That 5 secs timeout comes from the coordinator node I think - see
>> cassandra.yaml "read_request_timeout_in_ms" setting - that is influencing
>> this
>>
>> But it does not matter too much... The point is that none of the replicas
>> could completed your query within that 5 secs. And this is a clean
>> indication of something is slow with your query.
>> Maybe 4) is a bit less important here, or I would a bit make it more
>> precise: considered with your fetchSize together (driver setting on the
>> query level)
>>
>> By experience one reason could be if the query which used to works starts
>> not to work any longer is growing number of data. And a possible "wide
>> cluster" problem.
>> Do you have monitoring on the Cassandra machines? What does iowait show?
>> (for us when things like this will start happening is a clean indication)
>>
>> cheers
>> Attila Wind
>>
>> http://www.linkedin.com/in/attilaw
>> Mobile: +49 176 43556932
>>
>>
>> 14.09.2020 18:36 keltezéssel, Aaron Ploetz írta:
>>
>> Deepak,
>>
>> Can you reply with:
>>
>> 1) The query you are trying to run.
>> 2) The table definition (PRIMARY KEY, specifically).
>> 3) Maybe a little description of what the table is designed to do.
>> 4) How much data you're expecting returned (both # of rows and data size).
>>
>> Thanks,
>>
>> Aaron
>>
>>
>> On Mon, Sep 14, 2020 at 10:58 AM Deepak Sharma
>> <sharma.dee...@salesforce.com.invalid>
>> <sharma.dee...@salesforce.com.invalid> wrote:
>>
>>> Hi There,
>>>
>>> We are running into a strange issue in our Cassandra Cluster where one
>>> specific query is failing with following error:
>>>
>>> Cassandra timeout during read query at consistency QUORUM (3 responses
>>> were required but only 0 replica responded)
>>>
>>> This is not a typical query read timeout that we know for sure. This
>>> error is getting spit out within 5 seconds and the query timeout we have
>>> set is around 30 seconds
>>>
>>> Can we know what is happening here and how can we reproduce this in our
>>> local environment?
>>>
>>> Thanks,
>>> Deepak
>>>
>>>
