[
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701505#comment-13701505
]
Pavel Yaskevich commented on CASSANDRA-5661:
--------------------------------------------
bq. Sure, and this is supported, but this is not a case of contention which was
the concern I raised with Ben.
What contention are we talking? If that is memory pressure, then expiring items
in bulk is good and leaves room for new files. In case of LCS instances mostly
are going to be invalidated by compaction vs access time expiry, STCS observes
the same effect but better in terms of memory usage as smaller files are going
to be compacted into bigger one so each compaction reduces number of instances
and buffers.
I think the original problem in here was that with default settings LCS eats a
lot of memory because files are too small and even if minor portion of dataset
is read memory, overhead for caching is still unacceptable, this is why I was
talking about adding global memory cap and as a good bonus - expiry of unused
instances, because if file is expired by timer, it means that reads have turned
away from it or it's read in bursts so deallocating all of the cached instances
is the way to go, where with LRU we would only replace when new instances are
returned to the pull (as we don't pre-allocate) which could create a problem
when there is a burst of requests to the same file after long internal of
inactivity. But even after so many comments that we had here and people
([~tjake]) reporting that even current setup works for them on bigger files, it
still looks like the everybody is trying to solve different issues.
> Discard pooled readers for cold data
> ------------------------------------
>
> Key: CASSANDRA-5661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.2.1
> Reporter: Jonathan Ellis
> Assignee: Pavel Yaskevich
> Fix For: 1.2.7
>
> Attachments: CASSANDRA-5661.patch, DominatorTree.png, Histogram.png
>
>
> Reader pooling was introduced in CASSANDRA-4942 but pooled
> RandomAccessReaders are never cleaned up until the SSTableReader is closed.
> So memory use is "the worst case simultaneous RAR we had open for this file,
> forever."
> We should introduce a global limit on how much memory to use for RAR, and
> evict old ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
