[
https://issues.apache.org/jira/browse/CASSANDRA-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204394#comment-13204394
]
Sylvain Lebresne commented on CASSANDRA-3861:
---------------------------------------------
bq. Man, performance vs helping people do the Right Thing, that's a tough call
Yeah, but I kind of think the current "protection" is pretty lame. And we're
not protecting people the same way when they do a slice with Integer.MAX_VALUE,
do we? So I'm only mildly convinced that protecting users was the initial
intent here.
bq. Is allocating a 10K list array that much of a performance hit?
Not in isolation I suppose, but what when you do 10K queries/sec? Let's say
that I'm not convinced at all by the protective aspect of allocating the limit
(I can see as many people "protected" by the current code, than people feeding
some large value for the count (thus doing the wrong thing) that happens to not
OOM during tests but do during production because you have a load spike), and
so I think we shouldn't keep it given it has performance impact (even if it's a
small one).
Anyway, truth is I don't care too much because I think this won't be much of a
problem moving forward (adding automatic paging through CQL) so I can agree
with sticking with the status quo in the meantime, though my personal opinion
is still that we would be better off without the current allocation to the
limit.
> get_indexed_slices throws OOM Error when is called with too big
> indexClause.count
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-3861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3861
> Project: Cassandra
> Issue Type: Bug
> Components: API, Core
> Affects Versions: 1.0.7
> Reporter: Vladimir Tsanev
> Assignee: Sylvain Lebresne
> Fix For: 1.0.8
>
> Attachments: 3861.patch
>
>
> I tried to call get_index_slices with Integer.MAX_VALUE as IndexClause.count.
> Unfortunately the node died with OOM. In the log there si following error:
> ERROR [Thrift:4] 2012-02-06 17:43:39,224 Cassandra.java (line 3252) Internal
> error processing get_indexed_slices
> java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.<init>(ArrayList.java:112)
> at
> org.apache.cassandra.service.StorageProxy.scan(StorageProxy.java:1067)
> at
> org.apache.cassandra.thrift.CassandraServer.get_indexed_slices(CassandraServer.java:746)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$get_indexed_slices.process(Cassandra.java:3244)
> at
> org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889)
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Is it necessary to allocate all the memory in advance. I only have 3 KEYS
> that match my caluse. I do not known the exact number but in general I am
> sure that they wil fit in the memory.
> I can/will implement some calls with paging, but wanted to test and I am not
> happy with the fact the node disconnected.
> I wonder why ArrayList is used here?
> I think the result is never accessed by index (but only iterated) and the
> subList for non RandomAccess Lists (for example LinkedList) will do the same
> job if you are not using other operations than iteration.
> Is this related to the problem described in CASSANDRA-691.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
