[
https://issues.apache.org/jira/browse/CASSANDRA-7360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7360:
----------------------------------------
Attachment: 7360-v2.txt
bq. I would have wrapped it into a UncheckedException (e.g. SyncException) that
I would have caught in rawAddRow and rethrow as IOException as it is more direct
You're right, that's probably cleaner overall, made that change in the v2
attached.
bq. I think that in BufferedWriter.close() the code should probably try to call
super.close before trying to rethrow the exception
It's fixed by the virtue of BufferedWriter not overriding close() anymore.
bq. I have the following error message in my log when I run the Unit tests
That is something we need to fix but that's completely unrelated to this ticket
so not gonna fix it here. But created CASSANDRA-7889 so we don't forget to
scratch this itch.
> CQLSSTableWriter consumes all memory for table with compound primary key
> ------------------------------------------------------------------------
>
> Key: CASSANDRA-7360
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7360
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Xu Zhongxing
> Assignee: Sylvain Lebresne
> Fix For: 2.0.11
>
> Attachments: 7360-v2.txt, 7360.txt
>
>
> When using CQLSSTableWriter to write a table with compound primary key, if
> the partition key is identical for a huge amount of records, the sync()
> method is never called, and the memory usage keeps growing until the memory
> is exhausted.
> Could the code be improved to do sync() even when there is no new row
> created? The relevant code is in SSTableSimpleUnsortedWriter.java and
> AbstractSSTableSimpleWriter.java. I am new to the code and cannot produce a
> reasonable patch for now.
> The problem can be reproduced by the following test case:
> {code}
> import org.apache.cassandra.io.sstable.CQLSSTableWriter;
> import org.apache.cassandra.exceptions.InvalidRequestException;
> import java.io.IOException;
> import java.util.UUID;
> class SS {
> public static void main(String[] args) {
> String schema = "create table test.t (x uuid, y uuid, primary key (x,
> y))";
> String insert = "insert into test.t (x, y) values (?, ?)";
> CQLSSTableWriter writer = CQLSSTableWriter.builder()
> .inDirectory("/tmp/test/t")
> .forTable(schema).withBufferSizeInMB(32)
> .using(insert).build();
> UUID id = UUID.randomUUID();
> try {
> for (int i = 0; i < 50000000; i++) {
> UUID id2 = UUID.randomUUID();
> writer.addRow(id, id2);
> }
> writer.close();
> } catch (Exception e) {
> System.err.println("hell");
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
