[
https://issues.apache.org/jira/browse/CASSANDRA-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777597#comment-13777597
]
koray sariteke commented on CASSANDRA-6039:
-------------------------------------------
At SSTableSimpleUnsortedWriter, for every new row
buffer calculation takes place as
AbstractSSTableSimpleWriter->newRow()
SSTableSimpleUnsortedWriter->writeRow()
calculate columnfamily size and add columnfamily size to
currentsize
SSTableSimpleUnsortedWriter->getColumnFamily()
calculate columnfamily size if rowkey present and extract from
currentsize
I discard calculation of columnfamily size; put an counter and check value for
every new row activity like
protected void writeRow(DecoratedKey key, ColumnFamily columnFamily) throws
IOException
{
++counter;
//nearly 400000 record is 62MB
//pass bufferSize as nearly 400000
if (counter > bufferSize) {
counter = 0;
sync();
}
}
that already speeds up nearly 100 times
so; seems that calculation of column family slow down for new row insertion
exponential.
> creating sstable by SSTableSimpleUnsortedWriter for CompositeType is too slow.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-6039
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6039
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Environment: jdk7
> Reporter: koray sariteke
>
> Creating sstable for composite type is too slow. Dig into code and seems that
> inserting into TreeMapBackedSortedColumns takes too much time after a while.
> Similar table is like:
> CREATE TABLE demodb (
> id long,
> time long,
> login int,
> desc text,
> active boolean
> PRIMARY KEY (id, time)
> )
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
