[jira] [Commented] (CASSANDRA-5727) Evaluate default LCS sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724006#comment-13724006 ] T Jake Luciani commented on CASSANDRA-5727: --- [~danielmeyer] Did you track compaction time across sizes? Evaluate default LCS sstable size - Key: CASSANDRA-5727 URL: https://issues.apache.org/jira/browse/CASSANDRA-5727 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Daniel Meyer Fix For: 1.2.9 Attachments: BytesRead_vs_LCS.png, ReadLatency_vs_LCS.png, Throughtput_vs_LCS.png, UpdateLatency_vs_LCS.png What we're not sure about is the effect on compaction efficiency -- larger files mean that each level contains more data, so reads will have to touch less sstables, but we're also compacting less unchanged data when we merge forward. So the question is, how big can we make the sstables to get the benefits of the first effect, before the second effect starts to dominate? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5727) Evaluate default LCS sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13724016#comment-13724016 ] Jonathan Ellis commented on CASSANDRA-5727: --- Yes. Time was proportional to the bytes compacted [bytesread graph]. Evaluate default LCS sstable size - Key: CASSANDRA-5727 URL: https://issues.apache.org/jira/browse/CASSANDRA-5727 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Daniel Meyer Fix For: 1.2.9 Attachments: BytesRead_vs_LCS.png, ReadLatency_vs_LCS.png, Throughtput_vs_LCS.png, UpdateLatency_vs_LCS.png What we're not sure about is the effect on compaction efficiency -- larger files mean that each level contains more data, so reads will have to touch less sstables, but we're also compacting less unchanged data when we merge forward. So the question is, how big can we make the sstables to get the benefits of the first effect, before the second effect starts to dominate? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5727) Evaluate default LCS sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13722737#comment-13722737 ] Radim Kolar commented on CASSANDRA-5727: did you tried to measure standard compaction strategy to see if 160 MB LCS brings improvements? Evaluate default LCS sstable size - Key: CASSANDRA-5727 URL: https://issues.apache.org/jira/browse/CASSANDRA-5727 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Daniel Meyer Attachments: BytesRead_vs_LCS.png, ReadLatency_vs_LCS.png, Throughtput_vs_LCS.png, UpdateLatency_vs_LCS.png What we're not sure about is the effect on compaction efficiency -- larger files mean that each level contains more data, so reads will have to touch less sstables, but we're also compacting less unchanged data when we merge forward. So the question is, how big can we make the sstables to get the benefits of the first effect, before the second effect starts to dominate? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5727) Evaluate default LCS sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709580#comment-13709580 ] Radim Kolar commented on CASSANDRA-5727: You are missing 3 important points. There is no need to read lvldb source code, just read log file and you will notice differences in compaction, which contribute to speedup. I implemented lvldb 3 times. First into HDFS - where metadata are stored into path, second version was improvement of cassandra code, and 3rd version is improved version of current google record based backend, which is leveldb based with modifications allowing it run without filesystem and network part removed + 2 small performance tunings with major effect. You are right about concurrent compactions and non blocking writes for L0. K, you wanted hint, so here it is: Compare L0 in C 2.0 and leveldb. Evaluate default LCS sstable size - Key: CASSANDRA-5727 URL: https://issues.apache.org/jira/browse/CASSANDRA-5727 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Daniel Meyer What we're not sure about is the effect on compaction efficiency -- larger files mean that each level contains more data, so reads will have to touch less sstables, but we're also compacting less unchanged data when we merge forward. So the question is, how big can we make the sstables to get the benefits of the first effect, before the second effect starts to dominate? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5727) Evaluate default LCS sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708658#comment-13708658 ] Robert Coli commented on CASSANDRA-5727: Anecdotally, many people on @cassandra-user/#cassandra have been bitten by the current 5mb size. The types of negative experiences they have seem to relate to too many SSTables for small or medium data sizes. Even a relatively naive doubling of this default to 10mb seems likely to be a win for most of these users. Evaluate default LCS sstable size - Key: CASSANDRA-5727 URL: https://issues.apache.org/jira/browse/CASSANDRA-5727 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Daniel Meyer What we're not sure about is the effect on compaction efficiency -- larger files mean that each level contains more data, so reads will have to touch less sstables, but we're also compacting less unchanged data when we merge forward. So the question is, how big can we make the sstables to get the benefits of the first effect, before the second effect starts to dominate? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5727) Evaluate default LCS sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709054#comment-13709054 ] Radim Kolar commented on CASSANDRA-5727: Leveldb is using 2MB by default. Real problem is naive leveldb implementation in cassandra. Evaluate default LCS sstable size - Key: CASSANDRA-5727 URL: https://issues.apache.org/jira/browse/CASSANDRA-5727 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Daniel Meyer What we're not sure about is the effect on compaction efficiency -- larger files mean that each level contains more data, so reads will have to touch less sstables, but we're also compacting less unchanged data when we merge forward. So the question is, how big can we make the sstables to get the benefits of the first effect, before the second effect starts to dominate? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5727) Evaluate default LCS sstable size
[ https://issues.apache.org/jira/browse/CASSANDRA-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709056#comment-13709056 ] Jonathan Ellis commented on CASSANDRA-5727: --- Have you read the leveldb source? There is no magic there. Evaluate default LCS sstable size - Key: CASSANDRA-5727 URL: https://issues.apache.org/jira/browse/CASSANDRA-5727 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Daniel Meyer What we're not sure about is the effect on compaction efficiency -- larger files mean that each level contains more data, so reads will have to touch less sstables, but we're also compacting less unchanged data when we merge forward. So the question is, how big can we make the sstables to get the benefits of the first effect, before the second effect starts to dominate? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
