[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-8312: -- Assignee: (was: Yuki Morishita) > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Priority: Minor > Labels: lcs > Fix For: 2.1.x > > Attachments: cassandra-2.0-8312-1.txt > > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Deng updated CASSANDRA-8312: Labels: lcs (was: ) > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Assignee: Yuki Morishita >Priority: Minor > Labels: lcs > Fix For: 2.1.x > > Attachments: cassandra-2.0-8312-1.txt > > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-8312: --- Reviewer: (was: Paulo Motta) > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Priority: Minor > Labels: lcs > Fix For: 2.1.x > > Attachments: cassandra-2.0-8312-1.txt > > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-8312: --- Resolution: Won't Do Status: Resolved (was: Open) Closing for lack of activity. Please reopen if still relevant. > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Priority: Low > Labels: lcs, remove-reopen > Fix For: 2.1.x > > Attachments: cassandra-2.0-8312-1.txt > > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8312: --- Reviewer: Paulo Motta (was: Yuki Morishita) > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Assignee: Yuki Morishita >Priority: Minor > Fix For: 2.1.x > > Attachments: cassandra-2.0-8312-1.txt > > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8312: --- Assignee: Yuki Morishita (was: Jimmy Mårdell) > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Assignee: Yuki Morishita >Priority: Minor > Fix For: 2.1.x > > Attachments: cassandra-2.0-8312-1.txt > > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Mårdell updated CASSANDRA-8312: - Since Version: 2.0.11 > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Priority: Minor > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Mårdell updated CASSANDRA-8312: - Attachment: cassandra-2.0-8312-1.txt > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Priority: Minor > Attachments: cassandra-2.0-8312-1.txt > > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8312) Use live sstables in snapshot repair if possible
[ https://issues.apache.org/jira/browse/CASSANDRA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8312: -- Fix Version/s: (was: 2.0.12) 2.0.13 > Use live sstables in snapshot repair if possible > > > Key: CASSANDRA-8312 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8312 > Project: Cassandra > Issue Type: Improvement >Reporter: Jimmy Mårdell >Assignee: Jimmy Mårdell >Priority: Minor > Fix For: 3.0, 2.1.3, 2.0.13 > > Attachments: cassandra-2.0-8312-1.txt > > > Snapshot repair can be very much slower than parallel repairs because of the > overhead of opening the SSTables in the snapshot. This is particular true > when using LCS, as you typically have many smaller SSTables then. > I compared parallel and sequential repair on a small range on one of our > clusters (2*3 replicas). With parallel repair, this took 22 seconds. With > sequential repair (default in 2.0), the same range took 330 seconds! This is > an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were > 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely > causes lots of memory churning. > The idea would be to list the sstables in the snapshot, but use the > corresponding sstables in the live set if it's still available. For almost > all sstables, the original one should still exist. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
