Re: idea drive layout - 4 drives + RAID question
Thanks. Yep, I think OS + CL (2 drive RAID1) will provide the best balance of reduced headaches / performance. I'll also be pondering 1 drive OS, 1 drive CL as well. On Wed, Oct 31, 2012 at 9:27 PM, aaron morton aa...@thelastpickle.comwrote: Good question. The is a comment on the DS blog or docs somewhere that says on EC2 running the commit log on the raid-0 ephemeral is preferred. I think the recommendation was specifically about how the disks are setup on EC2. While the commit log will be competing with logs and everything else on the OS volume, it would be competing with C* reads, Memtable flushing, compacting and repairing on the data volume. The only way to be sure is to test both setups. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 31/10/2012, at 1:11 PM, Ran User ranuse...@gmail.com wrote: Is there a concern of a large falloff in commit log write performance (sequential) when sharing 2 drives (RAID 1) with the OS (os and services writing their own logs, etc)? Do you expect the hit to be marginal? On Tue, Oct 30, 2012 at 7:58 PM, aaron morton aa...@thelastpickle.comwrote: We also have 4-disk nodes, and we use the following layout: 2 x OS + Commit in RAID 1 2 x Data disk in RAID 0 +1 You are replicating data at the application level and want the fastest possible IO performance per node. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. There are some features coming in 1.2 that make using a JBOD setup easier. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 30/10/2012, at 9:23 PM, Pieter Callewaert pieter.callewa...@be-mobile.be wrote: We also have 4-disk nodes, and we use the following layout: 2 x OS + Commit in RAID 1 2 x Data disk in RAID 0 This gives us the advantage we never have to reinstall the node when a drive crashes. Kind regards, Pieter *From:* Ran User [mailto:ranuse...@gmail.com] *Sent:* dinsdag 30 oktober 2012 4:33 *To:* user@cassandra.apache.org *Subject:* Re: idea drive layout - 4 drives + RAID question Have you considered running RAID 10 for the data drives to improve MTBF? On one hand Cassandra is handling redundancy issues, on the other hand, reducing the frequency of dealing with failed nodes is attractive if cheap (switching RAID levels to 10). We have no experience with software RAID (have always used hardware raid with BBU). I'm assuming software RAID 1 or 10 (the mirroring part) is inherently reliable (perhaps minus some edge case). On Tue, Oct 30, 2012 at 1:07 AM, Tupshin Harper tups...@tupshin.com wrote: I would generally recommend 1 drive for OS and commit log and 3 drive raid 0 for data. The raid does give you good performance benefit, and it can be convenient to have the OS on a side drive for configuration ease and better MTBF. -Tupshin On Oct 29, 2012 8:56 PM, Ran User ranuse...@gmail.com wrote: I was hoping to achieve approx. 2x IO (write and read) performance via RAID 0 (by accepting a higher MTBF). Do believe the performance gains of RAID0 are much lower and/or are not worth it vs the increased server failure rate? From my understanding, RAID 10 would achieve the read performance benefits of RAID 0, but not the write benefits. I'm also considering RAID 10 to maximize server IO performance. Currently, we're working with 1 CF. Thank you On Mon, Oct 29, 2012 at 11:51 PM, Timmy Turner timm.t...@gmail.com wrote: I'm not sure whether the raid 0 gets you anything other than headaches should one of the drives fail. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. 2012/10/30 Ran User ranuse...@gmail.com: For a server with 4 drive slots only, I'm thinking: either: - OS (1 drive) - Commit Log (1 drive) - Data (2 drives, software raid 0) vs - OS + Data (3 drives, software raid 0) - Commit Log (1 drive) or something else? also, if I can spare the wasted storage, would RAID 10 for cassandra data improve read performance and have no effect on write performance? Thank you! ** **
RE: idea drive layout - 4 drives + RAID question
We also have 4-disk nodes, and we use the following layout: 2 x OS + Commit in RAID 1 2 x Data disk in RAID 0 This gives us the advantage we never have to reinstall the node when a drive crashes. Kind regards, Pieter From: Ran User [mailto:ranuse...@gmail.com] Sent: dinsdag 30 oktober 2012 4:33 To: user@cassandra.apache.org Subject: Re: idea drive layout - 4 drives + RAID question Have you considered running RAID 10 for the data drives to improve MTBF? On one hand Cassandra is handling redundancy issues, on the other hand, reducing the frequency of dealing with failed nodes is attractive if cheap (switching RAID levels to 10). We have no experience with software RAID (have always used hardware raid with BBU). I'm assuming software RAID 1 or 10 (the mirroring part) is inherently reliable (perhaps minus some edge case). On Tue, Oct 30, 2012 at 1:07 AM, Tupshin Harper tups...@tupshin.commailto:tups...@tupshin.com wrote: I would generally recommend 1 drive for OS and commit log and 3 drive raid 0 for data. The raid does give you good performance benefit, and it can be convenient to have the OS on a side drive for configuration ease and better MTBF. -Tupshin On Oct 29, 2012 8:56 PM, Ran User ranuse...@gmail.commailto:ranuse...@gmail.com wrote: I was hoping to achieve approx. 2x IO (write and read) performance via RAID 0 (by accepting a higher MTBF). Do believe the performance gains of RAID0 are much lower and/or are not worth it vs the increased server failure rate? From my understanding, RAID 10 would achieve the read performance benefits of RAID 0, but not the write benefits. I'm also considering RAID 10 to maximize server IO performance. Currently, we're working with 1 CF. Thank you On Mon, Oct 29, 2012 at 11:51 PM, Timmy Turner timm.t...@gmail.commailto:timm.t...@gmail.com wrote: I'm not sure whether the raid 0 gets you anything other than headaches should one of the drives fail. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. 2012/10/30 Ran User ranuse...@gmail.commailto:ranuse...@gmail.com: For a server with 4 drive slots only, I'm thinking: either: - OS (1 drive) - Commit Log (1 drive) - Data (2 drives, software raid 0) vs - OS + Data (3 drives, software raid 0) - Commit Log (1 drive) or something else? also, if I can spare the wasted storage, would RAID 10 for cassandra data improve read performance and have no effect on write performance? Thank you!
Re: idea drive layout - 4 drives + RAID question
We also have 4-disk nodes, and we use the following layout: 2 x OS + Commit in RAID 1 2 x Data disk in RAID 0 +1 You are replicating data at the application level and want the fastest possible IO performance per node. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. There are some features coming in 1.2 that make using a JBOD setup easier. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 30/10/2012, at 9:23 PM, Pieter Callewaert pieter.callewa...@be-mobile.be wrote: We also have 4-disk nodes, and we use the following layout: 2 x OS + Commit in RAID 1 2 x Data disk in RAID 0 This gives us the advantage we never have to reinstall the node when a drive crashes. Kind regards, Pieter From: Ran User [mailto:ranuse...@gmail.com] Sent: dinsdag 30 oktober 2012 4:33 To: user@cassandra.apache.org Subject: Re: idea drive layout - 4 drives + RAID question Have you considered running RAID 10 for the data drives to improve MTBF? On one hand Cassandra is handling redundancy issues, on the other hand, reducing the frequency of dealing with failed nodes is attractive if cheap (switching RAID levels to 10). We have no experience with software RAID (have always used hardware raid with BBU). I'm assuming software RAID 1 or 10 (the mirroring part) is inherently reliable (perhaps minus some edge case). On Tue, Oct 30, 2012 at 1:07 AM, Tupshin Harper tups...@tupshin.com wrote: I would generally recommend 1 drive for OS and commit log and 3 drive raid 0 for data. The raid does give you good performance benefit, and it can be convenient to have the OS on a side drive for configuration ease and better MTBF. -Tupshin On Oct 29, 2012 8:56 PM, Ran User ranuse...@gmail.com wrote: I was hoping to achieve approx. 2x IO (write and read) performance via RAID 0 (by accepting a higher MTBF). Do believe the performance gains of RAID0 are much lower and/or are not worth it vs the increased server failure rate? From my understanding, RAID 10 would achieve the read performance benefits of RAID 0, but not the write benefits. I'm also considering RAID 10 to maximize server IO performance. Currently, we're working with 1 CF. Thank you On Mon, Oct 29, 2012 at 11:51 PM, Timmy Turner timm.t...@gmail.com wrote: I'm not sure whether the raid 0 gets you anything other than headaches should one of the drives fail. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. 2012/10/30 Ran User ranuse...@gmail.com: For a server with 4 drive slots only, I'm thinking: either: - OS (1 drive) - Commit Log (1 drive) - Data (2 drives, software raid 0) vs - OS + Data (3 drives, software raid 0) - Commit Log (1 drive) or something else? also, if I can spare the wasted storage, would RAID 10 for cassandra data improve read performance and have no effect on write performance? Thank you!
Re: idea drive layout - 4 drives + RAID question
Is there a concern of a large falloff in commit log write performance (sequential) when sharing 2 drives (RAID 1) with the OS (os and services writing their own logs, etc)? Do you expect the hit to be marginal? On Tue, Oct 30, 2012 at 7:58 PM, aaron morton aa...@thelastpickle.comwrote: We also have 4-disk nodes, and we use the following layout: 2 x OS + Commit in RAID 1 2 x Data disk in RAID 0 +1 You are replicating data at the application level and want the fastest possible IO performance per node. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. There are some features coming in 1.2 that make using a JBOD setup easier. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 30/10/2012, at 9:23 PM, Pieter Callewaert pieter.callewa...@be-mobile.be wrote: We also have 4-disk nodes, and we use the following layout: 2 x OS + Commit in RAID 1 2 x Data disk in RAID 0 This gives us the advantage we never have to reinstall the node when a drive crashes. Kind regards, Pieter *From:* Ran User [mailto:ranuse...@gmail.com] *Sent:* dinsdag 30 oktober 2012 4:33 *To:* user@cassandra.apache.org *Subject:* Re: idea drive layout - 4 drives + RAID question Have you considered running RAID 10 for the data drives to improve MTBF? On one hand Cassandra is handling redundancy issues, on the other hand, reducing the frequency of dealing with failed nodes is attractive if cheap (switching RAID levels to 10). We have no experience with software RAID (have always used hardware raid with BBU). I'm assuming software RAID 1 or 10 (the mirroring part) is inherently reliable (perhaps minus some edge case). On Tue, Oct 30, 2012 at 1:07 AM, Tupshin Harper tups...@tupshin.com wrote: I would generally recommend 1 drive for OS and commit log and 3 drive raid 0 for data. The raid does give you good performance benefit, and it can be convenient to have the OS on a side drive for configuration ease and better MTBF. -Tupshin On Oct 29, 2012 8:56 PM, Ran User ranuse...@gmail.com wrote: I was hoping to achieve approx. 2x IO (write and read) performance via RAID 0 (by accepting a higher MTBF). Do believe the performance gains of RAID0 are much lower and/or are not worth it vs the increased server failure rate? From my understanding, RAID 10 would achieve the read performance benefits of RAID 0, but not the write benefits. I'm also considering RAID 10 to maximize server IO performance. Currently, we're working with 1 CF. Thank you On Mon, Oct 29, 2012 at 11:51 PM, Timmy Turner timm.t...@gmail.com wrote: I'm not sure whether the raid 0 gets you anything other than headaches should one of the drives fail. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. 2012/10/30 Ran User ranuse...@gmail.com: For a server with 4 drive slots only, I'm thinking: either: - OS (1 drive) - Commit Log (1 drive) - Data (2 drives, software raid 0) vs - OS + Data (3 drives, software raid 0) - Commit Log (1 drive) or something else? also, if I can spare the wasted storage, would RAID 10 for cassandra data improve read performance and have no effect on write performance? Thank you! ** **
idea drive layout - 4 drives + RAID question
For a server with 4 drive slots only, I'm thinking: either: - OS (1 drive) - Commit Log (1 drive) - Data (2 drives, software raid 0) vs - OS + Data (3 drives, software raid 0) - Commit Log (1 drive) or something else? also, if I can spare the wasted storage, would RAID 10 for cassandra data improve read performance and have no effect on write performance? Thank you!
Re: idea drive layout - 4 drives + RAID question
I'm not sure whether the raid 0 gets you anything other than headaches should one of the drives fail. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. 2012/10/30 Ran User ranuse...@gmail.com: For a server with 4 drive slots only, I'm thinking: either: - OS (1 drive) - Commit Log (1 drive) - Data (2 drives, software raid 0) vs - OS + Data (3 drives, software raid 0) - Commit Log (1 drive) or something else? also, if I can spare the wasted storage, would RAID 10 for cassandra data improve read performance and have no effect on write performance? Thank you!
Re: idea drive layout - 4 drives + RAID question
I was hoping to achieve approx. 2x IO (write and read) performance via RAID 0 (by accepting a higher MTBF). Do believe the performance gains of RAID0 are much lower and/or are not worth it vs the increased server failure rate? From my understanding, RAID 10 would achieve the read performance benefits of RAID 0, but not the write benefits. I'm also considering RAID 10 to maximize server IO performance. Currently, we're working with 1 CF. Thank you On Mon, Oct 29, 2012 at 11:51 PM, Timmy Turner timm.t...@gmail.com wrote: I'm not sure whether the raid 0 gets you anything other than headaches should one of the drives fail. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. 2012/10/30 Ran User ranuse...@gmail.com: For a server with 4 drive slots only, I'm thinking: either: - OS (1 drive) - Commit Log (1 drive) - Data (2 drives, software raid 0) vs - OS + Data (3 drives, software raid 0) - Commit Log (1 drive) or something else? also, if I can spare the wasted storage, would RAID 10 for cassandra data improve read performance and have no effect on write performance? Thank you!
Re: idea drive layout - 4 drives + RAID question
Have you considered running RAID 10 for the data drives to improve MTBF? On one hand Cassandra is handling redundancy issues, on the other hand, reducing the frequency of dealing with failed nodes is attractive if cheap (switching RAID levels to 10). We have no experience with software RAID (have always used hardware raid with BBU). I'm assuming software RAID 1 or 10 (the mirroring part) is inherently reliable (perhaps minus some edge case). On Tue, Oct 30, 2012 at 1:07 AM, Tupshin Harper tups...@tupshin.com wrote: I would generally recommend 1 drive for OS and commit log and 3 drive raid 0 for data. The raid does give you good performance benefit, and it can be convenient to have the OS on a side drive for configuration ease and better MTBF. -Tupshin On Oct 29, 2012 8:56 PM, Ran User ranuse...@gmail.com wrote: I was hoping to achieve approx. 2x IO (write and read) performance via RAID 0 (by accepting a higher MTBF). Do believe the performance gains of RAID0 are much lower and/or are not worth it vs the increased server failure rate? From my understanding, RAID 10 would achieve the read performance benefits of RAID 0, but not the write benefits. I'm also considering RAID 10 to maximize server IO performance. Currently, we're working with 1 CF. Thank you On Mon, Oct 29, 2012 at 11:51 PM, Timmy Turner timm.t...@gmail.comwrote: I'm not sure whether the raid 0 gets you anything other than headaches should one of the drives fail. You can already distribute the individual Cassandra column families on different drives by just setting up symlinks to the individual folders. 2012/10/30 Ran User ranuse...@gmail.com: For a server with 4 drive slots only, I'm thinking: either: - OS (1 drive) - Commit Log (1 drive) - Data (2 drives, software raid 0) vs - OS + Data (3 drives, software raid 0) - Commit Log (1 drive) or something else? also, if I can spare the wasted storage, would RAID 10 for cassandra data improve read performance and have no effect on write performance? Thank you!
