RE: hypothetical question about data storage

2013-07-30 Thread Johan De Meersman
Rick James wrote: > >When writing a random block, RAID-5 does not need to touch all the >drives, only the one with parity. Suitable XORs will update it >correctly. So, a write hits 2 drives, whether you have RAID-5 or -10. Only if the other blocks happen to be in the cache, otherwise a read is

Re: hypothetical question about data storage

2013-07-29 Thread Manuel Arostegui
2013/7/30 Rick James > Elevator... If the RAID _controller_ does the Elevator stuff, any OS > optimizations are wasted. > And there have been benchmarks backing that up. (Sorry, don't have any > links handy.) > > RAID 5/10 ... The testing I have done shows very little difference. > However, y

Re: hypothetical question about data storage

2013-07-29 Thread Carsten Pedersen
On 30-07-2013 01:16, Rick James wrote: Elevator... If the RAID _controller_ does the Elevator stuff, any OS optimizations are wasted. And there have been benchmarks backing that up. (Sorry, don't have any links handy.) RAID 5/10 ... The testing I have done shows very little difference. ...r

RE: hypothetical question about data storage

2013-07-29 Thread Rick James
Monday, July 29, 2013 3:38 PM > To: Rick James; will...@techservsys.com; mysql@lists.mysql.com > Subject: RE: hypothetical question about data storage > > Rick James wrote: > > > >For MySQL + RAID, a Linux elevator strategy of 'deadline' or 'noop' is &g

RE: hypothetical question about data storage

2013-07-29 Thread Johan De Meersman
Rick James wrote: > >For MySQL + RAID, a Linux elevator strategy of 'deadline' or 'noop' is >optimal. (The default, 'cfq', is not as good.) I should look into those again at some point. Do you have a brief word as to why they're better? >A RAID controller with multiple drives striped (and opt

RE: hypothetical question about data storage

2013-07-29 Thread Rick James
64TB limit per PARTITION. > -Original Message- > From: william drescher [mailto:will...@techservsys.com] > Sent: Saturday, July 27, 2013 4:32 AM > To: mysql@lists.mysql.com > Subject: Re: hypothetical question about data storage > > On 7/26/2013 6:58 PM, Chris Knipe wrote: >

Re: hypothetical question about data storage

2013-07-27 Thread william drescher
On 7/26/2013 6:58 PM, Chris Knipe wrote: The issue that we have identified is caused by seek time - hundreds of clients simultaneously searching for a single file. The only real way to explain this is to run 100 concurrent instances of bonnie++ doing random read/writes... Your disk utilization a

Re: hypothetical question about data storage

2013-07-26 Thread hsv
2013/07/27 00:58 +0200, Chris Knipe I would definately consider the md5 checksum as a PK (char(32) due to the hex nature), Well, not that it greatly matters, but you could convert it to BINARY(16). -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql

Re: hypothetical question about data storage

2013-07-26 Thread Chris Knipe
will have seeks. >> >>No, adding more RAM won't help much. Here's an argument: >>Suppose your data is 20 times as big as the buffer pool and you are >>doing random fetches (MD5, etc). Then 1/20 of fetches are cached; 95% >>cache miss. Estimated time: 0.95

RE: hypothetical question about data storage

2013-07-26 Thread Johan De Meersman
1/20 of fetches are cached; 95% >cache miss. Estimated time: 0.95 * 100ms = 95ms. >Now you double your RAM. 1/10 cached -> 90% cache miss -> 90ms average >-> Not much improvement over 95. > >> -Original Message----- >> From: ckn...@savage.za.org [mailto:ck

RE: hypothetical question about data storage

2013-07-26 Thread Rick James
ilto:ckn...@savage.za.org] On Behalf Of > Chris Knipe > Sent: Friday, July 26, 2013 12:30 AM > To: Johan De Meersman > Cc: mysql > Subject: Re: hypothetical question about data storage > > Hi All, > > Thanks for the responces, and I do concur. I was taking a stab in the &

Re: hypothetical question about data storage

2013-07-26 Thread Chris Knipe
ge - >> From: "Chris Knipe" >> To: mysql@lists.mysql.com >> Sent: Thursday, 25 July, 2013 11:53:53 PM >> Subject: hypothetical question about data storage >> >> Hi all, >> >> We run an VERY io intensive file application service. Curren

Re: hypothetical question about data storage

2013-07-26 Thread Johan De Meersman
mysql@lists.mysql.com > Sent: Thursday, 25 July, 2013 11:53:53 PM > Subject: hypothetical question about data storage > > Hi all, > > We run an VERY io intensive file application service. Currently, our > problem is that our disk spindles are being completely killed due to

Re: hypothetical question about data storage

2013-07-25 Thread Vahric Muhtaryan
Hi, Sorry but mysql is not the address of it , use riak instead of mysql With riak which is key and value based , all keys are on memory and just only one seek enough to handle it Consider to use riak VM On 7/26/13 12:53 AM, "Chris Knipe" wrote: >Hi all, > >We run an VERY io intensive file appli

hypothetical question about data storage

2013-07-25 Thread Chris Knipe
Hi all, We run an VERY io intensive file application service. Currently, our problem is that our disk spindles are being completely killed due to insufficient SEEK time on the hard drives (NOT physical read/write speeds). We have an directory structure where the files are stored based on the MD5