Re: SSD journal suggestion / rsockets

2012-11-09 Thread Joseph Glanville
On 10 November 2012 01:43, Atchley, Scott wrote: > On Nov 8, 2012, at 5:00 PM, Joseph Glanville > wrote: > >> On 9 November 2012 08:21, Dieter Kasper wrote: >>> Joseph, >>> >>> I've downloaded and read the presentation from 'Sean Hefty / Intel >>> Corporation' >>> about rsockets, which sounds

Re: SSD journal suggestion / rsockets

2012-11-09 Thread Atchley, Scott
On Nov 8, 2012, at 5:00 PM, Joseph Glanville wrote: > On 9 November 2012 08:21, Dieter Kasper wrote: >> Joseph, >> >> I've downloaded and read the presentation from 'Sean Hefty / Intel >> Corporation' >> about rsockets, which sounds very promising to me. >> Can you please teach me how to get

Re: SSD journal suggestion / rsockets

2012-11-08 Thread Dieter Kasper
Joseph, I've downloaded and read the presentation from 'Sean Hefty / Intel Corporation' about rsockets, which sounds very promising to me. Can you please teach me how to get access to the rsockets source ? Thanks, -Dieter On Thu, Nov 08, 2012 at 09:12:45PM +0100, Joseph Glanville wrote: > On 9

Re: SSD journal suggestion / rsockets

2012-11-08 Thread Joseph Glanville
On 9 November 2012 08:21, Dieter Kasper wrote: > Joseph, > > I've downloaded and read the presentation from 'Sean Hefty / Intel > Corporation' > about rsockets, which sounds very promising to me. > Can you please teach me how to get access to the rsockets source ? > > Thanks, > -Dieter > > rsock

Re: SSD journal suggestion

2012-11-08 Thread Joseph Glanville
On 9 November 2012 02:00, Atchley, Scott wrote: > On Nov 8, 2012, at 9:39 AM, Mark Nelson wrote: > >> On 11/08/2012 07:55 AM, Atchley, Scott wrote: >>> On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta >>> wrote: >>> 2012/11/8 Mark Nelson : > I haven't done much with IPoIB (just RDMA),

Re: SSD journal suggestion

2012-11-08 Thread Atchley, Scott
On Nov 8, 2012, at 11:19 AM, Andrey Korolyov wrote: > On Thu, Nov 8, 2012 at 7:02 PM, Atchley, Scott wrote: >> On Nov 8, 2012, at 10:00 AM, Scott Atchley wrote: >> >>> On Nov 8, 2012, at 9:39 AM, Mark Nelson wrote: >>> On 11/08/2012 07:55 AM, Atchley, Scott wrote: > On Nov 8, 2012,

Re: SSD journal suggestion

2012-11-08 Thread Andrey Korolyov
On Thu, Nov 8, 2012 at 7:02 PM, Atchley, Scott wrote: > On Nov 8, 2012, at 10:00 AM, Scott Atchley wrote: > >> On Nov 8, 2012, at 9:39 AM, Mark Nelson wrote: >> >>> On 11/08/2012 07:55 AM, Atchley, Scott wrote: On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta wrote: > 2012/

Re: SSD journal suggestion

2012-11-08 Thread Atchley, Scott
On Nov 8, 2012, at 10:00 AM, Scott Atchley wrote: > On Nov 8, 2012, at 9:39 AM, Mark Nelson wrote: > >> On 11/08/2012 07:55 AM, Atchley, Scott wrote: >>> On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta >>> wrote: >>> 2012/11/8 Mark Nelson : > I haven't done much with IPoIB (just R

Re: SSD journal suggestion

2012-11-08 Thread Atchley, Scott
On Nov 8, 2012, at 9:39 AM, Mark Nelson wrote: > On 11/08/2012 07:55 AM, Atchley, Scott wrote: >> On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta >> wrote: >> >>> 2012/11/8 Mark Nelson : I haven't done much with IPoIB (just RDMA), but my understanding is that it tends to top out at

Re: SSD journal suggestion

2012-11-08 Thread Mark Nelson
On 11/08/2012 07:55 AM, Atchley, Scott wrote: On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta wrote: 2012/11/8 Mark Nelson : I haven't done much with IPoIB (just RDMA), but my understanding is that it tends to top out at like 15Gb/s. Some others on this mailing list can probably speak mor

Re: SSD journal suggestion

2012-11-08 Thread Atchley, Scott
On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta wrote: > 2012/11/8 Mark Nelson : >> I haven't done much with IPoIB (just RDMA), but my understanding is that it >> tends to top out at like 15Gb/s. Some others on this mailing list can >> probably speak more authoritatively. Even with RDMA you

Re: SSD journal suggestion

2012-11-07 Thread Mark Nelson
On 11/07/2012 04:51 PM, Gandalf Corvotempesta wrote: 2012/11/7 Martin Mailand : But it looks good on paper, so it's definitely a try worth. is at least 4x times faster than 10gbe and AFAIK should have a lower latency. I'm planning to use infiniband as backend storage network, used for OSD repl

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
good question, probably we do not have enough experience with IPoIB. But it looks good on paper, so it's definitely a try worth. -martin Am 07.11.2012 23:28, schrieb Gandalf Corvotempesta: 2012/11/7 Martin Mailand : I tested a Arista 7150S-24, a HP5900 and in a few weeks I will get a Mellanox

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
Hi, I *think* the HP is Broadcom based, the Arista is Fulcrum based, and I don't know which chips Mellanox is using. Our NOC tested both of them, an the Arista was the clear winner, at least in our workload. -martin Am 07.11.2012 22:59, schrieb Stefan Priebe: HP told me they all use the s

Re: SSD journal suggestion

2012-11-07 Thread Stefan Priebe
Am 07.11.2012 22:55, schrieb Martin Mailand: Hi Stefan, deep buffers means latency spikes, you should go for fast switching latency. The HP5900 has a latency of 1ms, the Arista and Mellanox of 250ns. HP told me they all use the same ships and Arista measures latency while only one port is in

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
Hi Stefan, deep buffers means latency spikes, you should go for fast switching latency. The HP5900 has a latency of 1ms, the Arista and Mellanox of 250ns. And I you should think at the price the HP5900 cost 3 times of the Mellanox. -martin Am 07.11.2012 22:44, schrieb Stefan Priebe: Am 07.11

Re: SSD journal suggestion

2012-11-07 Thread Stefan Priebe
Am 07.11.2012 22:35, schrieb Martin Mailand: Hi, I tested a Arista 7150S-24, a HP5900 and in a few weeks I will get a Mellanox MSX1016. ATM the Arista is may favourite. For the dual 10GeB NICs I tested the Intel X520-DA2 and the Mellanox ConnectX-3. My favourite is the Intel X520-DA2. That's p

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
Hi, I tested a Arista 7150S-24, a HP5900 and in a few weeks I will get a Mellanox MSX1016. ATM the Arista is may favourite. For the dual 10GeB NICs I tested the Intel X520-DA2 and the Mellanox ConnectX-3. My favourite is the Intel X520-DA2. -martin Am 07.11.2012 22:14, schrieb Gandalf Corvot

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
Hi, I have 16 SAS disk on a LSI 9266-8i and 4 Intel 520 SSD on a HBA, the node has dual 10G Ethernet. The clients are 4 nodes with dual 10GeB, as test I use rados bench on each client. The aggregated write speed is around 1,6GB/s with single replication. In the first configuration, I had the

Re: SSD journal suggestion

2012-11-07 Thread Mark Nelson
On 11/07/2012 10:35 AM, Atchley, Scott wrote: On Nov 7, 2012, at 11:20 AM, Mark Nelson wrote: Right now I'm doing 3 journals per SSD, but topping out at about 1.2-1.4GB/s from the client perspective for the node with 15+ drives and 5 SSDs. It's possible newer versions of the code and tuning

Re: SSD journal suggestion

2012-11-07 Thread Atchley, Scott
On Nov 7, 2012, at 11:20 AM, Mark Nelson wrote: >>> Right now I'm doing 3 journals per SSD, but topping out at about >>> 1.2-1.4GB/s from the client perspective for the node with 15+ drives and >>> 5 SSDs. It's possible newer versions of the code and tuning may >>> increase that. >> >> What in

Re: SSD journal suggestion

2012-11-07 Thread Atchley, Scott
On Nov 7, 2012, at 10:01 AM, Mark Nelson wrote: > On 11/07/2012 06:28 AM, Gandalf Corvotempesta wrote: >> 2012/11/7 Sage Weil : >>> On Wed, 7 Nov 2012, Gandalf Corvotempesta wrote: I'm evaluating some SSD drives as journal. Samsung 840 Pro seems to be the fastest in sequential reads and

Re: SSD journal suggestion

2012-11-07 Thread Mark Nelson
On 11/07/2012 10:12 AM, Atchley, Scott wrote: On Nov 7, 2012, at 10:01 AM, Mark Nelson wrote: On 11/07/2012 06:28 AM, Gandalf Corvotempesta wrote: 2012/11/7 Sage Weil : On Wed, 7 Nov 2012, Gandalf Corvotempesta wrote: I'm evaluating some SSD drives as journal. Samsung 840 Pro seems to be th

Re: SSD journal suggestion

2012-11-07 Thread Mark Nelson
On 11/07/2012 06:28 AM, Gandalf Corvotempesta wrote: 2012/11/7 Sage Weil : On Wed, 7 Nov 2012, Gandalf Corvotempesta wrote: I'm evaluating some SSD drives as journal. Samsung 840 Pro seems to be the fastest in sequential reads and write. The 840 Pro seems to reach 485MB/s in sequential write:

Re: SSD journal suggestion

2012-11-07 Thread Sage Weil
On Wed, 7 Nov 2012, Gandalf Corvotempesta wrote: > I'm evaluating some SSD drives as journal. > Samsung 840 Pro seems to be the fastest in sequential reads and write. > > What parameter should I consider for a journal? I think that none of > read benchmark are influent because when dumping journal