Re: [Gluster-users] Why is xfs recommended?

[Lindsay Mathieson]

On Wed, 12 Nov 2014 10:24:34 AM Ravishankar N wrote:
> XFS scales well when there is lot of meta data and multi-threaded I/O
> involved [1]. Choosing a file system is mostly about running the  kind of
> workload you would expect your system to see, with your hardware
> configuration and your version of the OS.  If ext4 gives you better
> performance when used as back end for gluster with your settings and
> workload, there shouldn't be any reason why you cannot go with it.

In the end I went with Option B :) - ZFS. Built in support for journal and 
data ssd caches, and of course, all the ZFS goodies, which will be very useful 
in the future.

I tried dmCache, bcache and EnhanceIO with xfs and ext4. bcache was a buggy 
mess, dmcache was a pain in the ass to manage, EnhanceIO was pretty good. But 
I managed to generate data corruptions with all three.

Also disk benchmarks (bonnie++, dd, CrystalDiskMark) gave wildly varying 
results that didn't bear much relationship to observed improvements. 

In the end I went with a series of app benchmarks that matched our usage. For 
those all of the above including zfs gave good improvements in roughly the 
same range.

So all things being equal, zfs won out for being a well supported standard and 
all its other benefits.

thanks for all the help and advise folks, sorry for being a pain :) though no 
doubt I will continue to be so.

Cheers, from sunny BrizVegas.

-- 
Lindsay

signature.asc
Description: This is a digitally signed message part.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Why is xfs recommended?

[Pranith Kumar Karampuri]

On 11/12/2014 10:31 PM, Dennis Schafroth wrote:
Is it know whether the fix will work for other filesystems like btrfs 
and HFS+

hi Dennis,
This problem happens because of the way gluster handles 64-bit 
offsets introduced in ext4.


Pranith


cheers,
:-Dennis

On 12 Nov 2014, at 13:22 , Pranith Kumar Karampuri 
mailto:pkara...@redhat.com>> wrote:



n Wed, 12 Nov 2014 09:41:17 AM you wrote:

I have seen weirdness with ext4 and replicated volumes, see thread
"[Gluster-devel] Duplicate entries and other weirdness in a 3*4 volume"
started at 17 July.

Interesting, thanks.
It is a bug in gluster andhttp://review.gluster.com/#/c/8201/is the 
fix for it.
Some more information about this bug: Only 3.6.1 release is affected 
by this issue. Except that release no other release has this problem. 
I was under the impression that this patch was already merged. This 
patch will definitely be there for 3.6.2. I am sorry for the slippage 
:'-(




___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Why is xfs recommended?

[Dennis Schafroth]

Is it know whether the fix will work for other filesystems like btrfs and HFS+ 

cheers, 
:-Dennis

> On 12 Nov 2014, at 13:22 , Pranith Kumar Karampuri  
> wrote:
> 
>>> n Wed, 12 Nov 2014 09:41:17 AM you wrote:
 I have seen weirdness with ext4 and replicated volumes, see thread
 "[Gluster-devel] Duplicate entries and other weirdness in a 3*4 volume" 
 started at 17 July.
>>> Interesting, thanks.
>> It is a bug in gluster and http://review.gluster.com/#/c/8201/ 
>>  is the fix for it.
> Some more information about this bug: Only 3.6.1 release is affected by this 
> issue. Except that release no other release has this problem. I was under the 
> impression that this patch was already merged. This patch will definitely be 
> there for 3.6.2. I am sorry for the slippage :'-(

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Why is xfs recommended?

[Pranith Kumar Karampuri]

On 11/12/2014 05:41 PM, Pranith Kumar Karampuri wrote:


On 11/12/2014 03:35 PM, Lindsay Mathieson wrote:

On Wed, 12 Nov 2014 09:41:17 AM you wrote:

I have seen weirdness with ext4 and replicated volumes, see thread
"[Gluster-devel] Duplicate entries and other weirdness in a 3*4 volume"
started at 17 July.

Interesting, thanks.
It is a bug in gluster and http://review.gluster.com/#/c/8201/ is the 
fix for it.
Some more information about this bug: Only 3.6.1 release is affected by 
this issue. Except that release no other release has this problem. I was 
under the impression that this patch was already merged. This patch will 
definitely be there for 3.6.2. I am sorry for the slippage :'-(


Pranith


Pranith



___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users




___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users


___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Why is xfs recommended?

[Pranith Kumar Karampuri]

On 11/12/2014 03:35 PM, Lindsay Mathieson wrote:

On Wed, 12 Nov 2014 09:41:17 AM you wrote:

I have seen weirdness with ext4 and replicated volumes, see thread
"[Gluster-devel] Duplicate entries and other weirdness in a 3*4 volume"
started at 17 July.


Interesting, thanks.
It is a bug in gluster and http://review.gluster.com/#/c/8201/ is the 
fix for it.


Pranith



___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users


___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Why is xfs recommended?

[Lindsay Mathieson]

On Wed, 12 Nov 2014 09:41:17 AM you wrote:
> I have seen weirdness with ext4 and replicated volumes, see thread
> "[Gluster-devel] Duplicate entries and other weirdness in a 3*4 volume" 
> started at 17 July.


Interesting, thanks.
-- 
Lindsay

signature.asc
Description: This is a digitally signed message part.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Why is xfs recommended?

[Anders Blomdell]

On 2014-11-12 05:54, Ravishankar N wrote:
> On 11/12/2014 03:21 AM, Lindsay Mathieson wrote:
>> Just wondering about the usecases. In all my testing ext4 has been
>> consistently faster for sustained and random read/writes on large files  (VM
>> images).
>>
>> Tested with/without external ssd journals and caches.
> XFS scales well when there is lot of meta data and multi-threaded I/O 
> involved [1].
> Choosing a file system is mostly about running the  kind of workload you 
> would expect your system to see, with your hardware configuration and your 
> version of the OS.  If ext4 gives you better performance when used as back 
> end for gluster with your settings and workload, there shouldn't be any 
> reason why you cannot go with it.
> 
> [1] http://xfs.org/images/d/d1/Xfs-scalability-lca2012.pdf
I have seen weirdness with ext4 and replicated volumes, see thread
"[Gluster-devel] Duplicate entries and other weirdness in a 3*4 volume" 
started at 17 July.


/Anders
-- 
Anders Blomdell  Email: anders.blomd...@control.lth.se
Department of Automatic Control
Lund University  Phone:+46 46 222 4625
P.O. Box 118 Fax:  +46 46 138118
SE-221 00 Lund, Sweden

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Why is xfs recommended?

[Ravishankar N]

On 11/12/2014 03:21 AM, Lindsay Mathieson wrote:

Just wondering about the usecases. In all my testing ext4 has been
consistently faster for sustained and random read/writes on large files  (VM
images).

Tested with/without external ssd journals and caches.
XFS scales well when there is lot of meta data and multi-threaded I/O 
involved [1].
Choosing a file system is mostly about running the  kind of workload you 
would expect your system to see, with your hardware configuration and 
your version of the OS.  If ext4 gives you better performance when used 
as back end for gluster with your settings and workload, there shouldn't 
be any reason why you cannot go with it.


[1] http://xfs.org/images/d/d1/Xfs-scalability-lca2012.pdf


nb. While you can use a external journal with xfs I found the support and
tools for it too marginal to risk using. Unable to move, resize or remove the
journal without manually editing the partition bytes, whereas ext4 has tune2fs
for all of that. Plus builtin support for loading the journal via label or
uuid.


thanks,



___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users


___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Why is xfs recommended?

[Pranith Kumar Karampuri]

CC one of the xfs devs Brian Foster.

Pranith
On 11/12/2014 03:21 AM, Lindsay Mathieson wrote:

Just wondering about the usecases. In all my testing ext4 has been
consistently faster for sustained and random read/writes on large files  (VM
images).

Tested with/without external ssd journals and caches.

nb. While you can use a external journal with xfs I found the support and
tools for it too marginal to risk using. Unable to move, resize or remove the
journal without manually editing the partition bytes, whereas ext4 has tune2fs
for all of that. Plus builtin support for loading the journal via label or
uuid.


thanks,



___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users


___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users