Re: [Gluster-users] glusterfs 3.1.1 troubles on SLES11 SP1

2011-01-14 Thread Markus Fröhlich
hi amar! I made the test like you want me to do and the glusterfsd dies again. virt-zabbix-02:~ # mv /etc/glusterd /etc/glusterd.old virt-zabbix-03:~ # mv /etc/glusterd /etc/glusterd.old virt-zabbix-02:~ # rcglusterd start Starting glusterd:

Re: [Gluster-users] Share volume through rdma and tcp simultaneously

2011-01-14 Thread Amar Tumballi
With GlusterFS 3.1 and it's cool management frontend I didn't found a way to realize such a setup without hacking the volfile. *hint-to-the-developers* :-) On server side in 3.1.x version you can give option like below: option transport-type socket,rdma in protocol/server volume

[Gluster-users] Deleted files comes back

2011-01-14 Thread Georg Höllrigl
Hello, I delete a file on glusterfs but shortly after I have verified that it is away, it suddenly reappears. I'm using glusterfs 3.0.6. Two servers and several clients with client replication. I found out the following things: - The file only comes back when deleting on one glusterfs

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Burnash, James
Hello. Gluster - and in fact most (all?) parallel filesystems are optimized for very large files. That being the case, small files are not retrieved as efficiently, and result in a larger number of file operations in total because there are a fixed number for each file accessed. While the

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Łukasz Jagiełło
2011/1/14 Max Ivanov ivanov.ma...@gmail.com: Gluster - and in fact most (all?) parallel filesystems are optimized for very large files. That being the case, small files are not retrieved as efficiently, and result in a larger number of file operations in total because there are a fixed

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Andrew Latham
2011/1/14 Łukasz Jagiełło jagiello.luk...@gmail.com: 2011/1/14 Max Ivanov ivanov.ma...@gmail.com: Gluster - and in fact most (all?) parallel filesystems are optimized for very large files. That being the case, small files are not retrieved as efficiently, and result in a larger number of

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Joe Landman
On 01/14/2011 09:20 AM, Andrew Latham wrote: We are using it for image based storage with virtualization. What is the definition of large files for Gluster, that is a vague statement. MB size or larger -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics Inc. email:

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Jacob Shucart
For web hosting it is best to put user generated content(images, etc) on Gluster but to leave application files like PHP files on the local disk. This is because a single application file request could result in 20 other file requests since applications like PHP use includes/inherits, etc. This

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Joe Landman
On 01/14/2011 12:58 PM, Jacob Shucart wrote: For web hosting it is best to put user generated content(images, etc) on Gluster but to leave application files like PHP files on the local disk. This is because a single application file request could result in 20 other file requests since

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Rafiq Maniar
Hi, This can be offset by using a PHP bytecode caching system like Xcache or APC. Just make sure you have enough memory allocated. In this way you can have the benefit of a distributed filesystem for your application files while also having them accessed quickly (except for the 1st access).

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Marcus Bointon
On 14 Jan 2011, at 18:58, Jacob Shucart wrote: This kind of thing is fine on local disks, but when you're talking about a distributed filesystem the network latency starts to add up since 1 request to the web server results in a bunch of file requests. I think the main objection is that it

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Max Ivanov
I haven't seen Max's data, so I can't comment on this. Understand that performance is going to be bound by many things. One of many things is the speed of the spinning disk if thats what you use. Another will be network. It is very similair to kernel source tree - tons of small (2-20kb)

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Joe Landman
On 01/14/2011 05:19 PM, Max Ivanov wrote: I haven't seen Max's data, so I can't comment on this. Understand that performance is going to be bound by many things. One of many things is the speed of the spinning disk if thats what you use. Another will be network. It is very similair to

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Marcus Bointon
On 14 Jan 2011, at 23:12, Joe Landman wrote: If most of your file access times are dominated by latency (e.g. small, seeky like loads), and you are going over a gigabit connection, yeah, your performance is going to crater on any cluster file system. Local latency to traverse the storage

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Joe Landman
On 01/14/2011 06:26 PM, Marcus Bointon wrote: Our own experience has been generally that you are IOP constrained because of the stack you have to traverse. If you add more latency into this stack, you have more to traverse, and therefore, you have more you need to wait. Which will have a

Re: [Gluster-users] very bad performance on small files

2011-01-14 Thread Max Ivanov
Sure, and all that applies equally to both NFS and gluster, yet in Max's example NFS was ~50x faster than gluster for an identical small-file workload. So what's gluster doing over and above what NFS is doing that's taking so long, given that network and disk factors are equal? I'd buy a