Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-15 Thread Liam Slusser
Our application also stores the path of the file in a database. Accessing a file directly is normally pretty speedy. However, to get the files into the database required searching parts of the filesystem which was really slow. We also had users using the filesystem fixing things which was all un

Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-15 Thread Franco Broi
On 15 Apr 2014 18:15, Terada Michitaka wrote: > > >> To Liam: > > >I had about 100 million files in Gluster and it was unbelievably painfully > >slow. We had to ditch it for other technology. > > Has slow down occurred on writing file?, listing files, or both? > > In our application, path of th

Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-15 Thread Terada Michitaka
>> To Liam: >I had about 100 million files in Gluster and it was unbelievably painfully slow. We had to ditch it for other technology. Has slow down occurred on writing file?, listing files, or both? In our application, path of the data is managed in database. "ls" is slow, but not influence to

Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-15 Thread Franco Broi
My bug report is here https://bugzilla.redhat.com/show_bug.cgi?id=1067256 On Mon, 2014-04-14 at 23:51 -0700, Joe Julian wrote: > If you experience pain using any filesystem, you should see your > doctor. > > If you're not actually experiencing pain, perhaps you should avoid > hyperbole and inst

Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-14 Thread Joe Julian
If you experience pain using any filesystem, you should see your doctor. If you're not actually experiencing pain, perhaps you should avoid hyperbole and instead talk about what version you tried, what your tests were, how you tried to fix it, and what the results were. If you're using a curr

Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-14 Thread Liam Slusser
We consolidated hardware into a single large ZFS server with a redundant "hot" slave. thanks, liam On Mon, Apr 14, 2014 at 11:33 PM, Jeffrey 'jf' Lim wrote: > On Tue, Apr 15, 2014 at 2:30 PM, Liam Slusser wrote: > > > > I had about 100 million files in Gluster and it was unbelievably > painful

Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-14 Thread Jeffrey 'jf' Lim
On Tue, Apr 15, 2014 at 2:30 PM, Liam Slusser wrote: > > I had about 100 million files in Gluster and it was unbelievably painfully > slow. We had to ditch it for other technology. > and what is (or was) that other technology? -jf -- He who settles on the idea of the intelligent man as a stati

Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-14 Thread Liam Slusser
I had about 100 million files in Gluster and it was unbelievably painfully slow. We had to ditch it for other technology. On Mon, Apr 14, 2014 at 11:24 PM, Franco Broi wrote: > > I seriously doubt this is the right filesystem for you, we have problems > listing directories with a few hundred fi

Re: [Gluster-users] Writing is slow when there are 10 million files.

2014-04-14 Thread Franco Broi
I seriously doubt this is the right filesystem for you, we have problems listing directories with a few hundred files, never mind millions. On Tue, 2014-04-15 at 10:45 +0900, Terada Michitaka wrote: > Dear All, > > > > I have a problem with slow writing when there are 10 million files. > (To

[Gluster-users] Writing is slow when there are 10 million files.

2014-04-14 Thread Terada Michitaka
Dear All, I have a problem with slow writing when there are 10 million files. (Top level directories are 2,500.) I configured GlusterFS distributed cluster(3 nodes). Each node's spec is below. CPU: Xeon E5-2620 (2.00GHz 6 Core) HDD: SATA 7200rpm 4TB*12 (RAID 6) NW: 10GBEth GlusterFS : gluste