Re: [ceph-users] cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000

2017-09-11 Thread Yan, Zheng 
looks like http://tracker.ceph.com/issues/18314.


please try:

run "ceph fs set cephfs1 allow_dirfrags 1”

and

set mds config mds_bal_frag to 1, set mds_bal_split_size to 5000, and set 
mds_bal_fragment_size_max 5.



> On 11 Sep 2017, at 15:46, donglifec...@gmail.com wrote:
> 
> ZhengYan,
> 
> kernel client is 4.12.0.
> 
> [root@yj43959-ceph-dev ~]# uname -a
> Linux yj43959-ceph-dev.novalocal 4.12.0-1.el7.elrepo.x86_64 #1 SMP Sun Jul 2 
> 20:38:48 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
> 
> Thanks a lot.
> 
> donglifec...@gmail.com
>  
> From: Yan, Zheng
> Date: 2017-09-11 15:24
> To: donglifec...@gmail.com
> CC: ceph-users
> Subject: Re: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file 
> when one dir more than 10 files, mds_bal_fragment_size_max = 500
>  
> > On 11 Sep 2017, at 14:07, donglifec...@gmail.com wrote:
> >
> > ZhengYan,
> >
> > I set "mds_bal_fragment_size_max = 10, mds_bal_frag = true", then  I 
> > write  10 files named 512k.file$i, but there are still some file is 
> > missing. such as :
> > [root@yj43959-ceph-dev cephfs]# find ./volumes/ -type f | wc -l
> > 91070
>  
> which client do you use? kernel client or fuse client and version?
>  
> Yan, Zheng
>  
> >
> > 
> >
> > donglifec...@gmail.com
> >  
> > From: Yan, Zheng
> > Date: 2017-09-08 15:20
> > To: donglifec...@gmail.com
> > CC: ceph-users; marcus.haarmann
> > Subject: Re: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file 
> > when one dir more than 10 files, mds_bal_fragment_size_max = 500
> >  
> > > On 8 Sep 2017, at 13:54, donglifec...@gmail.com wrote:
> > >
> > > ZhengYan,
> > >
> > > I'm sorry, just a description of some questions.
> > >
> > > when one dir more than 10 files, I can continue to write it , but I 
> > > don't find file which wrote in the past. for example:
> > > 1.  I write  10 files named 512k.file$i
> > >
> > > 2. I continue to write  1 files named aaa.file$i
> > >
> > > 3. I continue to write  1 files named bbb.file$i
> > >
> > > 4.  I continue to write  1 files named ccc.file$i
> > >
> > > 5. I continue to write  1 files named ddd.file$i
> > >
> > > 6. I can't find all ddd.file$i, some ddd.file$i missing. such as:
> > >
> > > [root@yj43959-ceph-dev scripts]# find /mnt/cephfs/volumes -type f  |  
> > > grep 512k.file | wc -l
> > > 10
> > > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/aaa.file* | wc -l
> > > 1
> > > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/bbb.file* | wc -l
> > > 1
> > > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ccc.file* | wc -l
> > > 1
> > > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ddd.file* | wc -l 
> > >// some files missing
> > > 1072
> >  
> > It’s likely caused by http://tracker.ceph.com/issues/18314.  To support 
> > very large directory, you should enable directory fragment instead of 
> > enlarge mds_bal_fragment_size_max.
> >  
> > Regards
> > Yan, Zheng
> >  
> > >
> > > 
> > >
> > > donglifec...@gmail.com
> > >  
> > > From: donglifec...@gmail.com
> > > Date: 2017-09-08 13:30
> > > To: zyan
> > > CC: ceph-users
> > > Subject: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file 
> > > when one dir more than 10 files, mds_bal_fragment_size_max = 500
> > > ZhengYan,
> > >
> > > I test cephfs(Kraken 11.2.1),  I can't write more files when one dir more 
> > > than 10 files, I have already set up "mds_bal_fragment_size_max = 
> > > 500".
> > >
> > > why is this case? Is it a bug?
> > >
> > > Thanks a lot.
> > >
> > > donglifec...@gmail.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000

2017-09-11 Thread donglifec...@gmail.com 
ZhengYan,

kernel client is 4.12.0.

[root@yj43959-ceph-dev ~]# uname -a
Linux yj43959-ceph-dev.novalocal 4.12.0-1.el7.elrepo.x86_64 #1 SMP Sun Jul 2 
20:38:48 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

Thanks a lot.



donglifec...@gmail.com
 
From: Yan, Zheng
Date: 2017-09-11 15:24
To: donglifec...@gmail.com
CC: ceph-users
Subject: Re: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file when 
one dir more than 10 files, mds_bal_fragment_size_max = 500
 
> On 11 Sep 2017, at 14:07, donglifec...@gmail.com wrote:
> 
> ZhengYan,
> 
> I set "mds_bal_fragment_size_max = 10, mds_bal_frag = true", then  I 
> write  10 files named 512k.file$i, but there are still some file is 
> missing. such as :
> [root@yj43959-ceph-dev cephfs]# find ./volumes/ -type f | wc -l
> 91070
 
which client do you use? kernel client or fuse client and version?
 
Yan, Zheng
 
> 
> 
> 
> donglifec...@gmail.com
>  
> From: Yan, Zheng
> Date: 2017-09-08 15:20
> To: donglifec...@gmail.com
> CC: ceph-users; marcus.haarmann
> Subject: Re: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file 
> when one dir more than 10 files, mds_bal_fragment_size_max = 500
>  
> > On 8 Sep 2017, at 13:54, donglifec...@gmail.com wrote:
> >
> > ZhengYan,
> >
> > I'm sorry, just a description of some questions.
> >
> > when one dir more than 10 files, I can continue to write it , but I 
> > don't find file which wrote in the past. for example:
> > 1.  I write  10 files named 512k.file$i
> >
> > 2. I continue to write  1 files named aaa.file$i
> >
> > 3. I continue to write  1 files named bbb.file$i
> >
> > 4.  I continue to write  1 files named ccc.file$i
> >
> > 5. I continue to write  1 files named ddd.file$i
> >
> > 6. I can't find all ddd.file$i, some ddd.file$i missing. such as:
> >
> > [root@yj43959-ceph-dev scripts]# find /mnt/cephfs/volumes -type f  |  grep 
> > 512k.file | wc -l
> > 10
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/aaa.file* | wc -l
> > 1
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/bbb.file* | wc -l
> > 1
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ccc.file* | wc -l
> > 1
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ddd.file* | wc -l   
> >  // some files missing
> > 1072
>  
> It’s likely caused by http://tracker.ceph.com/issues/18314.  To support very 
> large directory, you should enable directory fragment instead of enlarge 
> mds_bal_fragment_size_max.
>  
> Regards
> Yan, Zheng
>  
> >
> > 
> >
> > donglifec...@gmail.com
> >  
> > From: donglifec...@gmail.com
> > Date: 2017-09-08 13:30
> > To: zyan
> > CC: ceph-users
> > Subject: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file when 
> > one dir more than 10 files, mds_bal_fragment_size_max = 500
> > ZhengYan,
> >
> > I test cephfs(Kraken 11.2.1),  I can't write more files when one dir more 
> > than 10 files, I have already set up "mds_bal_fragment_size_max = 
> > 500".
> >
> > why is this case? Is it a bug?
> >
> > Thanks a lot.
> >
> > donglifec...@gmail.com
 
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000

2017-09-11 Thread Yan, Zheng 

> On 11 Sep 2017, at 14:07, donglifec...@gmail.com wrote:
> 
> ZhengYan,
> 
> I set "mds_bal_fragment_size_max = 10, mds_bal_frag = true", then  I 
> write  10 files named 512k.file$i, but there are still some file is 
> missing. such as :
> [root@yj43959-ceph-dev cephfs]# find ./volumes/ -type f | wc -l
> 91070

which client do you use? kernel client or fuse client and version?

Yan, Zheng

> 
> 
> 
> donglifec...@gmail.com
>  
> From: Yan, Zheng
> Date: 2017-09-08 15:20
> To: donglifec...@gmail.com
> CC: ceph-users; marcus.haarmann
> Subject: Re: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file 
> when one dir more than 10 files, mds_bal_fragment_size_max = 500
>  
> > On 8 Sep 2017, at 13:54, donglifec...@gmail.com wrote:
> >
> > ZhengYan,
> >
> > I'm sorry, just a description of some questions.
> >
> > when one dir more than 10 files, I can continue to write it , but I 
> > don't find file which wrote in the past. for example:
> > 1.  I write  10 files named 512k.file$i
> >
> > 2. I continue to write  1 files named aaa.file$i
> >
> > 3. I continue to write  1 files named bbb.file$i
> >
> > 4.  I continue to write  1 files named ccc.file$i
> >
> > 5. I continue to write  1 files named ddd.file$i
> >
> > 6. I can't find all ddd.file$i, some ddd.file$i missing. such as:
> >
> > [root@yj43959-ceph-dev scripts]# find /mnt/cephfs/volumes -type f  |  grep 
> > 512k.file | wc -l
> > 10
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/aaa.file* | wc -l
> > 1
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/bbb.file* | wc -l
> > 1
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ccc.file* | wc -l
> > 1
> > [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ddd.file* | wc -l   
> >  // some files missing
> > 1072
>  
> It’s likely caused by http://tracker.ceph.com/issues/18314.  To support very 
> large directory, you should enable directory fragment instead of enlarge 
> mds_bal_fragment_size_max.
>  
> Regards
> Yan, Zheng
>  
> >
> > 
> >
> > donglifec...@gmail.com
> >  
> > From: donglifec...@gmail.com
> > Date: 2017-09-08 13:30
> > To: zyan
> > CC: ceph-users
> > Subject: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file when 
> > one dir more than 10 files, mds_bal_fragment_size_max = 500
> > ZhengYan,
> >
> > I test cephfs(Kraken 11.2.1),  I can't write more files when one dir more 
> > than 10 files, I have already set up "mds_bal_fragment_size_max = 
> > 500".
> >
> > why is this case? Is it a bug?
> >
> > Thanks a lot.
> >
> > donglifec...@gmail.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000

2017-09-08 Thread Yan, Zheng 

> On 8 Sep 2017, at 13:54, donglifec...@gmail.com wrote:
> 
> ZhengYan,
> 
> I'm sorry, just a description of some questions.
> 
> when one dir more than 10 files, I can continue to write it , but I don't 
> find file which wrote in the past. for example:
> 1.  I write  10 files named 512k.file$i
>
> 2. I continue to write  1 files named aaa.file$i
> 
> 3. I continue to write  1 files named bbb.file$i
> 
> 4.  I continue to write  1 files named ccc.file$i
> 
> 5. I continue to write  1 files named ddd.file$i
> 
> 6. I can't find all ddd.file$i, some ddd.file$i missing. such as:
> 
> [root@yj43959-ceph-dev scripts]# find /mnt/cephfs/volumes -type f  |  grep 
> 512k.file | wc -l
> 10
> [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/aaa.file* | wc -l
> 1
> [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/bbb.file* | wc -l
> 1
> [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ccc.file* | wc -l
> 1
> [root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ddd.file* | wc -l
> // some files missing
> 1072

It’s likely caused by http://tracker.ceph.com/issues/18314.  To support very 
large directory, you should enable directory fragment instead of enlarge 
mds_bal_fragment_size_max.

Regards
Yan, Zheng

> 
> 
> 
> donglifec...@gmail.com
>  
> From: donglifec...@gmail.com
> Date: 2017-09-08 13:30
> To: zyan
> CC: ceph-users
> Subject: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file when 
> one dir more than 10 files, mds_bal_fragment_size_max = 500
> ZhengYan,
> 
> I test cephfs(Kraken 11.2.1),  I can't write more files when one dir more 
> than 10 files, I have already set up "mds_bal_fragment_size_max = 
> 500".
> 
> why is this case? Is it a bug?
> 
> Thanks a lot.
> 
> donglifec...@gmail.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000

2017-09-07 Thread donglifec...@gmail.com 
ZhengYan,

I'm sorry, just a description of some questions.

when one dir more than 10 files, I can continue to write it , but I don't 
find file which wrote in the past. for example:
1.  I write  10 files named 512k.file$i
   
2. I continue to write  1 files named aaa.file$i

3. I continue to write  1 files named bbb.file$i

4.  I continue to write  1 files named ccc.file$i

5. I continue to write  1 files named ddd.file$i

6. I can't find all ddd.file$i, some ddd.file$i missing. such as:

[root@yj43959-ceph-dev scripts]# find /mnt/cephfs/volumes -type f  |  grep 
512k.file | wc -l
10
[root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/aaa.file* | wc -l
1
[root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/bbb.file* | wc -l
1
[root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ccc.file* | wc -l
1
[root@yj43959-ceph-dev scripts]# ls /mnt/cephfs/volumes/ddd.file* | wc -l// 
some files missing
1072




donglifec...@gmail.com
 
From: donglifec...@gmail.com
Date: 2017-09-08 13:30
To: zyan
CC: ceph-users
Subject: [ceph-users]cephfs(Kraken 11.2.1), Unable to write more file when one 
dir more than 10 files, mds_bal_fragment_size_max = 500
ZhengYan,

I test cephfs(Kraken 11.2.1),  I can't write more files when one dir more than 
10 files, I have already set up "mds_bal_fragment_size_max = 500".

why is this case? Is it a bug?

Thanks a lot.



donglifec...@gmail.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] cephfs(Kraken 11.2.1), Unable to write more file when one dir more than 100000 files, mds_bal_fragment_size_max = 5000000

2017-09-07 Thread Marcus Haarmann 
Its a feature ... 

http://docs.ceph.com/docs/master/cephfs/dirfrags/ 
https://www.spinics.net/lists/ceph-users/msg31473.html 

Marcus Haarmann 


Von: donglifec...@gmail.com 
An: "zyan"  
CC: "ceph-users"  
Gesendet: Freitag, 8. September 2017 07:30:53 
Betreff: [ceph-users] cephfs(Kraken 11.2.1), Unable to write more file when one 
dir more than 10 files, mds_bal_fragment_size_max = 500 

ZhengYan, 

I test cephfs( Kraken 11.2.1), I can't write more files when one dir more than 
10 files, I have already set up " mds_bal_fragment_size_max = 500". 

why is this case? Is it a bug? 

Thanks a lot. 


donglifec...@gmail.com 

___ 
ceph-users mailing list 
ceph-users@lists.ceph.com 
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com 
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com