Hi Sage,
I know about the setattr syntax from
https://github.com/ceph/ceph/blob/master/qa/workunits/fs/misc/layout_vxattrs.sh
=
setfattr -n ceph.dir.layout.pool -v data dir
setfattr -n ceph.dir.layout.pool -v 2 dir
But, in my case it is not working:
[root@rx37-1 ~]# setfattr -n ceph.dir.layout.pool -v 3 /mnt/cephfs/ssd-r2
setfattr: /mnt/cephfs/ssd-r2: Invalid argument
[root@rx37-1 ~]# setfattr -n ceph.dir.layout.pool -v SSD-r2 /mnt/cephfs/ssd-r2
setfattr: /mnt/cephfs/ssd-r2: Invalid argument
[root@rx37-1 ~]# strace setfattr -n ceph.dir.layout.pool -v SSD-r2
/mnt/cephfs/ssd-r2
(...)
setxattr("/mnt/cephfs/ssd-r2", "ceph.dir.layout.pool", "SSD-r2", 6, 0) = -1
EINVAL (Invalid argument)
Same with ceph-fuse:
[root@rx37-1 ~]# strace setfattr -n ceph.dir.layout.pool -v SSD-r2
/mnt/ceph-fuse/ssd-r2
(...)
setxattr("/mnt/ceph-fuse/ssd-r2", "ceph.dir.layout.pool", "SSD-r2", 6, 0) = -1
EINVAL (Invalid argument)
Setting all layout attribute at once does not work either:
[root@rx37-1 cephfs]# setfattr -n ceph.dir.layout -v "stripe_unit=2097152
stripe_count=1 object_size=4194304 pool=SSD-r2" /mnt/cephfs/ssd-r2
setfattr: /mnt/cephfs/ssd-r2: Invalid argument
How can I debug this further ?
It seems the Directory has no "layout" at all:
# getfattr -d -m - /mnt/cephfs/ssd-r2
# file: ssd-r2
ceph.dir.entries="0"
ceph.dir.files="0"
ceph.dir.rbytes="0"
ceph.dir.rctime="0.090"
ceph.dir.rentries="1"
ceph.dir.rfiles="0"
ceph.dir.rsubdirs="1"
ceph.dir.subdirs="0"
Kind Regards,
-Dieter
On Mon, Aug 18, 2014 at 09:37:39PM +0200, Sage Weil wrote:
> Hi Dieter,
>
> There is a new xattr based interface. See
>
>
> https://github.com/ceph/ceph/blob/master/qa/workunits/fs/misc/layout_vxattrs.sh
>
> The nice part about this interface is no new tools are necessary (just
> standard 'attr' or 'setfattr' commands) and it is the same with both
> ceph-fuse and the kernel client.
>
> sage
>
>
> On Mon, 18 Aug 2014, Kasper Dieter wrote:
>
> > Hi Sage,
> >
> > a couple of months ago (maybe last year) I was able to change the
> > assignment of Directorlies and Files of CephFS to different pools
> > back and forth (with cephfs set_layout as well as with setfattr).
> >
> > Now (with ceph v0.81 and Kernel 3.10 an the client side)
> > neither 'cephfs set_layout' nor 'setfattr' works anymore:
> >
> > # mount | grep ceph
> > ceph-fuse on /mnt/ceph-fuse type fuse.ceph-fuse
> > (rw,nosuid,nodev,allow_other,default_permissions)
> > 192.168.113.52:6789:/ on /mnt/cephfs type ceph (name=admin,key=client.admin)
> >
> > # ls -l /mnt/cephfs
> > total 0
> > -rw-r--r-- 1 root root 0 Aug 18 21:06 file
> > -rw-r--r-- 1 root root 0 Aug 18 21:10 file2
> > -rw-r--r-- 1 root root 0 Aug 18 21:11 file3
> > drwxr-xr-x 1 root root 0 Aug 18 20:54 sas-r2
> > drwxr-xr-x 1 root root 0 Aug 18 20:54 ssd-r2
> >
> > # getfattr -d -m - /mnt/cephfs
> > getfattr: Removing leading '/' from absolute path names
> > # file: mnt/cephfs
> > ceph.dir.entries="5"
> > ceph.dir.files="3"
> > ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304
> > pool=SAS-r2"
> > ceph.dir.rbytes="0"
> > ceph.dir.rctime="0.090"
> > ceph.dir.rentries="1"
> > ceph.dir.rfiles="0"
> > ceph.dir.rsubdirs="1"
> > ceph.dir.subdirs="2"
> >
> > # setfattr -n ceph.dir.layout.pool -v SSD-r2 /mnt/cephfs
> > setfattr: /mnt/cephfs: Invalid argument
> >
> > # ceph osd dump | grep pool
> > pool 0 'data' replicated size 3 min_size 2 crush_ruleset 0 object_hash
> > rjenkins pg_num 8064 pgp_num 8064 last_change 1 flags hashpspool
> > crash_replay_interval 45 stripe_width 0
> > pool 1 'metadata' replicated size 3 min_size 2 crush_ruleset 0 object_hash
> > rjenkins pg_num 8064 pgp_num 8064 last_change 1 flags hashpspool
> > stripe_width 0
> > pool 2 'rbd' replicated size 3 min_size 2 crush_ruleset 0 object_hash
> > rjenkins pg_num 8064 pgp_num 8064 last_change 1 flags hashpspool
> > stripe_width 0
> > pool 3 'SSD-r2' replicated size 2 min_size 2 crush_ruleset 3 object_hash
> > rjenkins pg_num 6000 pgp_num 6000 last_change 404 flags hashpspool
> > stripe_width 0
> > pool 4 'SAS-r2' replicated size 2 min_size 2 crush_ruleset 4 object_hash
> > rjenkins pg_num 6000 pgp_num 6000 last_change 408 flags hashpspool
> > stripe_width 0
> > pool 5 'SSD-r3' replicated size 3 min_size 2 crush_ruleset 3 object_hash
> > rjenkins pg_num 4000 pgp_num 4000 last_change 413 flags hashpspool
> > stripe_width 0
> > pool 6 'SAS-r3' replicated size 3 min_size 2 crush_ruleset 4 object_hash
> > rjenkins pg_num 4000 pgp_num 4000 last_change 416 flags hashpspool
> > stripe_width 0
> >
> > # getfattr -d -m - /mnt/cephfs/ssd-r2
> > getfattr: Removing leading '/' from absolute path names
> > # file: mnt/cephfs/ssd-r2
> > ceph.dir.entries="0"
> > ceph.dir.files="0"
> > ceph.dir.rbytes="0"
> > ceph.dir.rctime="0.090"
> > ceph.dir.rentries="1"
> > ceph.dir.rfiles="0"
> > ceph.dir.rsubdirs="1"
> >
> > # setfattr -n ceph.dir.layout.pool -v SSD-r2 /mnt/cephfs/ssd-r2
> > setfattr: /mnt/cephfs/ssd-r2: Invalid argument
> >
> > # cephfs /mnt/cephfs/ssd-r2 set_layout -p 3 -s 4194304 -u 4194304 -c 1
> > Error setting layout: (22) Invalid argument
> >
> >
> > Any recommendations ?
> > Is this a bug, or a new feature ?
> > Do I have to use a newer Kernel ?
> >
> >
> > Kind Regards,
> > -Dieter
> >
> >
> >
> > On Sat, Aug 31, 2013 at 02:26:48AM +0200, Sage Weil wrote:
> > > On Fri, 30 Aug 2013, Joao Pedras wrote:
> > > >
> > > > Greetings all!
> > > >
> > > > I am bumping into a small issue and I am wondering if someone has any
> > > > insight on it.
> > > >
> > > > I am trying to use a pool other than 'data' for cephfs. Said pool has
> > > > id #3
> > > > and I have run 'ceph mds add_data_pool 3'.
> > > >
> > > > After mounting cephfs seg faults when trying to set the layout:
> > > >
> > > > $> cephfs /path set_layout -p 3
> > > >
> > > > Segmentation fault
> > > >
> > > > Actually plainly running 'cephfs /path set_layout' without more options
> > > > will
> > > > seg fault as well.
> > > >
> > > > Version is 0.61.8 on ubuntu 12.04.
> > > >
> > > > A question that comes to mind here is if there is a way of accomplishing
> > > > this when using ceph-fuse (<3.x kernels).
> > >
> > > You can adjust this more easily using the xattr interface:
> > >
> > > getfattr -n ceph.dir.layout <dir>
> > > setfattr -n ceph.dir.layout.pool -v mypool
> > > getfattr -n ceph.dir.layout <dir>
> > >
> > > The interface tests are probably a decent reference given this isn't
> > > explicitly documented anywhere:
> > >
> > >
> > > https://github.com/ceph/ceph/blob/master/qa/workunits/misc/layout_vxattrs.sh
> > >
> > > sage
> > > _______________________________________________
> > > ceph-users mailing list
> > > ceph-users@lists.ceph.com
> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> >
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
