On Sun, 2019-07-28 at 18:20 +0000, Lee Norvall wrote: > Update to this I found that you cannot create a 2nd files system as yet and > it is still experimental. So I went down this route: > > Added a pool to the existing cephfs and then setfattr -n ceph.dir.layout.pool > -v SSD-NFS /mnt/cephfs/ssdnfs/ from a ceph-fuse client. > > I then nfs mounted from another box. I can see the files and dir etc from the > nfs client but my issue now is that I do not have permission to write, create > dir etc. The same goes for the default setup after running the ansible > playbook even when setting export to no_root_squash. I am missing a chain of > permission? ganesha-nfs is using admin userid, is this the same as the > client.admin or is this a user I need to create? Any info appreciated. > > Ceph is on CentOS 7 and SELinux is currently off as well. > > Copy of the ganesha conf below. Is secType correct or is it missing > something? > > RADOS_URLS { > ceph_conf = '/etc/ceph/ceph.conf'; > userid = "admin"; > } > %url rados://cephfs_data/ganesha-export-index > > NFSv4 { > RecoveryBackend = 'rados_kv'; > }
I your earlier email, you mentioned that you had more than one NFS server, but rados_kv is not safe in a multi-server configuration. The servers will be competing to store recovery information in the same objects, and won't honor each others' grace periods/ You may want to explore using "RecoveryBackend = rados_cluster" instead, which should handle that situation better. See this writeup, for some guidelines: https://jtlayton.wordpress.com/2018/12/10/deploying-an-active-active-nfs-cluster-over-cephfs/ Much of this is already automated too if you use k8s+rook. > RADOS_KV { > ceph_conf = '/etc/ceph/ceph.conf'; > userid = "admin"; > pool = "cephfs_data"; > } > > EXPORT > { > Export_id=20133; > Path = "/"; > Pseudo = /cephfile; > Access_Type = RW; > Protocols = 3,4; > Transports = TCP; > SecType = sys,krb5,krb5i,krb5p; > Squash = Root_Squash; > Attr_Expiration_Time = 0; > > FSAL { > Name = CEPH; > User_Id = "admin"; > } > > > } > > > > > > > > > > > On 28/07/2019 12:11, Lee Norvall wrote: > > Hi > > > > I am using ceph-ansible to deploy and just looking for best way/tips on > > how to export multiple pools/fs. > > > > Ceph: nautilus (14.2.2) > > NFS-Ganesha v 2.8 > > ceph-ansible stable 4.0 > > > > I have 3 x osd/NFS gateways running and NFS on the dashboard can see > > them in the cluster. I have managed to export for cephfs / and mounted > > it on another box. > > > > 1) can I add a new pool/fs to the export under that same NFS gateway > > cluster, or > > > > 2) do I have the to do something like add a new pool to the fs and then > > setfattr to make the layout /newfs_dir point to /new_pool? does this > > cause issues and false object count? > > > > 3) any other better ways... > > > > Rgds > > > > Lee > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > -- > > > Lee Norvall | CEO / Founder > Mob. +44 (0)7768 201884 > Tel. +44 (0)20 3026 8930 > Web. www.blocz.io > > Enterprise Cloud | Private Cloud | Hybrid/Multi Cloud | Cloud Backup > > > > This e-mail (and any attachment) has been sent from a PC belonging to My Mind > (Holdings) Limited. If you receive it in error, please tell us by return and > then delete it from your system; you may not rely on its contents nor > copy/disclose it to anyone. Opinions, conclusions and statements of intent in > this e-mail are those of the sender and will not bind My Mind (Holdings) > Limited unless confirmed by an authorised representative independently of > this message. We do not accept responsibility for viruses; you must scan for > these. Please note that e-mails sent to and from blocz IO Limited are > routinely monitored for record keeping, quality control and training > purposes, to ensure regulatory compliance and to prevent viruses and > unauthorised use of our computer systems. My Mind (Holdings) Limited is > registered in England & Wales under company number 10186410. Registered > office: 1st Floor Offices, 2a Highfield Road, Ringwood, Hampshire, United > Kingdom, BH24 1RQ. VAT Registration GB 244 9628 77 > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Jeff Layton <jlay...@poochiereds.net> _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com