On Sun, 2019-07-28 at 18:20 +0000, Lee Norvall wrote:
> Update to this I found that you cannot create a 2nd files system as yet and 
> it is still experimental.  So I went down this route:
> 
> Added a pool to the existing cephfs and then setfattr -n ceph.dir.layout.pool 
> -v SSD-NFS /mnt/cephfs/ssdnfs/ from a ceph-fuse client.
> 
> I then nfs mounted from another box. I can see the files and dir etc from the 
> nfs client but my issue now is that I do not have permission to write, create 
> dir etc.  The same goes for the default setup after running the ansible 
> playbook even when setting export to no_root_squash.  I am missing a chain of 
> permission?  ganesha-nfs is using admin userid, is this the same as the 
> client.admin or is this a user I need to create?  Any info appreciated.
> 
> Ceph is on CentOS 7 and SELinux is currently off as well.
> 
> Copy of the ganesha conf below.  Is secType correct or is it missing 
> something?
> 
> RADOS_URLS {
>    ceph_conf = '/etc/ceph/ceph.conf';
>    userid = "admin";
> }
> %url rados://cephfs_data/ganesha-export-index
> 
> NFSv4 {
>         RecoveryBackend = 'rados_kv';
> }

I your earlier email, you mentioned that you had more than one NFS
server, but rados_kv is not safe in a multi-server configuration. The
servers will be competing to store recovery information in the same
objects, and won't honor each others' grace periods/

You may want to explore using "RecoveryBackend = rados_cluster" instead,
which should handle that situation better. See this writeup, for some
guidelines:

    
https://jtlayton.wordpress.com/2018/12/10/deploying-an-active-active-nfs-cluster-over-cephfs/

Much of this is already automated too if you use k8s+rook.

> RADOS_KV {
>         ceph_conf = '/etc/ceph/ceph.conf';
>         userid = "admin";
>         pool = "cephfs_data";
> }
> 
> EXPORT
> {
>         Export_id=20133;
>         Path = "/";
>         Pseudo = /cephfile;
>         Access_Type = RW;
>         Protocols = 3,4;
>         Transports = TCP;
>         SecType = sys,krb5,krb5i,krb5p;
>         Squash = Root_Squash;
>         Attr_Expiration_Time = 0;
> 
>         FSAL {
>                 Name = CEPH;
>                 User_Id = "admin";
>         }
> 
> 
> }
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On 28/07/2019 12:11, Lee Norvall wrote:
> > Hi
> > 
> > I am using ceph-ansible to deploy and just looking for best way/tips on 
> > how to export multiple pools/fs.
> > 
> > Ceph: nautilus (14.2.2)
> > NFS-Ganesha v 2.8
> > ceph-ansible stable 4.0
> > 
> > I have 3 x osd/NFS gateways running and NFS on the dashboard can see 
> > them in the cluster.  I have managed to export for cephfs / and mounted 
> > it on another box.
> > 
> > 1) can I add a new pool/fs to the export under that same NFS gateway 
> > cluster, or
> > 
> > 2) do I have the to do something like add a new pool to the fs and then 
> > setfattr to make the layout /newfs_dir point to /new_pool?  does this 
> > cause issues and false object count?
> > 
> > 3) any other better ways...
> > 
> > Rgds
> > 
> > Lee
> > 
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
> -- 
>  
> 
> Lee Norvall | CEO / Founder 
> Mob. +44 (0)7768 201884 
> Tel. +44 (0)20 3026 8930 
> Web. www.blocz.io 
> 
> Enterprise Cloud | Private Cloud | Hybrid/Multi Cloud | Cloud Backup 
> 
>                 
> 
> This e-mail (and any attachment) has been sent from a PC belonging to My Mind 
> (Holdings) Limited. If you receive it in error, please tell us by return and 
> then delete it from your system; you may not rely on its contents nor 
> copy/disclose it to anyone. Opinions, conclusions and statements of intent in 
> this e-mail are those of the sender and will not bind My Mind (Holdings) 
> Limited unless confirmed by an authorised representative independently of 
> this message. We do not accept responsibility for viruses; you must scan for 
> these. Please note that e-mails sent to and from blocz IO Limited are 
> routinely monitored for record keeping, quality control and training 
> purposes, to ensure regulatory compliance and to prevent viruses and 
> unauthorised use of our computer systems. My Mind (Holdings) Limited is 
> registered in England & Wales under company number 10186410. Registered 
> office: 1st Floor Offices, 2a Highfield Road, Ringwood, Hampshire, United 
> Kingdom, BH24 1RQ. VAT Registration GB 244 
 9628 77
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

-- 
Jeff Layton <jlay...@poochiereds.net>

_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to