subject:"\[ceph\-users\] State of nfs\-ganesha CEPH fsal"

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-10-28 Thread Dennis Kramer (DT)

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Sorry for raising this topic from the dead, but i'm having the same
issues with NFS-GANESHA /w the wrong user/group information.

Do you maybe have a working ganesha.conf? I'm assuming I might
mis-configured something in this file. It's also nice to have some
reference config file from a working FSAL CEPH, the sample config is
very minimalistic.

I also have another issue with files that are not immediately visible
in a NFS folder after another system (using the same NFS) has created
it. There seems to be a slight delay before all system have the same
directory listing. This can be enforced by creating a *new* file in
this directory which will cause a refresh on this folder. Changing
directories also helps on affected system(s).

On 07/28/2015 11:30 AM, Haomai Wang wrote:
> On Tue, Jul 28, 2015 at 5:28 PM, Burkhard Linke 
>  wrote:
>> Hi,
>> 
>> On 07/28/2015 11:08 AM, Haomai Wang wrote:
>>> 
>>> On Tue, Jul 28, 2015 at 4:47 PM, Gregory Farnum
>>>  wrote:
 
 On Tue, Jul 28, 2015 at 8:01 AM, Burkhard Linke 
  wrote:
>> 
>> 
>> *snipsnap*
> 
> Can you give some details on that issues? I'm currently
> looking for a way to provide NFS based access to CephFS to
> our desktop machines.
 
 Ummm...sadly I can't; we don't appear to have any tracker
 tickets and I'm not sure where the report went to. :( I think
 it was from Haomai...
>>> 
>>> My fault, I should report this to ticket.
>>> 
>>> I have forgotten the details about the problem, I submit the
>>> infos to IRC :-(
>>> 
>>> It related to the "ls" output. It will print the wrong
>>> user/group owner as "-1", maybe related to root squash?
>> 
>> Are you sure this problem is related to the CephFS FSAL? I also
>> had a hard time setting up ganesha correctly, especially with
>> respect to user and group mappings, especially with a kerberized
>> setup.
>> 
>> I'm currently running a small test setup with one server and one
>> client to single out the last kerberos related problems
>> (nfs-ganesha 2.2.0 / Ceph Hammer 0.94.2 / Ubuntu 14.04).
>> User/group listings have been OK so far. Do you remember whether
>> the problem occurs every time or just arbitrarily?
>> 
> 
> Great!
> 
> I'm not sure the reason. I guess it may related to nfs-ganesha
> version or client distro version.
> 
>> Best regards, Burkhard 
>> ___ ceph-users
>> mailing list ceph-users@lists.ceph.com 
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
> 
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.20 (GNU/Linux)

iEYEARECAAYFAlYw1vgACgkQiJDTKUBxIRsrMACggkb1IZw7od43s9AFUMznwP6M
hW4AoJf2O11uM0F20TQwFJKPt76YcwhW
=PKLQ
-END PGP SIGNATURE-
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-10-28 Thread Lincoln Bryant

Hi Dennis,

We're using NFS Ganesha here as well. I can send you my configuration which is 
working but we squash users and groups down to a particular uid/gid, so it may 
not be super helpful for you.

I think files not being immediately visible is working as intended, due to 
directory caching. I _believe_ what you need to do is set the following 
(comments shamelessly stolen from the Gluster FSAL):
# If thuis flag is set to yes, a getattr is performed each time a readdir 
is done
# if mtime do not match, the directory is renewed. This will make the cache 
more
# synchronous to the FSAL, but will strongly decrease the directory cache 
performance
Use_Getattr_Directory_Invalidation = true;

Hope that helps.

Thanks,
Lincoln

> On Oct 28, 2015, at 9:08 AM, Dennis Kramer (DT)  wrote:
> 
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Sorry for raising this topic from the dead, but i'm having the same
> issues with NFS-GANESHA /w the wrong user/group information.
> 
> Do you maybe have a working ganesha.conf? I'm assuming I might
> mis-configured something in this file. It's also nice to have some
> reference config file from a working FSAL CEPH, the sample config is
> very minimalistic.
> 
> I also have another issue with files that are not immediately visible
> in a NFS folder after another system (using the same NFS) has created
> it. There seems to be a slight delay before all system have the same
> directory listing. This can be enforced by creating a *new* file in
> this directory which will cause a refresh on this folder. Changing
> directories also helps on affected system(s).
> 
> On 07/28/2015 11:30 AM, Haomai Wang wrote:
>> On Tue, Jul 28, 2015 at 5:28 PM, Burkhard Linke 
>>  wrote:
>>> Hi,
>>> 
>>> On 07/28/2015 11:08 AM, Haomai Wang wrote:
 
 On Tue, Jul 28, 2015 at 4:47 PM, Gregory Farnum
  wrote:
> 
> On Tue, Jul 28, 2015 at 8:01 AM, Burkhard Linke 
>  wrote:
>>> 
>>> 
>>> *snipsnap*
>> 
>> Can you give some details on that issues? I'm currently
>> looking for a way to provide NFS based access to CephFS to
>> our desktop machines.
> 
> Ummm...sadly I can't; we don't appear to have any tracker
> tickets and I'm not sure where the report went to. :( I think
> it was from Haomai...
 
 My fault, I should report this to ticket.
 
 I have forgotten the details about the problem, I submit the
 infos to IRC :-(
 
 It related to the "ls" output. It will print the wrong
 user/group owner as "-1", maybe related to root squash?
>>> 
>>> Are you sure this problem is related to the CephFS FSAL? I also
>>> had a hard time setting up ganesha correctly, especially with
>>> respect to user and group mappings, especially with a kerberized
>>> setup.
>>> 
>>> I'm currently running a small test setup with one server and one
>>> client to single out the last kerberos related problems
>>> (nfs-ganesha 2.2.0 / Ceph Hammer 0.94.2 / Ubuntu 14.04).
>>> User/group listings have been OK so far. Do you remember whether
>>> the problem occurs every time or just arbitrarily?
>>> 
>> 
>> Great!
>> 
>> I'm not sure the reason. I guess it may related to nfs-ganesha
>> version or client distro version.
>> 
>>> Best regards, Burkhard 
>>> ___ ceph-users
>>> mailing list ceph-users@lists.ceph.com 
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> 
>> 
>> 
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.20 (GNU/Linux)
> 
> iEYEARECAAYFAlYw1vgACgkQiJDTKUBxIRsrMACggkb1IZw7od43s9AFUMznwP6M
> hW4AoJf2O11uM0F20TQwFJKPt76YcwhW
> =PKLQ
> -END PGP SIGNATURE-
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-10-28 Thread Burkhard Linke


Hi,

On 10/28/2015 03:08 PM, Dennis Kramer (DT) wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Sorry for raising this topic from the dead, but i'm having the same
issues with NFS-GANESHA /w the wrong user/group information.

Do you maybe have a working ganesha.conf? I'm assuming I might
mis-configured something in this file. It's also nice to have some
reference config file from a working FSAL CEPH, the sample config is
very minimalistic.

I also have another issue with files that are not immediately visible
in a NFS folder after another system (using the same NFS) has created
it. There seems to be a slight delay before all system have the same
directory listing. This can be enforced by creating a *new* file in
this directory which will cause a refresh on this folder. Changing
directories also helps on affected system(s).


I've been testing ganesha with a kerberos setup as alternative to 
kernel-nfs and re-exporting a ceph/ceph-fuse mountpoint (side note: 
ceph-fuse and kernel-nfs do not play well, use kernel cephfs in this 
case...)


The ganesha.conf I've used looks like this:

NFS_KRB5
{
PrincipalName = "nfs";
KeytabPath = /etc/krb5.keytab ;
Active_krb5 = true ;
}

NFSv4
{
# Set an alternative path for libnfsidmap configuration file
IdmapConf = /etc/idmapd.conf;
}

NFS_CORE_PARAM {
NFS_Protocols = 4;
}

EXPORT_DEFAULT {
Protocols = 4;
Transports = TCP;
SecType = "krb5p";
}

EXPORT {
Export_ID = 2;
Path = "/ceph_subdiretory_to_mount";
Pseudo = "/exported_name_of_the_subdirectory";
SecType = "krb5p";

FSAL {
Name = CEPH;
}
CLIENT {
Clients = ;
Access_Type = RW;
}
}

On the testclient I've mounted it with

mount.nfs :/exported_name_ /mnt -o 
rw,noatime,fsc,nfsvers=4,intr,ac,sec=krb5p


Accessing files work as expected:

$ ls /mnt
-bash: cd: /mnt: Permission denied
$ klist
klist: Credentials cache file '/tmp/krb5cc_XYZ' not found
$ kinit
Password for XYZ@XYZ:
$ klist
< ticket details >
$ ls /mnt
< directory content >

The difficult part is setting up kerberos correctly (keytab, id mapping 
etc.). It took me some time to figure it out. You need a very recent 
version of ganesha (I'm using 2.1.0). And you should test the setup 
before trying to use the ceph fsal, e.g. with a local directory:


EXPORT
{
 Export_ID = 3;
 Path = "/opt";
 Pseudo = "/test";
 SecType = "krb5p";
 FSAL {
Name = VFS;
 }
 CLIENT {
Clients = ;
Access_Type = RW;
 }
}

(different Export_ID and pseudo are mandatory!)

No tests with root squash so far, but at least the kerberos part is working.

Regards,
Burkhard
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-07-28 Thread Burkhard Linke


Hi,

On 07/28/2015 11:08 AM, Haomai Wang wrote:

On Tue, Jul 28, 2015 at 4:47 PM, Gregory Farnum g...@gregs42.com wrote:

On Tue, Jul 28, 2015 at 8:01 AM, Burkhard Linke
burkhard.li...@computational.bio.uni-giessen.de wrote:


*snipsnap*
Can you give some details on that issues? I'm currently looking for 
a way to provide NFS based access to CephFS to our desktop machines. 

Ummm...sadly I can't; we don't appear to have any tracker tickets and
I'm not sure where the report went to. :( I think it was from
Haomai...

My fault, I should report this to ticket.

I have forgotten the details about the problem, I submit the infos to IRC :-(

It related to the ls output. It will print the wrong user/group
owner as -1, maybe related to root squash?
Are you sure this problem is related to the CephFS FSAL? I also had a 
hard time setting up ganesha correctly, especially with respect to user 
and group mappings, especially with a kerberized setup.


I'm currently running a small test setup with one server and one client 
to single out the last kerberos related problems (nfs-ganesha 2.2.0 / 
Ceph Hammer 0.94.2 / Ubuntu 14.04). User/group listings have been OK so 
far. Do you remember whether the problem occurs every time or just 
arbitrarily?


Best regards,
Burkhard
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-07-28 Thread Burkhard Linke


Hi,

On 07/27/2015 05:42 PM, Gregory Farnum wrote:

On Mon, Jul 27, 2015 at 4:33 PM, Burkhard Linke
burkhard.li...@computational.bio.uni-giessen.de wrote:

Hi,

the nfs-ganesha documentation states:

... This FSAL links to a modified version of the CEPH library that has been
extended to expose its distributed cluster and replication facilities to the
pNFS operations in the FSAL. ... The CEPH library modifications have not
been merged into the upstream yet. 

(https://github.com/nfs-ganesha/nfs-ganesha/wiki/Fsalsupport#ceph)

Is this still the case with the hammer release?

The FSAL has been upstream for quite a while, but it's not part of our
regular testing yet and I'm not sure what it gets from the Ganesha
side. I'd encourage you to test it, but be wary — we had a recent
report of some issues we haven't been able to set up to reproduce yet.
Can you give some details on that issues? I'm currently looking for a 
way to provide NFS based access to CephFS to our desktop machines.


The kernel NFS implementation in Ubuntu had some problems with CephFS in 
our setup, which I was not able to resolve yet. Ganesha seems to be more 
promising, since it uses libcephfs directly and does not need a 
mountpoint of its own.


Best regards,
Burkhard
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-07-28 Thread Haomai Wang

On Tue, Jul 28, 2015 at 5:28 PM, Burkhard Linke
burkhard.li...@computational.bio.uni-giessen.de wrote:
 Hi,

 On 07/28/2015 11:08 AM, Haomai Wang wrote:

 On Tue, Jul 28, 2015 at 4:47 PM, Gregory Farnum g...@gregs42.com wrote:

 On Tue, Jul 28, 2015 at 8:01 AM, Burkhard Linke
 burkhard.li...@computational.bio.uni-giessen.de wrote:


 *snipsnap*

 Can you give some details on that issues? I'm currently looking for a
 way to provide NFS based access to CephFS to our desktop machines.

 Ummm...sadly I can't; we don't appear to have any tracker tickets and
 I'm not sure where the report went to. :( I think it was from
 Haomai...

 My fault, I should report this to ticket.

 I have forgotten the details about the problem, I submit the infos to IRC
 :-(

 It related to the ls output. It will print the wrong user/group
 owner as -1, maybe related to root squash?

 Are you sure this problem is related to the CephFS FSAL? I also had a hard
 time setting up ganesha correctly, especially with respect to user and group
 mappings, especially with a kerberized setup.

 I'm currently running a small test setup with one server and one client to
 single out the last kerberos related problems (nfs-ganesha 2.2.0 / Ceph
 Hammer 0.94.2 / Ubuntu 14.04). User/group listings have been OK so far. Do
 you remember whether the problem occurs every time or just arbitrarily?


Great!

I'm not sure the reason. I guess it may related to nfs-ganesha version
or client distro version.

 Best regards,
 Burkhard
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



-- 
Best Regards,

Wheat
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-07-28 Thread Gregory Farnum

On Tue, Jul 28, 2015 at 8:01 AM, Burkhard Linke
burkhard.li...@computational.bio.uni-giessen.de wrote:
 Hi,

 On 07/27/2015 05:42 PM, Gregory Farnum wrote:

 On Mon, Jul 27, 2015 at 4:33 PM, Burkhard Linke
 burkhard.li...@computational.bio.uni-giessen.de wrote:

 Hi,

 the nfs-ganesha documentation states:

 ... This FSAL links to a modified version of the CEPH library that has
 been
 extended to expose its distributed cluster and replication facilities to
 the
 pNFS operations in the FSAL. ... The CEPH library modifications have not
 been merged into the upstream yet. 

 (https://github.com/nfs-ganesha/nfs-ganesha/wiki/Fsalsupport#ceph)

 Is this still the case with the hammer release?

 The FSAL has been upstream for quite a while, but it's not part of our
 regular testing yet and I'm not sure what it gets from the Ganesha
 side. I'd encourage you to test it, but be wary — we had a recent
 report of some issues we haven't been able to set up to reproduce yet.

 Can you give some details on that issues? I'm currently looking for a way to
 provide NFS based access to CephFS to our desktop machines.

Ummm...sadly I can't; we don't appear to have any tracker tickets and
I'm not sure where the report went to. :( I think it was from
Haomai...
-Greg


 The kernel NFS implementation in Ubuntu had some problems with CephFS in our
 setup, which I was not able to resolve yet. Ganesha seems to be more
 promising, since it uses libcephfs directly and does not need a mountpoint
 of its own.

 Best regards,
 Burkhard
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-07-28 Thread Haomai Wang

On Tue, Jul 28, 2015 at 4:47 PM, Gregory Farnum g...@gregs42.com wrote:
 On Tue, Jul 28, 2015 at 8:01 AM, Burkhard Linke
 burkhard.li...@computational.bio.uni-giessen.de wrote:
 Hi,

 On 07/27/2015 05:42 PM, Gregory Farnum wrote:

 On Mon, Jul 27, 2015 at 4:33 PM, Burkhard Linke
 burkhard.li...@computational.bio.uni-giessen.de wrote:

 Hi,

 the nfs-ganesha documentation states:

 ... This FSAL links to a modified version of the CEPH library that has
 been
 extended to expose its distributed cluster and replication facilities to
 the
 pNFS operations in the FSAL. ... The CEPH library modifications have not
 been merged into the upstream yet. 

 (https://github.com/nfs-ganesha/nfs-ganesha/wiki/Fsalsupport#ceph)

 Is this still the case with the hammer release?

 The FSAL has been upstream for quite a while, but it's not part of our
 regular testing yet and I'm not sure what it gets from the Ganesha
 side. I'd encourage you to test it, but be wary — we had a recent
 report of some issues we haven't been able to set up to reproduce yet.

 Can you give some details on that issues? I'm currently looking for a way to
 provide NFS based access to CephFS to our desktop machines.

 Ummm...sadly I can't; we don't appear to have any tracker tickets and
 I'm not sure where the report went to. :( I think it was from
 Haomai...

My fault, I should report this to ticket.

I have forgotten the details about the problem, I submit the infos to IRC :-(

It related to the ls output. It will print the wrong user/group
owner as -1, maybe related to root squash?

 -Greg


 The kernel NFS implementation in Ubuntu had some problems with CephFS in our
 setup, which I was not able to resolve yet. Ganesha seems to be more
 promising, since it uses libcephfs directly and does not need a mountpoint
 of its own.

 Best regards,
 Burkhard
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



-- 
Best Regards,

Wheat
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] State of nfs-ganesha CEPH fsal

2015-07-27 Thread Burkhard Linke


Hi,

the nfs-ganesha documentation states:

... This FSAL links to a modified version of the CEPH library that has 
been extended to expose its distributed cluster and replication 
facilities to the pNFS operations in the FSAL. ... The CEPH library 
modifications have not been merged into the upstream yet. 


(https://github.com/nfs-ganesha/nfs-ganesha/wiki/Fsalsupport#ceph)

Is this still the case with the hammer release?

Best regards,
Burkhard
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-07-27 Thread Gregory Farnum

On Mon, Jul 27, 2015 at 4:33 PM, Burkhard Linke
burkhard.li...@computational.bio.uni-giessen.de wrote:
 Hi,

 the nfs-ganesha documentation states:

 ... This FSAL links to a modified version of the CEPH library that has been
 extended to expose its distributed cluster and replication facilities to the
 pNFS operations in the FSAL. ... The CEPH library modifications have not
 been merged into the upstream yet. 

 (https://github.com/nfs-ganesha/nfs-ganesha/wiki/Fsalsupport#ceph)

 Is this still the case with the hammer release?

The FSAL has been upstream for quite a while, but it's not part of our
regular testing yet and I'm not sure what it gets from the Ganesha
side. I'd encourage you to test it, but be wary — we had a recent
report of some issues we haven't been able to set up to reproduce yet.
-Greg
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

2015-07-27 Thread Eric Eastman

We are looking at using Ganesha NFS with the Ceph file system.
Currently I am testing the FSAL interface on Ganesha NFS Release =
V2.2.0-2 running on Ceph 9.0.2.  This is all early work, as Ceph FS is
still not considered production ready, and Ceph 9.0.2 is a development
release.

Currently I am only seeing tracker issue 12334, but this is very early testing.

Eric

On Mon, Jul 27, 2015 at 9:42 AM, Gregory Farnum g...@gregs42.com wrote:
 On Mon, Jul 27, 2015 at 4:33 PM, Burkhard Linke
 burkhard.li...@computational.bio.uni-giessen.de wrote:
 Hi,

 the nfs-ganesha documentation states:

 ... This FSAL links to a modified version of the CEPH library that has been
 extended to expose its distributed cluster and replication facilities to the
 pNFS operations in the FSAL. ... The CEPH library modifications have not
 been merged into the upstream yet. 

 (https://github.com/nfs-ganesha/nfs-ganesha/wiki/Fsalsupport#ceph)

 Is this still the case with the hammer release?

 The FSAL has been upstream for quite a while, but it's not part of our
 regular testing yet and I'm not sure what it gets from the Ganesha
 side. I'd encourage you to test it, but be wary -- we had a recent
 report of some issues we haven't been able to set up to reproduce yet.
 -Greg
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

[ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

Re: [ceph-users] State of nfs-ganesha CEPH fsal

11 matches

Site Navigation

Mail list logo

Footer information