Re: [lustre-discuss] Apache via NFS via Lustre

2016-03-09 Thread John Dubinski 
Dear Martin,

Thanks, we do use ldap for our users and the mds is configured appropriately so 
I was confused that things seem to work normally for users but not apache. 

You and Philippe have sorted me out.  

Thanks for your help.

-John
 
> On Mar 9, 2016, at 4:13 AM, Martin Hecht  wrote:
> 
> I think, if the apache uid and gid needs to be known on the mds, this
> depends on the question if you have configured mdt.group_upcall or not.
> If not, the group memberships are checked on the lustre client against
> its /etc/group (or ldap if that's configured).
> 
> On 03/09/2016 06:59 AM, Philippe Weill wrote:
>> We use this on our lustre 2.5.3 but now your apache user uid and gid
>> have to be known on mds server
>> 
>> Le 09/03/2016 03:05, John Dubinski a écrit :
>>> Hi,
>>> 
>>> I'm wondering if there are any developments on this front.
>>> 
>>> We also NFS export some lustre filesystems from a client to an apache
>>> server so that users can link to their large datasets on their
>>> personal websites.  This has been working for years for us using
>>> lustre 1.8.
>>> 
>>> We recently built some new systems using lustre 2.5.3 and now this
>>> functionality is broken in the same way that Eric describes -
>>> symlinks to directories and files on the lustre filesystem are denied
>>> by the apache server.  This doesn't seem to be due to our apache
>>> configuration since symlinks to files and directories in ordinary
>>> (non-lustre) nfs-mounted filesystems work.  Also the nfs-exported
>>> filesystems behave normally - you can copy files in, as well as
>>> create and delete files as you wish.
>>> The only problems arise in relation to apache access.
>>> 
>>> We've also noticed that whenever the forbidden access messages comes
>>> up in the browser /var/log/messages on the lustre client spits out
>>> this error consistently:
>>> 
>>> Mar  8 19:53:14 nuexport02 kernel: LustreError:
>>> 2626:0:(mdc_locks.c:918:mdc_enqueue()) ldlm_cli_enqueue: -13
>>> 
>>> This appears to be related to file locking looking at the code...
>>> 
>>> We have also built a test apache server with lustre client modules
>>> that directly mount our lustre filesystems.  symlinks to the
>>> directories in the lustre fs within /var/www/html similarly return
>>> the forbidden access message with the above mdc_locks error.
>>> 
>>> We're running CentOS 6 with lustre 2.5.3 on the client and server
>>> side.  To repeat, direct client mounts of the lustre filesystems
>>> behave normally as well as nfs-exported mounts.  Only apache access
>>> to symlinks of files on a lustre filesystem give trouble.
>>> 
>>> Are there any special nfs export flags that can be set to help in
>>> /etc/exports?
>>> 
>>> Thanks for any help or insight!
>>> 
>>> Regards,
>>> 
>>> John
>>> 
>>> --
>>> John Dubinski
>>> Canadian Institute for Theoretical Astrophysics
>>> University of Torontophone:  416-946-7290
>>> 60 St. George St.fax:416-946-7287
>>> Toronto, Ontario e-mail: dubin...@cita.utoronto.ca
>>> CANADA M5S 3H8   url:www.cita.utoronto.ca
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> ___
>>> lustre-discuss mailing list
>>> lustre-discuss@lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>> 
>> 
> 
> 
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Apache via NFS via Lustre

2016-03-09 Thread John Dubinski 
Dear Philippe,

Of course!  I've added the uid/gid entries for apache in the mds server passwd 
and group files and now all works.  

Merci,

John


> On Mar 9, 2016, at 12:59 AM, Philippe Weill  
> wrote:
> 
> We use this on our lustre 2.5.3 but now your apache user uid and gid have to 
> be known on mds server
> 
> Le 09/03/2016 03:05, John Dubinski a écrit :
>> Hi,
>> 
>> I'm wondering if there are any developments on this front.
>> 
>> We also NFS export some lustre filesystems from a client to an apache server 
>> so that users can link to their large datasets on their personal websites.  
>> This has been working for years for us using lustre 1.8.
>> 
>> We recently built some new systems using lustre 2.5.3 and now this 
>> functionality is broken in the same way that Eric describes - symlinks to 
>> directories and files on the lustre filesystem are denied by the apache 
>> server.  This doesn't seem to be due to our apache configuration since 
>> symlinks to files and directories in ordinary (non-lustre) nfs-mounted 
>> filesystems work.  Also the nfs-exported filesystems behave normally - you 
>> can copy files in, as well as create and delete files as you wish.
>> The only problems arise in relation to apache access.
>> 
>> We've also noticed that whenever the forbidden access messages comes up in 
>> the browser /var/log/messages on the lustre client spits out this error 
>> consistently:
>> 
>> Mar  8 19:53:14 nuexport02 kernel: LustreError: 
>> 2626:0:(mdc_locks.c:918:mdc_enqueue()) ldlm_cli_enqueue: -13
>> 
>> This appears to be related to file locking looking at the code...
>> 
>> We have also built a test apache server with lustre client modules that 
>> directly mount our lustre filesystems.  symlinks to the directories in the 
>> lustre fs within /var/www/html similarly return the forbidden access message 
>> with the above mdc_locks error.
>> 
>> We're running CentOS 6 with lustre 2.5.3 on the client and server side.  To 
>> repeat, direct client mounts of the lustre filesystems behave normally as 
>> well as nfs-exported mounts.  Only apache access to symlinks of files on a 
>> lustre filesystem give trouble.
>> 
>> Are there any special nfs export flags that can be set to help in 
>> /etc/exports?
>> 
>> Thanks for any help or insight!
>> 
>> Regards,
>> 
>> John
>> 
>> --
>> John Dubinski
>> Canadian Institute for Theoretical Astrophysics
>> University of Torontophone:  416-946-7290
>> 60 St. George St.fax:416-946-7287
>> Toronto, Ontario e-mail: dubin...@cita.utoronto.ca
>> CANADA M5S 3H8   url:www.cita.utoronto.ca
>> 
>> 
>> 
>> 
>> 
>> 
>> ___
>> lustre-discuss mailing list
>> lustre-discuss@lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>> 
> 
> -- 
> Weill Philippe -  Administrateur Systeme et Reseaux
> CNRS/UPMC/IPSL   LATMOS (UMR 8190)
> Tour 45/46 3e Etage B302|4 Place Jussieu|75252 Paris Cedex 05 -  FRANCE
> Email:philippe.we...@latmos.ipsl.fr | tel:+33 0144274759
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Apache via NFS via Lustre

2016-03-09 Thread Martin Hecht 
I think, if the apache uid and gid needs to be known on the mds, this
depends on the question if you have configured mdt.group_upcall or not.
If not, the group memberships are checked on the lustre client against
its /etc/group (or ldap if that's configured).

On 03/09/2016 06:59 AM, Philippe Weill wrote:
> We use this on our lustre 2.5.3 but now your apache user uid and gid
> have to be known on mds server
>
> Le 09/03/2016 03:05, John Dubinski a écrit :
>> Hi,
>>
>> I'm wondering if there are any developments on this front.
>>
>> We also NFS export some lustre filesystems from a client to an apache
>> server so that users can link to their large datasets on their
>> personal websites.  This has been working for years for us using
>> lustre 1.8.
>>
>> We recently built some new systems using lustre 2.5.3 and now this
>> functionality is broken in the same way that Eric describes -
>> symlinks to directories and files on the lustre filesystem are denied
>> by the apache server.  This doesn't seem to be due to our apache
>> configuration since symlinks to files and directories in ordinary
>> (non-lustre) nfs-mounted filesystems work.  Also the nfs-exported
>> filesystems behave normally - you can copy files in, as well as
>> create and delete files as you wish.
>> The only problems arise in relation to apache access.
>>
>> We've also noticed that whenever the forbidden access messages comes
>> up in the browser /var/log/messages on the lustre client spits out
>> this error consistently:
>>
>> Mar  8 19:53:14 nuexport02 kernel: LustreError:
>> 2626:0:(mdc_locks.c:918:mdc_enqueue()) ldlm_cli_enqueue: -13
>>
>> This appears to be related to file locking looking at the code...
>>
>> We have also built a test apache server with lustre client modules
>> that directly mount our lustre filesystems.  symlinks to the
>> directories in the lustre fs within /var/www/html similarly return
>> the forbidden access message with the above mdc_locks error.
>>
>> We're running CentOS 6 with lustre 2.5.3 on the client and server
>> side.  To repeat, direct client mounts of the lustre filesystems
>> behave normally as well as nfs-exported mounts.  Only apache access
>> to symlinks of files on a lustre filesystem give trouble.
>>
>> Are there any special nfs export flags that can be set to help in
>> /etc/exports?
>>
>> Thanks for any help or insight!
>>
>> Regards,
>>
>> John
>>
>> --
>> John Dubinski
>> Canadian Institute for Theoretical Astrophysics
>> University of Torontophone:  416-946-7290
>> 60 St. George St.fax:416-946-7287
>> Toronto, Ontario e-mail: dubin...@cita.utoronto.ca
>> CANADA M5S 3H8   url:www.cita.utoronto.ca
>>
>>
>>
>>
>>
>>
>> ___
>> lustre-discuss mailing list
>> lustre-discuss@lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>




smime.p7s
Description: S/MIME Cryptographic Signature
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Apache via NFS via Lustre

2016-03-08 Thread Philippe Weill 

We use this on our lustre 2.5.3 but now your apache user uid and gid have to be 
known on mds server

Le 09/03/2016 03:05, John Dubinski a écrit :

Hi,

I'm wondering if there are any developments on this front.

We also NFS export some lustre filesystems from a client to an apache server so 
that users can link to their large datasets on their personal websites.  This 
has been working for years for us using lustre 1.8.

We recently built some new systems using lustre 2.5.3 and now this 
functionality is broken in the same way that Eric describes - symlinks to 
directories and files on the lustre filesystem are denied by the apache server. 
 This doesn't seem to be due to our apache configuration since symlinks to 
files and directories in ordinary (non-lustre) nfs-mounted filesystems work.  
Also the nfs-exported filesystems behave normally - you can copy files in, as 
well as create and delete files as you wish.
The only problems arise in relation to apache access.

We've also noticed that whenever the forbidden access messages comes up in the 
browser /var/log/messages on the lustre client spits out this error 
consistently:

Mar  8 19:53:14 nuexport02 kernel: LustreError: 
2626:0:(mdc_locks.c:918:mdc_enqueue()) ldlm_cli_enqueue: -13

This appears to be related to file locking looking at the code...

We have also built a test apache server with lustre client modules that 
directly mount our lustre filesystems.  symlinks to the directories in the 
lustre fs within /var/www/html similarly return the forbidden access message 
with the above mdc_locks error.

We're running CentOS 6 with lustre 2.5.3 on the client and server side.  To 
repeat, direct client mounts of the lustre filesystems behave normally as well 
as nfs-exported mounts.  Only apache access to symlinks of files on a lustre 
filesystem give trouble.

Are there any special nfs export flags that can be set to help in /etc/exports?

Thanks for any help or insight!

Regards,

John

--
John Dubinski
Canadian Institute for Theoretical Astrophysics
University of Torontophone:  416-946-7290
60 St. George St.fax:416-946-7287
Toronto, Ontario e-mail: dubin...@cita.utoronto.ca
CANADA M5S 3H8   url:www.cita.utoronto.ca






___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



--
Weill Philippe -  Administrateur Systeme et Reseaux
CNRS/UPMC/IPSL   LATMOS (UMR 8190)
Tour 45/46 3e Etage B302|4 Place Jussieu|75252 Paris Cedex 05 -  FRANCE
Email:philippe.we...@latmos.ipsl.fr | tel:+33 0144274759
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Apache via NFS via Lustre

2016-03-08 Thread John Dubinski 
Hi,

I'm wondering if there are any developments on this front. 

We also NFS export some lustre filesystems from a client to an apache server so 
that users can link to their large datasets on their personal websites.  This 
has been working for years for us using lustre 1.8.

We recently built some new systems using lustre 2.5.3 and now this 
functionality is broken in the same way that Eric describes - symlinks to 
directories and files on the lustre filesystem are denied by the apache server. 
 This doesn't seem to be due to our apache configuration since symlinks to 
files and directories in ordinary (non-lustre) nfs-mounted filesystems work.  
Also the nfs-exported filesystems behave normally - you can copy files in, as 
well as create and delete files as you wish.
The only problems arise in relation to apache access.

We've also noticed that whenever the forbidden access messages comes up in the 
browser /var/log/messages on the lustre client spits out this error 
consistently:

Mar  8 19:53:14 nuexport02 kernel: LustreError: 
2626:0:(mdc_locks.c:918:mdc_enqueue()) ldlm_cli_enqueue: -13

This appears to be related to file locking looking at the code...

We have also built a test apache server with lustre client modules that 
directly mount our lustre filesystems.  symlinks to the directories in the 
lustre fs within /var/www/html similarly return the forbidden access message 
with the above mdc_locks error.

We're running CentOS 6 with lustre 2.5.3 on the client and server side.  To 
repeat, direct client mounts of the lustre filesystems behave normally as well 
as nfs-exported mounts.  Only apache access to symlinks of files on a lustre 
filesystem give trouble.

Are there any special nfs export flags that can be set to help in /etc/exports?

Thanks for any help or insight!  

Regards,

John

--
John Dubinski
Canadian Institute for Theoretical Astrophysics
University of Torontophone:  416-946-7290
60 St. George St.fax:416-946-7287
Toronto, Ontario e-mail: dubin...@cita.utoronto.ca
CANADA M5S 3H8   url:www.cita.utoronto.ca






___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Apache via NFS via Lustre

2015-12-15 Thread Eric Kolb 

Hi Eli,

No, unfortunately we found no explanation as yet. We moved the web site 
off of Lustre/NFS but as time permits we may attempt tracing the problem 
further.


Eric

On 12/15/2015 10:50 AM, E.S. Rosenberg wrote:

Hi Eric,

Since we also have some issues with servers accessing files this issue
interests me...
Did you find anything on this?

Thanks,
Eli

On Mon, Nov 30, 2015 at 7:53 PM, Eric Kolb mailto:ek...@uvic.ca>> wrote:

Hello,

Perhaps someone has seen a similar issue? If so any insight would be
appreciated.

We recently migrated some websites from an NFS share by which the
NFS server is exporting a share sitting on a Lustre mount. Below is
the issue.

- sftp a file to website.ca  - the load balancer
will direct session to server1 or server2, let us say server1 in
this case

- Accessing the file with Apache via server1 works but via server2 a
NFS EIO -13 error is encountered

- On server1 (or server2) copy the file to /dev/null makes the file
accessible to Apache on either server1 or server2

- As work around sftp file to a temporary location, then ssh into
website.ca  and "cp tmp/file.jpeg www/file.jpeg",
this allows then
Apache on either server1 or server2 to access the file

- Created an lseek C program to try mimicking a captured Apache
strace but it shows success on any combination of file access scenarios

Prior to adding in Lustre no such such issues were encountered.

Thanks,
--
Eric Kolb
DCS - UVic Systems
University of Victoria
Office: 250-721-7658 
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org 
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org




___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Apache via NFS via Lustre

2015-12-15 Thread E.S. Rosenberg 
Hi Eric,

Since we also have some issues with servers accessing files this issue
interests me...
Did you find anything on this?

Thanks,
Eli

On Mon, Nov 30, 2015 at 7:53 PM, Eric Kolb  wrote:

> Hello,
>
> Perhaps someone has seen a similar issue? If so any insight would be
> appreciated.
>
> We recently migrated some websites from an NFS share by which the NFS
> server is exporting a share sitting on a Lustre mount. Below is the issue.
>
> - sftp a file to website.ca - the load balancer will direct session to
> server1 or server2, let us say server1 in this case
>
> - Accessing the file with Apache via server1 works but via server2 a NFS
> EIO -13 error is encountered
>
> - On server1 (or server2) copy the file to /dev/null makes the file
> accessible to Apache on either server1 or server2
>
> - As work around sftp file to a temporary location, then ssh into
> website.ca and "cp tmp/file.jpeg www/file.jpeg", this allows then
> Apache on either server1 or server2 to access the file
>
> - Created an lseek C program to try mimicking a captured Apache strace but
> it shows success on any combination of file access scenarios
>
> Prior to adding in Lustre no such such issues were encountered.
>
> Thanks,
> --
> Eric Kolb
> DCS - UVic Systems
> University of Victoria
> Office: 250-721-7658
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Apache via NFS via Lustre

2015-11-30 Thread Eric Kolb 

Hello,

Perhaps someone has seen a similar issue? If so any insight would be 
appreciated.


We recently migrated some websites from an NFS share by which the NFS 
server is exporting a share sitting on a Lustre mount. Below is the issue.


- sftp a file to website.ca - the load balancer will direct session to 
server1 or server2, let us say server1 in this case


- Accessing the file with Apache via server1 works but via server2 a NFS 
EIO -13 error is encountered


- On server1 (or server2) copy the file to /dev/null makes the file 
accessible to Apache on either server1 or server2


- As work around sftp file to a temporary location, then ssh into 
website.ca and "cp tmp/file.jpeg www/file.jpeg", this allows then

Apache on either server1 or server2 to access the file

- Created an lseek C program to try mimicking a captured Apache strace 
but it shows success on any combination of file access scenarios


Prior to adding in Lustre no such such issues were encountered.

Thanks,
--
Eric Kolb
DCS - UVic Systems
University of Victoria
Office: 250-721-7658
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org