Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

I neglected to mention that I had not set stat-prefetch to "off" on the 
newly created volume.


That's done, and output is as expected.

On 07/29/2013 05:16 PM, Nicholas Majeran wrote:

Thanks Kaushal.

A little more information:

I was never able to update that setting, so I deleted the gluster 
volume, updated to the latest CentOS 6.4 kernel 
(2.6.32-358.14.1.el6.x86_64), and reformatted the underlying XFS file 
system.


After that, I ran a few more tests, and noticed a caching issue 
whereby if I ran an "ls -l" I would sometimes see a + attached to the 
dir/file in question, and sometimes I would not.  See below.


[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x+ 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x+ 2 root root 40 Jul 29 16:43 1

After that, I set performance.cache-refresh-timeout to 0, and now I 
see the '+' next to the direntry each time I run 'ls -l' and getfacl.


Two things:

*  Is the above behavior expected?
*  What are the performance implications of setting 
performance.cache-refresh-timeout to 0?


Thanks.

On 07/24/2013 10:26 PM, Kaushal M wrote:

A client which mounts a volume has N connections with the N bricks and
1 connection to glusterd. The connection to glusterd is used to get
the volfiles and the ports of the bricks. For an already mounted
client this connection will still exist even if the volume was
stopped. Glusterd checks these connections when trying to set any
option. This is done to prevent older clients which don't support the
newer features from getting a newer volfile. If this isn't done, the
client will fetch the new volfile, try to understand it and fail, and
continue using the older volfile.
We could enhance this by telling exactly which clients are causing the
issue. I'll file a RFE for this.

But, that said stat-prefetch was a part of earlier releases as well,
and it was being disabled so the 'volume set' command should have gone
through. I'll check this out and reply here.

Thanks,
Kaushal

On Thu, Jul 25, 2013 at 2:44 AM, Nicholas Majeran
 wrote:
That would make sense, but when I had the volume stopped, nothing 
should
have been connected to that volume, correct?  The volume set 
operation still

failed with the volume stopped.



On 07/24/2013 03:49 PM, Joseph Landman wrote:

It says the clients don't support the ops.  Is it possible to
disconnect them one at a time until the command works? Might not be
possible but it probably would tell you which client was problematic.
Also these are native client as per the subject, right?

Sent from my iPad

On Jul 24, 2013, at 2:52 PM, Nicholas Majeran
 wrote:


Even with the volume stopped, I was unable to unset stat-prefetch.
Anything else I should look at?

On 07/23/2013 02:55 PM, Nicholas Majeran wrote:
FWIW, I tried to disable the parameter on a stopped volume, which 
was

successful.  I then started the volume and I could get/set the ACLs
normally.

I'm going to try the same procedure on the gv0 volume that threw the
error previously.
Thanks.

On 07/23/2013 02:29 PM, Nicholas Majeran wrote:

When I tried to disable that parameter, I receive the following:

gluster> volume set gv0 stat-prefetch off
volume set: failed: One or more connected clients cannot support 
the
feature being set. These clients need to be upgraded or 
disconnected before

running this command again

AFAICT, all the nodes are at the same version, but I did this 
this in

the logs I after I ran the command:

[2013-07-23 19:26:02.673304] E
[glusterd-op-sm.c:370:glusterd_check_client_op_version_support]
0-management: One or more clients don't support the required 
op-version

[2013-07-23 19:26:02.673325] E
[glusterd-syncop.c:767:gd_stage_op_phase] 0-management: Staging 
of operation
'Volume Set' failed on localhost : One or more connected clients 
cannot

support the feature being set. These clients need to be upgraded or
disconnected before running this command again
[2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)


On 07/23/2013 01:4

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

Thanks Kaushal.

A little more information:

I was never able to update that setting, so I deleted the gluster 
volume, updated to the latest CentOS 6.4 kernel 
(2.6.32-358.14.1.el6.x86_64), and reformatted the underlying XFS file 
system.


After that, I ran a few more tests, and noticed a caching issue whereby 
if I ran an "ls -l" I would sometimes see a + attached to the dir/file 
in question, and sometimes I would not.  See below.


[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x+ 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x 2 root root 40 Jul 29 16:43 1
[root@fs-copo05 tmp]# ls -l
total 0
drwxrwxr-x+ 2 root root 40 Jul 29 16:43 1

After that, I set performance.cache-refresh-timeout to 0, and now I see 
the '+' next to the direntry each time I run 'ls -l' and getfacl.


Two things:

*  Is the above behavior expected?
*  What are the performance implications of setting 
performance.cache-refresh-timeout to 0?


Thanks.

On 07/24/2013 10:26 PM, Kaushal M wrote:

A client which mounts a volume has N connections with the N bricks and
1 connection to glusterd. The connection to glusterd is used to get
the volfiles and the ports of the bricks. For an already mounted
client this connection will still exist even if the volume was
stopped. Glusterd checks these connections when trying to set any
option. This is done to prevent older clients which don't support the
newer features from getting a newer volfile. If this isn't done, the
client will fetch the new volfile, try to understand it and fail, and
continue using the older volfile.
We could enhance this by telling exactly which clients are causing the
issue. I'll file a RFE for this.

But, that said stat-prefetch was a part of earlier releases as well,
and it was being disabled so the 'volume set' command should have gone
through. I'll check this out and reply here.

Thanks,
Kaushal

On Thu, Jul 25, 2013 at 2:44 AM, Nicholas Majeran
 wrote:

That would make sense, but when I had the volume stopped, nothing should
have been connected to that volume, correct?  The volume set operation still
failed with the volume stopped.



On 07/24/2013 03:49 PM, Joseph Landman wrote:

It says the clients don't support the ops.  Is it possible to
disconnect them one at a time until the command works?  Might not be
possible but it probably would tell you which client was problematic.
Also these are native client as per the subject, right?

Sent from my iPad

On Jul 24, 2013, at 2:52 PM, Nicholas Majeran
 wrote:


Even with the volume stopped, I was unable to unset stat-prefetch.
Anything else I should look at?

On 07/23/2013 02:55 PM, Nicholas Majeran wrote:

FWIW, I tried to disable the parameter on a stopped volume, which was
successful.  I then started the volume and I could get/set the ACLs
normally.

I'm going to try the same procedure on the gv0 volume that threw the
error previously.
Thanks.

On 07/23/2013 02:29 PM, Nicholas Majeran wrote:

When I tried to disable that parameter, I receive the following:

gluster> volume set gv0 stat-prefetch off
volume set: failed: One or more connected clients cannot support the
feature being set. These clients need to be upgraded or disconnected before
running this command again

AFAICT, all the nodes are at the same version, but I did this this in
the logs I after I ran the command:

[2013-07-23 19:26:02.673304] E
[glusterd-op-sm.c:370:glusterd_check_client_op_version_support]
0-management: One or more clients don't support the required op-version
[2013-07-23 19:26:02.673325] E
[glusterd-syncop.c:767:gd_stage_op_phase] 0-management: Staging of operation
'Volume Set' failed on localhost : One or more connected clients cannot
support the feature being set. These clients need to be upgraded or
disconnected before running this command again
[2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect]
0-management: connection attempt failed (Connection refused)


On 07/23/2013 01:45 PM, Vijay Bellur wrote:

On 07/23/2013 09:28 PM, Nicholas Majeran wrote:

A little more detail... on files, I still can't get this to work, but
when I run ls and {get,set}facl on . it seems to wor

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Kaushal M]

A client which mounts a volume has N connections with the N bricks and
1 connection to glusterd. The connection to glusterd is used to get
the volfiles and the ports of the bricks. For an already mounted
client this connection will still exist even if the volume was
stopped. Glusterd checks these connections when trying to set any
option. This is done to prevent older clients which don't support the
newer features from getting a newer volfile. If this isn't done, the
client will fetch the new volfile, try to understand it and fail, and
continue using the older volfile.
We could enhance this by telling exactly which clients are causing the
issue. I'll file a RFE for this.

But, that said stat-prefetch was a part of earlier releases as well,
and it was being disabled so the 'volume set' command should have gone
through. I'll check this out and reply here.

Thanks,
Kaushal

On Thu, Jul 25, 2013 at 2:44 AM, Nicholas Majeran
 wrote:
> That would make sense, but when I had the volume stopped, nothing should
> have been connected to that volume, correct?  The volume set operation still
> failed with the volume stopped.
>
>
>
> On 07/24/2013 03:49 PM, Joseph Landman wrote:
>>
>> It says the clients don't support the ops.  Is it possible to
>> disconnect them one at a time until the command works?  Might not be
>> possible but it probably would tell you which client was problematic.
>> Also these are native client as per the subject, right?
>>
>> Sent from my iPad
>>
>> On Jul 24, 2013, at 2:52 PM, Nicholas Majeran
>>  wrote:
>>
>>> Even with the volume stopped, I was unable to unset stat-prefetch.
>>> Anything else I should look at?
>>>
>>> On 07/23/2013 02:55 PM, Nicholas Majeran wrote:

 FWIW, I tried to disable the parameter on a stopped volume, which was
 successful.  I then started the volume and I could get/set the ACLs
 normally.

 I'm going to try the same procedure on the gv0 volume that threw the
 error previously.
 Thanks.

 On 07/23/2013 02:29 PM, Nicholas Majeran wrote:
>
> When I tried to disable that parameter, I receive the following:
>
> gluster> volume set gv0 stat-prefetch off
> volume set: failed: One or more connected clients cannot support the
> feature being set. These clients need to be upgraded or disconnected 
> before
> running this command again
>
> AFAICT, all the nodes are at the same version, but I did this this in
> the logs I after I ran the command:
>
> [2013-07-23 19:26:02.673304] E
> [glusterd-op-sm.c:370:glusterd_check_client_op_version_support]
> 0-management: One or more clients don't support the required op-version
> [2013-07-23 19:26:02.673325] E
> [glusterd-syncop.c:767:gd_stage_op_phase] 0-management: Staging of 
> operation
> 'Volume Set' failed on localhost : One or more connected clients cannot
> support the feature being set. These clients need to be upgraded or
> disconnected before running this command again
> [2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect]
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect]
> 0-management: connection attempt failed (Connection refused)
>
>
> On 07/23/2013 01:45 PM, Vijay Bellur wrote:
>>
>> On 07/23/2013 09:28 PM, Nicholas Majeran wrote:
>>>
>>> A little more detail... on files, I still can't get this to work, but
>>> when I run ls and {get,set}facl on . it seems to work:
>>>
>>> [root@s1 tmp]# pwd
>>> /mnt/glusterfs/tmp
>>> [root@s1 tmp]# ls -ld .
>>> drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
>>> [root@suncosmbgw1 tmp]# ls -ld ../tmp
>>> drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
>>> [root@s1 tmp]# getfacl .
>>> # file: .
>>> # owner: nmajeran
>>> # group: root
>>> user::rwx
>>> user:root:rwx
>>> user:user1:rwx
>>> group::r-x
>>> group:g1:rwx
>>> group:g2:r-x
>>> group:g3:r-x
>>> mask::rwx
>>> other::r-x
>>>
>>> [root@s1 tmp]# getfacl ../tmp
>>> # file: ../tmp
>>> # owner: nmajeran
>>> # group: root
>>> user::rwx
>>> group::rwx
>>> other::r-x
>>
>>
>> Does disabling stat-prefetch address the problem permanently? You can
>> disable stat-prefetch via:
>>
>> gluster volume set  stat-prefetch off

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Joseph Landman]

Sent from my iPad

On Jul 24, 2013, at 5:14 PM, Nicholas Majeran
 wrote:

> That would make sense, but when I had the volume stopped, nothing should have 
> been connected to that volume, correct?  The volume set operation still 
> failed with the volume stopped.
>

I wonder if there was some left over state ... A hung mount that
didn't cleanup correctly?  You did a daemon restart as well?  If you
shut down the daemon, could you 'ps -ealf | grep -i gluster'  and see
if anything is still running that shouldn't be?

Basically trying to remove hung daemons in strange states as a possible cause.

>
> On 07/24/2013 03:49 PM, Joseph Landman wrote:
>> It says the clients don't support the ops.  Is it possible to
>> disconnect them one at a time until the command works?  Might not be
>> possible but it probably would tell you which client was problematic.
>> Also these are native client as per the subject, right?
>>
>> Sent from my iPad
>>
>> On Jul 24, 2013, at 2:52 PM, Nicholas Majeran
>>  wrote:
>>
>>> Even with the volume stopped, I was unable to unset stat-prefetch.
>>> Anything else I should look at?
>>>
>>> On 07/23/2013 02:55 PM, Nicholas Majeran wrote:
 FWIW, I tried to disable the parameter on a stopped volume, which was 
 successful.  I then started the volume and I could get/set the ACLs 
 normally.

 I'm going to try the same procedure on the gv0 volume that threw the error 
 previously.
 Thanks.

 On 07/23/2013 02:29 PM, Nicholas Majeran wrote:
> When I tried to disable that parameter, I receive the following:
>
> gluster> volume set gv0 stat-prefetch off
> volume set: failed: One or more connected clients cannot support the 
> feature being set. These clients need to be upgraded or disconnected 
> before running this command again
>
> AFAICT, all the nodes are at the same version, but I did this this in the 
> logs I after I ran the command:
>
> [2013-07-23 19:26:02.673304] E 
> [glusterd-op-sm.c:370:glusterd_check_client_op_version_support] 
> 0-management: One or more clients don't support the required op-version
> [2013-07-23 19:26:02.673325] E [glusterd-syncop.c:767:gd_stage_op_phase] 
> 0-management: Staging of operation 'Volume Set' failed on localhost : One 
> or more connected clients cannot support the feature being set. These 
> clients need to be upgraded or disconnected before running this command 
> again
> [2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect] 
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect] 
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect] 
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect] 
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect] 
> 0-management: connection attempt failed (Connection refused)
> [2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect] 
> 0-management: connection attempt failed (Connection refused)
>
>
> On 07/23/2013 01:45 PM, Vijay Bellur wrote:
>> On 07/23/2013 09:28 PM, Nicholas Majeran wrote:
>>> A little more detail... on files, I still can't get this to work, but
>>> when I run ls and {get,set}facl on . it seems to work:
>>>
>>> [root@s1 tmp]# pwd
>>> /mnt/glusterfs/tmp
>>> [root@s1 tmp]# ls -ld .
>>> drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
>>> [root@suncosmbgw1 tmp]# ls -ld ../tmp
>>> drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
>>> [root@s1 tmp]# getfacl .
>>> # file: .
>>> # owner: nmajeran
>>> # group: root
>>> user::rwx
>>> user:root:rwx
>>> user:user1:rwx
>>> group::r-x
>>> group:g1:rwx
>>> group:g2:r-x
>>> group:g3:r-x
>>> mask::rwx
>>> other::r-x
>>>
>>> [root@s1 tmp]# getfacl ../tmp
>>> # file: ../tmp
>>> # owner: nmajeran
>>> # group: root
>>> user::rwx
>>> group::rwx
>>> other::r-x
>>
>> Does disabling stat-prefetch address the problem permanently? You can 
>> disable stat-prefetch via:
>>
>> gluster volume set  stat-prefetch off
>>
>> -Vijay
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>>> --
>>> Nick Majeran
>>> Sun Trading LLC
>>> 312-229-9608
>>>
>>> ___
>>> Gluster-users mailing list
>>> Gluster-users@gluster.org
>>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
> --
> Nick Majeran
> Sun Trading LLC
> 312-229-9608
>
___
Gluster-user

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

That would make sense, but when I had the volume stopped, nothing should 
have been connected to that volume, correct?  The volume set operation 
still failed with the volume stopped.



On 07/24/2013 03:49 PM, Joseph Landman wrote:

It says the clients don't support the ops.  Is it possible to
disconnect them one at a time until the command works?  Might not be
possible but it probably would tell you which client was problematic.
Also these are native client as per the subject, right?

Sent from my iPad

On Jul 24, 2013, at 2:52 PM, Nicholas Majeran
 wrote:


Even with the volume stopped, I was unable to unset stat-prefetch.
Anything else I should look at?

On 07/23/2013 02:55 PM, Nicholas Majeran wrote:

FWIW, I tried to disable the parameter on a stopped volume, which was 
successful.  I then started the volume and I could get/set the ACLs normally.

I'm going to try the same procedure on the gv0 volume that threw the error 
previously.
Thanks.

On 07/23/2013 02:29 PM, Nicholas Majeran wrote:

When I tried to disable that parameter, I receive the following:

gluster> volume set gv0 stat-prefetch off
volume set: failed: One or more connected clients cannot support the feature 
being set. These clients need to be upgraded or disconnected before running 
this command again

AFAICT, all the nodes are at the same version, but I did this this in the logs 
I after I ran the command:

[2013-07-23 19:26:02.673304] E 
[glusterd-op-sm.c:370:glusterd_check_client_op_version_support] 0-management: 
One or more clients don't support the required op-version
[2013-07-23 19:26:02.673325] E [glusterd-syncop.c:767:gd_stage_op_phase] 
0-management: Staging of operation 'Volume Set' failed on localhost : One or 
more connected clients cannot support the feature being set. These clients need 
to be upgraded or disconnected before running this command again
[2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect] 0-management: 
connection attempt failed (Connection refused)
[2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect] 0-management: 
connection attempt failed (Connection refused)
[2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect] 0-management: 
connection attempt failed (Connection refused)
[2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect] 0-management: 
connection attempt failed (Connection refused)
[2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect] 0-management: 
connection attempt failed (Connection refused)
[2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect] 0-management: 
connection attempt failed (Connection refused)


On 07/23/2013 01:45 PM, Vijay Bellur wrote:

On 07/23/2013 09:28 PM, Nicholas Majeran wrote:

A little more detail... on files, I still can't get this to work, but
when I run ls and {get,set}facl on . it seems to work:

[root@s1 tmp]# pwd
/mnt/glusterfs/tmp
[root@s1 tmp]# ls -ld .
drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
[root@suncosmbgw1 tmp]# ls -ld ../tmp
drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
[root@s1 tmp]# getfacl .
# file: .
# owner: nmajeran
# group: root
user::rwx
user:root:rwx
user:user1:rwx
group::r-x
group:g1:rwx
group:g2:r-x
group:g3:r-x
mask::rwx
other::r-x

[root@s1 tmp]# getfacl ../tmp
# file: ../tmp
# owner: nmajeran
# group: root
user::rwx
group::rwx
other::r-x


Does disabling stat-prefetch address the problem permanently? You can disable 
stat-prefetch via:

gluster volume set  stat-prefetch off

-Vijay

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

--
Nick Majeran
Sun Trading LLC
312-229-9608

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users


--
Nick Majeran
Sun Trading LLC
312-229-9608

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Joseph Landman]

It says the clients don't support the ops.  Is it possible to
disconnect them one at a time until the command works?  Might not be
possible but it probably would tell you which client was problematic.
Also these are native client as per the subject, right?

Sent from my iPad

On Jul 24, 2013, at 2:52 PM, Nicholas Majeran
 wrote:

> Even with the volume stopped, I was unable to unset stat-prefetch.
> Anything else I should look at?
>
> On 07/23/2013 02:55 PM, Nicholas Majeran wrote:
>> FWIW, I tried to disable the parameter on a stopped volume, which was 
>> successful.  I then started the volume and I could get/set the ACLs normally.
>>
>> I'm going to try the same procedure on the gv0 volume that threw the error 
>> previously.
>> Thanks.
>>
>> On 07/23/2013 02:29 PM, Nicholas Majeran wrote:
>>> When I tried to disable that parameter, I receive the following:
>>>
>>> gluster> volume set gv0 stat-prefetch off
>>> volume set: failed: One or more connected clients cannot support the 
>>> feature being set. These clients need to be upgraded or disconnected before 
>>> running this command again
>>>
>>> AFAICT, all the nodes are at the same version, but I did this this in the 
>>> logs I after I ran the command:
>>>
>>> [2013-07-23 19:26:02.673304] E 
>>> [glusterd-op-sm.c:370:glusterd_check_client_op_version_support] 
>>> 0-management: One or more clients don't support the required op-version
>>> [2013-07-23 19:26:02.673325] E [glusterd-syncop.c:767:gd_stage_op_phase] 
>>> 0-management: Staging of operation 'Volume Set' failed on localhost : One 
>>> or more connected clients cannot support the feature being set. These 
>>> clients need to be upgraded or disconnected before running this command 
>>> again
>>> [2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect] 0-management: 
>>> connection attempt failed (Connection refused)
>>> [2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect] 0-management: 
>>> connection attempt failed (Connection refused)
>>> [2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect] 0-management: 
>>> connection attempt failed (Connection refused)
>>> [2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect] 0-management: 
>>> connection attempt failed (Connection refused)
>>> [2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect] 0-management: 
>>> connection attempt failed (Connection refused)
>>> [2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect] 0-management: 
>>> connection attempt failed (Connection refused)
>>>
>>>
>>> On 07/23/2013 01:45 PM, Vijay Bellur wrote:
 On 07/23/2013 09:28 PM, Nicholas Majeran wrote:
> A little more detail... on files, I still can't get this to work, but
> when I run ls and {get,set}facl on . it seems to work:
>
> [root@s1 tmp]# pwd
> /mnt/glusterfs/tmp
> [root@s1 tmp]# ls -ld .
> drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
> [root@suncosmbgw1 tmp]# ls -ld ../tmp
> drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
> [root@s1 tmp]# getfacl .
> # file: .
> # owner: nmajeran
> # group: root
> user::rwx
> user:root:rwx
> user:user1:rwx
> group::r-x
> group:g1:rwx
> group:g2:r-x
> group:g3:r-x
> mask::rwx
> other::r-x
>
> [root@s1 tmp]# getfacl ../tmp
> # file: ../tmp
> # owner: nmajeran
> # group: root
> user::rwx
> group::rwx
> other::r-x


 Does disabling stat-prefetch address the problem permanently? You can 
 disable stat-prefetch via:

 gluster volume set  stat-prefetch off

 -Vijay
>>>
>>> ___
>>> Gluster-users mailing list
>>> Gluster-users@gluster.org
>>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
> --
> Nick Majeran
> Sun Trading LLC
> 312-229-9608
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

Even with the volume stopped, I was unable to unset stat-prefetch.
Anything else I should look at?

On 07/23/2013 02:55 PM, Nicholas Majeran wrote:
FWIW, I tried to disable the parameter on a stopped volume, which was 
successful.  I then started the volume and I could get/set the ACLs 
normally.


I'm going to try the same procedure on the gv0 volume that threw the 
error previously.

Thanks.

On 07/23/2013 02:29 PM, Nicholas Majeran wrote:

When I tried to disable that parameter, I receive the following:

gluster> volume set gv0 stat-prefetch off
volume set: failed: One or more connected clients cannot support the 
feature being set. These clients need to be upgraded or disconnected 
before running this command again


AFAICT, all the nodes are at the same version, but I did this this in 
the logs I after I ran the command:


[2013-07-23 19:26:02.673304] E 
[glusterd-op-sm.c:370:glusterd_check_client_op_version_support] 
0-management: One or more clients don't support the required op-version
[2013-07-23 19:26:02.673325] E 
[glusterd-syncop.c:767:gd_stage_op_phase] 0-management: Staging of 
operation 'Volume Set' failed on localhost : One or more connected 
clients cannot support the feature being set. These clients need to 
be upgraded or disconnected before running this command again
[2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)



On 07/23/2013 01:45 PM, Vijay Bellur wrote:

On 07/23/2013 09:28 PM, Nicholas Majeran wrote:

A little more detail... on files, I still can't get this to work, but
when I run ls and {get,set}facl on . it seems to work:

[root@s1 tmp]# pwd
/mnt/glusterfs/tmp
[root@s1 tmp]# ls -ld .
drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
[root@suncosmbgw1 tmp]# ls -ld ../tmp
drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
[root@s1 tmp]# getfacl .
# file: .
# owner: nmajeran
# group: root
user::rwx
user:root:rwx
user:user1:rwx
group::r-x
group:g1:rwx
group:g2:r-x
group:g3:r-x
mask::rwx
other::r-x

[root@s1 tmp]# getfacl ../tmp
# file: ../tmp
# owner: nmajeran
# group: root
user::rwx
group::rwx
other::r-x




Does disabling stat-prefetch address the problem permanently? You 
can disable stat-prefetch via:


gluster volume set  stat-prefetch off

-Vijay



___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users




--
Nick Majeran
Sun Trading LLC
312-229-9608

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

I checked glusterfs -V before, but not glusterfsd, either way, all 3.4.0:

[root@pxe njm]# pssh -h copo -P "glusterfs -V | grep built" | grep -v SUCC
fs-copo06.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo05.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo07.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo08.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo09.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo10.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo11.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo12.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
[root@sunchipxe1 njm]# pssh -h copo -P "glusterfsd -V | grep built" | 
grep -v SUCC

fs-copo05.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo06.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo07.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo08.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo10.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo09.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo11.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08
fs-copo12.suntradingllc.com: glusterfs 3.4.0 built on Jul 12 2013 19:07:08

Is there another way to determine GlusterFS volume/protocol version in 
3.4.0?


On 07/23/2013 03:12 PM, Joe Landman wrote:

On 07/23/2013 03:55 PM, Nicholas Majeran wrote:

FWIW, I tried to disable the parameter on a stopped volume, which was
successful.  I then started the volume and I could get/set the ACLs
normally.

I'm going to try the same procedure on the gv0 volume that threw the
error previously.
Thanks.


The inconsistent version possibility worries me.  What does glusterfs 
-V and glusterfsd -V report?


[root@crunch glusterfs-3.4.0]# glusterfs -V
glusterfs 3.4.0 built on Jul 23 2013 16:02:16
...

[root@crunch glusterfs-3.4.0]# glusterfsd -V
glusterfs 3.4.0 built on Jul 23 2013 16:02:16
...

for each machine that is a brick (glusterfsd) and each client 
(glusterfs)?


We've had problems in the past with incomplete updates between major 
version revisions, to the point that we had to completely scrub any 
old libraries and config files off some units during updates (3.2.x 
and 3.3.x time periods).






--
Nick Majeran
Sun Trading LLC
312-229-9608

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Joe Landman]

On 07/23/2013 03:55 PM, Nicholas Majeran wrote:

FWIW, I tried to disable the parameter on a stopped volume, which was
successful.  I then started the volume and I could get/set the ACLs
normally.

I'm going to try the same procedure on the gv0 volume that threw the
error previously.
Thanks.


The inconsistent version possibility worries me.  What does glusterfs -V 
and glusterfsd -V report?


[root@crunch glusterfs-3.4.0]# glusterfs -V
glusterfs 3.4.0 built on Jul 23 2013 16:02:16
...

[root@crunch glusterfs-3.4.0]# glusterfsd -V
glusterfs 3.4.0 built on Jul 23 2013 16:02:16
...

for each machine that is a brick (glusterfsd) and each client (glusterfs)?

We've had problems in the past with incomplete updates between major 
version revisions, to the point that we had to completely scrub any old 
libraries and config files off some units during updates (3.2.x and 
3.3.x time periods).




--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: land...@scalableinformatics.com
web  : http://scalableinformatics.com
   http://scalableinformatics.com/siflash
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615
___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

FWIW, I tried to disable the parameter on a stopped volume, which was 
successful.  I then started the volume and I could get/set the ACLs 
normally.


I'm going to try the same procedure on the gv0 volume that threw the 
error previously.

Thanks.

On 07/23/2013 02:29 PM, Nicholas Majeran wrote:

When I tried to disable that parameter, I receive the following:

gluster> volume set gv0 stat-prefetch off
volume set: failed: One or more connected clients cannot support the 
feature being set. These clients need to be upgraded or disconnected 
before running this command again


AFAICT, all the nodes are at the same version, but I did this this in 
the logs I after I ran the command:


[2013-07-23 19:26:02.673304] E 
[glusterd-op-sm.c:370:glusterd_check_client_op_version_support] 
0-management: One or more clients don't support the required op-version
[2013-07-23 19:26:02.673325] E 
[glusterd-syncop.c:767:gd_stage_op_phase] 0-management: Staging of 
operation 'Volume Set' failed on localhost : One or more connected 
clients cannot support the feature being set. These clients need to be 
upgraded or disconnected before running this command again
[2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)



On 07/23/2013 01:45 PM, Vijay Bellur wrote:

On 07/23/2013 09:28 PM, Nicholas Majeran wrote:

A little more detail... on files, I still can't get this to work, but
when I run ls and {get,set}facl on . it seems to work:

[root@s1 tmp]# pwd
/mnt/glusterfs/tmp
[root@s1 tmp]# ls -ld .
drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
[root@suncosmbgw1 tmp]# ls -ld ../tmp
drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
[root@s1 tmp]# getfacl .
# file: .
# owner: nmajeran
# group: root
user::rwx
user:root:rwx
user:user1:rwx
group::r-x
group:g1:rwx
group:g2:r-x
group:g3:r-x
mask::rwx
other::r-x

[root@s1 tmp]# getfacl ../tmp
# file: ../tmp
# owner: nmajeran
# group: root
user::rwx
group::rwx
other::r-x




Does disabling stat-prefetch address the problem permanently? You can 
disable stat-prefetch via:


gluster volume set  stat-prefetch off

-Vijay



___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users


___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

When I tried to disable that parameter, I receive the following:

gluster> volume set gv0 stat-prefetch off
volume set: failed: One or more connected clients cannot support the 
feature being set. These clients need to be upgraded or disconnected 
before running this command again


AFAICT, all the nodes are at the same version, but I did this this in 
the logs I after I ran the command:


[2013-07-23 19:26:02.673304] E 
[glusterd-op-sm.c:370:glusterd_check_client_op_version_support] 
0-management: One or more clients don't support the required op-version
[2013-07-23 19:26:02.673325] E [glusterd-syncop.c:767:gd_stage_op_phase] 
0-management: Staging of operation 'Volume Set' failed on localhost : 
One or more connected clients cannot support the feature being set. 
These clients need to be upgraded or disconnected before running this 
command again
[2013-07-23 19:26:03.590547] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:06.591224] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:09.591912] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:12.592601] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:15.593282] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)
[2013-07-23 19:26:18.593946] E [socket.c:2788:socket_connect] 
0-management: connection attempt failed (Connection refused)



On 07/23/2013 01:45 PM, Vijay Bellur wrote:

On 07/23/2013 09:28 PM, Nicholas Majeran wrote:

A little more detail... on files, I still can't get this to work, but
when I run ls and {get,set}facl on . it seems to work:

[root@s1 tmp]# pwd
/mnt/glusterfs/tmp
[root@s1 tmp]# ls -ld .
drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
[root@suncosmbgw1 tmp]# ls -ld ../tmp
drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
[root@s1 tmp]# getfacl .
# file: .
# owner: nmajeran
# group: root
user::rwx
user:root:rwx
user:user1:rwx
group::r-x
group:g1:rwx
group:g2:r-x
group:g3:r-x
mask::rwx
other::r-x

[root@s1 tmp]# getfacl ../tmp
# file: ../tmp
# owner: nmajeran
# group: root
user::rwx
group::rwx
other::r-x




Does disabling stat-prefetch address the problem permanently? You can 
disable stat-prefetch via:


gluster volume set  stat-prefetch off

-Vijay



___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Vijay Bellur]

On 07/23/2013 09:28 PM, Nicholas Majeran wrote:

A little more detail... on files, I still can't get this to work, but
when I run ls and {get,set}facl on . it seems to work:

[root@s1 tmp]# pwd
/mnt/glusterfs/tmp
[root@s1 tmp]# ls -ld .
drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
[root@suncosmbgw1 tmp]# ls -ld ../tmp
drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
[root@s1 tmp]# getfacl .
# file: .
# owner: nmajeran
# group: root
user::rwx
user:root:rwx
user:user1:rwx
group::r-x
group:g1:rwx
group:g2:r-x
group:g3:r-x
mask::rwx
other::r-x

[root@s1 tmp]# getfacl ../tmp
# file: ../tmp
# owner: nmajeran
# group: root
user::rwx
group::rwx
other::r-x




Does disabling stat-prefetch address the problem permanently? You can 
disable stat-prefetch via:


gluster volume set  stat-prefetch off

-Vijay

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

A little more detail... on files, I still can't get this to work, but 
when I run ls and {get,set}facl on . it seems to work:


[root@s1 tmp]# pwd
/mnt/glusterfs/tmp
[root@s1 tmp]# ls -ld .
drwxrwxr-x+ 4 nmajeran root 4422 Jul 22 15:29 .
[root@suncosmbgw1 tmp]# ls -ld ../tmp
drwxrwxr-x 4 nmajeran root 4422 Jul 22 15:29 ../tmp
[root@s1 tmp]# getfacl .
# file: .
# owner: nmajeran
# group: root
user::rwx
user:root:rwx
user:user1:rwx
group::r-x
group:g1:rwx
group:g2:r-x
group:g3:r-x
mask::rwx
other::r-x

[root@s1 tmp]# getfacl ../tmp
# file: ../tmp
# owner: nmajeran
# group: root
user::rwx
group::rwx
other::r-x


On 07/23/2013 09:57 AM, Nicholas Majeran wrote:

Hello:

I'm wondering if there is a known issue with displaying / editing ACLs 
using the native client and 3.4.0?  I don't see the problem on ext4, 
although our installed base is primarily XFS.


The closest thing I've seen is this bug :

https://bugzilla.redhat.com/show_bug.cgi?id=962450

Which states exactly the problem I'm seeing using the native client on 
CentOS 6.4.


Any ideas?
Thanks.

-- nick

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users


--
Nick Majeran
Sun Trading LLC
312-229-9608

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] ACL issue with 3.4.0 GA + XFS + native client?

[Nicholas Majeran]

Hello:

I'm wondering if there is a known issue with displaying / editing ACLs 
using the native client and 3.4.0?  I don't see the problem on ext4, 
although our installed base is primarily XFS.


The closest thing I've seen is this bug :

https://bugzilla.redhat.com/show_bug.cgi?id=962450

Which states exactly the problem I'm seeing using the native client on 
CentOS 6.4.


Any ideas?
Thanks.

-- nick

___
Gluster-users mailing list
Gluster-users@gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL

[Thai. Ngo Bao]

Hi Anush,

Thanks for your quick reply. There was no error when set ACL. Below is what I 
has done at Client Side:

1.   Under root user: mount -t glusterfs IP_server:/volume_name  -o acl  
/mnt

2.   Under root user at client, create a folder named thainb: mkdir -p 
/mnt/thainb

3.   Under root user: chown -R thainb:thainb /mnt/thainb

4.   Under root user: set  -m u:thainb:rw /mnt/thainb

5.   su thainb

6.   create some folder and file under thainb folder

7.   su peter

8.   remove file and folder

9.   Permission denied log from glusterfs:

[2011-10-20 08:58:17.603705] W [fuse-bridge.c:847:fuse_err_cbk] 
0-glusterfs-fuse: 2646: ACCESS() /thainb/readme => -1 (Permission denied)

[2011-10-20 08:58:19.7369] W [fuse-bridge.c:908:fuse_unlink_cbk] 
0-glusterfs-fuse: 2648: UNLINK() /thainb/readme => -1 (Permission denied)



10.   Exit su and under root user, remove file and folder

11.   No log from gluster

---snip-
[root@GSO_DB_Local4 thainb]# su peter
[peter@GSO_DB_Local4 thainb]$ ls -al
total 56
drwxrwxr-x+  3 thainb thainb 8192 Oct 20 08:56 .
drwxr-xr-x  26 root   root   8192 Oct 20 08:34 ..
-rw-rw-r--   1 thainb thainb6 Oct 20 08:56 readme
drwxrwxr-x   2 thainb thainb 8192 Oct 20 08:40 test

[peter@GSO_DB_Local4 thainb]$ rm readme
rm: remove write-protected regular file `readme'? y
rm: cannot remove `readme': Permission denied
[peter@GSO_DB_Local4 thainb]$ exit
exit
[root@GSO_DB_Local4 thainb]# ls
readme  test
[root@GSO_DB_Local4 thainb]# rm readme
rm: remove regular file `readme'? y
[root@GSO_DB_Local4 thainb]# rm -f test
rm: cannot remove `test': Is a directory
[root@GSO_DB_Local4 thainb]# rm -rf test
[root@GSO_DB_Local4 thainb]# ls
[root@GSO_DB_Local4 thainb]# getfacl .
# file: .
# owner: thainb
# group: thainb
user::rwx
user:thainb:rw-
group::r-x
mask::rwx
other::r-x

[root@GSO_DB_Local4 thainb]# pwd
/mnt/thainb
[root@GSO_DB_Local4 thainb]# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda3  74G  7.1G   63G  11% /
/dev/sda1  99M   12M   82M  13% /boot
tmpfs 2.0G 0  2.0G   0% /dev/shm
/dev/sda3  74G  7.1G   63G  11% /data
glusterfs#lab3:/farm53   148G  9.5G  131G   7% /mnt

-end--

What do you suggest?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Wednesday, October 19, 2011 5:58 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi,

Do you see any error messages when you try to set ACL? Can you also paste the 
logs here?

-
Anush


From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 19 October 2011 16:16:36
To: gluster-users@gluster.org
Subject: [Gluster-users] ACL
Hi,

I am testing gluster 3.2.4 with ACL on small linux cluster:


1.   All exports (bricks) mounted with -o acl option

2.   All the glusterfs clients mounted with -o acl option

Acl works perfectly when clients mount under normal users but it will not work 
if clients mount under root user.  What am I missing? How can we force clients 
to mount under normal user rather than root user?

Thanks,
~Thai
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

[Gluster-users] ACL option ignored?

[James Yale]

Hi,

Yesterday I was having a problem with glusterfs-3.2.5-6.el6.x86_64
making shares mounted using the native not accessible by other users
than the one that performed the mount (allow_other option). To get
round the problem I passed some fuse options to the mount (mount -t
glusterfs -o fuse-opt=allow_other) but also upgraded my package to
glusterfs-3.2.5-7.el6.x86_64 - now the box has been rebooted all the
gluster mounts seem to get the allow_other option whether I specify it
or not, which is good, however now the ACL flag seems to be ignored.

Mounting from the command line or fstab results in a mount with no ACL support:

On client (I added some debug lines to /usr/sbin/mount.glusterfs to
check it's getting the acl option):

[root@puppet jim]# mount -t glusterfs -o acl
192.168.241.40:/gluster-webcontent /tmp/test/
DEBUG: found acl
DEBUG: cmd_line is /usr/sbin/glusterfs --log-level=INFO --acl
DEBUG: command line run is /usr/sbin/glusterfs --log-level=INFO --acl
--volfile-id=/gluster-webcontent --volfile-server=192.168.241.40
/tmp/test/

[root@puppet jim]# mount
/dev/xvda1 on / type ext4 (rw,noatime)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
192.168.241.40:/gluster-webcontent on /tmp/test type fuse.glusterfs
(rw,allow_other,max_read=131072)

On server (to confirm acl support):

[jim@ip-192-168-241-40 ~]$ mount | grep webcontent
/mnt/xvdf on /exports/webcontent type none (rw,bind,acl)

[jim@ip-192-168-241-40 ~]$ sudo gluster volume info

Volume Name: gluster-webcontent
Type: Replicate
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: 192.168.241.40:/exports/webcontent
Brick2: 192.168.241.41:/exports/webcontent

Full client --debug output here: http://pastie.org/3400564

Yesterday with glusterfs-3.2.5-6.el6.x86_64 I'm sure this worked, and
I saw acl options, today with glusterfs-3.2.5-7.el6.x86_64 I don't,
but I don't see any major changes in the change log.

Any ideas for further testing appreciated!

Thanks,

James
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL

[Gerald Brandt]

Hi,

- Original Message -

> From: "Thai. Ngo Bao" 
> To: "Anush Shetty" , gluster-users@gluster.org
> Sent: Friday, October 21, 2011 2:05:12 AM
> Subject: Re: [Gluster-users] ACL

> Hi Anush,

> Well, I was aware of this feature of glusterfs several months ago and
> one will lose the advantage of glusterfs (such as high performance,
> fail-over, etc) if he/she uses gluster nfs instead of gluster native
> protocol.
> I believe using gluster nfs will solve the problem I am concerning
> with (export some subdirectory per client). Could you please point
> me out what one will lose when using gluster nsf instead of native
> protocol in terms of performance?

> Thanks,
> ~Thai

Not necessarily. Take a look at this: 
http://community.gluster.org/p/nfs-performance-with-fuse-client-redundancy/

I plan on testing it shortly, with the NFS layer in a Citrix XenServer VM 
talking to two replicated bricks. I'm expecting higher read throughput, but 
higher latency as well. I can only give the VM NFS Server about 2 GB of RAM, so 
things will be tight,

Gerald

> From: Anush Shetty [mailto:an...@gluster.com]
> Sent: Friday, October 21, 2011 12:07 PM
> To: Thai. Ngo Bao; gluster-users@gluster.org
> Subject: RE: [Gluster-users] ACL

> Hi Thai,

> Yes, we do.

> http://community.gluster.org/q/export-subdirectories-of-volumes-with-nfs/

> -
> Anush

> From: Thai. Ngo Bao [tha...@vng.com.vn]
> Sent: 21 October 2011 08:21:32
> To: Anush Shetty; gluster-users@gluster.org
> Subject: RE: [Gluster-users] ACL

> Hi Anush,

> Thanks for the info. Also, I wonder if gluster concerns with some
> feature allowing glusterfs to export subdirectories for clients by
> gluster native protocol like NFS?

> Thanks,
> ~Thai

> From: Anush Shetty [mailto:an...@gluster.com]
> Sent: Thursday, October 20, 2011 1:37 PM
> To: Thai. Ngo Bao; gluster-users@gluster.org
> Subject: RE: [Gluster-users] ACL

> Hi Thai,

> As of now, there are no provisions to mount GlusterFS client as a
> normal user.

> -
> Anush

> From: gluster-users-boun...@gluster.org
> [gluster-users-boun...@gluster.org] on behalf of Thai. Ngo Bao
> [tha...@vng.com.vn]
> Sent: 20 October 2011 12:00:44
> To: Anush Shetty; gluster-users@gluster.org
> Subject: Re: [Gluster-users] ACL

> Hi Anush,

> Thanks for your response. I do know that there is almost no
> difference in behavior of these 2 cases (POSIX ACL) except the thing
> I concerned with by the test. Please correct me if I am wrong.
> It turns out that glusterfs ACL has no effect if client mounts
> gluster volume under root user. That’s why I asked in previous email
> if we can anyhow force (from glusterfs servers) clients to mount
> under a normal user instead of root.

> What do you think?

> ~Thai

> From: Anush Shetty [mailto:an...@gluster.com]
> Sent: Thursday, October 20, 2011 1:03 PM
> To: Thai. Ngo Bao; gluster-users@gluster.org
> Subject: RE: [Gluster-users] ACL

> Hi Thai,

> Have you tried these steps without glusterfs in the picture? I see no
> difference in behaviour when tried directly on the backend
> filesystem.

> -
> Anush

> From: Thai. Ngo Bao [tha...@vng.com.vn]
> Sent: 20 October 2011 08:04:31
> To: Anush Shetty; gluster-users@gluster.org
> Subject: RE: [Gluster-users] ACL

> Hi Anush,

> Thanks for your quick reply. There was no error when set ACL. Below
> is what I has done at Client Side:
> 1. Under root user: mount –t glusterfs IP_server:/volume_name –o acl
> /mnt
> 2. Under root user at client, create a folder named thainb: mkdir –p
> /mnt/thainb
> 3. Under root user: chown –R thainb:thainb /mnt/thainb
> 4. Under root user: set -m u:thainb:rw /mnt/thainb
> 5. su thainb
> 6. create some folder and file under thainb folder
> 7. su peter
> 8. remove file and folder
> 9. Permission denied log from glusterfs:
> [2011-10-20 08:58:17.603705] W [fuse-bridge.c:847:fuse_err_cbk]
> 0-glusterfs-fuse: 2646: ACCESS() /thainb/readme => -1 (Permission
> denied)
> [2011-10-20 08:58:19.7369] W [fuse-bridge.c:908:fuse_unlink_cbk]
> 0-glusterfs-fuse: 2648: UNLINK() /thainb/readme => -1 (Permission
> denied)

> 10. Exit su and under root user, remove file and folder
> 11. No log from gluster

> ---snip-
> [root@GSO_DB_Local4 thainb]# su peter
> [peter@GSO_DB_Local4 thainb]$ ls -al
> total 56
> drwxrwxr-x+ 3 thainb thainb 8192 Oct 20 08:56 .
> drwxr-xr-x 26 root root 8192 Oct 20 08:34 ..
> -rw-rw-r-- 1 thainb thainb 6 Oct 20 08:56 readme
> drwxrwxr-x 2 thainb thainb 8192 Oct 20 08:40 test

> [peter@GSO_DB_Local4 thainb]$ rm readme
> rm: remove write-protected regular file `readme'? y
> rm: cannot remove `readme': Permission denied
&g

Re: [Gluster-users] ACL

[Thai. Ngo Bao]

Hi Anush,

Well, I was aware of this feature of glusterfs several months ago and one will 
lose the advantage of glusterfs (such as high performance, fail-over, etc) if 
he/she uses gluster nfs instead of gluster native protocol.
I believe using gluster nfs will solve the problem I am concerning with (export 
some subdirectory per client). Could you please point me out what one will lose 
when using gluster nsf instead of native protocol in terms of performance?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Friday, October 21, 2011 12:07 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

Yes, we do.

http://community.gluster.org/q/export-subdirectories-of-volumes-with-nfs/

-
Anush


From: Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 21 October 2011 08:21:32
To: Anush Shetty; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL
Hi Anush,

Thanks for the info. Also, I wonder if gluster concerns with some feature 
allowing glusterfs to export subdirectories for clients by gluster native 
protocol like NFS?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Thursday, October 20, 2011 1:37 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

As of now, there are no provisions to mount GlusterFS client as a normal user.

-
Anush

From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 12:00:44
To: Anush Shetty; gluster-users@gluster.org
Subject: Re: [Gluster-users] ACL
Hi Anush,

Thanks for your response. I do know that there is almost no difference in 
behavior of these  2 cases (POSIX ACL) except the thing I concerned with by the 
test. Please correct me if I am wrong.
It turns out that glusterfs ACL has no effect if client mounts gluster volume 
under root user. That's why I asked in previous email if we can anyhow force 
(from glusterfs servers) clients to mount under a normal user instead of root.

What do you think?

~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Thursday, October 20, 2011 1:03 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

Have you tried these steps without glusterfs in the picture? I see no 
difference in behaviour when tried directly on the backend filesystem.

-
Anush

From: Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 08:04:31
To: Anush Shetty; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL
Hi Anush,

Thanks for your quick reply. There was no error when set ACL. Below is what I 
has done at Client Side:

1.   Under root user: mount -t glusterfs IP_server:/volume_name  -o acl  
/mnt

2.   Under root user at client, create a folder named thainb: mkdir -p 
/mnt/thainb

3.   Under root user: chown -R thainb:thainb /mnt/thainb

4.   Under root user: set  -m u:thainb:rw /mnt/thainb

5.   su thainb

6.   create some folder and file under thainb folder

7.   su peter

8.   remove file and folder

9.   Permission denied log from glusterfs:

[2011-10-20 08:58:17.603705] W [fuse-bridge.c:847:fuse_err_cbk] 
0-glusterfs-fuse: 2646: ACCESS() /thainb/readme => -1 (Permission denied)

[2011-10-20 08:58:19.7369] W [fuse-bridge.c:908:fuse_unlink_cbk] 
0-glusterfs-fuse: 2648: UNLINK() /thainb/readme => -1 (Permission denied)



10.   Exit su and under root user, remove file and folder

11.   No log from gluster

---snip-
[root@GSO_DB_Local4 thainb]# su peter
[peter@GSO_DB_Local4 thainb]$ ls -al
total 56
drwxrwxr-x+  3 thainb thainb 8192 Oct 20 08:56 .
drwxr-xr-x  26 root   root   8192 Oct 20 08:34 ..
-rw-rw-r--   1 thainb thainb6 Oct 20 08:56 readme
drwxrwxr-x   2 thainb thainb 8192 Oct 20 08:40 test

[peter@GSO_DB_Local4 thainb]$ rm readme
rm: remove write-protected regular file `readme'? y
rm: cannot remove `readme': Permission denied
[peter@GSO_DB_Local4 thainb]$ exit
exit
[root@GSO_DB_Local4 thainb]# ls
readme  test
[root@GSO_DB_Local4 thainb]# rm readme
rm: remove regular file `readme'? y
[root@GSO_DB_Local4 thainb]# rm -f test
rm: cannot remove `test': Is a directory
[root@GSO_DB_Local4 thainb]# rm -rf test
[root@GSO_DB_Local4 thainb]# ls
[root@GSO_DB_Local4 thainb]# getfacl .
# file: .
# owner: thainb
# group: thainb
user::rwx
user:thainb:rw-
group::r-x
mask::rwx
other::r-x

[root@GSO_DB_Local4 thainb]# pwd
/mnt/thainb
[root@GSO_DB_Local4 thainb]# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda3  74G  7.1G   63G  11% /
/dev/sda1  99M   12M   82M  13% /boot
tmpfs 2.0G 0  2.0G   0% /dev/shm
/dev/sda3  74G  7.1G   63G  11% /data
glusterfs#lab3:/farm53   148G  9.5G  131G   7% /mnt

-end--

What do you suggest?

Th

Re: [Gluster-users] ACL

[Anush Shetty]

Hi Thai,

Yes, we do.

http://community.gluster.org/q/export-subdirectories-of-volumes-with-nfs/

-
Anush



From: Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 21 October 2011 08:21:32
To: Anush Shetty; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Anush,

Thanks for the info. Also, I wonder if gluster concerns with some feature 
allowing glusterfs to export subdirectories for clients by gluster native 
protocol like NFS?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Thursday, October 20, 2011 1:37 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

As of now, there are no provisions to mount GlusterFS client as a normal user.

-
Anush

From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 12:00:44
To: Anush Shetty; gluster-users@gluster.org
Subject: Re: [Gluster-users] ACL
Hi Anush,

Thanks for your response. I do know that there is almost no difference in 
behavior of these  2 cases (POSIX ACL) except the thing I concerned with by the 
test. Please correct me if I am wrong.
It turns out that glusterfs ACL has no effect if client mounts gluster volume 
under root user. That’s why I asked in previous email if we can anyhow force 
(from glusterfs servers) clients to mount under a normal user instead of root.

What do you think?

~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Thursday, October 20, 2011 1:03 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

Have you tried these steps without glusterfs in the picture? I see no 
difference in behaviour when tried directly on the backend filesystem.

-
Anush

From: Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 08:04:31
To: Anush Shetty; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL
Hi Anush,

Thanks for your quick reply. There was no error when set ACL. Below is what I 
has done at Client Side:

1.   Under root user: mount –t glusterfs IP_server:/volume_name  –o acl  
/mnt

2.   Under root user at client, create a folder named thainb: mkdir –p 
/mnt/thainb

3.   Under root user: chown –R thainb:thainb /mnt/thainb

4.   Under root user: set  -m u:thainb:rw /mnt/thainb

5.   su thainb

6.   create some folder and file under thainb folder

7.   su peter

8.   remove file and folder

9.   Permission denied log from glusterfs:

[2011-10-20 08:58:17.603705] W [fuse-bridge.c:847:fuse_err_cbk] 
0-glusterfs-fuse: 2646: ACCESS() /thainb/readme => -1 (Permission denied)

[2011-10-20 08:58:19.7369] W [fuse-bridge.c:908:fuse_unlink_cbk] 
0-glusterfs-fuse: 2648: UNLINK() /thainb/readme => -1 (Permission denied)



10.   Exit su and under root user, remove file and folder

11.   No log from gluster

---snip-
[root@GSO_DB_Local4 thainb]# su peter
[peter@GSO_DB_Local4 thainb]$ ls -al
total 56
drwxrwxr-x+  3 thainb thainb 8192 Oct 20 08:56 .
drwxr-xr-x  26 root   root   8192 Oct 20 08:34 ..
-rw-rw-r--   1 thainb thainb6 Oct 20 08:56 readme
drwxrwxr-x   2 thainb thainb 8192 Oct 20 08:40 test

[peter@GSO_DB_Local4 thainb]$ rm readme
rm: remove write-protected regular file `readme'? y
rm: cannot remove `readme': Permission denied
[peter@GSO_DB_Local4 thainb]$ exit
exit
[root@GSO_DB_Local4 thainb]# ls
readme  test
[root@GSO_DB_Local4 thainb]# rm readme
rm: remove regular file `readme'? y
[root@GSO_DB_Local4 thainb]# rm -f test
rm: cannot remove `test': Is a directory
[root@GSO_DB_Local4 thainb]# rm -rf test
[root@GSO_DB_Local4 thainb]# ls
[root@GSO_DB_Local4 thainb]# getfacl .
# file: .
# owner: thainb
# group: thainb
user::rwx
user:thainb:rw-
group::r-x
mask::rwx
other::r-x

[root@GSO_DB_Local4 thainb]# pwd
/mnt/thainb
[root@GSO_DB_Local4 thainb]# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda3  74G  7.1G   63G  11% /
/dev/sda1  99M   12M   82M  13% /boot
tmpfs 2.0G 0  2.0G   0% /dev/shm
/dev/sda3  74G  7.1G   63G  11% /data
glusterfs#lab3:/farm53   148G  9.5G  131G   7% /mnt

-end--

What do you suggest?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Wednesday, October 19, 2011 5:58 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi,

Do you see any error messages when you try to set ACL? Can you also paste the 
logs here?

-
Anush

From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 19 October 2011 16:16:36
To: gluster-users@gluster.org
Subject: [Gluster-users] ACL
Hi,

I am testing gluster 3.2.4 with ACL on small linux cluster:


1.   All exports (bricks) mounted with –o ac

Re: [Gluster-users] ACL

[Thai. Ngo Bao]

Hi Anush,

Thanks for the info. Also, I wonder if gluster concerns with some feature 
allowing glusterfs to export subdirectories for clients by gluster native 
protocol like NFS?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Thursday, October 20, 2011 1:37 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

As of now, there are no provisions to mount GlusterFS client as a normal user.

-
Anush

From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 12:00:44
To: Anush Shetty; gluster-users@gluster.org
Subject: Re: [Gluster-users] ACL
Hi Anush,

Thanks for your response. I do know that there is almost no difference in 
behavior of these  2 cases (POSIX ACL) except the thing I concerned with by the 
test. Please correct me if I am wrong.
It turns out that glusterfs ACL has no effect if client mounts gluster volume 
under root user. That's why I asked in previous email if we can anyhow force 
(from glusterfs servers) clients to mount under a normal user instead of root.

What do you think?

~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Thursday, October 20, 2011 1:03 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

Have you tried these steps without glusterfs in the picture? I see no 
difference in behaviour when tried directly on the backend filesystem.

-
Anush

From: Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 08:04:31
To: Anush Shetty; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL
Hi Anush,

Thanks for your quick reply. There was no error when set ACL. Below is what I 
has done at Client Side:

1.   Under root user: mount -t glusterfs IP_server:/volume_name  -o acl  
/mnt

2.   Under root user at client, create a folder named thainb: mkdir -p 
/mnt/thainb

3.   Under root user: chown -R thainb:thainb /mnt/thainb

4.   Under root user: set  -m u:thainb:rw /mnt/thainb

5.   su thainb

6.   create some folder and file under thainb folder

7.   su peter

8.   remove file and folder

9.   Permission denied log from glusterfs:

[2011-10-20 08:58:17.603705] W [fuse-bridge.c:847:fuse_err_cbk] 
0-glusterfs-fuse: 2646: ACCESS() /thainb/readme => -1 (Permission denied)

[2011-10-20 08:58:19.7369] W [fuse-bridge.c:908:fuse_unlink_cbk] 
0-glusterfs-fuse: 2648: UNLINK() /thainb/readme => -1 (Permission denied)



10.   Exit su and under root user, remove file and folder

11.   No log from gluster

---snip-
[root@GSO_DB_Local4 thainb]# su peter
[peter@GSO_DB_Local4 thainb]$ ls -al
total 56
drwxrwxr-x+  3 thainb thainb 8192 Oct 20 08:56 .
drwxr-xr-x  26 root   root   8192 Oct 20 08:34 ..
-rw-rw-r--   1 thainb thainb6 Oct 20 08:56 readme
drwxrwxr-x   2 thainb thainb 8192 Oct 20 08:40 test

[peter@GSO_DB_Local4 thainb]$ rm readme
rm: remove write-protected regular file `readme'? y
rm: cannot remove `readme': Permission denied
[peter@GSO_DB_Local4 thainb]$ exit
exit
[root@GSO_DB_Local4 thainb]# ls
readme  test
[root@GSO_DB_Local4 thainb]# rm readme
rm: remove regular file `readme'? y
[root@GSO_DB_Local4 thainb]# rm -f test
rm: cannot remove `test': Is a directory
[root@GSO_DB_Local4 thainb]# rm -rf test
[root@GSO_DB_Local4 thainb]# ls
[root@GSO_DB_Local4 thainb]# getfacl .
# file: .
# owner: thainb
# group: thainb
user::rwx
user:thainb:rw-
group::r-x
mask::rwx
other::r-x

[root@GSO_DB_Local4 thainb]# pwd
/mnt/thainb
[root@GSO_DB_Local4 thainb]# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda3  74G  7.1G   63G  11% /
/dev/sda1  99M   12M   82M  13% /boot
tmpfs 2.0G 0  2.0G   0% /dev/shm
/dev/sda3  74G  7.1G   63G  11% /data
glusterfs#lab3:/farm53   148G  9.5G  131G   7% /mnt

-end--

What do you suggest?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Wednesday, October 19, 2011 5:58 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi,

Do you see any error messages when you try to set ACL? Can you also paste the 
logs here?

-
Anush

From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 19 October 2011 16:16:36
To: gluster-users@gluster.org
Subject: [Gluster-users] ACL
Hi,

I am testing gluster 3.2.4 with ACL on small linux cluster:


1.   All exports (bricks) mounted with -o acl option

2.   All the glusterfs clients mounted with -o acl option

Acl works perfectly when clients mount under normal users but it will not work 
if clients mount under root user.  What am I missing? How can we force clients 
to mount under normal user rather

Re: [Gluster-users] ACL

[Anush Shetty]

Hi Thai,

As of now, there are no provisions to mount GlusterFS client as a normal user.

-
Anush

From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 12:00:44
To: Anush Shetty; gluster-users@gluster.org
Subject: Re: [Gluster-users] ACL

Hi Anush,

Thanks for your response. I do know that there is almost no difference in 
behavior of these  2 cases (POSIX ACL) except the thing I concerned with by the 
test. Please correct me if I am wrong.
It turns out that glusterfs ACL has no effect if client mounts gluster volume 
under root user. That’s why I asked in previous email if we can anyhow force 
(from glusterfs servers) clients to mount under a normal user instead of root.

What do you think?

~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Thursday, October 20, 2011 1:03 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

Have you tried these steps without glusterfs in the picture? I see no 
difference in behaviour when tried directly on the backend filesystem.

-
Anush

From: Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 08:04:31
To: Anush Shetty; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL
Hi Anush,

Thanks for your quick reply. There was no error when set ACL. Below is what I 
has done at Client Side:

1.   Under root user: mount –t glusterfs IP_server:/volume_name  –o acl  
/mnt

2.   Under root user at client, create a folder named thainb: mkdir –p 
/mnt/thainb

3.   Under root user: chown –R thainb:thainb /mnt/thainb

4.   Under root user: set  -m u:thainb:rw /mnt/thainb

5.   su thainb

6.   create some folder and file under thainb folder

7.   su peter

8.   remove file and folder

9.   Permission denied log from glusterfs:

[2011-10-20 08:58:17.603705] W [fuse-bridge.c:847:fuse_err_cbk] 
0-glusterfs-fuse: 2646: ACCESS() /thainb/readme => -1 (Permission denied)

[2011-10-20 08:58:19.7369] W [fuse-bridge.c:908:fuse_unlink_cbk] 
0-glusterfs-fuse: 2648: UNLINK() /thainb/readme => -1 (Permission denied)



10.   Exit su and under root user, remove file and folder

11.   No log from gluster

---snip-
[root@GSO_DB_Local4 thainb]# su peter
[peter@GSO_DB_Local4 thainb]$ ls -al
total 56
drwxrwxr-x+  3 thainb thainb 8192 Oct 20 08:56 .
drwxr-xr-x  26 root   root   8192 Oct 20 08:34 ..
-rw-rw-r--   1 thainb thainb6 Oct 20 08:56 readme
drwxrwxr-x   2 thainb thainb 8192 Oct 20 08:40 test

[peter@GSO_DB_Local4 thainb]$ rm readme
rm: remove write-protected regular file `readme'? y
rm: cannot remove `readme': Permission denied
[peter@GSO_DB_Local4 thainb]$ exit
exit
[root@GSO_DB_Local4 thainb]# ls
readme  test
[root@GSO_DB_Local4 thainb]# rm readme
rm: remove regular file `readme'? y
[root@GSO_DB_Local4 thainb]# rm -f test
rm: cannot remove `test': Is a directory
[root@GSO_DB_Local4 thainb]# rm -rf test
[root@GSO_DB_Local4 thainb]# ls
[root@GSO_DB_Local4 thainb]# getfacl .
# file: .
# owner: thainb
# group: thainb
user::rwx
user:thainb:rw-
group::r-x
mask::rwx
other::r-x

[root@GSO_DB_Local4 thainb]# pwd
/mnt/thainb
[root@GSO_DB_Local4 thainb]# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda3  74G  7.1G   63G  11% /
/dev/sda1  99M   12M   82M  13% /boot
tmpfs 2.0G 0  2.0G   0% /dev/shm
/dev/sda3  74G  7.1G   63G  11% /data
glusterfs#lab3:/farm53   148G  9.5G  131G   7% /mnt

-end--

What do you suggest?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Wednesday, October 19, 2011 5:58 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi,

Do you see any error messages when you try to set ACL? Can you also paste the 
logs here?

-
Anush

From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 19 October 2011 16:16:36
To: gluster-users@gluster.org
Subject: [Gluster-users] ACL
Hi,

I am testing gluster 3.2.4 with ACL on small linux cluster:


1.   All exports (bricks) mounted with –o acl option

2.   All the glusterfs clients mounted with –o acl option

Acl works perfectly when clients mount under normal users but it will not work 
if clients mount under root user.  What am I missing? How can we force clients 
to mount under normal user rather than root user?

Thanks,
~Thai

No virus found in this message.
Checked by AVG - www.avg.com<http://www.avg.com>
Version: 2012.0.1831 / Virus Database: 2092/4562 - Release Date: 10/19/11
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL

[Thai. Ngo Bao]

Hi Anush,

Thanks for your response. I do know that there is almost no difference in 
behavior of these  2 cases (POSIX ACL) except the thing I concerned with by the 
test. Please correct me if I am wrong.
It turns out that glusterfs ACL has no effect if client mounts gluster volume 
under root user. That's why I asked in previous email if we can anyhow force 
(from glusterfs servers) clients to mount under a normal user instead of root.

What do you think?

~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Thursday, October 20, 2011 1:03 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Thai,

Have you tried these steps without glusterfs in the picture? I see no 
difference in behaviour when tried directly on the backend filesystem.

-
Anush

From: Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 08:04:31
To: Anush Shetty; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL
Hi Anush,

Thanks for your quick reply. There was no error when set ACL. Below is what I 
has done at Client Side:

1.   Under root user: mount -t glusterfs IP_server:/volume_name  -o acl  
/mnt

2.   Under root user at client, create a folder named thainb: mkdir -p 
/mnt/thainb

3.   Under root user: chown -R thainb:thainb /mnt/thainb

4.   Under root user: set  -m u:thainb:rw /mnt/thainb

5.   su thainb

6.   create some folder and file under thainb folder

7.   su peter

8.   remove file and folder

9.   Permission denied log from glusterfs:

[2011-10-20 08:58:17.603705] W [fuse-bridge.c:847:fuse_err_cbk] 
0-glusterfs-fuse: 2646: ACCESS() /thainb/readme => -1 (Permission denied)

[2011-10-20 08:58:19.7369] W [fuse-bridge.c:908:fuse_unlink_cbk] 
0-glusterfs-fuse: 2648: UNLINK() /thainb/readme => -1 (Permission denied)



10.   Exit su and under root user, remove file and folder

11.   No log from gluster

---snip-
[root@GSO_DB_Local4 thainb]# su peter
[peter@GSO_DB_Local4 thainb]$ ls -al
total 56
drwxrwxr-x+  3 thainb thainb 8192 Oct 20 08:56 .
drwxr-xr-x  26 root   root   8192 Oct 20 08:34 ..
-rw-rw-r--   1 thainb thainb6 Oct 20 08:56 readme
drwxrwxr-x   2 thainb thainb 8192 Oct 20 08:40 test

[peter@GSO_DB_Local4 thainb]$ rm readme
rm: remove write-protected regular file `readme'? y
rm: cannot remove `readme': Permission denied
[peter@GSO_DB_Local4 thainb]$ exit
exit
[root@GSO_DB_Local4 thainb]# ls
readme  test
[root@GSO_DB_Local4 thainb]# rm readme
rm: remove regular file `readme'? y
[root@GSO_DB_Local4 thainb]# rm -f test
rm: cannot remove `test': Is a directory
[root@GSO_DB_Local4 thainb]# rm -rf test
[root@GSO_DB_Local4 thainb]# ls
[root@GSO_DB_Local4 thainb]# getfacl .
# file: .
# owner: thainb
# group: thainb
user::rwx
user:thainb:rw-
group::r-x
mask::rwx
other::r-x

[root@GSO_DB_Local4 thainb]# pwd
/mnt/thainb
[root@GSO_DB_Local4 thainb]# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda3  74G  7.1G   63G  11% /
/dev/sda1  99M   12M   82M  13% /boot
tmpfs 2.0G 0  2.0G   0% /dev/shm
/dev/sda3  74G  7.1G   63G  11% /data
glusterfs#lab3:/farm53   148G  9.5G  131G   7% /mnt

-end--

What do you suggest?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Wednesday, October 19, 2011 5:58 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi,

Do you see any error messages when you try to set ACL? Can you also paste the 
logs here?

-
Anush

From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 19 October 2011 16:16:36
To: gluster-users@gluster.org
Subject: [Gluster-users] ACL
Hi,

I am testing gluster 3.2.4 with ACL on small linux cluster:


1.   All exports (bricks) mounted with -o acl option

2.   All the glusterfs clients mounted with -o acl option

Acl works perfectly when clients mount under normal users but it will not work 
if clients mount under root user.  What am I missing? How can we force clients 
to mount under normal user rather than root user?

Thanks,
~Thai

No virus found in this message.
Checked by AVG - www.avg.com<http://www.avg.com>
Version: 2012.0.1831 / Virus Database: 2092/4562 - Release Date: 10/19/11
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL

[Anush Shetty]

Hi Thai,

Have you tried these steps without glusterfs in the picture? I see no 
difference in behaviour when tried directly on the backend filesystem.

-
Anush

From: Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 20 October 2011 08:04:31
To: Anush Shetty; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi Anush,

Thanks for your quick reply. There was no error when set ACL. Below is what I 
has done at Client Side:

1.   Under root user: mount –t glusterfs IP_server:/volume_name  –o acl  
/mnt

2.   Under root user at client, create a folder named thainb: mkdir –p 
/mnt/thainb

3.   Under root user: chown –R thainb:thainb /mnt/thainb

4.   Under root user: set  -m u:thainb:rw /mnt/thainb

5.   su thainb

6.   create some folder and file under thainb folder

7.   su peter

8.   remove file and folder

9.   Permission denied log from glusterfs:

[2011-10-20 08:58:17.603705] W [fuse-bridge.c:847:fuse_err_cbk] 
0-glusterfs-fuse: 2646: ACCESS() /thainb/readme => -1 (Permission denied)

[2011-10-20 08:58:19.7369] W [fuse-bridge.c:908:fuse_unlink_cbk] 
0-glusterfs-fuse: 2648: UNLINK() /thainb/readme => -1 (Permission denied)



10.   Exit su and under root user, remove file and folder

11.   No log from gluster

---snip-
[root@GSO_DB_Local4 thainb]# su peter
[peter@GSO_DB_Local4 thainb]$ ls -al
total 56
drwxrwxr-x+  3 thainb thainb 8192 Oct 20 08:56 .
drwxr-xr-x  26 root   root   8192 Oct 20 08:34 ..
-rw-rw-r--   1 thainb thainb6 Oct 20 08:56 readme
drwxrwxr-x   2 thainb thainb 8192 Oct 20 08:40 test

[peter@GSO_DB_Local4 thainb]$ rm readme
rm: remove write-protected regular file `readme'? y
rm: cannot remove `readme': Permission denied
[peter@GSO_DB_Local4 thainb]$ exit
exit
[root@GSO_DB_Local4 thainb]# ls
readme  test
[root@GSO_DB_Local4 thainb]# rm readme
rm: remove regular file `readme'? y
[root@GSO_DB_Local4 thainb]# rm -f test
rm: cannot remove `test': Is a directory
[root@GSO_DB_Local4 thainb]# rm -rf test
[root@GSO_DB_Local4 thainb]# ls
[root@GSO_DB_Local4 thainb]# getfacl .
# file: .
# owner: thainb
# group: thainb
user::rwx
user:thainb:rw-
group::r-x
mask::rwx
other::r-x

[root@GSO_DB_Local4 thainb]# pwd
/mnt/thainb
[root@GSO_DB_Local4 thainb]# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda3  74G  7.1G   63G  11% /
/dev/sda1  99M   12M   82M  13% /boot
tmpfs 2.0G 0  2.0G   0% /dev/shm
/dev/sda3  74G  7.1G   63G  11% /data
glusterfs#lab3:/farm53   148G  9.5G  131G   7% /mnt

-end--

What do you suggest?

Thanks,
~Thai

From: Anush Shetty [mailto:an...@gluster.com]
Sent: Wednesday, October 19, 2011 5:58 PM
To: Thai. Ngo Bao; gluster-users@gluster.org
Subject: RE: [Gluster-users] ACL

Hi,

Do you see any error messages when you try to set ACL? Can you also paste the 
logs here?

-
Anush


From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 19 October 2011 16:16:36
To: gluster-users@gluster.org
Subject: [Gluster-users] ACL
Hi,

I am testing gluster 3.2.4 with ACL on small linux cluster:


1.   All exports (bricks) mounted with –o acl option

2.   All the glusterfs clients mounted with –o acl option

Acl works perfectly when clients mount under normal users but it will not work 
if clients mount under root user.  What am I missing? How can we force clients 
to mount under normal user rather than root user?

Thanks,
~Thai
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL

[Anush Shetty]

Hi,

Do you see any error messages when you try to set ACL? Can you also paste the 
logs here?

-
Anush



From: gluster-users-boun...@gluster.org [gluster-users-boun...@gluster.org] on 
behalf of Thai. Ngo Bao [tha...@vng.com.vn]
Sent: 19 October 2011 16:16:36
To: gluster-users@gluster.org
Subject: [Gluster-users] ACL

Hi,

I am testing gluster 3.2.4 with ACL on small linux cluster:


1.   All exports (bricks) mounted with –o acl option

2.   All the glusterfs clients mounted with –o acl option

Acl works perfectly when clients mount under normal users but it will not work 
if clients mount under root user.  What am I missing? How can we force clients 
to mount under normal user rather than root user?

Thanks,
~Thai

From: gluster-users-boun...@gluster.org 
[mailto:gluster-users-boun...@gluster.org] On Behalf Of Amar Tumballi
Sent: Tuesday, August 02, 2011 2:57 AM
To: Papp Tamas
Cc: gluster-users@gluster.org
Subject: Re: [Gluster-users] xfs+acl

To use 'acl' below thing should be minimum setup:

* glusterfs version 3.2.2 or higher
* all exports (ie bricks) mounted with '-oacl' option
* all the glusterfs clients mounted with '-oacl' option

Regards,
Amar
On Tue, Aug 2, 2011 at 1:06 AM, Papp Tamas 
mailto:tom...@martos.bme.hu>> wrote:
On 2011-08-01 21:32, Liam Slusser wrote:
We use XFS with Gluster with great success.

We also do.
I'm asking about acl.


Thank you,
tamas
___
Gluster-users mailing list
Gluster-users@gluster.org<mailto:Gluster-users@gluster.org>
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users


No virus found in this message.
Checked by AVG - www.avg.com<http://www.avg.com>
Version: 2012.0.1831 / Virus Database: 2092/4560 - Release Date: 10/18/11
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

[Gluster-users] ACL

[Thai. Ngo Bao]

Hi,

I am testing gluster 3.2.4 with ACL on small linux cluster:


1.   All exports (bricks) mounted with –o acl option

2.   All the glusterfs clients mounted with –o acl option

Acl works perfectly when clients mount under normal users but it will not work 
if clients mount under root user.  What am I missing? How can we force clients 
to mount under normal user rather than root user?

Thanks,
~Thai

From: gluster-users-boun...@gluster.org 
[mailto:gluster-users-boun...@gluster.org] On Behalf Of Amar Tumballi
Sent: Tuesday, August 02, 2011 2:57 AM
To: Papp Tamas
Cc: gluster-users@gluster.org
Subject: Re: [Gluster-users] xfs+acl

To use 'acl' below thing should be minimum setup:

* glusterfs version 3.2.2 or higher
* all exports (ie bricks) mounted with '-oacl' option
* all the glusterfs clients mounted with '-oacl' option

Regards,
Amar
On Tue, Aug 2, 2011 at 1:06 AM, Papp Tamas 
mailto:tom...@martos.bme.hu>> wrote:
On 2011-08-01 21:32, Liam Slusser wrote:
We use XFS with Gluster with great success.

We also do.
I'm asking about acl.


Thank you,
tamas
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users


No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.1831 / Virus Database: 2092/4560 - Release Date: 10/18/11
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

Re: [Gluster-users] ACL with GlusterFS 3.1?

[Shehjar Tikoo]

Hi

ACLs are not supported  as yet.

Thanks

Mike Hanby wrote:

Howdy,

Are access control lists (ACL, i.e. setfacl / getfacl) supported in
GlusterFS 3.1?

If yes, beyond mounting the bricks with "defaults,acl" what do I need
to do to enable ACL for both NFS and native Gluster clients?

Google isn't returning anything useful on this topic.

Thanks,

Mike

= Mike Hanby mha...@uab.edu UAB
School of Engineering Information Systems Specialist II IT HPCS /
Research Computing


___ Gluster-users mailing
list Gluster-users@gluster.org 
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users


___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

[Gluster-users] ACL with GlusterFS 3.1?

[Mike Hanby]

Howdy,

Are access control lists (ACL, i.e. setfacl / getfacl) supported in GlusterFS 
3.1?

If yes, beyond mounting the bricks with "defaults,acl" what do I need to do to 
enable ACL for both NFS and native Gluster clients?

Google isn't returning anything useful on this topic.

Thanks,

Mike

=
Mike Hanby
mha...@uab.edu
UAB School of Engineering
Information Systems Specialist II
IT HPCS / Research Computing


___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users