Re: [Gluster-devel] fstest for fuse mount gluster

[qingwei wei]

Hi Saravana,

I submitted just now.
https://bugzilla.redhat.com/show_bug.cgi?id=1444537

Cw

On Fri, Apr 21, 2017 at 8:15 PM, Saravanakumar Arumugam  wrote:

> Hi,
>
>
> On 04/20/2017 08:58 PM, qingwei wei wrote:
>
> Hi,
>
> Posted this in gluster-user mailing list but got no response so far, so i
> post in gluster-devel.
>
> I found this test suite ( 
> https://github.com/Hnasar/pjdfstest) for me to test fuse mount gluster
> and there is some reported issue from the test. One of the error is as
> follow.
>
> When i chmod  to a file in fuse mounted gluster volume. the return
> stat value for the file is not  instead of 4777.
>
> root@ubuntu16d:/mnt/g310mp# touch test
> root@ubuntu16d:/mnt/g310mp# chmod  test
> root@ubuntu16d:/mnt/g310mp# stat test
>   File: 'test'
>   Size: 0   Blocks: 0  IO Block: 131072 regular empty
> file
> Device: 29h/41d Inode: 9618589997017543511  Links: 1
> Access: (4777/-rwsrwxrwx)  Uid: (0/root)   Gid: (0/root)
> Access: 2017-11-30 14:21:23.374871207 +0800
> Modify: 2017-11-30 14:21:16.974871000 +0800
> Change: 2017-11-30 14:21:23.374871207 +0800
>  Birth: -
>
> Performing this operation in normal ext4 system produce correct result.
>
> root@ubuntu16d:/mnt/g310mp# touch ~/testfile
> root@ubuntu16d:/mnt/g310mp# chmod  ~/testfile
> root@ubuntu16d:/mnt/g310mp# stat ~/testfile
>   File: '/home/ubuntu/testfile'
>   Size: 0   Blocks: 0  IO Block: 4096   regular empty
> file
> Device: fc00h/64512dInode: 662649  Links: 1
> Access: (/-rwsrwsrwt)  Uid: (0/root)   Gid: (0/root)
> Access: 2017-11-30 14:23:00.518867795 +0800
> Modify: 2017-11-30 14:23:00.518867795 +0800
> Change: 2017-11-30 14:23:08.742867507 +0800
>  Birth: -
>
> Besides , 3777 also an issue. The stat return is 0777.
>
> My OS is Ubuntu 16.04 and my gluster version is 3.10.1 and the it is a
> simple volume with default parameter.
>
> root@ubuntu16d:/mnt/g310mp# gluster volume info
>
> Volume Name: g310
> Type: Distribute
> Volume ID: 114666c6-4884-436a-81a8-2deb3c0923ba
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 1
> Transport-type: tcp
> Bricks:
> Brick1: 192.168.36.130:/mnt/g310brick
> Options Reconfigured:
> transport.address-family: inet
> nfs.disable: on
>
> for me, i seldom use those mode (3777 & ) but i cannot say for sure
> for others. So is this something i should be concerned about?
>
>
> This looks like a bug and needs to be verified.
>
> Please raise a  bug here:
> https://bugzilla.redhat.com/enter_bug.cgi?product=
> GlusterFS=mainline=core
>
> - Saravana
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Proposal for an extended READDIRPLUS operation via gfAPI

[Soumya Koduri]

Hi,

We currently have readdirplus operation to fetch stat for each of the 
dirents. But that may not be sufficient and often applications may need 
extra information, like for eg., NFS-Ganesha like applications which 
operate on handles need to generate handles for each of those dirents 
returned. So this would require extra calls to the backend, in this case 
LOOKUP (which is very expensive operation) resulting in quite slow

readdir performance.

To address that, introducing this new API using which applications can
request for any extra information to be returned as part of
readdirplus response [1]

Patch: https://review.gluster.org/#/c/15663
Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1442950
Github issue#  https://github.com/gluster/glusterfs/issues/174

The interface is designed to be a bit aligned with xstat [2] format as 
suggested by Niels. This helps in extending this operation in future.


The synopsis of this new API - the arguments to be passed and how it can 
be used is mentioned in the patch [3].


Since the initial requirement is to return handles, I have defined it 
with glfs_h_* prefix for now, but it may well be used by applications 
not having to use them (like SMB). Suggestions are welcome.


The current changes are POC and need to tested extensively but we had 
seen a huge performance improvement (with initial patch-set, at least on 
a single brick volume).


Request for comments/suggestions on any improvements needed on the 
interface.


Thanks,
Soumya

[1] https://review.gluster.org/#/c/15663
[2] https://lwn.net/Articles/394298/
[3] https://review.gluster.org/#/c/15663/8/api/src/glfs-handles.h
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Nightly regression job with enabling brick multiplexing

[Atin Mukherjee]

As we don't run our .t files with brick mux being enabled for every patches
can we ensure that there is a nightly regression trigger with brick
multiplexing feature being enabled. The reason for this ask is very simple,
we have no control on the regression for this feature. I've already seen a
very basic test (volume status not reflecting bricks online after glusterd
restart) breaking recently which used to work earlier.

~Atin
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] fstest for fuse mount gluster

[Saravanakumar Arumugam]

Hi,

On 04/20/2017 08:58 PM, qingwei wei wrote:

Hi,

Posted this in gluster-user mailing list but got no response so far, 
so i post in gluster-devel.


I found this test suite (https://github.com/Hnasar/pjdfstest 
) for me to test fuse mount 
gluster and there is some reported issue from the test. One of the 
error is as follow.


When i chmod  to a file in fuse mounted gluster volume. the return 
stat value for the file is not  instead of 4777.


root@ubuntu16d:/mnt/g310mp# touch test
root@ubuntu16d:/mnt/g310mp# chmod  test
root@ubuntu16d:/mnt/g310mp# stat test
  File: 'test'
  Size: 0   Blocks: 0  IO Block: 131072 regular 
empty file

Device: 29h/41d Inode: 9618589997017543511  Links: 1
Access: (4777/-rwsrwxrwx)  Uid: (0/root)   Gid: (  0/root)
Access: 2017-11-30 14:21:23.374871207 +0800
Modify: 2017-11-30 14:21:16.974871000 +0800
Change: 2017-11-30 14:21:23.374871207 +0800
 Birth: -

Performing this operation in normal ext4 system produce correct result.

root@ubuntu16d:/mnt/g310mp# touch ~/testfile
root@ubuntu16d:/mnt/g310mp# chmod  ~/testfile
root@ubuntu16d:/mnt/g310mp# stat ~/testfile
  File: '/home/ubuntu/testfile'
  Size: 0   Blocks: 0  IO Block: 4096 regular 
empty file

Device: fc00h/64512dInode: 662649  Links: 1
Access: (/-rwsrwsrwt)  Uid: (0/root)   Gid: (  0/root)
Access: 2017-11-30 14:23:00.518867795 +0800
Modify: 2017-11-30 14:23:00.518867795 +0800
Change: 2017-11-30 14:23:08.742867507 +0800
 Birth: -

Besides , 3777 also an issue. The stat return is 0777.

My OS is Ubuntu 16.04 and my gluster version is 3.10.1 and the it is a 
simple volume with default parameter.


root@ubuntu16d:/mnt/g310mp# gluster volume info

Volume Name: g310
Type: Distribute
Volume ID: 114666c6-4884-436a-81a8-2deb3c0923ba
Status: Started
Snapshot Count: 0
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: 192.168.36.130:/mnt/g310brick
Options Reconfigured:
transport.address-family: inet
nfs.disable: on

for me, i seldom use those mode (3777 & ) but i cannot say for 
sure for others. So is this something i should be concerned about?




This looks like a bug and needs to be verified.

Please raise a  bug here:
https://bugzilla.redhat.com/enter_bug.cgi?product=GlusterFS=mainline=core

- Saravana
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] About locking subvolume in dht translator

[Tahereh Fattahi]

Thank you
When I get the layout from the parent of a file inode (directory) I can see
all of the subvolume, so I had thought maybe this work is correct!
So is there any way to do a directory operation from a file? I had test
creating parent frame and parent_loc, but there was no correct
functionality

On Fri, Apr 21, 2017 at 11:42 AM, Amar Tumballi  wrote:

>
>
> On Fri, Apr 21, 2017 at 12:21 PM, Tahereh Fattahi 
> wrote:
>
>> Hi
>> I want add some code in dht_create before doing create in hashed_subvol.
>> I want write something in all subvols with setxattr operation.
>> My problem is that when I want get lock (write lock) on subvols, I get
>> error:
>>
>> pending frames:
>> frame : type(1) op(CREATE)
>> frame : type(1) op(CREATE)
>> frame : type(0) op(0)
>> patchset: git://git.gluster.com/glusterfs.git
>> signal received: 11
>> time of crash:
>> 2017-04-21 06:38:23
>> configuration details:
>> argp 1
>> backtrace 1
>> dlfcn 1
>> .
>> .
>> .
>> But if I just get lock on one subvol, the step of getting locks do
>> correctly.
>> I dont know the problem! Are subvols that get from layout from a file
>> parent inode fake?? (from dht_get_layout function)
>> If I get these locks from a directory for example in a setxattr
>> operation, there is no problem.
>> The function of lock is dht_blocking_inodelk in dht_commom.c and the
>> operation
>>
>> STACK_WIND_COOKIE (frame, dht_blocking_inodelk_cbk,
>>(void *) (long) i,
>>local->lock.locks[i]->xl,
>>local->lock.locks[i]->xl->fops->inodelk,
>>local->lock.locks[i]->domain,
>>>lock.locks[i]->loc, F_SETLKW, ,
>> NULL);
>>
>> (in dht_blocking_inodelk_rec function )raise error.
>>
>
> Mostly because its a file and inode context is not present on all the
> subvols?
>
>
>
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
>
> --
> Amar Tumballi (amarts)
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] About locking subvolume in dht translator

[Amar Tumballi]

On Fri, Apr 21, 2017 at 12:21 PM, Tahereh Fattahi 
wrote:

> Hi
> I want add some code in dht_create before doing create in hashed_subvol.
> I want write something in all subvols with setxattr operation.
> My problem is that when I want get lock (write lock) on subvols, I get
> error:
>
> pending frames:
> frame : type(1) op(CREATE)
> frame : type(1) op(CREATE)
> frame : type(0) op(0)
> patchset: git://git.gluster.com/glusterfs.git
> signal received: 11
> time of crash:
> 2017-04-21 06:38:23
> configuration details:
> argp 1
> backtrace 1
> dlfcn 1
> .
> .
> .
> But if I just get lock on one subvol, the step of getting locks do
> correctly.
> I dont know the problem! Are subvols that get from layout from a file
> parent inode fake?? (from dht_get_layout function)
> If I get these locks from a directory for example in a setxattr operation,
> there is no problem.
> The function of lock is dht_blocking_inodelk in dht_commom.c and the
> operation
>
> STACK_WIND_COOKIE (frame, dht_blocking_inodelk_cbk,
>(void *) (long) i,
>local->lock.locks[i]->xl,
>local->lock.locks[i]->xl->fops->inodelk,
>local->lock.locks[i]->domain,
>>lock.locks[i]->loc, F_SETLKW, ,
> NULL);
>
> (in dht_blocking_inodelk_rec function )raise error.
>

Mostly because its a file and inode context is not present on all the
subvols?



>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Amar Tumballi (amarts)
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] High load on glusterfsd process

[ABHISHEK PALIWAL]

If the patch provided in that case will resolve my bug as well then please
provide the patch so that I will backport it on 3.7.6

On Fri, Apr 21, 2017 at 11:30 AM, ABHISHEK PALIWAL 
wrote:

> Hi Team,
>
> I have noticed that there are so many glusterfsd threads are running in my
> system and we observed some of those thread consuming more cpu. I did
> “strace” on two such threads (before the problem disappeared by itself) and
> found that there is a continuous activity like below:
>
> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-
> 4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz",
> {st_mode=S_IFREG|0670, st_size=1995, ...}) = 0
> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-
> 246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz",
> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data
> available)
> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-
> 246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz",
> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data
> available)
> lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-
> 4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz",
> {st_mode=S_IFREG|0670, st_size=169, ...}) = 0
> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-
> 246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz",
> "trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data
> available)
> lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-
> 246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz",
> "trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data
> available)
>
> I have found the below existing issue which is very similar to my scenario.
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1298258
>
> We are using the gluster-3.7.6 and it seems that the issue is fixed in
> 3.8.4 version.
>
> Could you please let me know why it showing the number of above logs and
> reason behind it as it is not explained in the above bug.
>
> Regards,
> Abhishek
>
> --
>
>
>
>
> Regards
> Abhishek Paliwal
>



-- 




Regards
Abhishek Paliwal
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] About locking subvolume in dht translator

[Tahereh Fattahi]

Hi
I want add some code in dht_create before doing create in hashed_subvol.
I want write something in all subvols with setxattr operation.
My problem is that when I want get lock (write lock) on subvols, I get
error:

pending frames:
frame : type(1) op(CREATE)
frame : type(1) op(CREATE)
frame : type(0) op(0)
patchset: git://git.gluster.com/glusterfs.git
signal received: 11
time of crash:
2017-04-21 06:38:23
configuration details:
argp 1
backtrace 1
dlfcn 1
.
.
.
But if I just get lock on one subvol, the step of getting locks do
correctly.
I dont know the problem! Are subvols that get from layout from a file
parent inode fake?? (from dht_get_layout function)
If I get these locks from a directory for example in a setxattr operation,
there is no problem.
The function of lock is dht_blocking_inodelk in dht_commom.c and the
operation

STACK_WIND_COOKIE (frame, dht_blocking_inodelk_cbk,
   (void *) (long) i,
   local->lock.locks[i]->xl,
   local->lock.locks[i]->xl->fops->inodelk,
   local->lock.locks[i]->domain,
   >lock.locks[i]->loc, F_SETLKW, ,
NULL);

(in dht_blocking_inodelk_rec function )raise error.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Priority based ping packet for 3.10

[Raghavendra G]

Summing up various discussions I had on this,

1. Current ping frame work should measure just the responsiveness of
network and rpc layer. This means poller threads shouldn't be winding the
individual fops at all (as it might add delay in reading the ping
requests). Instead, they can queue the requests to a common work queue and
other threads should pick up the requests.
2. We also need another tool to measure the responsiveness of the entire
Brick xlator stack. This tool can have a slightly larger time than ping
timeout as responses naturally will be delayed. Whether this tool should
measure the responsiveness of the backend fs is an open question as we
already have a posix health checker that measures the responsiveness and
sends a CHILD_DOWN when backend fs in not responsive. Also, there are open
questions here like what data structures various xlators are accessing as
part of this fop (like inode, fd, mem-pools etc). Accessing various data
structures will result in a different latency.
3. Currently ping packets are not sent by a client when there is no I/O
from it. As per the discussions above, client should measure the
responsiveness even when there is no traffic to/from it. May be the
interval during which ping packets are sent can be increased.
4. We've fixed some lock contention issues on the brick stack due to high
latency on backend fs. However, this is on-going work as contentions can be
found in various codepaths (mem-pool etc).

We'll shortly send a fix for 1. The other things will be picked based on
the bandwidth. Contributions are welcome :).

regards,
Raghavendra.

On Wed, Jan 25, 2017 at 11:01 AM, Joe Julian  wrote:

> Yes, the earlier a fault is detected the better.
>
> On January 24, 2017 9:21:27 PM PST, Jeff Darcy  wrote:
>>
>>  If there are no responses to be received and no requests being
>>>  sent to a brick, why would be a client be interested in the health of
>>>  server/brick?
>>>
>>
>> The client (code) might not, but the user might want to find out and fix
>> the fault before the brick gets busy again.
>> --
>>
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-devel
>>
>>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Raghavendra G
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] High load on glusterfsd process

[ABHISHEK PALIWAL]

Hi Team,

I have noticed that there are so many glusterfsd threads are running in my
system and we observed some of those thread consuming more cpu. I did
“strace” on two such threads (before the problem disappeared by itself) and
found that there is a continuous activity like below:

lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz",
{st_mode=S_IFREG|0670, st_size=1995, ...}) = 0
lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz",
"trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data
available)
lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/firewall_-J208482-425_20170126T113552+.log.gz",
"trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data
available)
lstat("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz",
{st_mode=S_IFREG|0670, st_size=169, ...}) = 0
lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz",
"trusted.bit-rot.bad-file", 0x3fff81f58550, 255) = -1 ENODATA (No data
available)
lgetxattr("/opt/lvmdir/c2/brick/.glusterfs/e7/7d/e77d12b3-92f8-4dfe-9a7f-246e901cbdf1/002700/tcli_-J208482-425_20170123T180550+.log.gz",
"trusted.bit-rot.signature", 0x3fff81f58550, 255) = -1 ENODATA (No data
available)

I have found the below existing issue which is very similar to my scenario.

https://bugzilla.redhat.com/show_bug.cgi?id=1298258

We are using the gluster-3.7.6 and it seems that the issue is fixed in
3.8.4 version.

Could you please let me know why it showing the number of above logs and
reason behind it as it is not explained in the above bug.

Regards,
Abhishek

-- 




Regards
Abhishek Paliwal
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-devel