Which version of gluster is this? On Wed, Sep 5, 2018 at 1:27 PM Zhou, Cynthia (NSB - CN/Hangzhou) < cynthia.z...@nokia-sbell.com> wrote:
> Hi glusterfs experts: > > Good day! > > Recently when I do some test on my gluster env, I found that there > are some remaining entries in command “gluster v heal mstate info” *even > after reboot*. > > fstest_035ffc492ec43551a64087f9280ffe3e is a folder in /mnt/mstate > and in this folder only one file(fstest_458bb82d8884ed5c9dadec4ed93bec4e) > exists. > > When I dbg by gdb on sn-0 I found that: > > Parent dir changelog(fstest_035ffc492ec43551a64087f9280ffe3e) says need > to heal entry, but the changelog/gfid/filetype of the only entry in parent > dir shows there is nothing to be healed, so glustershd does nothing every > round of heal. And this entry will remain. > > My gdb shows that each round of heal on sn-0 , it exits in function > __afr_selfheal_entry (if (AFR_COUNT(healed_sinks, priv->child_count) == > 0)), because in this case all three healed_sinks are zero. > > Have you any idea how to solve this issue from glusterfs pov? > Thanks! > > > > [test steps] > > Reboot three sn nodes( sn-0, sn-1, sn-2(arbiter)) sequentially, and > on another node (with glusterfs clients) run fstest. > > > > [problem description] > > > > Remaining entries in “gluster v heal mstate info” command even after > reboot sn-0 many times, the entries are still there! > > > > [root@sn-0:/home/robot] > > # gluster v heal mstate info > > Brick sn-0.local:/mnt/bricks/mstate/brick > > /fstest_035ffc492ec43551a64087f9280ffe3e > > Status: Connected > > Number of entries: 1 > > > > Brick sn-1.local:/mnt/bricks/mstate/brick > > Status: Connected > > Number of entries: 0 > > > > Brick sn-2.local:/mnt/bricks/mstate/brick > > /fstest_035ffc492ec43551a64087f9280ffe3e > > Status: Connected > > Number of entries: 1 > > > > > > > > ////////////////////////////////////////////////////////////////////some > env informations/////////////////////////////////////////////////////// > > # gluster v info mstate > > Volume Name: mstate > > Type: Replicate > > Volume ID: 1d896674-17a2-4ae7-aa7c-c6e22013df99 > > Status: Started > > Snapshot Count: 0 > > Number of Bricks: 1 x (2 + 1) = 3 > > Transport-type: tcp > > Bricks: > > Brick1: sn-0.local:/mnt/bricks/mstate/brick > > Brick2: sn-1.local:/mnt/bricks/mstate/brick > > Brick3: sn-2.local:/mnt/bricks/mstate/brick (arbiter) > > Options Reconfigured: > > performance.client-io-threads: off > > nfs.disable: on > > transport.address-family: inet > > cluster.server-quorum-type: none > > cluster.quorum-reads: no > > cluster.favorite-child-policy: mtime > > cluster.consistent-metadata: on > > network.ping-timeout: 42 > > cluster.quorum-type: auto > > server.allow-insecure: on > > cluster.server-quorum-ratio: 51% > > [root@sn-1:/home/robot] > > > > > > [root@sn-2:/root] > > # getfattr -m . -d -e hex > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/ > > getfattr: Removing leading '/' from absolute path names > > # file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/ > > trusted.afr.dirty=0x000000000000000000000000 > > trusted.afr.mstate-client-0=0x000000010000000000000003 > > trusted.gfid=0xa0975560eaef4cb299467101de00446a > > trusted.glusterfs.dht=0x000000010000000000000000ffffffff > > > > [root@sn-2:/root] > > # getfattr -m . -d -e hex > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > getfattr: Removing leading '/' from absolute path names > > # file: > mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > trusted.afr.mstate-client-0=0x000000000000000000000000 > > trusted.gfid=0x9fc20f587f094182816390f056f7370f > > > trusted.gfid2path.864159d77373ad5f=0x61303937353536302d656165662d346362322d393934362d3731303164653030343436612f6673746573745f3435386262383264383838346564356339646164656334656439336265633465 > > > > # cd /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e > > [root@sn-2 > :/mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e] > > # ls > > fstest_458bb82d8884ed5c9dadec4ed93bec4e > > [root@sn-2 > :/mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e] > > # stat fstest_458bb82d8884ed5c9dadec4ed93bec4e > > File: fstest_458bb82d8884ed5c9dadec4ed93bec4e > > Size: 0 Blocks: 8 IO Block: 4096 fifo > > Device: fd31h/64817d Inode: 22086 Links: 2 > > Access: (0644/prw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > Access: 2018-08-30 04:33:17.552870661 +0300 > > Modify: 2018-08-30 04:33:17.552870661 +0300 > > Change: 2018-08-30 04:33:17.553870661 +0300 > > Birth: - > > [root@sn-2 > :/mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e] > > [root@sn-2:/root] > > # exit > > logout > > Connection to sn-2.local closed. > > [root@sn-0:/home/robot] > > # getfattr -m . -d -e hex > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/ > > getfattr: Removing leading '/' from absolute path names > > # file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/ > > trusted.afr.dirty=0x000000000000000000000000 > > trusted.afr.mstate-client-2=0x00000000000000000000001c > > trusted.gfid=0xa0975560eaef4cb299467101de00446a > > trusted.glusterfs.dht=0x000000010000000000000000ffffffff > > > > [root@sn-0:/home/robot] > > # getfattr -m . -d -e hex > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > getfattr: Removing leading '/' from absolute path names > > # file: > mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > trusted.gfid=0x9fc20f587f094182816390f056f7370f > > > trusted.gfid2path.864159d77373ad5f=0x61303937353536302d656165662d346362322d393934362d3731303164653030343436612f6673746573745f3435386262383264383838346564356339646164656334656439336265633465 > > > > # ls /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e > > fstest_458bb82d8884ed5c9dadec4ed93bec4e > > [root@sn-0:/home/robot] > > # stat > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > File: > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > Size: 0 Blocks: 8 IO Block: 4096 fifo > > Device: fd31h/64817d Inode: 21899 Links: 2 > > Access: (0644/prw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > Access: 2018-08-30 04:33:17.552870661 +0300 > > Modify: 2018-08-30 04:33:17.552870661 +0300 > > Change: 2018-08-30 04:33:17.263809048 +0300 > > Birth: - > > [root@sn-0:/home/robot] > > [root@sn-0:/home/robot] > > # ssh sn-1.local > > > > USAGE OF THE ROOT ACCOUNT AND THE FULL BASH IS RECOMMENDED ONLY FOR > LIMITED USE. PLEASE USE A NON-ROOT ACCOUNT AND THE SCLI SHELL (fsclish) > AND/OR LIMITED BASH SHELL. > > > > Read /opt/nokia/share/security/readme_root.txt for more details. > > > > [root@sn-1:/root] > > # getfattr -m . -d -e hex > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/ > > getfattr: Removing leading '/' from absolute path names > > # file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/ > > trusted.afr.dirty=0x000000000000000000000000 > > trusted.afr.mstate-client-0=0x000000000000000000000000 > > trusted.afr.mstate-client-2=0x000000000000000000000000 > > trusted.gfid=0xa0975560eaef4cb299467101de00446a > > trusted.glusterfs.dht=0x000000010000000000000000ffffffff > > > > [root@sn-1:/root] > > # getfattr -m . -d -e hex > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > getfattr: Removing leading '/' from absolute path names > > # file: > mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > trusted.afr.mstate-client-0=0x000000000000000000000000 > > trusted.gfid=0x9fc20f587f094182816390f056f7370f > > > trusted.gfid2path.864159d77373ad5f=0x61303937353536302d656165662d346362322d393934362d3731303164653030343436612f6673746573745f3435386262383264383838346564356339646164656334656439336265633465 > > > > [root@sn-1:/root] > > [root@sn-1:/root] > > # ls /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e > > fstest_458bb82d8884ed5c9dadec4ed93bec4e > > [root@sn-1:/root] > > # stat > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > > File: > /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e > > Size: 0 Blocks: 8 IO Block: 4096 fifo > > Device: fd31h/64817d Inode: 22168 Links: 2 > > Access: (0644/prw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) > > Access: 2018-08-30 04:33:17.552870661 +0300 > > Modify: 2018-08-30 04:33:17.552870661 +0300 > > Change: 2018-08-30 04:33:17.037673648 +0300 > > Birth: - > > [root@sn-1:/root] > > > > > > > -- Pranith
_______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel