Re: [Gluster-users] gfid generation

2016-11-15 Thread Pranith Kumar Karampuri 
On Wed, Nov 16, 2016 at 3:31 AM, Ankireddypalle Reddy 
wrote:

> Kaushal/Pranith,
>   Thanks for clarifying this. As I
> understand there are 2 id's. Please correct if there is a mistake in my
> assumptions:
>   1) HASH generated by DHT and this will
> generate the same id for a given file all the time.
>   2) GFID which is an version 4 UUID. As
> per the below links this is supposed to contain a time stamp field in it.
> So this will not generate the same id for a given file all the time.
>https://en.wikipedia.org/wiki/
> Universally_unique_identifier
>https://tools.ietf.org/html/rfc4122


That is correct. There is no involvement of parent gfid in either of this
:-).


>
>
> Thanks and Regards,
> ram
> -Original Message-
> From: Kaushal M [mailto:kshlms...@gmail.com]
> Sent: Tuesday, November 15, 2016 1:21 PM
> To: Ankireddypalle Reddy
> Cc: Pranith Kumar Karampuri; gluster-users@gluster.org; Gluster Devel
> Subject: Re: [Gluster-users] gfid generation
>
> On Tue, Nov 15, 2016 at 11:33 PM, Ankireddypalle Reddy <
> are...@commvault.com> wrote:
> > Pranith,
> >
> >  Thanks for getting back on this. I am trying to see
> > how gfid can be generated programmatically. Given a file name how do
> > we generate gfid for it. I was reading some of the email threads about
> > it where it was mentioned that gfid is generated based upon parent
> > directory gfid and the file name. Given a same parent gfid and file
> > name do we always end up with the same gfid.
>
> You're probably confusing the hash as generated for the elastic hash
> algorithm in DHT, with UUID. That is a combination of
>
> I always thought that the GFID was a UUID, which was randomly generated.
> (The random UUID might be being modified a little to allow some leeway with
> directory listing, IIRC).
>
> Adding gluster-devel to get more eyes on this.
>
> >
> >
> >
> > Thanks and Regards,
> >
> > ram
> >
> >
> >
> > From: Pranith Kumar Karampuri [mailto:pkara...@redhat.com]
> > Sent: Tuesday, November 15, 2016 12:58 PM
> > To: Ankireddypalle Reddy
> > Cc: gluster-users@gluster.org
> > Subject: Re: [Gluster-users] gfid generation
> >
> >
> >
> > Sorry, didn't understand the question. Are you saying give a file on
> > gluster how to get gfid of the file?
> >
> > #getfattr -d -m. -e hex /path/to/file shows it
> >
> >
> >
> > On Fri, Nov 11, 2016 at 9:47 PM, Ankireddypalle Reddy
> > 
> > wrote:
> >
> > Hi,
> >
> > Is the mapping from file name to gfid an idempotent operation.
> > If so please point me to the function that does this.
> >
> >
> >
> > Thanks and Regards,
> >
> > Ram
> >
> > ***Legal Disclaimer***
> >
> > "This communication may contain confidential and privileged material
> > for the
> >
> > sole use of the intended recipient. Any unauthorized review, use or
> > distribution
> >
> > by others is strictly prohibited. If you have received the message by
> > mistake,
> >
> > please advise the sender by reply email and delete the message. Thank
> you."
> >
> > **
> >
> >
> > ___
> > Gluster-users mailing list
> > Gluster-users@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
> >
> >
> >
> >
> > --
> >
> > Pranith
> >
> > ***Legal Disclaimer***
> > "This communication may contain confidential and privileged material
> > for the sole use of the intended recipient. Any unauthorized review,
> > use or distribution by others is strictly prohibited. If you have
> > received the message by mistake, please advise the sender by reply
> > email and delete the message. Thank you."
> > **
> >
> > ___
> > Gluster-users mailing list
> > Gluster-users@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
> ***Legal Disclaimer***
> "This communication may contain confidential and privileged material for
> the
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
> by others is strictly prohibited. If you have received the message by
> mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **
>



-- 
Pranith
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gfid generation

2016-11-15 Thread Ankireddypalle Reddy 
Kaushal/Pranith,
  Thanks for clarifying this. As I understand 
there are 2 id's. Please correct if there is a mistake in my assumptions:
  1) HASH generated by DHT and this will 
generate the same id for a given file all the time.
  2) GFID which is an version 4 UUID. As per 
the below links this is supposed to contain a time stamp field in it.  So this 
will not generate the same id for a given file all the time.
   
https://en.wikipedia.org/wiki/Universally_unique_identifier
   https://tools.ietf.org/html/rfc4122
 
Thanks and Regards,
ram
-Original Message-
From: Kaushal M [mailto:kshlms...@gmail.com] 
Sent: Tuesday, November 15, 2016 1:21 PM
To: Ankireddypalle Reddy
Cc: Pranith Kumar Karampuri; gluster-users@gluster.org; Gluster Devel
Subject: Re: [Gluster-users] gfid generation

On Tue, Nov 15, 2016 at 11:33 PM, Ankireddypalle Reddy  
wrote:
> Pranith,
>
>  Thanks for getting back on this. I am trying to see 
> how gfid can be generated programmatically. Given a file name how do 
> we generate gfid for it. I was reading some of the email threads about 
> it where it was mentioned that gfid is generated based upon parent 
> directory gfid and the file name. Given a same parent gfid and file 
> name do we always end up with the same gfid.

You're probably confusing the hash as generated for the elastic hash algorithm 
in DHT, with UUID. That is a combination of

I always thought that the GFID was a UUID, which was randomly generated. (The 
random UUID might be being modified a little to allow some leeway with 
directory listing, IIRC).

Adding gluster-devel to get more eyes on this.

>
>
>
> Thanks and Regards,
>
> ram
>
>
>
> From: Pranith Kumar Karampuri [mailto:pkara...@redhat.com]
> Sent: Tuesday, November 15, 2016 12:58 PM
> To: Ankireddypalle Reddy
> Cc: gluster-users@gluster.org
> Subject: Re: [Gluster-users] gfid generation
>
>
>
> Sorry, didn't understand the question. Are you saying give a file on 
> gluster how to get gfid of the file?
>
> #getfattr -d -m. -e hex /path/to/file shows it
>
>
>
> On Fri, Nov 11, 2016 at 9:47 PM, Ankireddypalle Reddy 
> 
> wrote:
>
> Hi,
>
> Is the mapping from file name to gfid an idempotent operation.  
> If so please point me to the function that does this.
>
>
>
> Thanks and Regards,
>
> Ram
>
> ***Legal Disclaimer***
>
> "This communication may contain confidential and privileged material 
> for the
>
> sole use of the intended recipient. Any unauthorized review, use or 
> distribution
>
> by others is strictly prohibited. If you have received the message by 
> mistake,
>
> please advise the sender by reply email and delete the message. Thank you."
>
> **
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
> --
>
> Pranith
>
> ***Legal Disclaimer***
> "This communication may contain confidential and privileged material 
> for the sole use of the intended recipient. Any unauthorized review, 
> use or distribution by others is strictly prohibited. If you have 
> received the message by mistake, please advise the sender by reply 
> email and delete the message. Thank you."
> **
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
***Legal Disclaimer***
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
**
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] md-cache invalidation

2016-11-15 Thread Andrea Fogazzi 
Hi all,

I just subscribed ot the list; we have been testing GlusterFS from some time 
for a typical  workload small files: more than 90%-95% of accesses are stats, 
of an around 1 TB and we have millions of dirs and few kb files.


I am  doing some tests with md-cache, following instructions availble on blog 
post

http://blog.gluster.org/2016/10/gluster-tiering-and-small-file-performance/


Our installation is 3.8.5 upgraded from 3.7.14.


I am a bit confused if we can use invalidation mechanism of "upcalls"  with 
with 3.8.5; when I run "gluster volume set vol performance.cache-invalidation 
on", I get error that option is not available; instead I was able to enable 
features.cache-invalidation but I wasn't able to make md cache invalidation 
work correctly; in fact, with the mentioned configuration, what we find is that 
metadata are cached until timeout expires, but even when files are updated from 
one of the client other clients do not receive the updated version until the md 
cache expires at the timeout.


Could someone clarify  if the md cache invalidation upcall mechanism is 
supposed to work on 3.8.x, or instead it's only available in 3.9?


Would change the behaviour if using NFS instead of gluterfs on the client side?


Thanks in advance.


Kind regards,

andrea

--

Andrea Fogazzi
fo...@fogazzi.com
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gfid generation

2016-11-15 Thread Pranith Kumar Karampuri 
On Tue, Nov 15, 2016 at 11:33 PM, Ankireddypalle Reddy  wrote:

> Pranith,
>
>  Thanks for getting back on this. I am trying to see how
> gfid can be generated programmatically. Given a file name how do we
> generate gfid for it. I was reading some of the email threads about it
> where it was mentioned that gfid is generated based upon parent directory
> gfid and the file name. Given a same parent gfid and file name do we always
> end up with the same gfid.
>

No, gfid is a UUID, so there is correlation.
https://en.wikipedia.org/wiki/Universally_unique_identifier can help you I
guess. Where did you see the above information that Parent gfid and name
are involved?

PS: These kinds of mails people in gluster-devel are more suited to answer
IMO.


>
> Thanks and Regards,
>
> ram
>
>
>
> *From:* Pranith Kumar Karampuri [mailto:pkara...@redhat.com]
> *Sent:* Tuesday, November 15, 2016 12:58 PM
> *To:* Ankireddypalle Reddy
> *Cc:* gluster-users@gluster.org
> *Subject:* Re: [Gluster-users] gfid generation
>
>
>
> Sorry, didn't understand the question. Are you saying give a file on
> gluster how to get gfid of the file?
>
> #getfattr -d -m. -e hex /path/to/file shows it
>
>
>
> On Fri, Nov 11, 2016 at 9:47 PM, Ankireddypalle Reddy <
> are...@commvault.com> wrote:
>
> Hi,
>
> Is the mapping from file name to gfid an idempotent operation.  If
> so please point me to the function that does this.
>
>
>
> Thanks and Regards,
>
> Ram
>
> ***Legal Disclaimer***
>
> "This communication may contain confidential and privileged material for
> the
>
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
>
> by others is strictly prohibited. If you have received the message by
> mistake,
>
> please advise the sender by reply email and delete the message. Thank you."
>
> **
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
> --
>
> Pranith
> ***Legal Disclaimer***
> "This communication may contain confidential and privileged material for
> the
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
> by others is strictly prohibited. If you have received the message by
> mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **
>



-- 
Pranith
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gfid generation

2016-11-15 Thread Kaushal M 
On Tue, Nov 15, 2016 at 11:33 PM, Ankireddypalle Reddy
 wrote:
> Pranith,
>
>  Thanks for getting back on this. I am trying to see how
> gfid can be generated programmatically. Given a file name how do we generate
> gfid for it. I was reading some of the email threads about it where it was
> mentioned that gfid is generated based upon parent directory gfid and the
> file name. Given a same parent gfid and file name do we always end up with
> the same gfid.

You're probably confusing the hash as generated for the elastic hash
algorithm in DHT, with UUID. That is a combination of

I always thought that the GFID was a UUID, which was randomly
generated. (The random UUID might be being modified a little to allow
some leeway with directory listing, IIRC).

Adding gluster-devel to get more eyes on this.

>
>
>
> Thanks and Regards,
>
> ram
>
>
>
> From: Pranith Kumar Karampuri [mailto:pkara...@redhat.com]
> Sent: Tuesday, November 15, 2016 12:58 PM
> To: Ankireddypalle Reddy
> Cc: gluster-users@gluster.org
> Subject: Re: [Gluster-users] gfid generation
>
>
>
> Sorry, didn't understand the question. Are you saying give a file on gluster
> how to get gfid of the file?
>
> #getfattr -d -m. -e hex /path/to/file shows it
>
>
>
> On Fri, Nov 11, 2016 at 9:47 PM, Ankireddypalle Reddy 
> wrote:
>
> Hi,
>
> Is the mapping from file name to gfid an idempotent operation.  If
> so please point me to the function that does this.
>
>
>
> Thanks and Regards,
>
> Ram
>
> ***Legal Disclaimer***
>
> "This communication may contain confidential and privileged material for the
>
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
>
> by others is strictly prohibited. If you have received the message by
> mistake,
>
> please advise the sender by reply email and delete the message. Thank you."
>
> **
>
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
> --
>
> Pranith
>
> ***Legal Disclaimer***
> "This communication may contain confidential and privileged material for the
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
> by others is strictly prohibited. If you have received the message by
> mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] After upgrade from 3.4.2 to 3.8.5 - High CPU usage resulting in disconnects and split-brain

2016-11-15 Thread Pranith Kumar Karampuri 
On Tue, Nov 15, 2016 at 4:33 AM, Micha Ober  wrote:

> Hi,
>
> I upgraded an installation of GlusterFS on Ubuntu 14.04.3 from version
> 3.4.2 to 3.8.5.
> Few hours after the upgrade, I noticed files in "split-brain" state. I
> never had split-brain files in months of operation before, with the old
> version.
>
> Using htop, I observed the "glusterfs" process jumping from 0% to 100+%
> CPU usage every now and then.
> Using iostat, I confirmed there is no bottleneck on the local disks (util
> is well below 10%)
>
> Inspecting the logfiles, it looks like clients are losing connection quite
> often:
>
> [2016-11-14 16:34:56.685349] C 
> [rpc-clnt-ping.c:160:rpc_clnt_ping_timer_expired]
> 0-gv0-client-1: server X.X.X.62:49152 has not responded in the last 42
> seconds, disconnecting.
> [2016-11-14 16:35:47.690348] C 
> [rpc-clnt-ping.c:160:rpc_clnt_ping_timer_expired]
> 0-gv0-client-8: server X.X.X.219:49153 has not responded in the last 42
> seconds, disconnecting.
> [2016-11-14 17:09:33.903096] C 
> [rpc-clnt-ping.c:160:rpc_clnt_ping_timer_expired]
> 0-gv0-client-7: server X.X.X.62:49153 has not responded in the last 42
> seconds, disconnecting.
>

You need to find out what is leading to the disconnects above. These could
be the reason for split-brain.


>
> There are a total of 6 servers with 2 bricks each (Distribute-Replicate)
>
> The result of a 60 second gluster volume profile can be seen here:
> http://pastebin.com/5WN5S63B
>
> After upgrading, I set:
>
> cluster.granular-entry-heal yes
> cluster.locking-scheme granular
>
> I now reverted to no/full to see if files are still going "split-brain".
>
> Best regards
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>



-- 
Pranith
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gfid generation

2016-11-15 Thread Ankireddypalle Reddy 
Pranith,
 Thanks for getting back on this. I am trying to see how gfid 
can be generated programmatically. Given a file name how do we generate gfid 
for it. I was reading some of the email threads about it where it was mentioned 
that gfid is generated based upon parent directory gfid and the file name. 
Given a same parent gfid and file name do we always end up with the same gfid.

Thanks and Regards,
ram

From: Pranith Kumar Karampuri [mailto:pkara...@redhat.com]
Sent: Tuesday, November 15, 2016 12:58 PM
To: Ankireddypalle Reddy
Cc: gluster-users@gluster.org
Subject: Re: [Gluster-users] gfid generation

Sorry, didn't understand the question. Are you saying give a file on gluster 
how to get gfid of the file?
#getfattr -d -m. -e hex /path/to/file shows it

On Fri, Nov 11, 2016 at 9:47 PM, Ankireddypalle Reddy 
> wrote:
Hi,
Is the mapping from file name to gfid an idempotent operation.  If so 
please point me to the function that does this.

Thanks and Regards,
Ram
***Legal Disclaimer***
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
**

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users



--
Pranith
***Legal Disclaimer***
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
**
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] autofs' wildcard key

2016-11-15 Thread Niels de Vos 
On Tue, Nov 15, 2016 at 04:34:15PM +, lejeczek wrote:
> hi everyone
> 
> it does not work for me, is it supposed to work?
> Autofs does not complain nor report any errors nor problems.
> 
> * -fstype=glusterfs -rw 10.5.6.49,10.5.6.100:/USER-HOME/&
> 
> but this does:
> 
> everything -fstype=glusterfs -rw 10.5.6.49,10.5.6.100:/USER-HOME
> 
> so wildcard keys does not work? What's broken here?

This would use subdirectory mounts, namely /USER-HOME/$USERNAME.
Gluster/fuse does not support subdirectory mounts. It is a feature we
would like to get included in an upcoming release. You may want to
follow https://bugzilla.redhat.com/show_bug.cgi?id=892808 to get updates
on the progress.

Niels


signature.asc
Description: PGP signature
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gfid generation

2016-11-15 Thread Pranith Kumar Karampuri 
Sorry, didn't understand the question. Are you saying give a file on
gluster how to get gfid of the file?
#getfattr -d -m. -e hex /path/to/file shows it

On Fri, Nov 11, 2016 at 9:47 PM, Ankireddypalle Reddy 
wrote:

> Hi,
>
> Is the mapping from file name to gfid an idempotent operation.  If
> so please point me to the function that does this.
>
>
>
> Thanks and Regards,
>
> Ram
> ***Legal Disclaimer***
> "This communication may contain confidential and privileged material for
> the
> sole use of the intended recipient. Any unauthorized review, use or
> distribution
> by others is strictly prohibited. If you have received the message by
> mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>



-- 
Pranith
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster File Abnormalities

2016-11-15 Thread Nithya Balachandran 
On 15 November 2016 at 21:59, Kevin Leigeb  wrote:

> Nithya -
>
>
>
> Thanks for the reply, I will send this at the top to keep the thread from
> getting really ugly.
>
>
>
> We did indeed copy from the individual bricks in an effort to speed up the
> copy. We had one rsync running from each brick to the mount point for the
> new cluster. As stated, we skipped all files with size 0 so that stub files
> wouldn’t be copied. Some files with permissions of 1000 (equivalent to
> -T) were larger than 0 and were also copied.
>
>
>
I’m mostly trying to ascertain why such files would exist (failed
> rebalance?) and what we can do about this problem.
>
>
>
Rebalance creates these linkto files as targets for the file migration -
while the file size displayed would be nonzero, the actual disk usage for
such files would be zero.
The rebalance process first creates these files and then checks if it is
possible to migrate the file. If not, it just leaves it there as it is not
using up any space and will be ignored by gluster clients. It is not a
failed rebalance - sometimes some files are not migrated because of space
constraints and they are considered to have been skipped.

This behaviour was modified as part of http://review.gluster.org/#/c/12347.
We now reset the size to 0.

I'm afraid, if those files were overwritten by their linkto files, the only
way forward would be to restore from a backup.

Regards,
Nithya






> Thanks,
>
> Kevin
>
>
>
> *From:* Nithya Balachandran [mailto:nbala...@redhat.com]
> *Sent:* Tuesday, November 15, 2016 10:21 AM
> *To:* Kevin Leigeb 
> *Cc:* gluster-users@gluster.org
> *Subject:* Re: [Gluster-users] Gluster File Abnormalities
>
>
>
>
>
> Hi kevin,
>
>
>
> On 15 November 2016 at 20:56, Kevin Leigeb  wrote:
>
> All -
>
>
>
> We recently moved from an old cluster running 3.7.9 to a new one running
> 3.8.4. To move the data we rsync’d all files from the old gluster nodes
> that were not in the .glusterfs directory and had a size of greater-than
> zero (to avoid stub files) through the front-end of the new cluster.
>
>
>
> Did you rsync via the mount point or directly from the bricks?
>
>
>
> However, it has recently come to our attention that some of the files
> copied over were already “corrupted” on the old back-end. That is, these
> files had permissions of 1000 (like a stub file) yet were the full size of
> the actual file.
>
>
>
> Does this correspond to a file permission of ___T when viewed using ls? If
> yes, these are dht linkto files. They were possibly created during a
> rebalance and left behind because the file was skipped. They should be
> ignored when accessing the gluster volume via the mount point.
>
>
>
> In some cases, these were the only copies of the file that existed at all
> on any of the bricks, in others, another version of the file existed that
> was also full size and had the proper permissions. In some cases, we
> believe, these correct files were rsync’d but then overwritten by the 1000
> permission version resulting in a useless file on the new cluster.
>
>
>
> This sounds like you were running rsync directly on the bricks. Can you
> please confirm if that is the case?
>
>
>
> These files are thought by the OS to be binaries when trying to open them
> using vim, but they are actually text files (or at least were originally).
> We can cat the file to see that it has a length of zero and so far that is
> our only reliable test to find which files are indeed corrupted (find .
> -type f | xargs wc -l). With nearly 50 million files on our cluster, this
> is really a non-starter because of the speed.
>
>
>
> Has anyone seen this issue previously? We’re hoping to find a solution
> that doesn’t involve overthinking the problem and thought this might be a
> great place to start.
>
>
>
> Let me know if there’s any info I may have omitted that could be of
> further use.
>
>
>
> Thanks,
>
> Kevin
>
>
>
> Thanks,
>
> Nithya
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
>
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] autofs' wildcard key

2016-11-15 Thread lejeczek 

hi everyone

it does not work for me, is it supposed to work?
Autofs does not complain nor report any errors nor problems.

* -fstype=glusterfs -rw 10.5.6.49,10.5.6.100:/USER-HOME/&

but this does:

everything -fstype=glusterfs -rw 10.5.6.49,10.5.6.100:/USER-HOME

so wildcard keys does not work? What's broken here?
many thanks.
L
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster File Abnormalities

2016-11-15 Thread Kevin Leigeb 
Nithya -

Thanks for the reply, I will send this at the top to keep the thread from 
getting really ugly.

We did indeed copy from the individual bricks in an effort to speed up the 
copy. We had one rsync running from each brick to the mount point for the new 
cluster. As stated, we skipped all files with size 0 so that stub files 
wouldn’t be copied. Some files with permissions of 1000 (equivalent to 
-T) were larger than 0 and were also copied.

I’m mostly trying to ascertain why such files would exist (failed rebalance?) 
and what we can do about this problem.

Thanks,
Kevin

From: Nithya Balachandran [mailto:nbala...@redhat.com]
Sent: Tuesday, November 15, 2016 10:21 AM
To: Kevin Leigeb 
Cc: gluster-users@gluster.org
Subject: Re: [Gluster-users] Gluster File Abnormalities


Hi kevin,

On 15 November 2016 at 20:56, Kevin Leigeb 
> wrote:
All -

We recently moved from an old cluster running 3.7.9 to a new one running 3.8.4. 
To move the data we rsync’d all files from the old gluster nodes that were not 
in the .glusterfs directory and had a size of greater-than zero (to avoid stub 
files) through the front-end of the new cluster.

Did you rsync via the mount point or directly from the bricks?

However, it has recently come to our attention that some of the files copied 
over were already “corrupted” on the old back-end. That is, these files had 
permissions of 1000 (like a stub file) yet were the full size of the actual 
file.

Does this correspond to a file permission of ___T when viewed using ls? If yes, 
these are dht linkto files. They were possibly created during a rebalance and 
left behind because the file was skipped. They should be ignored when accessing 
the gluster volume via the mount point.

In some cases, these were the only copies of the file that existed at all on 
any of the bricks, in others, another version of the file existed that was also 
full size and had the proper permissions. In some cases, we believe, these 
correct files were rsync’d but then overwritten by the 1000 permission version 
resulting in a useless file on the new cluster.

This sounds like you were running rsync directly on the bricks. Can you please 
confirm if that is the case?

These files are thought by the OS to be binaries when trying to open them using 
vim, but they are actually text files (or at least were originally). We can cat 
the file to see that it has a length of zero and so far that is our only 
reliable test to find which files are indeed corrupted (find . -type f | xargs 
wc -l). With nearly 50 million files on our cluster, this is really a 
non-starter because of the speed.

Has anyone seen this issue previously? We’re hoping to find a solution that 
doesn’t involve overthinking the problem and thought this might be a great 
place to start.

Let me know if there’s any info I may have omitted that could be of further use.

Thanks,
Kevin

Thanks,
Nithya
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Gluster File Abnormalities

2016-11-15 Thread Nithya Balachandran 
Hi kevin,

On 15 November 2016 at 20:56, Kevin Leigeb  wrote:

> All -
>
>
>
> We recently moved from an old cluster running 3.7.9 to a new one running
> 3.8.4. To move the data we rsync’d all files from the old gluster nodes
> that were not in the .glusterfs directory and had a size of greater-than
> zero (to avoid stub files) through the front-end of the new cluster.
>

Did you rsync via the mount point or directly from the bricks?


> However, it has recently come to our attention that some of the files
> copied over were already “corrupted” on the old back-end. That is, these
> files had permissions of 1000 (like a stub file) yet were the full size of
> the actual file.
>

Does this correspond to a file permission of ___T when viewed using ls? If
yes, these are dht linkto files. They were possibly created during a
rebalance and left behind because the file was skipped. They should be
ignored when accessing the gluster volume via the mount point.

In some cases, these were the only copies of the file that existed at all
> on any of the bricks, in others, another version of the file existed that
> was also full size and had the proper permissions. In some cases, we
> believe, these correct files were rsync’d but then overwritten by the 1000
> permission version resulting in a useless file on the new cluster.
>
>
>
This sounds like you were running rsync directly on the bricks. Can you
please confirm if that is the case?


> These files are thought by the OS to be binaries when trying to open them
> using vim, but they are actually text files (or at least were originally).
> We can cat the file to see that it has a length of zero and so far that is
> our only reliable test to find which files are indeed corrupted (find .
> -type f | xargs wc -l). With nearly 50 million files on our cluster, this
> is really a non-starter because of the speed.
>
>
>
> Has anyone seen this issue previously? We’re hoping to find a solution
> that doesn’t involve overthinking the problem and thought this might be a
> great place to start.
>
>
>
> Let me know if there’s any info I may have omitted that could be of
> further use.
>
>
>
> Thanks,
>
> Kevin
>
> Thanks,
Nithya

> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Glusterfs readonly Issue

2016-11-15 Thread Nag Pavan Chilakam 
Hi Atul,
In Short: it is due to client side quorum behavior
Detailed info:
I see that there are 3 nodes in the cluster ie master1, master2, compute01
However the volume is being hosted only on master1 and master2. 
Also,  see that you have enabled server side quorum, and client side quorum 
from vol info
cluster.quorum-type: auto  =>client side quorum options
cluster.server-quorum-type: server  =>server side quorum options
cluster.server-quorum-ratio: 51%  =>server side quorum options
Given that you are using compute01 (which is more of a dummy node in this 
case), hence even though one node is down, still server side quorum is 
maintained

Client side quorum means, that the nodes are reachable by the client(IO path). 
If set to "auto", this option allows writes to the file only if number of 
bricks that are up >= ceil (of the total number of bricks that constitute that 
replica/2). If the number of replicas is even, then there is a further check: 
If the number of up bricks is exactly equal to n/2, then the first brick must 
be one of the bricks that is up. If it is more than n/2 then it is not 
necessary that the first brick is one of the up bricks.
In x2 case, the first brick of a replica pair must be always up for data to be 
written from client.

Hence when you bring down node1 you get readonly, but when you bring down node2 
you can still write to the volume.


- Original Message -
From: "Atul Yadav" 
To: "Atin Mukherjee" , "gluster-users" 

Sent: Monday, 14 November, 2016 8:04:24 PM
Subject: [Gluster-users] Glusterfs readonly Issue

Dear Team, 

In the event of the failure of master1, master 2 glusterfs home directory will 
become read only fs. 

If we manually shutdown the master 2, then there is no impact on the file 
system and all io operation will complete with out any problem. 

can you please provide some guidance to isolate the problem. 



# gluster peer status 
Number of Peers: 2 

Hostname: master1-ib.dbt.au 
Uuid: a5608d66-a3c6-450e-a239-108668083ff2 
State: Peer in Cluster (Connected) 

Hostname: compute01-ib.dbt.au 
Uuid: d2c47fc2-f673-4790-b368-d214a58c59f4 
State: Peer in Cluster (Connected) 



# gluster vol info home 

Volume Name: home 
Type: Replicate 
Volume ID: 2403ddf9-c2e0-4930-bc94-734772ef099f 
Status: Started 
Number of Bricks: 1 x 2 = 2 
Transport-type: tcp,rdma 
Bricks: 
Brick1: master1-ib.dbt.au:/glusterfs/home/brick1 
Brick2: master2-ib.dbt.au:/glusterfs/home/brick2 
Options Reconfigured: 
performance.quick-read: off 
performance.read-ahead: off 
performance.io-cache: off 
performance.stat-prefetch: off 
network.remote-dio: enable 
cluster.quorum-type: auto  
nfs.disable: on 
performance.readdir-ahead: on 
cluster.server-quorum-type: server 
config.transport: tcp,rdma 
network.ping-timeout: 10 
cluster.server-quorum-ratio: 51% 
cluster.enable-shared-storage: disable 



# gluster vol heal home info 
Brick master1-ib.dbt.au:/glusterfs/home/brick1 
Status: Connected 
Number of entries: 0 

Brick master2-ib.dbt.au:/glusterfs/home/brick2 
Status: Connected 
Number of entries: 0 


# gluster vol heal home info heal-failed 
Gathering list of heal failed entries on volume home has been unsuccessful on 
bricks that are down. Please check if all brick processes are 
running[root@master2 


Thank You 
Atul Yadav 

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Gluster File Abnormalities

2016-11-15 Thread Kevin Leigeb 
All -

We recently moved from an old cluster running 3.7.9 to a new one running 3.8.4. 
To move the data we rsync'd all files from the old gluster nodes that were not 
in the .glusterfs directory and had a size of greater-than zero (to avoid stub 
files) through the front-end of the new cluster. However, it has recently come 
to our attention that some of the files copied over were already "corrupted" on 
the old back-end. That is, these files had permissions of 1000 (like a stub 
file) yet were the full size of the actual file. In some cases, these were the 
only copies of the file that existed at all on any of the bricks, in others, 
another version of the file existed that was also full size and had the proper 
permissions. In some cases, we believe, these correct files were rsync'd but 
then overwritten by the 1000 permission version resulting in a useless file on 
the new cluster.

These files are thought by the OS to be binaries when trying to open them using 
vim, but they are actually text files (or at least were originally). We can cat 
the file to see that it has a length of zero and so far that is our only 
reliable test to find which files are indeed corrupted (find . -type f | xargs 
wc -l). With nearly 50 million files on our cluster, this is really a 
non-starter because of the speed.

Has anyone seen this issue previously? We're hoping to find a solution that 
doesn't involve overthinking the problem and thought this might be a great 
place to start.

Let me know if there's any info I may have omitted that could be of further use.

Thanks,
Kevin
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] GlusterFS geo-replication brick KeyError

2016-11-15 Thread Saravanakumar Arumugam 



On 11/15/2016 03:46 AM, Shirwa Hersi wrote:

Hi,

I'm using glusterfs geo-replication on version 3.7.11, one of the 
bricks becomes faulty and does not replicated to slave bricks after i 
start geo-replication session.
Following are the logs related to the faulty brick, can someone please 
advice me on how to resolve this issue.


[2016-06-11 09:41:17.359086] E 
[syncdutils(/var/glusterfs/gluster_b2/brick):276:log_raise_exception] : 
FAIL:
Traceback (most recent call last):
   File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 166, in main
 main_i()
   File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 663, in 
main_i
 local.service_loop(*[r for r in [remote] if r])
   File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 1497, in 
service_loop
 g3.crawlwrap(oneshot=True)
   File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 571, in 
crawlwrap
 self.crawl()
   File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1201, in 
crawl
 self.changelogs_batch_process(changes)
   File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1107, in 
changelogs_batch_process
 self.process(batch)
   File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 984, in 
process
 self.datas_in_batch.remove(unlinked_gfid)
KeyError: '.gfid/757b0ad8-b6f5-44da-b71a-1b1c25a72988'
The bug mentioned is fixed in upstream. Refer this link: 
http://www.gluster.org/pipermail/bugs/2016-June/061785.html You can 
update gluster to get the fix. Alternatively, you can try to restart 
geo-rep session using "start force" to overcome the error. But updating 
is better. Thanks, Saravana
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] REMINDER: Gluster Community Bug Triage meeting at 12:00 UTC (~in 5 minutes)

2016-11-15 Thread Soumya Koduri 

Hi all,

Apologies for the late notice.

This meeting is scheduled for anyone who is interested in learning more
about, or assisting with the Bug Triage.

Meeting details:
- location: #gluster-meeting on Freenode IRC
  (https://webchat.freenode.net/?channels=gluster-meeting  )
- date: every Tuesday
- time: 12:00 UTC
  (in your terminal, run: date -d "12:00 UTC")
- agenda: https://public.pad.fsfe.org/p/gluster-bug-triage

Currently the following items are listed:
* Roll Call
* Status of last weeks action items
* Group Triage
* Open Floor

The last two topics have space for additions. If you have a suitable bug
or topic to discuss, please add it to the agenda.

Appreciate your participation.

Thanks,
Soumya
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Volume ping-timeout parameter and client side mount timeouts

2016-11-15 Thread Mohammed Rafi K C 
If I understand the query correctly, the problem is that gluster takes
more than 20seconds to timeout even though the brick was offline for
more than 35s. With that assumptions I have some

How did you understand that the timer has expired after 35s only, by log
file? If so glusterfs wait  some time to flush the logs to console to
push it as batch, not sure how long. So the actual timing in the logs
may not be accurate.

If you have already confirmed that by using a wireshark or similar tools
that it takes more than 20seconds to disconnect the socket, then there
could be some thing else which we need to look into.

Can you conform that using wireshark or similar tools if not already done.


Rafi KC


On 11/14/2016 09:13 PM, Martin Schlegel wrote:
> Hello Gluster Community
>
> We have 2x brick nodes running with replication for a volume gv0 for which 
> set a
> "gluster volume set gv0 ping-timeout 20".
>
> In our tests it seemed there is unknown delay with this ping-timeout - we see 
> it
> timing out much later after about 35 seconds and not at around 20 seconds (see
> test below).
>
> Our distributed database cluster is using Gluster as a secondary file system 
> for
> backups etc. - it's Pacemaker cluster manager needs to know how long to wait
> before giving up on the glusterfs mounted file system to become available 
> again
> or when to failover to another node.
>
> 1. When do we know when to give up waiting on the glusterfs mount point to
> become accessible again following an outage on the brick server this client 
> was
> connected to ?
> 2. Is there a timeout / interval setting on the client side that we could
> reduce, so that it more quickly tries to switch the mount point to a 
> different,
> available brick server ?
>
>
> Regards,
> Martin Schlegel
>
> __
>
> Here is how we tested this:
>
> As a test we blocked the entire network on one of these brick nodes:
> root@glusterfs-brick-node1 $ date;iptables -A INPUT -i bond0 -j DROP ; 
> iptables
> -A OUTPUT -o bond0 -j DROP
> Mon Nov 14 08:26:55 UTC 2016
>
> From the syslog on the glusterfs-client-node
> Nov 14 08:27:30 glusterfs-client-node1 pgshared1[26783]: [2016-11-14
> 08:27:30.275694] C [rpc-clnt-ping.c:160:rpc_clnt_ping_timer_expired]
> 0-gv0-client-0: server glusterfs-brick-node1:49152 has not responded in the 
> last
> 20 seconds, disconnecting.
>
> <--- This last message "has not responded in the last 20 seconds" is confusing
> to me, because the brick node was clearly blocked for 35 seconds already ! Is
> there some client-side check interval that can be reduced ?
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] question about info and info.tmp

2016-11-15 Thread songxin 
ok, thank you.




在 2016-11-15 16:12:34，"Atin Mukherjee"  写道：





On Tue, Nov 15, 2016 at 12:47 PM, songxin  wrote:



Hi Atin,


I think the root cause is in the function glusterd_import_friend_volume as 
below. 

int32_t 
glusterd_import_friend_volume (dict_t *peer_data, size_t count) 
{ 
... 
ret = glusterd_volinfo_find (new_volinfo->volname, _volinfo); 
if (0 == ret) { 
(void) gd_check_and_update_rebalance_info (old_volinfo, 
   new_volinfo); 
(void) glusterd_delete_stale_volume (old_volinfo, new_volinfo); 
} 
... 
ret = glusterd_store_volinfo (new_volinfo, 
GLUSTERD_VOLINFO_VER_AC_NONE); 
if (ret) { 
gf_msg (this->name, GF_LOG_ERROR, 0, 
GD_MSG_VOLINFO_STORE_FAIL, "Failed to store " 
"volinfo for volume %s", new_volinfo->volname); 
goto out; 
} 
... 
} 

glusterd_delete_stale_volume will remove the info and bricks/* and the 
glusterd_store_volinfo will create the new one. 
But if glusterd is killed before rename the info will is empty. 


And glusterd will start failed because the infois empty in the next time you 
start the glusterd.


Any idea， Atin？


Give me some time, will check it out, but reading at this analysis looks very 
well possible if a volume is changed when the glusterd was done on node a and 
when the same comes up during peer handshake we update the volinfo and during 
that time glusterd goes down once again. I'll confirm it by tomorrow.


BTW, excellent work Xin!




Thanks,
Xin



在 2016-11-15 12:07:05，"Atin Mukherjee"  写道：





On Tue, Nov 15, 2016 at 8:58 AM, songxin  wrote:

Hi Atin,
I have some clues about this issue.
I could reproduce this issue use the scrip that mentioned in 
https://bugzilla.redhat.com/show_bug.cgi?id=1308487 .


I really appreciate your help in trying to nail down this issue. While I am at 
your email and going through the code to figure out the possible cause for it, 
unfortunately I don't see any script in the attachment of the bug.  Could you 
please cross check?
 



After I added some debug print,which like below, in glusterd-store.c and I 
found that the /var/lib/glusterd/vols/xxx/info and 
/var/lib/glusterd/vols/xxx/bricks/* are removed. 
But other files in /var/lib/glusterd/vols/xxx/ will not be remove.


int32_t
glusterd_store_volinfo (glusterd_volinfo_t *volinfo, glusterd_volinfo_ver_ac_t 
ac)
{
int32_t ret = -1;


GF_ASSERT (volinfo)


ret = access("/var/lib/glusterd/vols/gv0/info", F_OK);
if(ret < 0)
{
gf_msg (THIS->name, GF_LOG_ERROR, 0, 0, "info is not exit(%d)", 
errno);
}
else
{
ret = stat("/var/lib/glusterd/vols/gv0/info", );
if(ret < 0)
{
gf_msg (THIS->name, GF_LOG_ERROR, 0, 0, "stat info 
error");
}
else
{
gf_msg (THIS->name, GF_LOG_ERROR, 0, 0, "info size is 
%lu, inode num is %lu", buf.st_size, buf.st_ino);
}
}


glusterd_perform_volinfo_version_action (volinfo, ac);
ret = glusterd_store_create_volume_dir (volinfo);
if (ret)
goto out;


...
}


So it is easy to understand why  the info or 10.32.1.144.-opt-lvmdir-c2-brick 
sometimes is empty.
It is becaue the info file is not exist, and it will be create by “fd = open 
(path, O_RDWR | O_CREAT | O_APPEND, 0600);” in function gf_store_handle_new.
And the info file is empty before rename.
So the info file is empty if glusterd shutdown before rename.
 



My question is following.
1.I did not find the point the info is removed.Could you tell me the point 
where the info and /bricks/* are removed?
2.why the file info and bricks/* is removed?But other files in 
var/lib/glusterd/vols/xxx/ are not be removed?

AFAIK, we never delete the info file and hence this file is opened with 
O_APPEND flag. As I said I will go back and cross check the code once again.






Thanks,
Xin



在 2016-11-11 20:34:05，"Atin Mukherjee"  写道：





On Fri, Nov 11, 2016 at 4:00 PM, songxin  wrote:

Hi Atin,



Thank you for your support.
Sincerely wait for your reply.


By the way, could you make sure that the issue, file info is empty, cause by 
rename is interrupted in kernel?


As per my RCA on that bug, it looked to be.
 



Thanks,
Xin

在 2016-11-11 15:49:02，"Atin Mukherjee"  写道：





On Fri, Nov 11, 2016 at 1:15 PM, songxin  wrote:

Hi Atin,
Thank you for your reply.
Actually it is very difficult to reproduce because I don't know when there was 
an ongoing commit happening.It is just a coincidence.
But I want to make sure the root

Re: [Gluster-users] question about info and info.tmp

2016-11-15 Thread Atin Mukherjee 
On Tue, Nov 15, 2016 at 12:47 PM, songxin  wrote:

>
> Hi Atin,
>
> I think the root cause is in the function glusterd_import_friend_volume as
> below.
>
> int32_t
> glusterd_import_friend_volume (dict_t *peer_data, size_t count)
> {
> ...
> ret = glusterd_volinfo_find (new_volinfo->volname, _volinfo);
> if (0 == ret) {
> (void) gd_check_and_update_rebalance_info (old_volinfo,
>new_volinfo);
> (void) glusterd_delete_stale_volume (old_volinfo,
> new_volinfo);
> }
> ...
> ret = glusterd_store_volinfo (new_volinfo,
> GLUSTERD_VOLINFO_VER_AC_NONE);
> if (ret) {
> gf_msg (this->name, GF_LOG_ERROR, 0,
> GD_MSG_VOLINFO_STORE_FAIL, "Failed to store "
> "volinfo for volume %s", new_volinfo->volname);
> goto out;
> }
> ...
> }
>
> glusterd_delete_stale_volume will remove the info and bricks/* and the
> glusterd_store_volinfo will create the new one.
> But if glusterd is killed before rename the info will is empty.
>
> And glusterd will start failed because the infois empty in the next time
> you start the glusterd.
>
> Any idea， Atin？
>

Give me some time, will check it out, but reading at this analysis looks
very well possible if a volume is changed when the glusterd was done on
node a and when the same comes up during peer handshake we update the
volinfo and during that time glusterd goes down once again. I'll confirm it
by tomorrow.

BTW, excellent work Xin!


> Thanks,
> Xin
>
>
> 在 2016-11-15 12:07:05，"Atin Mukherjee"  写道：
>
>
>
> On Tue, Nov 15, 2016 at 8:58 AM, songxin  wrote:
>
>> Hi Atin,
>> I have some clues about this issue.
>> I could reproduce this issue use the scrip that mentioned in
>> https://bugzilla.redhat.com/show_bug.cgi?id=1308487 .
>>
>
> I really appreciate your help in trying to nail down this issue. While I
> am at your email and going through the code to figure out the possible
> cause for it, unfortunately I don't see any script in the attachment of the
> bug.  Could you please cross check?
>
>
>>
>> After I added some debug print,which like below, in glusterd-store.c and
>> I found that the /var/lib/glusterd/vols/xxx/info and
>> /var/lib/glusterd/vols/xxx/bricks/* are removed.
>> But other files in /var/lib/glusterd/vols/xxx/ will not be remove.
>>
>> int32_t
>> glusterd_store_volinfo (glusterd_volinfo_t *volinfo,
>> glusterd_volinfo_ver_ac_t ac)
>> {
>> int32_t ret = -1;
>>
>> GF_ASSERT (volinfo)
>>
>> ret = access("/var/lib/glusterd/vols/gv0/info", F_OK);
>> if(ret < 0)
>> {
>> gf_msg (THIS->name, GF_LOG_ERROR, 0, 0, "info is not
>> exit(%d)", errno);
>> }
>> else
>> {
>> ret = stat("/var/lib/glusterd/vols/gv0/info", );
>> if(ret < 0)
>> {
>> gf_msg (THIS->name, GF_LOG_ERROR, 0, 0, "stat
>> info error");
>> }
>> else
>> {
>> gf_msg (THIS->name, GF_LOG_ERROR, 0, 0, "info
>> size is %lu, inode num is %lu", buf.st_size, buf.st_ino);
>> }
>> }
>>
>> glusterd_perform_volinfo_version_action (volinfo, ac);
>> ret = glusterd_store_create_volume_dir (volinfo);
>> if (ret)
>> goto out;
>>
>> ...
>> }
>>
>> So it is easy to understand why  the info or 10.32.1.144.-opt-lvmdir-c2-
>> brick sometimes is empty.
>> It is becaue the info file is not exist, and it will be create by “fd =
>> open (path, O_RDWR | O_CREAT | O_APPEND, 0600);” in function
>> gf_store_handle_new.
>> And the info file is empty before rename.
>> So the info file is empty if glusterd shutdown before rename.
>>
>>
>
>> My question is following.
>> 1.I did not find the point the info is removed.Could you tell me the
>> point where the info and /bricks/* are removed?
>> 2.why the file info and bricks/* is removed?But other files in 
>> var/lib/glusterd/vols/xxx/
>> are not be removed?
>>
>
> AFAIK, we never delete the info file and hence this file is opened with
> O_APPEND flag. As I said I will go back and cross check the code once again.
>
>
>
>
>> Thanks,
>> Xin
>>
>>
>> 在 2016-11-11 20:34:05，"Atin Mukherjee"  写道：
>>
>>
>>
>> On Fri, Nov 11, 2016 at 4:00 PM, songxin  wrote:
>>
>>> Hi Atin,
>>>
>>> Thank you for your support.
>>> Sincerely wait for your reply.
>>>
>>> By the way, could you make sure that the issue, file info is empty,
>>> cause by rename is interrupted in kernel?
>>>
>>
>> As per my RCA on that bug, it looked to be.
>>
>>
>>>
>>> Thanks,
>>> Xin
>>>
>>> 在 2016-11-11 15:49:02，"Atin Mukherjee"  写道：
>>>
>>>
>>>
>>> On Fri, Nov 11, 2016 at 1:15 PM, songxin