Re: [Gluster-devel] could you help to check about a glusterfs issue seems to be related to ctime

[Zhou, Cynthia (NSB - CN/Hangzhou)]

I have not tested with option attribute-timeout=0 -o entry-timeout=0,
But I think to set that two options to 0 will have impact on performance.
Also set consistent-metadata on will also have impact for performance, so I’d 
rather keep consistent-metadata to be off and use ctime feature to solve the 
tar issue.

cynthia

From: Ravishankar N 
Sent: 2020年3月17日 17:08
To: Zhou, Cynthia (NSB - CN/Hangzhou) ; Kotresh 
Hiremath Ravishankar 
Cc: Gluster Devel 
Subject: Re: [Gluster-devel] could you help to check about a glusterfs issue 
seems to be related to ctime



On 17/03/20 12:56 pm, Zhou, Cynthia (NSB - CN/Hangzhou) wrote:
Hi,
I know that this may not be your presumption of this ctime feature, however, 
our delima is that

  1.  product need to support date change scenario.
  2.  if we disable ctime feature, we will meet tar issue.
  3.  if we set consistent-metadata to on to work around this tar issue, there 
is another even worse problem: 
https://bugzilla.redhat.com/show_bug.cgi?id=1773476, app just cranshed!
So I have to make this patch.

For what it is worth, the 'file changed as we read it' warning from tar can be 
suppressed with the --warning=no-file-changed flag.

For 3, do you see any visible usability/performance issues with the work-around 
mentioned in the bug (comment 5 and 6)? Just curious.
Regards,
Ravi

Maybe I will keep this as internal use just for our local use scenarios.
Cynthia.

From: Kotresh Hiremath Ravishankar 

Sent: 2020年3月17日 15:17
To: Zhou, Cynthia (NSB - CN/Hangzhou) 

Cc: Amar Tumballi ; Gluster Devel 

Subject: Re: [Gluster-devel] could you help to check about a glusterfs issue 
seems to be related to ctime

The whole ctime features relies on time provided by clients which are time 
synchronized. This patch brings in the time from server to compare against the 
time sent from client.
As Amar mentioned, this doesn't fit well into the scheme of how ctime is 
designed. Definitely keeping it optional and disabling it by default is one 
way. But is that what your intention
here?

On Tue, Mar 17, 2020 at 10:56 AM Zhou, Cynthia (NSB - CN/Hangzhou) 
mailto:cynthia.z...@nokia-sbell.com>> wrote:
Ok, thanks for your feedback!
I will do local test to verify this patch first.

cynthia

From: Amar Tumballi mailto:a...@kadalu.io>>
Sent: 2020年3月17日 13:18
To: Zhou, Cynthia (NSB - CN/Hangzhou) 
mailto:cynthia.z...@nokia-sbell.com>>
Cc: Kotresh Hiremath Ravishankar 
mailto:khire...@redhat.com>>; Gluster Devel 
mailto:gluster-devel@gluster.org>>
Subject: Re: [Gluster-devel] could you help to check about a glusterfs issue 
seems to be related to ctime


On Tue, Mar 17, 2020 at 10:18 AM Zhou, Cynthia (NSB - CN/Hangzhou) 
mailto:cynthia.z...@nokia-sbell.com>> wrote:
Hi glusterfs expert,
Our product need to tolerate change date to future and then change back.
How about change like this ?
https://review.gluster.org/#/c/glusterfs/+/24229/1/xlators/storage/posix/src/posix-metadata.c

when time change to future and change back , should still be able to update 
mdata, so the following changes to file can be populated to other clients.


We do like to have people integrating with GlusterFS. But this change is not 
inline with the 'assumptions' we had about the feature.

If you have verified this change works for you, please add it as an 'option' in 
posix, which can be changed through volume set, and keep this option 
disable/off by default. That should be an easier way to get the patch reviewed 
and take it further. Please make sure to provide the 'description' for the 
option with details.

Regards,
Amar


cynthia

From: Zhou, Cynthia (NSB - CN/Hangzhou)
Sent: 2020年3月12日 17:31
To: 'Kotresh Hiremath Ravishankar' 
mailto:khire...@redhat.com>>
Cc: 'Gluster Devel' 
mailto:gluster-devel@gluster.org>>
Subject: RE: could you help to check about a glusterfs issue seems to be 
related to ctime

Hi,
One more question, I find each client has the same future time stamp where are 
those time stamps from, since Since it is different from any brick stored time 
stamp. And after I modify files  from clients, it remains the same.
[root@mn-0:/home/robot]
# stat /mnt/export/testfile
  File: /mnt/export/testfile
  Size: 193 Blocks: 1  IO Block: 131072 regular file
Device: 28h/40d Inode: 10383279039841136109  Links: 1
Access: (0644/-rw-r--r--)  Uid: (0/root)   Gid: (  
615/_nokfsuifileshare)
Access: 2020-04-11 12:20:22.114365172 +0300
Modify: 2020-04-11 12:20:22.121552573 +0300
Change: 2020-04-11 12:20:22.121552573 +0300

[root@mn-0:/home/robot]
# date
Thu Mar 12 11:27:33 EET 2020
[root@mn-0:/home/robot]

[root@mn-0:/home/robot]
# stat /mnt/bricks/export/brick/testfile
  File: /mnt/bricks/export/brick/testfile
  Size: 193 Blocks: 16 IO Block: 4096   regular file
Device: fc02h/64514dInode: 512015  Links: 2
Access: (0644/-rw-r--r--)  Uid: (0/

Re: [Gluster-devel] could you help to check about a glusterfs issue seems to be related to ctime

[Zhou, Cynthia (NSB - CN/Hangzhou)]

Hi,
I know that this may not be your presumption of this ctime feature, however, 
our delima is that

  1.  product need to support date change scenario.
  2.  if we disable ctime feature, we will meet tar issue.
  3.  if we set consistent-metadata to on to work around this tar issue, there 
is another even worse problem: 
https://bugzilla.redhat.com/show_bug.cgi?id=1773476, app just cranshed!
So I have to make this patch.
Maybe I will keep this as internal use just for our local use scenarios.
Cynthia.

From: Kotresh Hiremath Ravishankar 
Sent: 2020年3月17日 15:17
To: Zhou, Cynthia (NSB - CN/Hangzhou) 
Cc: Amar Tumballi ; Gluster Devel 
Subject: Re: [Gluster-devel] could you help to check about a glusterfs issue 
seems to be related to ctime

The whole ctime features relies on time provided by clients which are time 
synchronized. This patch brings in the time from server to compare against the 
time sent from client.
As Amar mentioned, this doesn't fit well into the scheme of how ctime is 
designed. Definitely keeping it optional and disabling it by default is one 
way. But is that what your intention
here?

On Tue, Mar 17, 2020 at 10:56 AM Zhou, Cynthia (NSB - CN/Hangzhou) 
mailto:cynthia.z...@nokia-sbell.com>> wrote:
Ok, thanks for your feedback!
I will do local test to verify this patch first.

cynthia

From: Amar Tumballi mailto:a...@kadalu.io>>
Sent: 2020年3月17日 13:18
To: Zhou, Cynthia (NSB - CN/Hangzhou) 
mailto:cynthia.z...@nokia-sbell.com>>
Cc: Kotresh Hiremath Ravishankar 
mailto:khire...@redhat.com>>; Gluster Devel 
mailto:gluster-devel@gluster.org>>
Subject: Re: [Gluster-devel] could you help to check about a glusterfs issue 
seems to be related to ctime


On Tue, Mar 17, 2020 at 10:18 AM Zhou, Cynthia (NSB - CN/Hangzhou) 
mailto:cynthia.z...@nokia-sbell.com>> wrote:
Hi glusterfs expert,
Our product need to tolerate change date to future and then change back.
How about change like this ?
https://review.gluster.org/#/c/glusterfs/+/24229/1/xlators/storage/posix/src/posix-metadata.c

when time change to future and change back , should still be able to update 
mdata, so the following changes to file can be populated to other clients.


We do like to have people integrating with GlusterFS. But this change is not 
inline with the 'assumptions' we had about the feature.

If you have verified this change works for you, please add it as an 'option' in 
posix, which can be changed through volume set, and keep this option 
disable/off by default. That should be an easier way to get the patch reviewed 
and take it further. Please make sure to provide the 'description' for the 
option with details.

Regards,
Amar


cynthia

From: Zhou, Cynthia (NSB - CN/Hangzhou)
Sent: 2020年3月12日 17:31
To: 'Kotresh Hiremath Ravishankar' 
mailto:khire...@redhat.com>>
Cc: 'Gluster Devel' 
mailto:gluster-devel@gluster.org>>
Subject: RE: could you help to check about a glusterfs issue seems to be 
related to ctime

Hi,
One more question, I find each client has the same future time stamp where are 
those time stamps from, since Since it is different from any brick stored time 
stamp. And after I modify files  from clients, it remains the same.
[root@mn-0:/home/robot]
# stat /mnt/export/testfile
  File: /mnt/export/testfile
  Size: 193 Blocks: 1  IO Block: 131072 regular file
Device: 28h/40d Inode: 10383279039841136109  Links: 1
Access: (0644/-rw-r--r--)  Uid: (0/root)   Gid: (  
615/_nokfsuifileshare)
Access: 2020-04-11 12:20:22.114365172 +0300
Modify: 2020-04-11 12:20:22.121552573 +0300
Change: 2020-04-11 12:20:22.121552573 +0300

[root@mn-0:/home/robot]
# date
Thu Mar 12 11:27:33 EET 2020
[root@mn-0:/home/robot]

[root@mn-0:/home/robot]
# stat /mnt/bricks/export/brick/testfile
  File: /mnt/bricks/export/brick/testfile
  Size: 193 Blocks: 16 IO Block: 4096   regular file
Device: fc02h/64514dInode: 512015  Links: 2
Access: (0644/-rw-r--r--)  Uid: (0/root)   Gid: (  
615/_nokfsuifileshare)
Access: 2020-04-11 12:20:22.100395536 +0300
Modify: 2020-03-12 11:25:04.095981276 +0200
Change: 2020-03-12 11:25:04.095981276 +0200
Birth: 2020-04-11 08:53:26.805163816 +0300


[root@mn-1:/root]
# stat /mnt/bricks/export/brick/testfile
  File: /mnt/bricks/export/brick/testfile
  Size: 193 Blocks: 16 IO Block: 4096   regular file
Device: fc02h/64514dInode: 512015  Links: 2
Access: (0644/-rw-r--r--)  Uid: (0/root)   Gid: (  
615/_nokfsuifileshare)
Access: 2020-04-11 12:20:22.100395536 +0300
Modify: 2020-03-12 11:25:04.094913452 +0200
Change: 2020-03-12 11:25:04.095913453 +0200
Birth: 2020-03-12 07:53:26.803783053 +0200



From: Zhou, Cynthia (NSB - CN/Hangzhou)
Sent: 2020年3月12日 16:09
To: 'Kotresh Hiremath Ravishankar' 
mailto:khire...@redhat.com>>
Cc: Gluster Devel mailto:gluster-devel@gluster.org>>
Subject: RE: could you help to check about a glusterfs issue seems to be 
related to ctime

Hi,
This is abnormal test case, however, when this happened 

Re: [Gluster-devel] could you help to check about a glusterfs issue seems to be related to ctime

[Kotresh Hiremath Ravishankar]

The whole ctime features relies on time provided by clients which are time
synchronized. This patch brings in the time from server to compare against
the time sent from client.
As Amar mentioned, this doesn't fit well into the scheme of how ctime is
designed. Definitely keeping it optional and disabling it by default is one
way. But is that what your intention
here?

On Tue, Mar 17, 2020 at 10:56 AM Zhou, Cynthia (NSB - CN/Hangzhou) <
cynthia.z...@nokia-sbell.com> wrote:

> Ok, thanks for your feedback!
>
> I will do local test to verify this patch first.
>
>
>
> cynthia
>
>
>
> *From:* Amar Tumballi 
> *Sent:* 2020年3月17日 13:18
> *To:* Zhou, Cynthia (NSB - CN/Hangzhou) 
> *Cc:* Kotresh Hiremath Ravishankar ; Gluster Devel <
> gluster-devel@gluster.org>
> *Subject:* Re: [Gluster-devel] could you help to check about a glusterfs
> issue seems to be related to ctime
>
>
>
>
>
> On Tue, Mar 17, 2020 at 10:18 AM Zhou, Cynthia (NSB - CN/Hangzhou) <
> cynthia.z...@nokia-sbell.com> wrote:
>
> Hi glusterfs expert,
>
> Our product need to tolerate change date to future and then change back.
>
> How about change like this ?
>
>
> https://review.gluster.org/#/c/glusterfs/+/24229/1/xlators/storage/posix/src/posix-metadata.c
>
>
>
> when time change to future and change back , should still be able to
> update mdata, so the following changes to file can be populated to other
> clients.
>
>
>
>
>
> We do like to have people integrating with GlusterFS. But this change is
> not inline with the 'assumptions' we had about the feature.
>
>
>
> If you have verified this change works for you, please add it as an
> 'option' in posix, which can be changed through volume set, and keep this
> option disable/off by default. That should be an easier way to get the
> patch reviewed and take it further. Please make sure to provide the
> 'description' for the option with details.
>
>
>
> Regards,
>
> Amar
>
>
>
>
>
> cynthia
>
>
>
> *From:* Zhou, Cynthia (NSB - CN/Hangzhou)
> *Sent:* 2020年3月12日 17:31
> *To:* 'Kotresh Hiremath Ravishankar' 
> *Cc:* 'Gluster Devel' 
> *Subject:* RE: could you help to check about a glusterfs issue seems to
> be related to ctime
>
>
>
> Hi,
>
> One more question, I find each client has the same future time stamp where
> are those time stamps from, since Since it is different from any brick
> stored time stamp. And after I modify files  from clients, it remains the
> same.
>
> [root@mn-0:/home/robot]
>
> # stat /mnt/export/testfile
>
>   File: /mnt/export/testfile
>
>   Size: 193 Blocks: 1  IO Block: 131072 regular file
>
> Device: 28h/40d Inode: 10383279039841136109  Links: 1
>
> Access: (0644/-rw-r--r--)  Uid: (0/root)   Gid: (
> 615/_nokfsuifileshare)
>
> Access: 2020-04-11 12:20:22.114365172 +0300
>
> Modify: 2020-04-11 12:20:22.121552573 +0300
>
> Change: 2020-04-11 12:20:22.121552573 +0300
>
>
>
> [root@mn-0:/home/robot]
>
> # date
>
> Thu Mar 12 11:27:33 EET 2020
>
> [root@mn-0:/home/robot]
>
>
>
> [root@mn-0:/home/robot]
>
> # stat /mnt/bricks/export/brick/testfile
>
>   File: /mnt/bricks/export/brick/testfile
>
>   Size: 193 Blocks: 16 IO Block: 4096   regular file
>
> Device: fc02h/64514dInode: 512015  Links: 2
>
> Access: (0644/-rw-r--r--)  Uid: (0/root)   Gid: (
> 615/_nokfsuifileshare)
>
> Access: 2020-04-11 12:20:22.100395536 +0300
>
> Modify: 2020-03-12 11:25:04.095981276 +0200
>
> Change: 2020-03-12 11:25:04.095981276 +0200
>
> Birth: 2020-04-11 08:53:26.805163816 +0300
>
>
>
>
>
> [root@mn-1:/root]
>
> # stat /mnt/bricks/export/brick/testfile
>
>   File: /mnt/bricks/export/brick/testfile
>
>   Size: 193 Blocks: 16 IO Block: 4096   regular file
>
> Device: fc02h/64514dInode: 512015  Links: 2
>
> Access: (0644/-rw-r--r--)  Uid: (0/root)   Gid: (
> 615/_nokfsuifileshare)
>
> Access: 2020-04-11 12:20:22.100395536 +0300
>
> Modify: 2020-03-12 11:25:04.094913452 +0200
>
> Change: 2020-03-12 11:25:04.095913453 +0200
>
> Birth: 2020-03-12 07:53:26.803783053 +0200
>
>
>
>
>
>
>
> *From:* Zhou, Cynthia (NSB - CN/Hangzhou)
> *Sent:* 2020年3月12日 16:09
> *To:* 'Kotresh Hiremath Ravishankar' 
> *Cc:* Gluster Devel 
> *Subject:* RE: could you help to check about a glusterfs issue seems to
> be related to ctime
>
>
>
> Hi,
>
> This is abnormal test case, however, when this happened it will have big
> impact on the apps using those files. And this can not be restored
> automatically unless disable some xlator, I think it is unacceptable for
> the user apps.
>
>
>
>
>
> cynthia
>
>
>
> *From:* Kotresh Hiremath Ravishankar 
> *Sent:* 2020年3月12日 14:37
> *To:* Zhou, Cynthia (NSB - CN/Hangzhou) 
> *Cc:* Gluster Devel 
> *Subject:* Re: could you help to check about a glusterfs issue seems to
> be related to ctime
>
>
>
> All the perf xlators depend on time (mostly mtime I guess). In my setup,
> only quick read was enabled and hence disabling it worked for me.
> All perf xlators needs to be disabled to make it work