Re: [Bug-tar] Bug? Where? Why? Why so many files changing as we read them?
Paul Eggert wrote: One possibility is that the file system is buggy, and that reading a file (and therefore modifying st_atime) also modifies st_ctime as a side effect. Often this sort of thing is delayed, that is, you read a file and its st_atime is updated immediately, but its st_ctime is updated after a delay of a few seconds. This would explain the observed behavior. AFAIK (but with MS, who really knows anything?), MS adopted a similar system to the linux kernel in regards to updating last-access times when atime updating is turned off. I.e. I thought that the kernel opportunistically updates last-access if something else changes in the inode to be no earlier than the ctime or modtime fields. I just read last week in looking at MS notes on NTFS implementation, that it does something similar when don't have last-access turned off -- but it may hold off on the writing out of the last access time for up to an hour -- trying to wait for a time when one of the other times is updated. But that said -- I don't get the same messages when I run tar on the client-MS machine. Instead I get failures to read ACL's and ext-attrs... which is maybe worse. But it's only when I use the linux CIFS to access the remote NTFS that tar gets upset. So I don't think it is the source FS. See, for example: https://bugzilla.redhat.com/show_bug.cgi?id=1058526 which does indeed illustrate remote file-system bugs that can cause the behavior in question. (Mark O'Keefe's proposed change to tar isn't right; there are security reasons to worry about ctime as opposed to mtime). Well -- I have yet to try to use stat on all the files b4 and aftr ... keep getting sidetracked.
Re: [Bug-tar] Bug? Where? Why? Why so many files changing as we read them?
One possibility is that the file system is buggy, and that reading a file (and therefore modifying st_atime) also modifies st_ctime as a side effect. Often this sort of thing is delayed, that is, you read a file and its st_atime is updated immediately, but its st_ctime is updated after a delay of a few seconds. This would explain the observed behavior. See, for example: https://bugzilla.redhat.com/show_bug.cgi?id=1058526 which does indeed illustrate remote file-system bugs that can cause the behavior in question. (Mark O'Keefe's proposed change to tar isn't right; there are security reasons to worry about ctime as opposed to mtime).
Re: [Bug-tar] Bug? Where? Why? Why so many files changing as we read them?
On Monday 26 of October 2015 14:30:14 Linda A. Walsh wrote:
> > Try 'stat' on this file before/after running 'tar -c'. See '10.1.3 Race
> > conditions' section in documentation [1].
>
> Well the 78 messages come out sprinkled throughout a 60-70K file dump.
> It doesn't seem a 'stat' would be practical to try to isolate 1 in 1000
> files for a change that "might" be occurring...
One file is enough to prove that the "at least one file" has really
changed and tar is most likely not guilty, src/create.c source:
if ((timespec_cmp (get_stat_ctime (&final_stat), original_ctime) != 0
/* Original ctime will change if the file is a directory and
--remove-files is given */
&& !(remove_files_option && is_dir))
|| original_size < final_stat.st_size)
{
WARNOPT (WARN_FILE_CHANGED,
(0, 0, _("%s: file changed as we read it"),
quotearg_colon (p)));
set_exit_status (TAREXIT_DIFFERS);
}
So in case the "guilty" file has not changed its 'ctime' or its size, tar
should not complain. If there is a bug in GNU tar, I would suggest you to
use debugger and walk precisely through the cited code.
Re: [Bug-tar] Bug? Where? Why? Why so many files changing as we read them?
Pavel Raiskup wrote: Try 'stat' on this file before/after running 'tar -c'. See '10.1.3 Race conditions' section in documentation [1]. Well the 78 messages come out sprinkled throughout a 60-70K file dump. It doesn't seem a 'stat' would be practical to try to isolate 1 in 1000 files for a change that "might" be occurring... I'm already familiar with the concepts in section 10.1.3, but as I mentioned, those files belonged to apps that weren't in use, and in come cases haven't been used for quite some time. Their last-mod times indicated that all the files and the directory hadn't been changed in some time. FWIW, the volume it was accessing has 'last-accessed' time turned off, but I think it also has semantics similar to linux kernel's default last-access time update (i.e. if writing to inode and updating other stuff, then make sure last-access time is no earlier than other time/date fields in the inode. The only think accessing them would have been tar, and the occasional cosmic ray, but tar should handle the 1st case... ;-) Either make sure tar is the only process working with those files (globally as that is CIFS mount point, not only on your machine) That's the thing. It *is* only a CIFS mount point on my machine. I'm the only user on either of the machines. The link between the machines is a cross-over wire (no routers or switches) @ 10Gb, so the connection is the Win7 workstation's is a dedicated link to a linux server, who's only function is to handle the Win7's internet & disk needs so -- single user. I.e. "--warning=no-file-changed" would only hide the symptom -- not explain or prevent the cause. ;-(
Re: [Bug-tar] Bug? Where? Why? Why so many files changing as we read them?
On Sunday 25 of October 2015 20:23:57 Linda A. Walsh wrote: > This may not be a tar bug, but I am trying to figure out > what is going on and/or why so many 'changes'... [..] > Trying to run tar, as such: > > > tar cf ~/Appdata.tar --acls --xattrs Appdata > > And got **78** lines (out of 64389 files in 12357 directories) > that claim "something changed". > Note: I wasn't running any of the below related apps as tar ran: > > tar: Appdata/Local/Google/Chrome/User Data/Default/Cache: file changed > as we read it Try 'stat' on this file before/after running 'tar -c'. See '10.1.3 Race conditions' section in documentation [1]. Either make sure tar is the only process working with those files (globally as that is CIFS mount point, not only on your machine) or pass there '--warning=no-file-changed'. [1] https://www.gnu.org/software/tar/manual/tar.html Pavel
[Bug-tar] Bug? Where? Why? Why so many files changing as we read them?
This may not be a tar bug, but I am trying to figure out what is going on and/or why so many 'changes'... using tar V1.28 on linux-4.1.0 Reading from a Win7-SP1 workstation via 'CIFS' (mounted fs) trying to do: tar cf ~/Appdata.tar --acls --xattrs Appdata In my "Windows Home directory" as seen as a mounted file system via CIFS. //Athenae/C/ /athenae/ cifs user,noauto,rw,uid=x,gid=x,nocase,serverino,creds=xxx,setuids 0 0 Trying to run tar, as such: tar cf ~/Appdata.tar --acls --xattrs Appdata And got **78** lines (out of 64389 files in 12357 directories) that claim "something changed". Note: I wasn't running any of the below related apps as tar ran: tar: Appdata/Local/Google/Chrome/User Data/Default/Cache: file changed as we read it tar: Appdata/Local/Google/Chrome/User Data/Default: file changed as we read it tar: Appdata/Local/Google/Chrome/User Data: file changed as we read it tar: Appdata/Local/Microsoft/ehome/Art Cache: file changed as we read it tar: Appdata/Local/Microsoft/Media Player/Art Cache/LocalMLS: file changed as we read it tar: Appdata/Local/Microsoft/OneNote/12.0/Backup/Personal Notebook: file changed as we read it tar: Appdata/Local/Microsoft/OneNote/12.0/Backup: file changed as we read it ... tar: Exiting with failure status due to previous errors Of minor note, "find" commands run locally via cygwin and on linux (via CIFS) both show the same # files+dirs (76746) -- showing a 1:1 mapping. I did an "ls -l" from both sides (cyg v. linux+CIFS) that shows the last mod times -- Looked at "Appdata/Local/Microsoft/OneNote/12.0/Backup" and the subdir "Personal Notebook": Oct 15 2012 Backup Mar 18 2015 Personal Notebook I.e. neither looks like they changed recently. So why the messages? Thanks!
