Re: Unison 2.10.2 fast update check broken?
Rolf Campbell wrote: Marcus Picasso wrote: Seems that Cygwin port of the unison file synchronizer does not do the -fastcheck very well. Transcript follows: ... Can somebody confirm / explain this behaviour? I have a large tree that I'm synchronizing across two hard-disks, and got suspicious when re-running synchronization takes longer than expected. The above transcript functions as expected using linux or native Win32 unison builds. Regards, -Marcus. I have noticed a change in how -fastcheck which seems to be caused by my upgrade from cygwin 1.5.14 -> 1.5.16. I tried doing a unison sync between a maching running 1.5.14 and a machine running 1.5.16 when I noticed the 1.5.16 machine spent a lot of time grinding the disk. So, I upgraded the 1.5.14 machine to 1.5.16 and it too went from a 10 second scan time to a half hour of heavy disk access. It seems that this bug is back again. I'm not sure if it was the 1.5.18-1 upgrade that caused it, but now my unison is back to slow file scanning. Cygwin Configuration Diagnostics Current System Time: Tue Jul 19 10:31:32 2005 Windows XP Professional Ver 5.1 Build 2600 Service Pack 2 Path: C:\cygwin\usr\local\bin C:\cygwin\bin C:\cygwin\bin C:\cygwin\usr\X11R6\bin C:\WINDOWS\system32 C:\WINDOWS C:\WINDOWS\System32\Wbem C:\Program Files\Common Files\GTK\2.0\bin C:\Program Files\Perforce Output from C:\cygwin\bin\id.exe (nontsec) UID: 11643(rcampbell)GID: 10513(Domain Users) 0(root) 544(Administrators) 545(Users) 10513(Domain Users) Output from C:\cygwin\bin\id.exe (ntsec) UID: 11643(rcampbell)GID: 10513(Domain Users) 0(root) 544(Administrators) 545(Users) 10513(Domain Users) SysDir: C:\WINDOWS\system32 WinDir: C:\WINDOWS USER = `rcampbell' PWD = `/tmp' HOME = `/home/rcampbell' MAKE_MODE = `unix' HOMEPATH = `\Documents and Settings\rcampbell' MANPATH = `/usr/local/man:/usr/share/man:/usr/man::/usr/ssl/man' APPDATA = `C:\Documents and Settings\rcampbell\Application Data' HOSTNAME = `desk-rcampbell2' TERM = `xterm' PROCESSOR_IDENTIFIER = `x86 Family 15 Model 3 Stepping 3, GenuineIntel' WINDIR = `C:\WINDOWS' WINDOWID = `4819248' OLDPWD = `/home/rcampbell' USERDOMAIN = `TROPICNETWORKS' OS = `Windows_NT' ALLUSERSPROFILE = `C:\Documents and Settings\All Users' TEMP = `/tmp' COMMONPROGRAMFILES = `C:\Program Files\Common Files' USERNAME = `rcampbell' PROCESSOR_LEVEL = `15' FP_NO_HOST_CHECK = `NO' SYSTEMDRIVE = `C:' USERPROFILE = `C:\Documents and Settings\rcampbell' PS1 = `\[\e]0;[EMAIL PROTECTED] \[\e[33m\]\w\[\e[0m\]\n\$ ' LOGONSERVER = `\\EXCHANGE' PROCESSOR_ARCHITECTURE = `x86' SHLVL = `1' COLORFGBG = `0;default;15' TROPIC_UNIQUE_ID = `156' USERDNSDOMAIN = `TROPICNETWORKS.COM' PATHEXT = `.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH' HOMEDRIVE = `C:' COMSPEC = `C:\WINDOWS\system32\cmd.exe' TMP = `/tmp' SYSTEMROOT = `C:\WINDOWS' PRINTER = `\\spooler\135MC-4th' CVS_RSH = `/bin/ssh' PROCESSOR_REVISION = `0303' INFOPATH = `/usr/local/info:/usr/share/info:/usr/info:' PROGRAMFILES = `C:\Program Files' DISPLAY = `:0' COSMIC = `t' NUMBER_OF_PROCESSORS = `2' SESSIONNAME = `Console' P4CONFIG = `.p4config' COMPUTERNAME = `DESK-RCAMPBELL2' COLORTERM = `rxvt-xpm' _ = `/usr/bin/cygcheck' POSIXLY_CORRECT = `1' HKEY_CURRENT_USER\Software\Cygnus Solutions HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2 HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2 (default) = `/cygdrive' cygdrive flags = 0x0022 HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/ (default) = `C:\cygwin' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/a (default) = `A:' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/bin (default) = `C:\cygwin\bin' flags = 0x004a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/bin/cygcheck (default) = `C:\cygwin\bin\cygcheck' flags = 0x001a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/bin/cygcheck.exe (default) = `C:\cygwin\bin\cygcheck.exe' flags = 0x001a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/bin/strace (default) = `C:\cygwin\bin\strace' flags = 0x001a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/bin/strace.exe (default) = `C:\cygwin\bin\strace.exe' flags = 0x001a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/c (default) = `C:' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/d (default) = `C:\d' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/bin (default) = `C:\cygwin\bin' flags = 0x00
Re: Unison 2.10.2 fast update check broken?
Rolf Campbell wrote: Marcus Picasso wrote: > Seems that Cygwin port of the unison file synchronizer does not do the > -fastcheck very well. Transcript follows: > > ... > > Can somebody confirm / explain this behaviour? I have a large tree that > I'm synchronizing across two hard-disks, and got suspicious when > re-running synchronization takes longer than expected. The above > transcript functions as expected using linux or native Win32 unison > builds. > > Regards, > -Marcus. I have noticed a change in how -fastcheck which seems to be caused by my upgrade from cygwin 1.5.14 -> 1.5.16. I tried doing a unison sync between a maching running 1.5.14 and a machine running 1.5.16 when I noticed the 1.5.16 machine spent a lot of time grinding the disk. So, I upgraded the 1.5.14 machine to 1.5.16 and it too went from a 10 second scan time to a half hour of heavy disk access. ... That's exactly my issue also. I think it's the recent ctime changes in Cygwin that has broken unison. The ctime stamp that gets recorded in the unison archive database is slightly off, compared to the actual ctime stamp of the file that got modified by unison. Could it be that unison reads the ctime stamp, then closes the file, which results in an update of the stamp, causing the mismatch of the stamps? Andrew, any ideas how to fix this? :) -Marcus. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Unison 2.10.2 fast update check broken?
Marcus Picasso wrote: Seems that Cygwin port of the unison file synchronizer does not do the -fastcheck very well. Transcript follows: ... Can somebody confirm / explain this behaviour? I have a large tree that I'm synchronizing across two hard-disks, and got suspicious when re-running synchronization takes longer than expected. The above transcript functions as expected using linux or native Win32 unison builds. Regards, -Marcus. I have noticed a change in how -fastcheck which seems to be caused by my upgrade from cygwin 1.5.14 -> 1.5.16. I tried doing a unison sync between a maching running 1.5.14 and a machine running 1.5.16 when I noticed the 1.5.16 machine spent a lot of time grinding the disk. So, I upgraded the 1.5.14 machine to 1.5.16 and it too went from a 10 second scan time to a half hour of heavy disk access. Cygwin Configuration Diagnostics Current System Time: Sun May 01 13:04:20 2005 Windows 2000 Professional Ver 5.0 Build 2195 Service Pack 4 Path: C:\cygwin\usr\local\bin C:\cygwin\bin C:\cygwin\bin C:\cygwin\usr\X11R6\bin C:\WINNT\system32 C:\WINNT C:\WINNT\System32\Wbem C:\Program Output from C:\cygwin\bin\id.exe (nontsec) UID: 11643(rcampbell)GID: 10513(Domain Users) 0(root) 544(Administrators) 545(Users) 10513(Domain Users) Output from C:\cygwin\bin\id.exe (ntsec) UID: 11643(rcampbell)GID: 10513(Domain Users) 0(root) 544(Administrators) 545(Users) 10513(Domain Users) SysDir: C:\WINNT\system32 WinDir: C:\WINNT HOME = `C:\cygwin\home\rcampbell' MAKE_MODE = `unix' PWD = `/tmp' USER = `rcampbell' ALLUSERSPROFILE = `C:\Documents and Settings\All Users' APPDATA = `C:\Documents and Settings\rcampbell\Application Data' COLORFGBG = `0;default;15' COLORTERM = `rxvt-xpm' COMMONPROGRAMFILES = `C:\Program Files\Common Files' COMPUTERNAME = `PCRCAMPBELL3' COMSPEC = `C:\WINNT\system32\cmd.exe' COSMIC = `t' CVS_RSH = `/bin/ssh' DISPLAY = `:0' HOMEDRIVE = `C:' HOMEPATH = `\Documents and Settings\rcampbell' HOSTNAME = `pcrcampbell3' INFOPATH = `/usr/local/info:/usr/info:/usr/share/info:/usr/autotool/devel/info:/usr/autotool/stable/info:' LOGONSERVER = `\\PCRCAMPBELL3' MANPATH = `/usr/local/man:/usr/man:/usr/share/man:/usr/autotool/devel/man::/usr/ssl/man' NUMBER_OF_PROCESSORS = `1' OLDPWD = `/home/rcampbell' OS2LIBPATH = `C:\WINNT\system32\os2\dll;' OS = `Windows_NT' P4CONFIG = `.p4config' PATHEXT = `.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH' PROCESSOR_ARCHITECTURE = `x86' PROCESSOR_IDENTIFIER = `x86 Family 6 Model 8 Stepping 6, GenuineIntel' PROCESSOR_LEVEL = `6' PROCESSOR_REVISION = `0806' PROGRAMFILES = `C:\Program Files' PS1 = `\[\033]0;\w\007 [EMAIL PROTECTED] \[\033[33m\w\033[0m\] $ ' SHLVL = `1' SYSTEMDRIVE = `C:' SYSTEMROOT = `C:\WINNT' TEMP = `C:\DOCUME~1\RCAMPB~1\LOCALS~1\Temp' TERM = `xterm' TMP = `C:\DOCUME~1\RCAMPB~1\LOCALS~1\Temp' TROPIC_UNIQUE_ID = `156' USERDNSDOMAIN = `tropicnetworks.com' USERDOMAIN = `TROPICNETWORKS' USERNAME = `rcampbell' USERPROFILE = `C:\Documents and Settings\rcampbell' WINDIR = `C:\WINNT' WINDOWID = `168188080' _ = `/usr/bin/cygcheck' POSIXLY_CORRECT = `1' HKEY_CURRENT_USER\Software\Cygnus Solutions HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2 HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2 (default) = `/cygdrive' cygdrive flags = 0x0022 HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/ (default) = `C:\cygwin' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/c (default) = `C:' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/d (default) = `d:' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/e (default) = `E:' flags = 0x001a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/f (default) = `f:' flags = 0x010a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/bin (default) = `C:\cygwin/bin' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/lib (default) = `C:\cygwin/lib' flags = 0x000a HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\Program Options c: hd NTFS 19069Mb 84% CP CS UN PA FC d: cd N/AN/A f: netN/AN/A C:\cygwin / system binmode C: /c system binmode d: /d system binmode E: /e system binmode,exec f: /f system binmode C:\cygwin/bin /usr/bin system binmode C:\cygwin/lib /usr/lib system binmode . /cygdrive system binmode,cygdrive Found: C:\cygwin\bin\awk.
Re: Unison 2.10.2 fast update check broken?
Hi Andrew, I just rerun the transcript below with both linux and native Win32 builds of unison, and the difference is, that those versions actually transfer the modification times even if the content of the file is unchanged. This results in the following synchronizations to not to have to read the contents of files, because the modification times match. So the problem actually is in the Cygwin version, and if you (or somebody) somehow could get it to transfer the modification times, that would be great. It probably hasn't got anything to do with your patches, more likely with Cygwin file modification times (maybe the ctime issues mentioned on this list?), but maybe a patch for this problem is still possible? Too bad I don't know oCaml. :) I'm using unison to backup a large tree to a spare hard-drive, and from there on to another PC, and noticed that lots of files get reread unnecessarily at each syncrhonization, which slows it down considerably. The Cygwin port is valuable, because I have greater control on the file-permissions of files that get created by unison, and it also can spawn external processes (which the native one can't), for instance to do a merge in an editor. Thanks, -Marcus. Hi Marcus. Thanks for the report. > Seems that Cygwin port of the unison file synchronizer does not do the > -fastcheck very well. Transcript follows: > > # Start of transcript > > # creates archives for first time > $ cd /tmp ; touch a b ; /bin/unison-2.10.2 ./a ./b > ... > > $ touch a > > $ /bin/unison-2.10.2 -fastcheck true -times -debug verbose ./a ./b > ... > > # now output shows that file contents is checked, ie. the "Double-check > # possibly updated file" line, which is correct, since we did a touch > > $ /bin/unison-2.10.2 -fastcheck true -times -debug verbose ./a ./b > ... > > # BUG: outputs again the "Double-check possibly updated file" line for > file > # 'b', ie. file content is checked even if no mods. > > # End of transcript I think this is the expected behavior, since a and b have the same contents. When Unison observes the different timestamps, it flags the files as possibly different and prints the "Double-check" message. Then it looks more carefully, and determines that their contents are in fact the same, so no update is propagated. That includes the timestamps, even though you specified -times. So then when you run the operation again, the same thing happens because nothing was changed on the previous run. The effect of -times is to make Unison synchronize the timestamps when an update is needed because the file contents are different. It doesn't make the timestamps sufficient to determine that files are different. At least, that's my reading of the manual. > The above transcript functions as > expected using linux or native Win32 unison builds. Hm, are you sure? If so, then that blows away my carefully constructed explanation above. I've looked at the set of patches that I applied to get Unison 2.10.2 running in Cygwin, and I don't see anything there that would obviously affect the -fastcheck option. But it's possible, and if you show me the evidence I'll look into it. HTH, Andrew. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Unison 2.10.2 fast update check broken?
Hi Marcus. Thanks for the report. > Seems that Cygwin port of the unison file synchronizer does not do the > -fastcheck very well. Transcript follows: > > # Start of transcript > > # creates archives for first time > $ cd /tmp ; touch a b ; /bin/unison-2.10.2 ./a ./b > ... > > $ touch a > > $ /bin/unison-2.10.2 -fastcheck true -times -debug verbose ./a ./b > ... > > # now output shows that file contents is checked, ie. the "Double-check > # possibly updated file" line, which is correct, since we did a touch > > $ /bin/unison-2.10.2 -fastcheck true -times -debug verbose ./a ./b > ... > > # BUG: outputs again the "Double-check possibly updated file" line for file > # 'b', ie. file content is checked even if no mods. > > # End of transcript I think this is the expected behavior, since a and b have the same contents. When Unison observes the different timestamps, it flags the files as possibly different and prints the "Double-check" message. Then it looks more carefully, and determines that their contents are in fact the same, so no update is propagated. That includes the timestamps, even though you specified -times. So then when you run the operation again, the same thing happens because nothing was changed on the previous run. The effect of -times is to make Unison synchronize the timestamps when an update is needed because the file contents are different. It doesn't make the timestamps sufficient to determine that files are different. At least, that's my reading of the manual. > The above transcript functions as > expected using linux or native Win32 unison builds. Hm, are you sure? If so, then that blows away my carefully constructed explanation above. I've looked at the set of patches that I applied to get Unison 2.10.2 running in Cygwin, and I don't see anything there that would obviously affect the -fastcheck option. But it's possible, and if you show me the evidence I'll look into it. HTH, Andrew. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Unison 2.10.2 fast update check broken?
Seems that Cygwin port of the unison file synchronizer does not do the -fastcheck very well. Transcript follows: # Start of transcript # creates archives for first time $ cd /tmp ; touch a b ; /bin/unison-2.10.2 ./a ./b ... $ touch a $ /bin/unison-2.10.2 -fastcheck true -times -debug verbose ./a ./b ... # now output shows that file contents is checked, ie. the "Double-check # possibly updated file" line, which is correct, since we did a touch $ /bin/unison-2.10.2 -fastcheck true -times -debug verbose ./a ./b ... # BUG: outputs again the "Double-check possibly updated file" line for file # 'b', ie. file content is checked even if no mods. # End of transcript Can somebody confirm / explain this behaviour? I have a large tree that I'm synchronizing across two hard-disks, and got suspicious when re-running synchronization takes longer than expected. The above transcript functions as expected using linux or native Win32 unison builds. Regards, -Marcus. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
