I see very odd results from rsync 2.4.7pre1, the latest cvs version (sept 12, i think was the last modified file). We have a number of network-attached storage devices. 10/100 ethernet, nfs2 mounted (under nfs3, they buffer deletes, and recursive deletions fail). Usually, these are kept syncronized across a wan by a nightly cronjob, We have a few we keep in reserve, which we syncronize locally. They're all the same, aside from whatever is left different by rsync failures. they are syncronized, one at a time, during the week. They fail on different files. as you see, bildmax4 failed on only one file. bildmax2 failed on several screensful, but i trimmed it down. i don't see any duplication between systems. At the bottom, i show a ls of one of the files that failed on bildmax2, both on the master (/big1/....) and on the bildmax itself (/bildmax/bildmax2/...). What value is too big? Oh, the cmdline: /cadappl/encap/bin/rsync -Wa --delete --force --bwlimit=524288 source destination It's about 128Gb of data in about 2.8M files. Any idea what this randomness is? might the "Value too large for defined data type" be thrown if the system runs out of memory? These jobs get up to over a half a gig memory used. It was compiled (and is running) on a 64-bit machine. My life would be greatly simplified if i could run these syncs as single chunks, as there are things being removed, as well as added, so automatically breaking up the runs may leave things out. Anybody got any ideas? I think i've heard of others running much larger distributions. Incidentally, from this test, it's plain that the old false timeout problem, where one process waiting for another to finish its work would timeout and stop the run, even though the other threads were still working, at the 60-second select_timeout value that is set when no io_timeout is set, is fixed in this version. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ starting /bildmax/bildmax2 at Wed Oct 31 02:42:46 PST 2001 readlink big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/.22-4.htm.IFaG5g: Value too large for defined data type readlink big1/cadappl1/hpux/ictools/arm_ads/1.1/common/include/.sstream.MLaG5g: Value too large for defined data type rsync error: partial transfer (code 23) at main.c(537) finished /bildmax/bildmax2 at Wed Oct 31 12:46:00 PST 2001 starting /bildmax/bildmax3 at Wed Oct 31 12:46:00 PST 2001 readlink big1/cadappl1/hpux/ictools/tmpTempo: No such file or directory symlink big/tools/DI/factory_integrator2.2/data -> /cadappl/ictools/factory_integrator/2.2/data : File exists symlink big/tools/synopsys/synopsys1999.05/doc -> /cadappl/ictools/synopsys/1999.05/doc : File exists symlink big1/tools1/DI/dis2.2.2/DI/documentation -> /cadappl/ictools/Design_Integrator/2.2.2/documentation : File exists symlink big1/tools1/DI/dis2.2.2/DI/system -> /cadappl/ictools/Design_Integrator/2.2.2/system : File exists rsync error: partial transfer (code 23) at main.c(537) finished /bildmax/bildmax3 at Wed Oct 31 18:07:04 PST 2001 starting /bildmax/bildmax4 at Wed Oct 31 18:07:04 PST 2001 readlink big/tools/DI/dis2.2.1/DI/system/product/vsc983/spice_models/.vsc9a_wire.inc.enc.1.1.0.fobG1u: Value too large for defined data type rsync error: partial transfer (code 23) at main.c(537) finished /bildmax/bildmax4 at Thu Nov 1 05:09:48 PST 2001 starting /bildmax/bildmax5 at Thu Nov 1 05:09:48 PST 2001 readlink big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18corelib_danger_p/2.0/lib/corelib_danger_p/dly6x3pd/auLvs/.master.tag.dNsOZO: Value too large for defined data type readlink big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18corelib_p/2.0.1/lib/corelib_p/ors2pd/datasheet/.master.tag.U8zOZO: Value too large for defined data type rsync error: partial transfer (code 23) at main.c(537) finished /bildmax/bildmax5 at Fri Nov 2 00:41:47 PST 2001 starting /bildmax/bildmax6 at Fri Nov 2 00:41:47 PST 2001 readlink big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18corelib_p/2.0.1/lib/corelib_p/ao6anx4pd/abstract/.layout.cdb.hSuGZO: Value too large for defined data type Tools@willy /site/local/share/ToolSync/localrep>ls -l /big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/22-4.htm -rw-r--r-- 1 Tools Tools 12025 Apr 27 1999 /big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/22-4.htm Tools@willy /site/local/share/ToolSync/localrep>ls -l /bildmax/bildmax2/big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/22-4.htm -rw-r--r-- 1 Tools Tools 12025 Apr 27 1999 /bildmax/bildmax2/big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/22-4.htm Tools@willy /site/local/share/ToolSync/localrep> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ main.c line 537 is just exit_cleanup(status) +++++++++++++++++++++++++++++++++++++++Memory usage+++++++++++++++++++++++++++++++++++++++++++++++++++++ load averages: 0.25, 0.33, 0.35 08:46:15 102 processes: 100 sleeping, 1 zombie, 1 on cpu CPU states: 94.1% idle, 0.5% user, 5.3% kernel, 0.0% iowait, 0.0% swap Memory: 3072M real, 1709M free, 644M swap in use, 5501M swap free
PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 10838 Tools 1 33 0 535M 176M sleep 55:19 0.00% rsync 19393 Tools 1 33 0 535M 1728K sleep 5:04 0.59% rsync 10837 Tools 1 33 0 285M 78M sleep 26:46 0.28% rsync ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(nnnnnnnnnnnn, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), ".\n" ' "There are some who call me.... Tim?"