Orcallator, procallator and friends aren't shiny new toys Adrian Cockroft wrote initial version of orcallator in the early 90s for his book "Solaris Performance Tuning. The 2nd edition is I think 1998. The current version of ORCA (processes the collected data) is from I believe 2007 or so www.orcaware.org i think it was Sent from my BlackBerry® smartphone with Nextel Direct Connect
-----Original Message----- From: Arthur Corliss <acorl...@nevaeh-linux.org> Date: Mon, 29 Mar 2010 00:31:50 To: Dana Hudes<dhu...@hudes.org> Cc: <module-authors@perl.org> Subject: Re: Trimming the CPAN - "Automatic Purging" On Sun, 28 Mar 2010, Dana Hudes wrote: > Why is rsync a problem? Where is the bottleneck in the protocol or the code > implementing it? > Specifics! > SAR is antiquated doesn't give the info you really need. Using a linux > system? Use procallator and feed resulting collected data to ORCA. Better > yet, use DTrace or at least truss. Compile rsync with profiling code -- use > Sun Studio 12 it runs on Linux as well as Solaris and its a free download. Wow. You kids and your new shiny toys... Look, here's a nice little specific example for you. I run an rsync server that contains 8,700+ files and directories. Now, say I want to sync a mere thirty-two new files. Making that request on my server causes the rsync daemon to stat the entire hierarchy to the tune of 18,000+ f & lstats. Per request. Freaking ouch. And that's a tolerable use-case in my mind for rsync. That's a hell of alot I/O generated which would take but a couple of stats to retrieve via HTTP or FTP. Assuming you knew what you needed already. Now, when you add in a file set of sufficient size to exhaust filesystem caching, plus a crap load of concurrent requests, my archaic SAR reports written on stone tables tend to say your I/O wait states starts pushing the load levels unacceptably high, not to mention the pages being thrashed from memory's cache pool, high interrupts and excessive seeks on the drives, and so on and so forth. <sniff> Cavemen are people, too. Now, look at the size of CPAN with *hundreds* of thousands of files. Can you imagine that amount of I/O *per* request?! > From a network protocol perspective rsync is quite good. If your network > capacity is so large that it exceeds bandwidth or IOPs of your disks you > probably can afford better disks or a more efficient disk storage layout. > Are mirrors like nic.funet.fi running multiple gigabit WAN connections? If > so they could sure demand stream more than a bunch of SATA2 disks can provide. > > Without performance data its a waste of time to argue against rsync And without having had examined how rsync works on both ends it should have been a waste of time to argue the merits of rsync. --Arthur Corliss Live Free or Die