While I agree that security takes longer, I'd suggest that a lot of the time is spent on the highly syncronous nature of the network sync operation. Looking at strace output, the procedure appears to be something like:
1: stat a file 2. checksum a file 3. send request to master server to SYNC file 4. server checksums file 5. server returns results 6. client compares. I think performance would be increased significantly if #3 was somehow batched up into blocks larger then 1. Perhaps at every directory level, all the files in the directory (non-recursively) could be batched up and send in a single request, without sacrificing security. Thank you, -Jason Martin > -----Original Message----- > From: Mark Burgess [mailto:[EMAIL PROTECTED] > Sent: Friday, September 23, 2005 7:05 AM > To: Baker, Darryl > Cc: Martin, Jason H; [email protected] > Subject: RE: Optimizing large copies > > > > I concur with this. Cfengine insists on the correctness and > security of each operation -- and that takes time. A program > like rsync can achieve significantly better performance in > large copies if you have a trusted base. > > M > > On Fri, 2005-09-23 at 08:59 -0400, Baker, Darryl wrote: > > I understand your concerns but with the number and size of > the files > > you are talking about you really should investigate a > program that is > > optimized for files copying. While cfengine can maintain > binary files > > this is not its primary focus. Its main focus is maintaining the > > system configuration. Rsync was designed to keep files identical on > > several machines. Using rsync in conjunction with ssh for > security you > > end up with a much better solution for maintaining binary > files. It is > > optimized just for this purpose. The use of rsync with ssh is well > > documented and in use in a large number of large sites. > > > > > > _______________________________________________ > > Help-cfengine mailing list > > [email protected] > > http://lists.gnu.org/mailman/listinfo/help-cfengine > > _______________________________________________ Help-cfengine mailing list [email protected] http://lists.gnu.org/mailman/listinfo/help-cfengine
