> Hi, > > 14.04.2009 09:23, Kern Sibbald wrote: > > Hello James, > > > > It is hard to respond because I don't know what a PV driver is or how it is > > related to network drivers. > > Neither do I :-)
ParaVirtualised drivers. Much faster than an emulation of a PCI device in a virtual machine. > > > On Tuesday 14 April 2009 08:41:37 James Harper wrote: > >> I'm writing Windows PV drivers for Xen, and bacula is the only app that > >> I have found that can reliably trigger a certain bug in the network > >> driver. While debugging that, I have noticed that bacula may not be > >> sending data in the most efficient way possible... > > > > That depends on how you define "the most efficient way possible". > > > > Bacula makes a large number of small network I/O calls, but it expects the > OS > > to deal with getting them efficiently across the network. > > That said, there were some important optimizations between 2.2.8 and 3.0.0, > > which reduce the number of network writes by a factor of 2. > > This seems to be present in 2.5.4x as I can't see what James describes > in my test setup. If you are using wireshark then you won't. I'm talking about one level below what you would see with a packet sniffer. When a windows app (probably similar to Linux) sends data to a socket, windows will not necessarily send the data straight away but will instead coalesce the data into (up to) a ~61K to send to the hardware (or 1460 bytes if the hardware doesn't support large send offloading). Because Bacula is doing lots of small writes, the network driver is handed a chained list of buffers for the size of each write, so for example in one case we have separate writes of 12, 195, 4, 11, 20, 4, 11, 234, 4, 12, 194, 4, 11, 20, 4, 11, 234, 4, 12, 238, 4, 11, 20, 4, 11, and 171 bytes, so we get a list of 26 fairly tiny buffers. My driver has a limit of 19 buffers that it can handle, and so in the above case Windows has to copy a whole load of data around to make up a 1514 byte packet. Also, there are only 256 'slots' in the communications ring with the virtual hardware, and each buffer uses up one of those 'slots' leaving less room for other packets. A physical network adapter probably has less restrictions on scatter-gather lists but there is still going to be overhead with making a packet out of lots of small buffers vs one big one. > > We would be happy to consider something more, but I would recommend two > > things: > > > > 1. Do your testing with version 3.0.0. > > Seems well worth it, considering the difference of my observations to > what James reports. > > > 2. Test real file transfers and get some performance data that shows that it > > is really a bottle neck. At least for a local network, I am sure you will > > find that the comm part of Bacula is not the restraining factor. > > Agreed :-) > I'll try both of those. Thanks James ------------------------------------------------------------------------------ This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel
