On 27/06/15 10:45, Dmitri Maziuk wrote:
> Compressing data on the client means fewer bytes to send over the
> wire. Block-level compression like bzip2 tends to be completely
> cpu-bound and anything bigger than a cellphone tends to have plenty of
> cycles to spare.
Not entirely true and certainly not accurate for Gb/s and faster links.
Individual CPU cores haven't changed much in terms of speed for nearly a
decade and most compression algorithms are single threaded when fed from
stdin (even pigz and pbzip). On top of that you're facing memory <-> CPU
bottlenecks.
The result when running gzip compression over a 1Gb/s link is that
you'll usually achieve a maximum end-to-end rate of 30MB/s, vs the wire
speed of 110MB/s ( b = bit, B=BYTE, aka o = octet if you're french )
bzip is significantly worse. Even LZO gives little gain at 1Gb/s in
exchange for making your CPU run hot and drawing more power than you
need to.
At 10Gb/s, LZO is a significant bottleneck.
The takeaway: Unless you're backing up over a WAN (and a slow or
overloaded one at that), wireside compression is not a net win.
We have 300 people on our campus linking to the main site at 1Gb/s and
whilst backing up machines on the main site to our satellite one we
still achieve faster backups without compression than with it.
The other side of compression is that it's a huge CPU drain. I already
have enough problems with people rebooting windows systems mid-backup
without adding even more incentive for them to hit the Big Red Button.
------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
