On Wednesday 21 July 2010 11:59, Berry van Sleeuwen wrote:
>Sparse files. OK. Then the next question, how can I store a 26G file in
>a machine that isn't that large? And to add to this, why does the
>filesystem backup really dump 26G into our TSM server?
Because it isn't really using 26GB of disk space. The *length* of the file is
26GB because the program writing it seeked out that far and wrote something.
But it didn't write all the data between zero and 26GB, so Linux didn't
allocate disk space for the parts of the file that were never written to.
Run "du -h /var/log/lastlog" to see just how little disk space that file
uses. Here's what it says on my system, for example:
# ll -h /var/log/lastlog
-rw-r--r-- 1 root tty 1.2M Jul 20 08:30 /var/log/lastlog
# du -h /var/log/lastlog
48K /var/log/lastlog
So even though the file is 1.2MB long, it's only using up 48KB (or 12 blocks)
of disk space. The file is "sparse" because it does not have blocks
allocated for its entire length.
The backup dumps a 26GB file because when a program reads a part of a sparse
file that was never written, it gets back a block of all zeros. So TSM is
reading all that unallocated space, and writing out lots of blocks of zeros
to the backup file. Thus the backup file is not a sparse file, because TSM
wrote every block of that 26GB. Perhaps there's some TSM option to get it to
recognise sparse files?
Rick pointed out that rsync and tar have options that deal with sparse files
intelligently: when they copy a sparse file, they do not write out blocks of
all zeros. Instead, they seek past such "empty" blocks to avoid writing to
them, thus creating a sparse output file. That's how a proper Linux file
copy is done. The cp command also does that.
- MacK.
-----
Edmund R. MacKenty
Software Architect
Rocket Software
275 Grove Street · Newton, MA 02466-2272 · USA
Tel: +1.617.614.4321
Email: m...@rs.com
Web: www.rocketsoftware.com
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
