2012/11/8 andrea crotti <andrea.crott...@gmail.com>: > > > > Yes yes I saw the answer, but now I was thinking that what I need is > simply this: > tar czpvf - /path/to/archive | split -d -b 100M - tardisk > > since it should run only on Linux it's probably way easier, my script > will then only need to create the list of files to tar.. > > The only doubt is if this is more or less reliably then doing it in > Python, when can this fail with some bad broken pipe? > (the filesystem is not very good as I said and it's mounted with NFS)
In the meanwhile I tried a couple of things, and using the pipe on Linux actually works very nicely, it's even faster than simple tar for some reasons.. [andrea@andreacrotti isos]$ time tar czpvf - file1.avi file2.avi | split -d -b 1000M - inchunks file1.avi file2.avi real 1m39.242s user 1m14.415s sys 0m7.140s [andrea@andreacrotti isos]$ time tar czpvf total.tar.gz file1.avi file2.avi file1.avi file2.avi real 1m41.190s user 1m13.849s sys 0m5.723s [andrea@andreacrotti isos]$ time split -d -b 1000M total.tar.gz inchunks real 0m55.282s user 0m0.020s sys 0m3.553s -- http://mail.python.org/mailman/listinfo/python-list
