RFC: new cp option: --efficient-sparse=HOW

2011-01-31 Thread Jim Meyering
Now that we have can read sparse files efficiently, what if I want to copy a 20PiB sparse file, and yet I want to be sure that it does so efficiently. Few people can afford to wait around while a normal processor and storage system process that much raw data. But if it's a sparse file and the src

Re: RFC: new cp option: --efficient-sparse=HOW

2011-01-31 Thread Jim Meyering
Jim Meyering wrote: > Now that we have can read sparse files efficiently, > what if I want to copy a 20PiB sparse file, and yet I want to > be sure that it does so efficiently. Few people can afford > to wait around while a normal processor and storage system process > that much raw data. But if

Re: RFC: new cp option: --efficient-sparse=HOW

2011-01-31 Thread Eric Blake
On 01/31/2011 02:46 PM, Jim Meyering wrote: > Now that we have can read sparse files efficiently, > what if I want to copy a 20PiB sparse file, and yet I want to > be sure that it does so efficiently. Few people can afford > to wait around while a normal processor and storage system process > that

Re: RFC: new cp option: --efficient-sparse=HOW

2011-01-31 Thread Jim Meyering
Eric Blake wrote: > On 01/31/2011 02:46 PM, Jim Meyering wrote: >> Now that we have can read sparse files efficiently, >> what if I want to copy a 20PiB sparse file, and yet I want to >> be sure that it does so efficiently. Few people can afford >> to wait around while a normal processor and stor

Re: RFC: new cp option: --efficient-sparse=HOW

2011-01-31 Thread Eric Blake
On 01/31/2011 03:17 PM, Eric Blake wrote: >> The default is --efficient-sparse=auto, and for symmetry, >> I've provided --efficient-sparse=never, in case someone finds >> a reason to want to skip the ioctl. > > Conversely, what happens if I have a file that contains large blocks of > zeros but is

Re: RFC: new cp option: --efficient-sparse=HOW

2011-01-31 Thread Eric Blake
On 01/31/2011 03:27 PM, Jim Meyering wrote: > It's orthogonal. > > --sparse=always still does the hole-punching, independently > of whether we're copying normally or via the efficient FIEMAP-based > code. > > E.g., if you have a sparse file, where one non-sparse chunk > contains all-zero blocks (

Re: RFC: new cp option: --efficient-sparse=HOW

2011-01-31 Thread Pádraig Brady
On 31/01/11 21:46, Jim Meyering wrote: > Now that we have can read sparse files efficiently, > what if I want to copy a 20PiB sparse file, and yet I want to > be sure that it does so efficiently. Few people can afford > to wait around while a normal processor and storage system process > that much

Re: RFC: new cp option: --efficient-sparse=HOW

2011-02-03 Thread Jim Meyering
Pádraig Brady wrote: > On 31/01/11 21:46, Jim Meyering wrote: >> Now that we have can read sparse files efficiently, >> what if I want to copy a 20PiB sparse file, and yet I want to >> be sure that it does so efficiently. Few people can afford >> to wait around while a normal processor and storage

Re: RFC: new cp option: --efficient-sparse=HOW

2011-02-04 Thread Pádraig Brady
On 03/02/11 20:29, Jim Meyering wrote: > Does anyone know how to determine if a file system (say the one with ".") > supports the FIEMAP ioctl, but without compiling/running a C program? > I.e., via perl or python? I've written a tiny C program that works > and a Perl one that is supposed to be id