bug#73784: [PATCH] cp: new option --nocache-source

[Pádraig Brady]

On 13/10/2024 05:56, Masatake YAMATO wrote:

When copying files, the system data cache are consumed, the system
data cache is utilized for both the source and destination files. In
scenarios such as creating backup files for old, unused files, it is
clear to users that these files will not be needed in the near
future. In such cases, retaining the data for these files in the cache
constitutes a waste of computer resources, especially when running
applications that require significant memory in the foreground.

With the new option, users will have the ability to request the
discarding of the system data cache, thereby avoiding the unwanted
swapping out of data from foreground processes.

I evaluated cache consumption using a script called
run.bash. Initially, run.bash creates many small files, each 8 KB in
size. It then copies these files using the cp command, both with and
without the specified option. Finally, it reports the difference in
the total size of the caches before and after the copying process.

run.bash:

     #!/bin/bash
     CP=$1
     shift

     [[ -e "$CP" ]] || {
        echo "no file found: $CP" 1>&2
        exit 1
     }

     N=8
     S=drop-src
     D=${HOME}/drop-dst

     mkdir -p $S
     mkdir -p $D

     start=
     end=
     print_cached()
     {
        grep ^Cached: /proc/meminfo
     }

     start()
     {
        start=$(print_cached | awk '{print $2}')
     }

     end()
     {
        end=$(print_cached | awk '{print $2}')
     }

     report()
     {
        echo -n "delta[$N:$1/$2]: "
        expr "$end" - "$start"
     }

     cleanup()
     {
        local i
        local j
        for ((i = 0; i < 10; i++)); do
            for ((j = 0; j < 10; j++)); do
                rm -f $S/F-${i}${j}*
                rm -f $D/F-${i}${j}*
            done
        done
        rm -f $S/F-*
        rm -f $D/F-*
     }

     prep()
     {
        local i
        for ((i = 0; i < 1024 * $N; i++ )); do
            if ! dd if=/dev/zero of=$S/F-$i bs=4096 count=2 \
                    status=none; then
                echo "failed in dd of=$S/F-$F" 1>&2
                exit 1
            fi
        done
        sync
     }

     run_cp()
     {
        start

        local i
        time for ((i = 0; i < 1024 * $N; i++ )); do
            if ! "${CP}" "$@" "$S/F-$i" "$D/F-$i"; then
                echo "failed in cp " "$@" "$S/F-$i" " $D/F-$i" 1>&2
                exit 1
            fi
        done

        end
        report "$1" $2
     }

     cleanup
     sync

     prep
     run_cp "$@"

running:

     ~/coreutils/nocache$  ./run.bash ../src/cp

     real       0m16.051s
     user       0m4.249s
     sys        0m12.437s
     delta[8:/]: 65548
     ~/coreutils/nocache$  ./run.bash ../src/cp --nocache-source

     real       0m17.109s
     user       0m4.492s
     sys        0m13.317s
     delta[8:--nocache-source/]: 620

--nocache-source option suppresses the consumption of the cache
massively.

Thanks for the patch.
I have some reservations/notes though...

There is nothing particularly special about cp, that it might need this option.
I.e. it would be nice to be able to wrap any program so that it streamed
data through the cache, rather than aggressively cached.  I'm not sure how to 
do that,
but also I'd be reluctant to start adding such options to individual commands 
though.
Perhaps Linux' open() may gain an O_STREAM flag in future that might be
more generally applied with a wrapper or something.

For single (large) files, one already has this functionality in dd.

On the write side, you'd also have to worry about syncing, to make the
drop cache advisory effective, and this could impact performance.

Might this drop caches for already cached files,
which cp may just happen to be copying,
thus potentially impacting performance for other programs.

If reflinking we probably would not want to do this operation,
since we're not reading the source.

thanks,
Pádraig