try bsdiff maybe? https://www.daemonology.net/bsdiff/
it will probably be very slow if you're not going through the filesystem cache On Fri, Apr 14, 2023, 4:59 PM Ralf Mardorf <[email protected]> wrote: > Hi, > > my google search was "does linux diff compare data using a cache". > > I'm trying to figure out what's going on. The first diff of 10 GiB of > data copied from a SATA3 SSD to an USB 2 stick connected to an USB 3 > port took around a minute, right after the copy finished. A second diff > needed 3 seconds. Both returned exit status 0. > > It's impossible to read 10 GiB of data in 3 seconds from an USB 2 stick. > Does diff use cached data instead of comparing the "real" files line by > line? > > Google returned "diff isn't doing any caching. The OS is. If you are > using Linux, you can flush the disk buffers and cache". > > I expected that diff ensures to compare the "real" files line by line, > but seemingly diff isn't aimed to check integrity of data. > > Does a command exist that compares "real" files, not just cached files > by default? > > I experience weird things with Raptor Lake hardware, especially if USB > is involved and I want to check the integrity of USB transferred, saved > files by using a tool, without manually clearing cached data manually. > > Regards, > Ralf >
