On Tuesday, 30 March 2021 18:11:56 BST Dr Rainer Woitok wrote:
> On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote:
>
> ("> >" refers to Michael <confabul...@kintzios.com>)
>
> > Michael,
> >
> > On Friday, 2020-11-27 19:07:17 +0000, you wrote:
> > > ...
> > > A 4k block size is recommended for ntfs-3g which is the default sector
> > > created by fdisk and friends on Linux these days. This will align your
> > > partition optimally. In addition, mkfs.ntfs will use 4096 bytes as the
> > > default cluster size, so you should be good in that respect.
> > >
> > > Another setting you may want to try is mounting the USB with
> > > 'big_writes' -
> > > check the man page. This should help particularly with large files,
> > > which
> > > will use larger blocks up to 128KB when copying data to the NTFS.
> >
> > Both, the VeraCrypt command line (--fs-options=big_writes) and the Vera-
> > Crypt GUI (under "Settings --> Preferences") allow setting this mount
> > option. But
> >
> > $ mount | grep veracrypt
> >
> > never shows it, initially causing me to erroneously believe it wasn't
> > set and to try finding on the web another way of setting it. By pure
> > chance I finally found out that
> >
> > $ ps -ef | grep veracrypt
> >
> > lists a "/usr/sbin/mount.ntfs" task which shows the options really in
> > effect. However, I haven't yet had the time to test the effect of this
> > option when writing plenty of really big files. I will report on that
> > later.
>
> Well, it's been quite a while, due to my being almost permanently con-
> fronted with more pressing tasks ... :-(
>
> To sum up my experience with my new 128 GB Philips USB 3.0 sticks: while
> the Philips sticks are significantly faster for reading operations than
> my old 64 GB Verbatim ones (probably USB 2.0), writing operations to the
> Philips sticks are unbearably slow, regardless of whether I created a
> normal unencrypted NTFS filesystem on them or an encrypted NTFS filesys-
> tem using VeraCrypt. Writing to the USB stick while at the same time
> reading from it in a different terminal window caused commands like "cd"
> or "ls" to simply stall. Thus while running
>
> $ cp --preserve=timestamps -ru $source_dir .
>
> in one terminal window, I ran
>
> $ while true
>
> > do n=$(ps -ef|g 'cp --preserve'|g -v grep)
> >
> > if [[ "$n" = "${o-}" ]]
> > then sleep 10
> > else o="$n"
> >
> > echo "$n"
> >
> > fi
> >
> > done
>
> in another, to get the wall clock times when copying a new file began.
> That way I found that copying a 30 MB file took about 40 minutes.OK, unless you made a typo and the "minutes" were meant to say seconds, this is ridiculously slow. You could run some tests to see what is causing the delay. The veracrypt algos & cipher iterations, the fuse based ntfs-3g, or the USB stick's controller. However if, as I understand it, all other variables are the same and the only change was to replace your Verbatim 64G USB 2.0 sticks with Philips 128G USB 3.0 sticks, then the slow writes point to the Philips devices being the culprit. Some years ago I tested some USB 2.0 sticks of various sizes, from 256M up to 32G and recall the smaller the USB stick the faster the write performance, so differences in writing speed are normal. The writing speed you're describing however is a clear indication of something being wrong. > So what are my options? > > - Stay away from Philips USB 3.0 sticks? > > - Stay away from Philips USB sticks in general? Without knowing the internals, a brand may offer only an unwarranted assumption of performance. We saw Western Digital disks being sold as CMR, while having SMR internals. A brand could switch OEM suppliers, or components, making benchmarking unreliable. > - Stay away from USB 3.0 sticks in general? USB 3.0 is faster and USB 3.2 when available will be even faster. So use whatever the USB ports on your PC offer. > - Stay away from Filesystem in User Space using a non-stable 5.10 or > 5.11 kernel (currently I'm using stable 5.4.97)? > > - Stay away from Gentoo? > > - Stay away from Linux in general and go back to OTOS (aka the Only > True Operating System aka Windoze)? > > - ...? In-kernel fs drivers are measurably faster than fuse based fs for well understood reasons. However, if needs must and the fs you require is not available on Linux, then some compromise will be required. > Any ideas and comments welcome ... > > Sincerely, > Rainer You may want to run some tests on the sticks you have, if only to bottom out what their performance is on different PCs and USB ports: dd if=/dev/zero of=/run/media/<USER>/<STICK_LABEL>/TESTFILE bs=512 count=600000 oflag=direct conv=notrunc,fsync status=progress Use a large enough file to make sure the USB controller cache gets saturated. You could use a ramdisk/tmpfs as an input file. If you write directly to the device as Dale suggested it will wipe data, so keep a backup of anything you need first. You can experiment with different filesystems and in the first opportunity with a different make of USB 3.0 stick. You'd soon be able to determine how good the real world performance can get and if the Philips or something else is causing the problem you've experienced. A note about UDF: it works and it is versatile - but ensuring interoperability between different OSs can be tricky. Check some suggestions here: https://askubuntu.com/questions/27936/can-and-should-udf-be-used-as-a-hard-drive-format and here: https://superuser.com/questions/39942/using-udf-on-a-usb-flash-drive Ultimately such tests are an attempt to eliminate methodically all other factors, until you isolate the cause of the problem you are experiencing.
signature.asc
Description: This is a digitally signed message part.
