For the record, I reinstalled the machine in question with Ubuntu 18.04 a few days ago, and upgraded it since to 20.04, and the problem was not present there.
This make me suspect this issue is fixed in kernel 5.4.0 and possible earlier versions. I am unable to test with a Debian kernel. :( -- Happy hacking Petter Reinholdtsen
