It turns out that v0.9.0 of nftables userspace does not have tproxy support at all and the errors are just misleading.
The support was committed in 2be1d52644cf 2018-08-03 12:17:31 +0200 src: Add tproxy support but the release is: cd21a243162a (tag: v0.9.0) 2018-06-08 14:46:00 +0200
