I think I found the relevant fixes: First and foremost it's 741a11d9e410. (net: ipv6: Add RT6_LOOKUP_F_IFACE flag if oif is set)
This seems to have already solved my problem, however there were two followup fixes that I should probably also apply: d46a9d678e4c net: ipv6: Dont add RT6_LOOKUP_F_IFACE flag if saddr set 6f21c96a78b8 ipv6: enforce flowi6_oif usage in ip6_dst_lookup_tail() So again, sorry for the noise and thanks for your help! Andreas