Hello Kenny, thank you for submitting this and for helping to improve
Ubuntu.
Would it be possible for you to run the following command on an affected
system? This will collect logs that will be helpful for debugging the
issue.
apport-collect 2101914
Thank you!
** Changed in: linux-aws (Ubuntu)
Status: New => Triaged
** Changed in: linux-aws (Ubuntu)
Assignee: (unassigned) => Philip Cox (philcox)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-aws in Ubuntu.
https://bugs.launchpad.net/bugs/2101914
Title:
nt_tables reporting unknown option "--xor-mark" in 6.8.0-1025.27
Status in linux-aws package in Ubuntu:
Triaged
Bug description:
After upgrading from 6.8.0-1023.25~22.04.1 to 6.8.0-1025.27~22.04.1 in some
Kubernetes worker nodes, our kube-proxy started failing with these logs:
```
2025-03-06T00:55:59.784251404Z stderr F E0306 00:55:59.784126 1
proxier.go:1432] "Failed to execute iptables-restore" err=<
2025-03-06T00:55:59.784266755Z stderr F exit status 2: Warning: Extension
MARK revision 0 not supported, missing kernel module?
2025-03-06T00:55:59.784269955Z stderr F ip6tables-restore v1.8.9
(nf_tables): unknown option "--xor-mark"
2025-03-06T00:55:59.784272495Z stderr F Error occurred at line: 11
2025-03-06T00:55:59.784274584Z stderr F Try `ip6tables-restore -h' or
'ip6tables-restore --help' for more information.
2025-03-06T00:55:59.784277344Z stderr F > rules="*nat\n:KUBE-SERVICES -
[0:0]\n:KUBE-POSTROUTING - [0:0]\n:KUBE-NODE-PORT - [0:0]\n:KUBE-LOAD-BALANCER
- [0:0]\n:KUBE-MARK-MASQ - [0:0]\n-A KUBE-SERVICES -s ::1/128 -j RETURN\n-A
KUBE-SERVICES -m addrtype --dst-type LOCAL -j KUBE-NODE-PORT\n-A
KUBE-LOAD-BALANCER -j KUBE-MARK-MASQ\n-A KUBE-POSTROUTING -m mark ! --mark
0x00004000/0x00004000 -j RETURN\n-A KUBE-POSTROUTING -j MARK --xor-mark
0x00004000\n-A KUBE-POSTROUTING -m comment --comment \"kubernetes service
traffic requiring SNAT\" -j MASQUERADE --random-fully\n-A KUBE-MARK-MASQ -j
MARK --or-mark 0x00004000\nCOMMIT\n*filter\n:KUBE-FORWARD -
[0:0]\n:KUBE-NODE-PORT - [0:0]\n:KUBE-PROXY-FIREWALL -
[0:0]\n:KUBE-SOURCE-RANGES-FIREWALL - [0:0]\n:KUBE-IPVS-FILTER -
[0:0]\n:KUBE-IPVS-OUT-FILTER - [0:0]\n-A KUBE-SOURCE-RANGES-FIREWALL -j
DROP\n-A KUBE-FORWARD -m comment --comment \"kubernetes forwarding rules\" -m
mark --mark 0x00004000/0x00004000 -j ACCEPT\n-A KUBE-FORWARD -m comment --co
mment \"kubernetes forwarding conntrack rule\" -m conntrack --ctstate
RELATED,ESTABLISHED -j ACCEPT\n-A KUBE-NODE-PORT -m comment --comment
\"Kubernetes health check node port\" -m set --match-set
KUBE-6-HEALTH-CHECK-NODE-PORT dst -j ACCEPT\n-A KUBE-IPVS-FILTER -m set
--match-set KUBE-6-LOAD-BALANCER dst,dst -j RETURN\n-A KUBE-IPVS-FILTER -m set
--match-set KUBE-6-CLUSTER-IP dst,dst -j RETURN\n-A KUBE-IPVS-FILTER -m set
--match-set KUBE-6-EXTERNAL-IP dst,dst -j RETURN\n-A KUBE-IPVS-FILTER -m set
--match-set KUBE-6-EXTERNAL-IP-LOCAL dst,dst -j RETURN\n-A KUBE-IPVS-FILTER -m
set --match-set KUBE-6-HEALTH-CHECK-NODE-PORT dst -j RETURN\n-A
KUBE-IPVS-FILTER -m conntrack --ctstate NEW -m set --match-set KUBE-6-IPVS-IPS
dst -j REJECT\nCOMMIT\n"
```
This error about "--xor-mark" being unknown looks very similar to what
was reported in https://github.com/bottlerocket-
os/bottlerocket/issues/4295. That issue mentioned that
https://web.git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.15.170&id=90baa455aa7e099152898cfa5eb3928d6152da12
should fix it. I verified that ip6tables-restore works fine on
6.8.0-1024.26~22.04.1 (without changing any Kubernetes-related
package), so I think this issue is specific to 1025.
That fix commit says that it: Fixes: 0bfcb7b71e73 ("netfilter:
xtables: avoid NFPROTO_UNSPEC where needed"). It looks like the buggy
commit is the latest commit to touch xt_mark.c in the jammy
aws-6.8-next branch: https://git.launchpad.net/~canonical-
kernel/ubuntu/+source/linux-
aws/+git/jammy/log/net/netfilter/xt_mark.c?h=aws-6.8-next
Is there any way to fast-track the fix commit into linux-aws? Will
1025 (without the fix) get promoted from jammy-proposed to jammy?
I'm not familiar with the process by which commits are merged into
linux-aws and published, so I apologize if this is not the right place
for this, and would appreciate pointers to the right place to ask.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-aws/+bug/2101914/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : [email protected]
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
