Vladimir Ostrovsky created CLOUDSTACK-74:
--------------------------------------------
Summary: CloudStack distributes incorrect IPtables rules to its
KVM hosts
Key: CLOUDSTACK-74
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-74
Project: CloudStack
Issue Type: Bug
Components: Install and Setup, KVM, Network Controller
Affects Versions: pre-4.0.0
Reporter: Vladimir Ostrovsky
Some strange issue with IPtables rules created by CloudStack 3.0.4 on KVM
hosts:
I have a Basic zone of 3 KVM hosts:
• 1st host runs Console Proxy, v-2-VM
• 2nd host is unused
• 3rd host runs Secondary Storage VM, s-1-VM
Each host has two network interfaces:
• eth0 (bridge cloudbr0), connected to isolated private network (used to
communicate with CloudStack management server, with Primary & Secondary
storages and so on).
• eth1 (bridge cloudbr1), connected to public / external network (goes to
Internet, used by customers to communicate with VMs in the Basic zone and with
Console Proxy, used by SSVM to upload / download templates).
The problem:
• Console Proxy doesn’t have communication with private network (with
CloudStack, in first place).
• SSVM doesn’t have communication with external network (for example,
Internet).
Why it happens:
On 1st host, the IPtables rules of FORWARD chain look like this:
Chain FORWARD (policy DROP 4690 packets, 242K bytes)
pkts bytes target prot opt in out source destination
996 34620 BF-cloudbr1 all -- * cloudbr1 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-is-bridged
0 0 BF-cloudbr1 all -- cloudbr1 * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-is-bridged
0 0 DROP all -- * cloudbr1 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- cloudbr1 * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * virbr0 0.0.0.0/0 192.168.122.0/24 state
RELATED,ESTABLISHED
0 0 ACCEPT all -- virbr0 * 192.168.122.0/24 0.0.0.0/0
0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
So we see that there are rules for traffic passing via cloudbr1 (public
interface), but no rules for cloudbr0 (private interface), and the default is
DROP.
On 3rd host, however, it’s exactly vice versa:
Chain FORWARD (policy DROP 887 packets, 28760 bytes)
pkts bytes target prot opt in out source destination
597K 4206M BF-cloudbr0 all -- * cloudbr0 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-is-bridged
0 0 BF-cloudbr0 all -- cloudbr0 * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-is-bridged
0 0 DROP all -- * cloudbr0 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- cloudbr0 * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * virbr0 0.0.0.0/0 192.168.122.0/24 state
RELATED,ESTABLISHED
0 0 ACCEPT all -- virbr0 * 192.168.122.0/24 0.0.0.0/0
0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
And on 2nd host, there are no rules for both interfaces:
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- * virbr0 0.0.0.0/0 192.168.122.0/24 state
RELATED,ESTABLISHED
0 0 ACCEPT all -- virbr0 * 192.168.122.0/24 0.0.0.0/0
0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
After I migrated the SSVM to the 1st host, the rules changed to:
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
98 15967 BF-cloudbr0 all -- * cloudbr0 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-is-bridged
6 192 BF-cloudbr0 all -- cloudbr0 * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-is-bridged
6 192 DROP all -- * cloudbr0 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- cloudbr0 * 0.0.0.0/0 0.0.0.0/0
1107 38228 BF-cloudbr1 all -- * cloudbr1 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-is-bridged
3 96 BF-cloudbr1 all -- cloudbr1 * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match
--physdev-is-bridged
3 96 DROP all -- * cloudbr1 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- cloudbr1 * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * virbr0 0.0.0.0/0 192.168.122.0/24 state
RELATED,ESTABLISHED
0 0 ACCEPT all -- virbr0 * 192.168.122.0/24 0.0.0.0/0
0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with
icmp-port-unreachable
It seems like the logic that CloudStack implements is this:
• Console Proxy doesn’t need access to private network – so only rules
for cloudbr1 are installed
• SSVM doesn’t need access to public network– so only rules for cloudbr0
are installed
But it’s totally incorrect:
• Both need to communicate with CloudStack management server via private
network (to TCP port 8250).
• Both need to communicate with external networks: SSVM to
upload/download templates, Console Proxy – to provide VM consoles to users.
Regards,
Vladimir.
P.S. Link to the bug report on the old site:
http://bugs.cloudstack.org/browse/CS-16205
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
