[
https://issues.apache.org/jira/browse/CLOUDSTACK-9946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059057#comment-16059057
]
Jayapal Reddy edited comment on CLOUDSTACK-9946 at 6/29/17 6:22 AM:
--------------------------------------------------------------------
When two PF rules delete concurrently then there is race condition. When second
rule delete command is send, the second rule revoke is set to true where the
first one revoke set to false. Due to this rule is still there in the VR.
{noformat}
{"rules":[{"revoke":false,"protocol":"tcp","source_ip_address":"10.147.52.103","source_port_range":"82:82","destination_ip_address":"10.1.1.68","destination_port_range":"82:82"},{"revoke":true,"protocol":"tcp","source_ip_address":"10.147.52.103","source_port_range":"83:83","destination_ip_address":"10.1.1.68","destination_port_range":"83:83"}],"type":"forwardrules"}
{noformat}
was (Author: jayapal):
When two PF rules delete concurrently then there is race condition. The second
rule revoke is set to true where the first one revoke set to true. Due to this
rule is still there in the VR.
{noformat}
{"rules":[{"revoke":false,"protocol":"tcp","source_ip_address":"10.147.52.103","source_port_range":"82:82","destination_ip_address":"10.1.1.68","destination_port_range":"82:82"},{"revoke":true,"protocol":"tcp","source_ip_address":"10.147.52.103","source_port_range":"83:83","destination_ip_address":"10.1.1.68","destination_port_range":"83:83"}],"type":"forwardrules"}
{noformat}
> When multiple PF rules are deleted , the 1st PF rule added is still retained
> in forwardingrules.json file in VPC VR .
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-9946
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9946
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Virtual Router
> Affects Versions: 4.10.0.0
> Reporter: DeepthiMachiraju
> Labels: PVR
> Fix For: 4.10.0.0
>
> Attachments: MS_log_deletion_pf_rules.txt
>
>
> - Create a VPC , and deploy a VM in the Tier.
> - Navigate to PUblick IP address in the VPC and acquire an IP.
> - Create Multiple PF rules as below . Was able to sucessfully ssh and access
> HTTP to the VM.
> - Now delete all the rules configured .
> Observation :
> - All the rules are cleaned up in the UI & DB . But the 1st rule added is
> still retained in the IPtables and forwardingrules.json file .
> - and user is still able to access the rule.
> Logs when rules are added :
> acquired ip and assigned 5 pf rules :
> root@r-53-VM:/etc/cloudstack# ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
> qlen 1000
> link/ether 0e:00:a9:fe:02:c3 brd ff:ff:ff:ff:ff:ff
> inet 169.254.2.195/16 brd 169.254.255.255 scope global eth0
> 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
> qlen 1000
> link/ether 06:3d:52:00:00:0d brd ff:ff:ff:ff:ff:ff
> inet 10.147.30.112/24 brd 10.147.30.255 scope global eth1
> inet 10.147.30.113/24 brd 10.147.30.255 scope global secondary eth1
> 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
> qlen 1000
> link/ether 02:00:4b:3f:00:14 brd ff:ff:ff:ff:ff:ff
> inet 172.16.1.1/24 brd 172.16.1.255 scope global eth2
> 5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
> qlen 1000
> link/ether 02:00:67:bb:00:04 brd ff:ff:ff:ff:ff:ff
> inet 172.16.2.1/24 brd 172.16.2.255 scope global eth3
> ********************************
> root@r-53-VM:/etc/cloudstack# cat forwardingrules.json
> {
> "10.147.30.113": [
> {
> "internal_ip": "172.16.2.10",
> "internal_ports": "10:10",
> "protocol": "tcp",
> "public_ip": "10.147.30.113",
> "public_ports": "10:10",
> "type": "forward"
> },
> {
> "internal_ip": "172.16.2.10",
> "internal_ports": "20:20",
> "protocol": "tcp",
> "public_ip": "10.147.30.113",
> "public_ports": "20:20",
> "type": "forward"
> },
> {
> "internal_ip": "172.16.2.10",
> "internal_ports": "30:30",
> "protocol": "tcp",
> "public_ip": "10.147.30.113",
> "public_ports": "30:30",
> "type": "forward"
> },
> {
> "internal_ip": "172.16.2.10",
> "internal_ports": "22:22",
> "protocol": "tcp",
> "public_ip": "10.147.30.113",
> "public_ports": "22:22",
> "type": "forward"
> },
> {
> "internal_ip": "172.16.2.10",
> "internal_ports": "80:80",
> "protocol": "tcp",
> "public_ip": "10.147.30.113",
> "public_ports": "80:80",
> "type": "forward"
> }
> ],
> "id": "forwardingrules"
> ********************************
> root@r-53-VM:/etc/cloudstack# iptables -t nat -L
> Chain PREROUTING (policy ACCEPT)
> target prot opt source destination
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:10
> to:172.16.2.10:10
> DNAT tcp -- anywhere 10.147.30.113 tcp
> dpt:ftp-data to:172.16.2.10:20
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:30
> to:172.16.2.10:30
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:ssh
> to:172.16.2.10:22
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:http
> to:172.16.2.10:80
> Chain INPUT (policy ACCEPT)
> target prot opt source destination
> Chain OUTPUT (policy ACCEPT)
> target prot opt source destination
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:10
> to:172.16.2.10:10
> DNAT tcp -- anywhere 10.147.30.113 tcp
> dpt:ftp-data to:172.16.2.10:20
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:30
> to:172.16.2.10:30
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:ssh
> to:172.16.2.10:22
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:http
> to:172.16.2.10:80
> Chain POSTROUTING (policy ACCEPT)
> target prot opt source destination
> SNAT all -- 172.16.2.0/24 anywhere to:172.16.2.1
> SNAT all -- 172.16.1.0/24 anywhere to:172.16.1.1
> SNAT all -- anywhere anywhere to:10.147.30.112
> SNAT all -- anywhere anywhere to:10.147.30.113
> SNAT tcp -- anywhere 10.147.30.113 tcp dpt:10
> to:172.16.2.10:10
> SNAT tcp -- anywhere 10.147.30.113 tcp
> dpt:ftp-data to:172.16.2.10:20
> SNAT tcp -- anywhere 10.147.30.113 tcp dpt:30
> to:172.16.2.10:30
> SNAT tcp -- anywhere 10.147.30.113 tcp dpt:ssh
> to:172.16.2.10:22
> SNAT tcp -- anywhere 10.147.30.113 tcp dpt:http
> to:172.16.2.10:80
> ********************************
> mysql> select * from port_forwarding_rules;
> +-----+-------------+-----------------+-----------------+---------------+
> | id | instance_id | dest_ip_address | dest_port_start | dest_port_end |
> +-----+-------------+-----------------+-----------------+---------------+
> | 113 | 24 | 172.16.2.10 | 10 | 10 |
> | 114 | 24 | 172.16.2.10 | 20 | 20 |
> | 115 | 24 | 172.16.2.10 | 30 | 30 |
> | 116 | 24 | 172.16.2.10 | 22 | 22 |
> | 117 | 24 | 172.16.2.10 | 80 | 80 |
> +-----+-------------+-----------------+-----------------+---------------+
> 5 rows in set (0.00 sec)
> ********************************
> ================== Logs post deleting the pf rules ========================
> root@r-53-VM:/etc/cloudstack# ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
> qlen 1000
> link/ether 0e:00:a9:fe:02:c3 brd ff:ff:ff:ff:ff:ff
> inet 169.254.2.195/16 brd 169.254.255.255 scope global eth0
> 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
> qlen 1000
> link/ether 06:3d:52:00:00:0d brd ff:ff:ff:ff:ff:ff
> inet 10.147.30.112/24 brd 10.147.30.255 scope global eth1
> 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
> qlen 1000
> link/ether 02:00:4b:3f:00:14 brd ff:ff:ff:ff:ff:ff
> inet 172.16.1.1/24 brd 172.16.1.255 scope global eth2
> 5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP
> qlen 1000
> link/ether 02:00:67:bb:00:04 brd ff:ff:ff:ff:ff:ff
> inet 172.16.2.1/24 brd 172.16.2.255 scope global eth3
> root@r-53-VM:/etc/cloudstack#
> ********************************
> root@r-53-VM:/etc/cloudstack# iptables -t nat -L
> Chain PREROUTING (policy ACCEPT)
> target prot opt source destination
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:10
> to:172.16.2.10:10
> Chain INPUT (policy ACCEPT)
> target prot opt source destination
> Chain OUTPUT (policy ACCEPT)
> target prot opt source destination
> DNAT tcp -- anywhere 10.147.30.113 tcp dpt:10
> to:172.16.2.10:10
> Chain POSTROUTING (policy ACCEPT)
> target prot opt source destination
> SNAT all -- 172.16.2.0/24 anywhere to:172.16.2.1
> SNAT all -- 172.16.1.0/24 anywhere to:172.16.1.1
> SNAT all -- anywhere anywhere to:10.147.30.112
> SNAT tcp -- anywhere 10.147.30.113 tcp dpt:10
> to:172.16.2.10:10
> root@r-53-VM:/etc/cloudstack#
> +-********************************
> root@r-53-VM:/etc/cloudstack#
> root@r-53-VM:/etc/cloudstack# cat forwardingrules.json
> {
> "10.147.30.113": [
> {
> "internal_ip": "172.16.2.10",
> "internal_ports": "10:10",
> "protocol": "tcp",
> "public_ip": "10.147.30.113",
> "public_ports": "10:10",
> "type": "forward"
> }
> ],
> "id": "forwardingrules"
> }root@r-53-VM:/etc/cloudstack#
> Attached MS.log
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
