[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106502#comment-14106502 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit c4a32d94165bd0b43b777fad749cf071c270cd23 in cloudstack's branch refs/heads/master from sanjeev [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=c4a32d9 ] CLOUDSTACK-5986: Test script for verifying fix CS-5986 for vpc networks Signed-off-by: sanjeev Made changes to test script as per the review comments in Review Request 23928 > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071677#comment-14071677 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 9f4f9211d0aecbfeadfa626b6202109f40200125 in cloudstack's branch refs/heads/master from sanjeev [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=9f4f921 ] CLOUDSTACK-5986: Test scipt to verify the fix in isolated network for non-vpc networks Signed-off-by: sanjeev > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938561#comment-13938561 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 5973a4269143679693fc5f1fe89c7b3d70c26b6f in cloudstack's branch refs/heads/4.3 from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=5973a42 ] CLOUDSTACK-5986: Fix dnsmasq lease for VPC (cherry picked from commit 657d042953e40e3220c5b0b86511f3bf6a1e186f) Signed-off-by: Animesh Chaturvedi > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935601#comment-13935601 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 657d042953e40e3220c5b0b86511f3bf6a1e186f in cloudstack's branch refs/heads/4.3-forward from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=657d042 ] CLOUDSTACK-5986: Fix dnsmasq lease for VPC > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932742#comment-13932742 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 6baa0538a852f59d17f9b2372ec6ae96381e4b3d in cloudstack's branch refs/heads/4.2 from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=6baa053 ] CLOUDSTACK-5986: Fix dnsmasq lease judgement > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932523#comment-13932523 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 92b9951df3050c8934b882b0b4c70828ac01f9af in cloudstack's branch refs/heads/4.3 from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=92b9951 ] CLOUDSTACK-5986: Fix dnsmasq lease for VPC (cherry picked from commit a253ff24682a7d405bfa8428a0833423bf19f9b9) Signed-off-by: Animesh Chaturvedi > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931126#comment-13931126 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 8937434fce6958155e9871caa1648504b7c34999 in cloudstack's branch refs/heads/master from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=8937434 ] CLOUDSTACK-5986: Fix dnsmasq lease for VPC > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931127#comment-13931127 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit a253ff24682a7d405bfa8428a0833423bf19f9b9 in cloudstack's branch refs/heads/4.3-forward from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=a253ff2 ] CLOUDSTACK-5986: Fix dnsmasq lease for VPC > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931130#comment-13931130 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 122761db348d06214105f7a4abc37ea8685372b7 in cloudstack's branch refs/heads/4.2 from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=122761d ] CLOUDSTACK-5986: Fix dnsmasq lease for VPC > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886173#comment-13886173 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 121d88743a764e009717cc84440b1c7d5b1e42a4 in branch refs/heads/master from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=121d887 ] CLOUDSTACK-5986: Make dnsmasq handle dnsmasq.leases when dhcp_release is available The original issue has been exposed due to CloudStack VR would modify the dnsmasq.leases, thus make it unsync with dnsmasq's memory lease. Make the modification to let dnsmasq handle the lease file if dhcp_release is available. > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886170#comment-13886170 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 30419cac26993d279e589674faa084c15f2694ee in branch refs/heads/4.2 from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=30419ca ] CLOUDSTACK-5986: Make dnsmasq handle dnsmasq.leases when dhcp_release is available The original issue has been exposed due to CloudStack VR would modify the dnsmasq.leases, thus make it unsync with dnsmasq's memory lease. Make the modification to let dnsmasq handle the lease file if dhcp_release is available. > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886164#comment-13886164 ] ASF subversion and git services commented on CLOUDSTACK-5986: - Commit 441d02dd402e632c3eba1062a55fc0d752e76003 in branch refs/heads/4.3-forward from [~yasker] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=441d02d ] CLOUDSTACK-5986: Make dnsmasq handle dnsmasq.leases when dhcp_release is available The original issue has been exposed due to CloudStack VR would modify the dnsmasq.leases, thus make it unsync with dnsmasq's memory lease. Make the modification to let dnsmasq handle the lease file if dhcp_release is available. > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CLOUDSTACK-5986) dnsmasq racy condition result in dnsmasq failed to handout IP address
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13886044#comment-13886044 ] Sheng Yang commented on CLOUDSTACK-5986: To reproduce the issue(or verify the fix), you can: 1. Create a set of vms (e.g. 10), use API to assign each one of them a pre-determined IP(e.g. 10.1.1.10-10.1.1.19). 2. Destroy them 3. Create another set of VM, use the same IP address range but different IP for each of them For example. VM1 used name: VM1, 10.1.1.10 VM2 used name: VM2, 10.1.1.11 VM3 used name: VM3, 10.1.1.12 VM4 used name: VM4, 10.1.1.13 VM5 used name: VM5, 10.1.1.14 Then the new one can be: VM1 used name: VM1, 10.1.1.14 VM2 used name: VM2, 10.1.1.13 VM3 used name: VM3, 10.1.1.12 VM4 used name: VM4, 10.1.1.11 VM5 used name: VM5, 10.1.1.10 Then you would find some of VMs cannot get the IP. > dnsmasq racy condition result in dnsmasq failed to handout IP address > - > > Key: CLOUDSTACK-5986 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5986 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Virtual Router >Affects Versions: 4.2.0, 4.2.1 >Reporter: Sheng Yang >Assignee: Sheng Yang >Priority: Critical > Fix For: 4.2.1, 4.3.0 > > > In the past, dnsmasq.leases is managed by cloudstack, and would be updated > after new entry is added(and the entry with same IP or host would be removed). > In 4.2, we introduced dhcp_release try to speed up the dnsmasq reload > process, thus result in this issue. > So in this scenario: > 1. VM1 would create entry: "mac A, IP A, host A" > 2. VM2 would create entry: "mac B, IP B, host B" > 3. VM1 destroyed and VM 2 destroyed, no change to lease file, two records > still existed. > 4. VM1 created with IP B: "mac C, IP B, host A". > At this time, lease file would only contain: "mac C, IP B, host A", because > the original entries would be removed by either IP match or host name match. > In fact dnsmasq still holds lease of IP A with mac A in memory, but record is > already removed from lease file by cloudstack. The lease file is out of sync > now. > 5. VM2 recreated with IP A, dhcp_release would be used to release the lease. > But there is no "mac A, IP A" record in the lease file, so dhcp_release > failed. > So result in dnsmasq cannot handle out the IP A to new mac D, because dnsmasq > was still holding lease for IP A with mac A and haven't been released yet. > So it only happened when user create and destroy different VMs which share > the same name but get different IPs. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
