Re: Review Request: Remove filterwin2k from virtual router default config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10843/ --- (Updated May 3, 2013, 9:05 p.m.) Review request for cloudstack and Frank Zhang. Changes --- Rebased patch against master! Description --- Commenting out filterwin2k. Should this be pushed back as a patch for older branches as well? This addresses bugs CLOUDSTACK-2224 and CLOUDSTACK-282. Diffs (updated) - patches/systemvm/debian/config/etc/dnsmasq.conf.tmpl 7d656cb patches/systemvm/debian/config/etc/vpcdnsmasq.conf 3717fc8 Diff: https://reviews.apache.org/r/10843/diff/ Testing --- Verified in local environment that SRV records are accessible, enabling Windows KMS services. Thanks, Dennis Lawler
Pending reviews
Still pending review: https://reviews.apache.org/r/10843/ Should to good to commit. Any takers? :) https://reviews.apache.org/r/10884/
Re: Review Request: Release old DHCP entries
> On May 1, 2013, 10:05 p.m., Chiradeep Vittal wrote: > > patches/systemvm/debian/config/root/edithosts.sh, line 104 > > <https://reviews.apache.org/r/10884/diff/2/?file=286739#file286739line104> > > > > Please check for the existence of 'dhcp_release' binary. You will have > > folks upgrading from 4.1 and expecting their 'old' system vms to work until > > they finish upgrading to the new system vm. That's right, my apologies. I didn't think about that until after Marcus posted his proposed changes to the mailing list, so I incorporated them with a few minor changes. - Dennis --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10884/#review20031 ------- On May 2, 2013, 5:38 a.m., Dennis Lawler wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/10884/ > --- > > (Updated May 2, 2013, 5:38 a.m.) > > > Review request for cloudstack. > > > Description > --- > > Release old DHCP entries when adding new entries. > > Add dnsmasq-utils to System VM template. > > > This addresses bug CLOUDSTACK-2299. > > > Diffs > - > > patches/systemvm/debian/config/root/edithosts.sh 1f98fbf > scripts/network/exdhcp/dnsmasq_edithosts.sh 05285d9 > tools/appliance/definitions/systemvmtemplate/postinstall.sh ae8f1ad > tools/appliance/definitions/systemvmtemplate64/postinstall.sh ae8f1ad > > Diff: https://reviews.apache.org/r/10884/diff/ > > > Testing > --- > > Tested in staging VM destroyed and re-instantiated with same IP. > Did not test adding dnsmasq-utils to system VM build (yet). > > > Thanks, > > Dennis Lawler > >
Re: Review Request: Release old DHCP entries
> On May 1, 2013, 8:36 p.m., Marcus Sorensen wrote: > > patches/systemvm/debian/config/root/edithosts.sh, line 103 > > <https://reviews.apache.org/r/10884/diff/2/?file=286739#file286739line103> > > > > shouldn't this be $ipv4?, and if it fails to find the mac in the leases > > file, will it kill the whole script? That's right - bad typo. If it fails to find the MAC in the leases file (the usual case), dhcp_release does not get enough args and fails. It shouldn't kill the whole script, or at least it hasn't in testing. - Dennis --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10884/#review20025 ------- On May 2, 2013, 5:38 a.m., Dennis Lawler wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/10884/ > --- > > (Updated May 2, 2013, 5:38 a.m.) > > > Review request for cloudstack. > > > Description > --- > > Release old DHCP entries when adding new entries. > > Add dnsmasq-utils to System VM template. > > > This addresses bug CLOUDSTACK-2299. > > > Diffs > - > > patches/systemvm/debian/config/root/edithosts.sh 1f98fbf > scripts/network/exdhcp/dnsmasq_edithosts.sh 05285d9 > tools/appliance/definitions/systemvmtemplate/postinstall.sh ae8f1ad > tools/appliance/definitions/systemvmtemplate64/postinstall.sh ae8f1ad > > Diff: https://reviews.apache.org/r/10884/diff/ > > > Testing > --- > > Tested in staging VM destroyed and re-instantiated with same IP. > Did not test adding dnsmasq-utils to system VM build (yet). > > > Thanks, > > Dennis Lawler > >
Re: Review Request: Release old DHCP entries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10884/ --- (Updated May 2, 2013, 5:38 a.m.) Review request for cloudstack. Changes --- Incorporated proposed changes from Marcus Sorensen, but used "command -v" to determine existence of dhcp_release. Fixed $ipv4 usage in patch. Fixed exdhcp dnsmasq update to reflect correct args. Description --- Release old DHCP entries when adding new entries. Add dnsmasq-utils to System VM template. This addresses bug CLOUDSTACK-2299. Diffs (updated) - patches/systemvm/debian/config/root/edithosts.sh 1f98fbf scripts/network/exdhcp/dnsmasq_edithosts.sh 05285d9 tools/appliance/definitions/systemvmtemplate/postinstall.sh ae8f1ad tools/appliance/definitions/systemvmtemplate64/postinstall.sh ae8f1ad Diff: https://reviews.apache.org/r/10884/diff/ Testing --- Tested in staging VM destroyed and re-instantiated with same IP. Did not test adding dnsmasq-utils to system VM build (yet). Thanks, Dennis Lawler
Review Request: Release old DHCP entries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10884/ --- Review request for cloudstack. Description --- Release old DHCP entries when adding new entries. Add dnsmasq-utils to System VM template. This addresses bug CLOUDSTACK-2299. Diffs - patches/systemvm/debian/config/root/edithosts.sh 1f98fbf scripts/network/exdhcp/dnsmasq_edithosts.sh 05285d9 tools/appliance/definitions/systemvmtemplate/postinstall.sh ae8f1ad tools/appliance/definitions/systemvmtemplate64/postinstall.sh ae8f1ad Diff: https://reviews.apache.org/r/10884/diff/ Testing --- Tested in staging VM destroyed and re-instantiated with same IP. Did not test adding dnsmasq-utils to system VM build (yet). Thanks, Dennis Lawler
Re: Virtual Router: DHCP and 2-second DNS outages
Something like this:
--- a/patches/systemvm/debian/config/root/edithosts.sh
+++ b/patches/systemvm/debian/config/root/edithosts.sh
@@ -99,6 +99,10 @@ wait_for_dnsmasq () {
logger -t cloud "edithosts: update $1 $2 $3 to hosts"
+#release previous dhcp lease if present
+dhcp_release eth0 $ip $(grep $ip $DHCP_LEASES | awk '{print $2}')
(same for dnsmasq_edithosts.sh)
--- a/tools/appliance/definitions/systemvmtemplate/postinstall.sh
+++ b/tools/appliance/definitions/systemvmtemplate/postinstall.sh
@@ -40,7 +40,7 @@ install_packages() {
# haproxy
apt-get --no-install-recommends -q -y --force-yes install haproxy
# dnsmasq
- apt-get --no-install-recommends -q -y --force-yes install dnsmasq
+ apt-get --no-install-recommends -q -y --force-yes install dnsmasq
dnsmasq-utils
Would need a rebuild of system VM, perhaps documentation update to notify
users that external dnsmasq should also have dnsmasq-utils package
installed.
On Wed, May 1, 2013 at 11:29 AM, Marcus Sorensen wrote:
> How do we go about requesting that dnsmasq-utils be installed on the new
> system VM?
>
>
> On Wed, May 1, 2013 at 11:15 AM, Marcus Sorensen >wrote:
>
> > I think on new system VM edithosts should preemptively expire lease for
> > the passed ip and then sighup. That avoids complications in having to put
> > in separate calls to the router VM in each agent resource just to expire.
> > On May 1, 2013 9:10 AM, "Dennis Lawler" wrote:
> >
> >> It does reconfigure the available leases for new IP allocations. It
> just
> >> doesn't expire the leases it has already handed out.
> >>
> >> If you replace the "service dnsmasq restart" in edithosts.sh with "kill
> -s
> >> 1" on the router VM, you'll start seeing these log messages when a VM is
> >> destroyed and re-allocated:
> >>
> >> dnsmasq-dhcp[pid]: not using configured address 192.168.1.100 because it
> >> is
> >> leased to aa:bb:cc:11:22:33
> >> dnsmasq-dhcp[pid]: DHCPDISCOVER(eth0) aa:bb:cc:22:33:44 no address
> >> available
> >>
> >>
> >>
> >>
> >> On Tue, Apr 30, 2013 at 10:10 PM, Marcus Sorensen >> >wrote:
> >>
> >> > that's strange, because the dnsmasq man page explicitly calls out the
> >> > SIGHUP as a way to reconfigure DHCP hosts entries from a
> >> --dhcp-hostsfile
> >> > parameter. Or are these not the same thing?
> >> >
> >> >
> >> > On Tue, Apr 30, 2013 at 5:52 PM, Chiradeep Vittal <
> >> > chiradeep.vit...@citrix.com> wrote:
> >> >
> >> > >
> >> > >
> >> > > On 4/30/13 3:26 PM, "Dennis Lawler" wrote:
> >> > >
> >> > > >Every time a new VM is started up, there is a 2 second outage in
> DNS
> >> > > >services that can cause problems in guest VMs that use the router
> VM
> >> for
> >> > > >DNS.
> >> > > >
> >> > > >
> >> > > >
> >> > > >For Cloudstack configurations using both DHCP and DNS services on
> the
> >> > > >router
> >> > > >VM (both implemented with dnsmasq), there is currently a 2 second
> DNS
> >> > > >service outage every time a new VM is instantiated
> >> > > >
> >> > > >
> >> > > >
> >> > > >The source of this outage is in edithosts.sh, which uses "service
> >> > dnsmasq
> >> > > >restart" to pick up the freshly added DNS and DHCP entries.
> >> > > >
> >> > > >Restarting the dnsmasq service triggers a sleep for 2 seconds after
> >> > > >killing
> >> > > >dnsmasq before starting it back up again.
> >> > > >
> >> > > >
> >> > > >
> >> > > >An obvious solution would be to replace "service dnsmasq restart"
> >> with
> >> > > >"kill
> >> > > >-s 1 $pid" (SIGHUP) so that dnsmasq reads the new DHCP entries
> >> without
> >> > > >restarting, as in dnsmasq_edithosts.sh (external dhcp).
> >> > > >
> >> > > >
> >> > > >Unfortunately, this solution is flawed because dnsmasq SIGHUP
> >> handling
> >> > > >does
> >> > > >not expire in-memory DHCP leases in dnsmasq and all leases are
> >> infinite
> >> > by
> >> > > >defau
Re: Review Request: Remove filterwin2k from virtual router default config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10843/ --- (Updated May 1, 2013, 4:47 p.m.) Review request for cloudstack and Frank Zhang. Changes --- Missed vpcdnsmasq.conf in original patch. Description --- Commenting out filterwin2k. Should this be pushed back as a patch for older branches as well? This addresses bugs CLOUDSTACK-2224 and CLOUDSTACK-282. Diffs (updated) - patches/systemvm/debian/config/etc/dnsmasq.conf 7d656cb patches/systemvm/debian/config/etc/vpcdnsmasq.conf 3717fc8 Diff: https://reviews.apache.org/r/10843/diff/ Testing --- Verified in local environment that SRV records are accessible, enabling Windows KMS services. Thanks, Dennis Lawler
Re: Review Request: Remove filterwin2k from virtual router default config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10843/ --- (Updated May 1, 2013, 4:16 p.m.) Review request for cloudstack and Frank Zhang. Description --- Commenting out filterwin2k. Should this be pushed back as a patch for older branches as well? This addresses bugs CLOUDSTACK-2224 and CLOUDSTACK-282. Diffs - patches/systemvm/debian/config/etc/dnsmasq.conf 7d656cb Diff: https://reviews.apache.org/r/10843/diff/ Testing --- Verified in local environment that SRV records are accessible, enabling Windows KMS services. Thanks, Dennis Lawler
Re: Virtual Router: DHCP and 2-second DNS outages
It does reconfigure the available leases for new IP allocations. It just doesn't expire the leases it has already handed out. If you replace the "service dnsmasq restart" in edithosts.sh with "kill -s 1" on the router VM, you'll start seeing these log messages when a VM is destroyed and re-allocated: dnsmasq-dhcp[pid]: not using configured address 192.168.1.100 because it is leased to aa:bb:cc:11:22:33 dnsmasq-dhcp[pid]: DHCPDISCOVER(eth0) aa:bb:cc:22:33:44 no address available On Tue, Apr 30, 2013 at 10:10 PM, Marcus Sorensen wrote: > that's strange, because the dnsmasq man page explicitly calls out the > SIGHUP as a way to reconfigure DHCP hosts entries from a --dhcp-hostsfile > parameter. Or are these not the same thing? > > > On Tue, Apr 30, 2013 at 5:52 PM, Chiradeep Vittal < > chiradeep.vit...@citrix.com> wrote: > > > > > > > On 4/30/13 3:26 PM, "Dennis Lawler" wrote: > > > > >Every time a new VM is started up, there is a 2 second outage in DNS > > >services that can cause problems in guest VMs that use the router VM for > > >DNS. > > > > > > > > > > > >For Cloudstack configurations using both DHCP and DNS services on the > > >router > > >VM (both implemented with dnsmasq), there is currently a 2 second DNS > > >service outage every time a new VM is instantiated > > > > > > > > > > > >The source of this outage is in edithosts.sh, which uses "service > dnsmasq > > >restart" to pick up the freshly added DNS and DHCP entries. > > > > > >Restarting the dnsmasq service triggers a sleep for 2 seconds after > > >killing > > >dnsmasq before starting it back up again. > > > > > > > > > > > >An obvious solution would be to replace "service dnsmasq restart" with > > >"kill > > >-s 1 $pid" (SIGHUP) so that dnsmasq reads the new DHCP entries without > > >restarting, as in dnsmasq_edithosts.sh (external dhcp). > > > > > > > > >Unfortunately, this solution is flawed because dnsmasq SIGHUP handling > > >does > > >not expire in-memory DHCP leases in dnsmasq and all leases are infinite > by > > >default. > > > > Aha! That's why SIGHUP didn't work consistently. This has been bugging me > > for a long time. > > > > >Thus, this will only work if the guest VM performs a DHCP release on > > >shutdown, which cannot always be guaranteed. > > > > > > > > > > > >A few possible solutions off the top of my head: > > > > > >1. Separate DNS and DHCP services. While DHCP services still > > >experience an outage during VM, DNS will not necessarily be impacted if > > >implemented correctly. > > > > > >2. Use SIGHUP with dnsmasq and implement a removeDhcpEntry > interface > > >for network appliances to force a DHCP release whenever a NIC / IP is > > >deallocated. This can use dhcp_release to simulate a DHCP release on > the > > >router VM. > > >Catch: dhcp_release is not available for Debian 6.0. The System VM > needs > > >to > > >be updated to at least Debian 7.0, or the dnsmasq-tools .deb from 7.0 > > >would > > >need to be included in the System VM image. > > > > There is going to be a new system vm based on 7.0 for the upcoming > > release. This should work with earlier releases as well. > > https://cwiki.apache.org/confluence/x/UlHVAQ > > > > > > > >3. Change DHCP to have a shorter lease, track de-allocation of IPs > > >separately from VM destruction. > > >Catch: This may cause occasional IP pool exhaustion depending on > > >allocation > > >of the guest IP range and the rate of VM destruction / instantiation in > > >the > > >network. > > > > > > > > > > > >Thoughts? > > > > > > > >
Virtual Router: DHCP and 2-second DNS outages
Every time a new VM is started up, there is a 2 second outage in DNS services that can cause problems in guest VMs that use the router VM for DNS. For Cloudstack configurations using both DHCP and DNS services on the router VM (both implemented with dnsmasq), there is currently a 2 second DNS service outage every time a new VM is instantiated The source of this outage is in edithosts.sh, which uses "service dnsmasq restart" to pick up the freshly added DNS and DHCP entries. Restarting the dnsmasq service triggers a sleep for 2 seconds after killing dnsmasq before starting it back up again. An obvious solution would be to replace "service dnsmasq restart" with "kill -s 1 $pid" (SIGHUP) so that dnsmasq reads the new DHCP entries without restarting, as in dnsmasq_edithosts.sh (external dhcp). Unfortunately, this solution is flawed because dnsmasq SIGHUP handling does not expire in-memory DHCP leases in dnsmasq and all leases are infinite by default. Thus, this will only work if the guest VM performs a DHCP release on shutdown, which cannot always be guaranteed. A few possible solutions off the top of my head: 1. Separate DNS and DHCP services. While DHCP services still experience an outage during VM, DNS will not necessarily be impacted if implemented correctly. 2. Use SIGHUP with dnsmasq and implement a removeDhcpEntry interface for network appliances to force a DHCP release whenever a NIC / IP is deallocated. This can use dhcp_release to simulate a DHCP release on the router VM. Catch: dhcp_release is not available for Debian 6.0. The System VM needs to be updated to at least Debian 7.0, or the dnsmasq-tools .deb from 7.0 would need to be included in the System VM image. 3. Change DHCP to have a shorter lease, track de-allocation of IPs separately from VM destruction. Catch: This may cause occasional IP pool exhaustion depending on allocation of the guest IP range and the rate of VM destruction / instantiation in the network. Thoughts?
Review Request: Remove filterwin2k from virtual router default config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10843/ --- Review request for cloudstack. Description --- Commenting out filterwin2k. Should this be pushed back as a patch for older branches as well? This addresses bugs CLOUDSTACK-2224 and CLOUDSTACK-282. Diffs - patches/systemvm/debian/config/etc/dnsmasq.conf 7d656cb Diff: https://reviews.apache.org/r/10843/diff/ Testing --- Verified in local environment that SRV records are accessible, enabling Windows KMS services. Thanks, Dennis Lawler
RE: CS dnsmasq DNS services - filterwin2k
Thanks Frank, will do! -Original Message- From: Frank Zhang [mailto:frank.zh...@citrix.com] Sent: Monday, April 29, 2013 2:12 PM To: cloudstack-...@incubator.apache.org Subject: RE: CS dnsmasq DNS services - filterwin2k > > On 4/26/13 3:27 PM, "Dennis Lawler" wrote: > > >The dnsmasq "filterwin2k" option disables SRV record requests, > >affecting Kerberos, SIP, Windows KMS licensing, XMPP, and Google Talk. > > > >This was disabled by default in dnsmasq config, but was re-added via > >commit 3b75abb, which does not exactly say why. > > > >Does anyone know why dnsmasq is configured this way for Cloudstack by > >default? Should it be? Dennis, it's disabled two years for some magic reason that I cannot recall now. As you said it causes lots of problems, I think you can remove it. Patch is welcome! > > > >--Dennis smime.p7s Description: S/MIME cryptographic signature
CS dnsmasq DNS services - filterwin2k
The dnsmasq "filterwin2k" option disables SRV record requests, affecting Kerberos, SIP, Windows KMS licensing, XMPP, and Google Talk. This was disabled by default in dnsmasq config, but was re-added via commit 3b75abb, which does not exactly say why. Does anyone know why dnsmasq is configured this way for Cloudstack by default? Should it be? --Dennis
