Hi Indra Could you check the routing table from the guest VM ( route -n ) and I would also like you to take a tcpdump from VM to VR with port 53 to check if you are able to get the reply from the VM.
Regards Vihar K On Jul 22, 2014 9:10 AM, "Indra Pramana" <in...@sg.or.id> wrote: > Hi Santhosh, > > Here it is: > > Supported ServicesDhcp, Dns, UserDataService CapabilitiesDhcp: > VirtualRouter, Dns: VirtualRouter, UserData: VirtualRouter > The zone has been running for quite some time, I created the zone almost a > year ago and there was no issues only until recently. So I don't think the > issue is due to the zone or service/network offering's default > configuration, since I didn't make any changes to the zone configuration. > > Any advice on what should I investigate next? > > Looking forward to your reply, thank you. > > Cheers. > > > > On Mon, Jul 21, 2014 at 11:53 PM, Santhosh Edukulla < > santhosh.eduku...@citrix.com> wrote: > > > While creating zone, you would have selected network offering, we can see > > the supported services for each network offering, available under Service > > Offerings->Select Network Offerings, some thing like below, so you may > want > > to check the network offering you associated to your datacenter and > > corresponding capabilities , below are the supported services for one of > > the shared network offering. > > > > Supported Services Dns, Dhcp, UserData > > Service Capabilities Dns: VirtualRouter, Dhcp: VirtualRouter, > UserData: > > VirtualRouter > > > > Santhosh > > ________________________________________ > > From: Indra Pramana [in...@sg.or.id] > > Sent: Monday, July 21, 2014 10:37 AM > > To: users@cloudstack.apache.org > > Subject: Re: DNS service on VR not responding > > > > Hi Santhosh, Vihar, > > > > The network which this VR is responsible is a shared, not isolated > network. > > It seems there's no network offering being tagged to a shared network? > How > > do I know if the DNS service is being chosen or not? > > > > Looking forward to your reply, thank you. > > > > Cheers. > > > > > > > > On Mon, Jul 21, 2014 at 6:45 PM, Vihar <vih1...@gmail.com> wrote: > > > > > Yes, not choosing DNS service from network offering may be one of the > > > reason it is not resolving the DNS queries. > > > > > > Regards > > > Vihar K > > > On Jul 21, 2014 4:09 PM, "Santhosh Edukulla" < > > santhosh.eduku...@citrix.com > > > > > > > wrote: > > > > > > > Below are points so far, > > > > > > > > Have we selected the dns under network offering? As well, can you > check > > > > whether your dns queries are reaching VR by enabling VR in > resolv.conf > > ( > > > > guest vm ) and running trace route for some example domain? > > > > > > > > Santhosh > > > > ________________________________________ > > > > From: Indra Pramana [in...@sg.or.id] > > > > Sent: Monday, July 21, 2014 6:33 AM > > > > To: users@cloudstack.apache.org > > > > Subject: Re: DNS service on VR not responding > > > > > > > > Hi Vihar, > > > > > > > > Have tried: > > > > > > > > - Restarting dnsmasq service > > > > - Stopping and starting the VR from CloudStack GUI. > > > > > > > > Problem still persists. :( > > > > > > > > Any other hints or suggestions? > > > > > > > > Looking forward to to your reply, thank you. > > > > > > > > Cheers. > > > > > > > > > > > > > > > > On Mon, Jul 21, 2014 at 6:19 PM, Vihar <vih1...@gmail.com> wrote: > > > > > > > > > Hi Indra, > > > > > > > > > > Are you referring to /etc/resolv.conf on the VR itself or on the > > guest > > > > VM? > > > > > >> I was referring to the VR itself. I thought there was 3 IP > address > > > in > > > > VR > > > > > itself. > > > > > > > > > > Have you tried stopping and starting the VR if not can you give a > > try. > > > > > > > > > > Regards > > > > > Vihar K > > > > > On Jul 21, 2014 3:26 PM, "Indra Pramana" <in...@sg.or.id> wrote: > > > > > > > > > > > Hi Vihar, > > > > > > > > > > > > Are you referring to /etc/resolv.conf on the VR itself or on the > > > guest > > > > > VM? > > > > > > > > > > > > On the VR itself, there are only two entries on /etc/resolv.conf > > > > pointing > > > > > > to both Google public DNS servers. > > > > > > > > > > > > On the guest VM, there are 3 entries, one to the VR and two to > the > > > > Google > > > > > > public DNS servers. If I commented out both Google DNS servers > and > > > only > > > > > > leaving the VR IP there, I cannot resolve anything. If the VR IP > is > > > > > > commented out and leaving both Google DNS servers there, then I > can > > > > > > resolve. So the issue is confirmed due to DNS service on the VR. > > > > > > > > > > > > But I am not too sure why it doesn't respond even though the > > dnsmasq > > > > > > service is running. > > > > > > > > > > > > Thank you. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jul 21, 2014 at 1:00 PM, Vihar <vih1...@gmail.com> > wrote: > > > > > > > > > > > > > Hi , > > > > > > > > > > > > > > I would like you to comment second and third IP address I.e > > 4.2.2.2 > > > > and > > > > > > > 8.8.8.8 and uncomment the first IP which is allocated to DNS > and > > > try > > > > to > > > > > > > resolve the internet. It might be resolving the queries from > > > external > > > > > DNS > > > > > > > server. > > > > > > > > > > > > > > If you are not able to resolve the names from VR, check if the > > DNS > > > > > > service > > > > > > > is running properly for the IP which act as a DNS server. > > > > > > > > > > > > > > Regards > > > > > > > Vihar > > > > > > > On Jul 21, 2014 10:19 AM, "Indra Pramana" <in...@sg.or.id> > > wrote: > > > > > > > > > > > > > > > Hi Sanjeev, > > > > > > > > > > > > > > > > Good day to you, and thank you for your reply. > > > > > > > > > > > > > > > > Yes, I can resolve domains without any issues from within the > > VR > > > > > > itself. > > > > > > > > > > > > > > > > root@r-2606-VM:/etc# ping yahoo.com > > > > > > > > PING yahoo.com (98.139.183.24): 56 data bytes > > > > > > > > 64 bytes from 98.139.183.24: icmp_seq=0 ttl=47 time=250.473 > ms > > > > > > > > 64 bytes from 98.139.183.24: icmp_seq=1 ttl=47 time=239.240 > ms > > > > > > > > 64 bytes from 98.139.183.24: icmp_seq=2 ttl=45 time=247.605 > ms > > > > > > > > 64 bytes from 98.139.183.24: icmp_seq=3 ttl=45 time=244.913 > ms > > > > > > > > ^C--- yahoo.com ping statistics --- > > > > > > > > 4 packets transmitted, 4 packets received, 0% packet loss > > > > > > > > round-trip min/avg/max/stddev = 239.240/245.558/250.473/4.144 > > ms > > > > > > > > > > > > > > > > root@r-2606-VM:/etc# ping google.com > > > > > > > > PING google.com (74.125.68.102): 56 data bytes > > > > > > > > 64 bytes from 74.125.68.102: icmp_seq=0 ttl=52 time=1.353 ms > > > > > > > > 64 bytes from 74.125.68.102: icmp_seq=1 ttl=52 time=1.199 ms > > > > > > > > 64 bytes from 74.125.68.102: icmp_seq=2 ttl=52 time=1.268 ms > > > > > > > > 64 bytes from 74.125.68.102: icmp_seq=3 ttl=52 time=1.287 ms > > > > > > > > ^C--- google.com ping statistics --- > > > > > > > > 4 packets transmitted, 4 packets received, 0% packet loss > > > > > > > > round-trip min/avg/max/stddev = 1.199/1.277/1.353/0.055 ms > > > > > > > > > > > > > > > > The VR uses 8.8.8.8 and 8.8.4.4 to resolve domains. > > > > > > > > > > > > > > > > root@r-2606-VM:/etc# cat /etc/resolv.conf > > > > > > > > nameserver 8.8.8.8 > > > > > > > > nameserver 8.8.4.4 > > > > > > > > > > > > > > > > I can ping both name servers without any issues. > > > > > > > > > > > > > > > > root@r-2606-VM:/etc# ping 8.8.8.8 > > > > > > > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > > > > > > > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=52 time=4.693 ms > > > > > > > > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=52 time=2.390 ms > > > > > > > > 64 bytes from 8.8.8.8: icmp_seq=2 ttl=52 time=2.523 ms > > > > > > > > 64 bytes from 8.8.8.8: icmp_seq=3 ttl=52 time=2.458 ms > > > > > > > > ^C--- 8.8.8.8 ping statistics --- > > > > > > > > 4 packets transmitted, 4 packets received, 0% packet loss > > > > > > > > round-trip min/avg/max/stddev = 2.390/3.016/4.693/0.969 ms > > > > > > > > > > > > > > > > root@r-2606-VM:/etc# ping 8.8.4.4 > > > > > > > > PING 8.8.4.4 (8.8.4.4): 56 data bytes > > > > > > > > 64 bytes from 8.8.4.4: icmp_seq=0 ttl=52 time=2.649 ms > > > > > > > > 64 bytes from 8.8.4.4: icmp_seq=1 ttl=52 time=2.458 ms > > > > > > > > 64 bytes from 8.8.4.4: icmp_seq=2 ttl=52 time=2.436 ms > > > > > > > > 64 bytes from 8.8.4.4: icmp_seq=3 ttl=52 time=2.393 ms > > > > > > > > ^C--- 8.8.4.4 ping statistics --- > > > > > > > > 4 packets transmitted, 4 packets received, 0% packet loss > > > > > > > > round-trip min/avg/max/stddev = 2.393/2.484/2.649/0.098 ms > > > > > > > > > > > > > > > > Looking forward to your reply, thank you. > > > > > > > > > > > > > > > > Cheers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jul 21, 2014 at 12:18 PM, Sanjeev Neelarapu < > > > > > > > > sanjeev.neelar...@citrix.com> wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > Can you check if the VR is able to resolve the domain names > > by > > > > > > pinging > > > > > > > > > from VR ? > > > > > > > > > > > > > > > > > > -Sanjeev > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: Vihar [mailto:vih1...@gmail.com] > > > > > > > > > Sent: Monday, July 21, 2014 5:43 AM > > > > > > > > > To: users@cloudstack.apache.org > > > > > > > > > Cc: d...@cloudstack.apache.org > > > > > > > > > Subject: RE: DNS service on VR not responding > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > Yes, if I remove or comment out the first nameserver entry > > for > > > > the > > > > > > VR's > > > > > > > > > IP, and only leaving 8.8.8.8 and 8.8.4.4, guest VMs will be > > > > running > > > > > > > fine > > > > > > > > > and will be able to resolve domains properly." > > > > > > > > > > > > > > > > > > Are you able to ping the first DNS server IP address that > you > > > > > > commented > > > > > > > > > out? > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > Vihar K > > > > > > > > > On Jul 20, 2014 11:29 PM, "Santhosh Edukulla" < > > > > > > > > > santhosh.eduku...@citrix.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Do a traceroute to an external domain say google.com > from > > > > guest > > > > > > vm, > > > > > > > as > > > > > > > > > > you mentioned below, both by commenting out vr ip and > not, > > in > > > > > > > > > > resolv.conf, you may see the difference. > > > > > > > > > > > > > > > > > > > > "Yes, if I remove or comment out the first nameserver > entry > > > for > > > > > the > > > > > > > > > > VR's IP, and only leaving 8.8.8.8 and 8.8.4.4, guest VMs > > will > > > > be > > > > > > > > > > running fine and will be able to resolve domains > properly." > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Santhosh > > > > > > > > > > ________________________________________ > > > > > > > > > > From: Indra Pramana [in...@sg.or.id] > > > > > > > > > > Sent: Sunday, July 20, 2014 1:48 PM > > > > > > > > > > To: users@cloudstack.apache.org > > > > > > > > > > Cc: d...@cloudstack.apache.org > > > > > > > > > > Subject: Re: DNS service on VR not responding > > > > > > > > > > > > > > > > > > > > Hi Santhosh, > > > > > > > > > > > > > > > > > > > > Good day to you, and thank you for your email. > > > > > > > > > > > > > > > > > > > > Traceroute packets seems to be dropped, I think it's by > > > > default. > > > > > > See > > > > > > > > > > result > > > > > > > > > > below: > > > > > > > > > > > > > > > > > > > > # traceroute X.X.X.2 > > > > > > > > > > traceroute to X.X.X.2 (X.X.X.2), 30 hops max, 60 byte > > packets > > > > > > > > > > 1 * * * > > > > > > > > > > 2 * * * > > > > > > > > > > 3 * * * > > > > > > > > > > > > > > > > > > > > However, I am able to ping, and there is a response when > I > > > > tried > > > > > to > > > > > > > > > > telnet to port 53. > > > > > > > > > > > > > > > > > > > > 64 bytes from X.X.X.2: icmp_req=4 ttl=64 time=2.00 ms > > > > > > > > > > 64 bytes from X.X.X.2: icmp_req=5 ttl=64 time=0.291 ms > > > > > > > > > > 64 bytes from X.X.X.2: icmp_req=6 ttl=64 time=0.384 ms ^C > > > > > > > > > > --- X.X.X.2 ping statistics --- > > > > > > > > > > 6 packets transmitted, 6 received, 0% packet loss, time > > > 4999ms > > > > > rtt > > > > > > > > > > min/avg/max/mdev = 0.270/0.603/2.006/0.628 ms > > > > > > > > > > > > > > > > > > > > # telnet X.X.X.2 53 > > > > > > > > > > Trying X.X.X.2... > > > > > > > > > > Connected to X.X.X.2. > > > > > > > > > > Escape character is '^]'. > > > > > > > > > > > > > > > > > > > > netstat -a on the VR shows the service is listening on > > domain > > > > > port > > > > > > > > (53). > > > > > > > > > > > > > > > > > > > > tcp 0 0 r-2606-VM:domain *:* > > > > > > > > > LISTEN > > > > > > > > > > > > > > > > > > > > tcp 0 0 X.X.X.2:domain *:* > > > > > > LISTEN > > > > > > > > > > > > > > > > > > > > udp 156992 0 r-2606-VM:domain *:* > > > > > > > > > > > > > > > > > > > > udp 164032 0 X.X.X.2:domain *:* > > > > > > > > > > > > > > > > > > > > Can you advise if there's anything else I need to check? > > > > > > > > > > > > > > > > > > > > Looking forward to your reply, thank you. > > > > > > > > > > > > > > > > > > > > Cheers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jul 21, 2014 at 1:17 AM, Santhosh Edukulla < > > > > > > > > > > santhosh.eduku...@citrix.com> wrote: > > > > > > > > > > > > > > > > > > > > > Run trace route from guest vms, the result will yield > to > > > the > > > > > > point > > > > > > > > > > > where packet drop is happening, could be a network acl > > rule > > > > > > issue, > > > > > > > > > > > but tracert command can lead to some answers. > > > > > > > > > > > > > > > > > > > > > > List running ports as well on VR, do a telnet to dns > port > > > on > > > > > > router > > > > > > > > > > > from guest vm to verify for its response. > > > > > > > > > > > > > > > > > > > > > > Santhosh > > > > > > > > > > > ________________________________________ > > > > > > > > > > > From: Indra Pramana [in...@sg.or.id] > > > > > > > > > > > Sent: Sunday, July 20, 2014 1:06 PM > > > > > > > > > > > To: users@cloudstack.apache.org > > > > > > > > > > > Cc: d...@cloudstack.apache.org > > > > > > > > > > > Subject: Re: DNS service on VR not responding > > > > > > > > > > > > > > > > > > > > > > Hi Rafael, > > > > > > > > > > > > > > > > > > > > > > Good day to you, and thank you for your reply. > > > > > > > > > > > > > > > > > > > > > > Can't find anything wrong on dnsmasq.log / daemon.log, > > just > > > > > some > > > > > > > log > > > > > > > > > > > entries related to DHCP, nothing on DNS. I masked the > IP > > > > > > addresses > > > > > > > > > > > since they are public. > > > > > > > > > > > > > > > > > > > > > > === > > > > > > > > > > > Jul 20 16:21:51 r-2606-VM dnsmasq[3519]: > > DHCPDISCOVER(eth0) > > > > > > X.X.X.X > > > > > > > > > > > 06:62:a8:01:13:37 > > > > > > > > > > > Jul 20 16:21:51 r-2606-VM dnsmasq[3519]: > DHCPOFFER(eth0) > > > > > X.X.X.X > > > > > > > > > > > 06:62:a8:01:13:37 > > > > > > > > > > > Jul 20 16:21:51 r-2606-VM dnsmasq[3519]: > > DHCPREQUEST(eth0) > > > > > > X.X.X.X > > > > > > > > > > > 06:62:a8:01:13:37 > > > > > > > > > > > Jul 20 16:21:51 r-2606-VM dnsmasq[3519]: DHCPACK(eth0) > > > > X.X.X.X > > > > > > > > > > > 06:62:a8:01:13:37 yyyyyy > > > > > > > > > > > Jul 20 16:23:53 r-2606-VM dnsmasq[3519]: > DHCPINFORM(eth0) > > > > > X.X.X.X > > > > > > > > > > > 06:43:4a:01:12:65 > > > > > > > > > > > Jul 20 16:23:53 r-2606-VM dnsmasq[3519]: DHCPACK(eth0) > > > > X.X.X.X > > > > > > > > > > > 06:43:4a:01:12:65 zzzzzz > > > > > > > > > > > === > > > > > > > > > > > > > > > > > > > > > > Yes, the guest VMs are having difficulties resolving > > > domains > > > > > into > > > > > > > IP > > > > > > > > > > > addresses because of the problem on the VR's DNS > server. > > > > > > > > > > > > > > > > > > > > > > $ host www.google.com X.X.X.X (where X.X.X.X is the IP > > > > address > > > > > > of > > > > > > > > > > > the > > > > > > > > > > VR) > > > > > > > > > > > ;; connection timed out; no servers could be reached > > > > > > > > > > > > > > > > > > > > > > However, from within the VR, I am able to resolve > domains > > > > just > > > > > > > fine. > > > > > > > > > > > > > > > > > > > > > > Any advise where can I start troubleshooting this? > > > > > > > > > > > > > > > > > > > > > > Looking forward to your reply, thank you. > > > > > > > > > > > > > > > > > > > > > > Cheers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jul 20, 2014 at 11:26 PM, Rafael Weingartner < > > > > > > > > > > > rafaelweingart...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > Have you taken a look at dnsmasq.log in the VR ? > > > > > > > > > > > > What do you mean with not responding? The addresses > are > > > not > > > > > > being > > > > > > > > > > > resolved > > > > > > > > > > > > to ip addresses? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jul 20, 2014 at 11:53 AM, Indra Pramana < > > > > > > in...@sg.or.id> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Dear all, > > > > > > > > > > > > > > > > > > > > > > > > > > All our guest VMs are having our virtual router > > (VR)'s > > > IP > > > > > > > > > > > > > address on /etc/resolv.conf. In the past two > weeks, I > > > > just > > > > > > > > > > > > > realised that the DNS service on the VR is not > > working, > > > > and > > > > > > > > > > > > > doesn't respond to DNS queries > > > > > > > > > > > from > > > > > > > > > > > > > the DNS clients on the guest VM. > > > > > > > > > > > > > > > > > > > > > > > > > > I have tried to stop and start back the VR, but the > > > > problem > > > > > > > > > persists. > > > > > > > > > > > > > > > > > > > > > > > > > > DHCP services seems to be running fine, only DNS > > > services > > > > > are > > > > > > > > > > > > > not > > > > > > > > > > > > working. > > > > > > > > > > > > > From what I understand, both services are provided > by > > > > > > dnsmasq, > > > > > > > > > > correct? > > > > > > > > > > > > > > > > > > > > > > > > > > Any advice on how can I resolve the problem? > > > > > > > > > > > > > > > > > > > > > > > > > > Looking forward to your reply, thank you. > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Rafael Weingärtner > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
