Hello,
I'm trying to understand this error, I created new network with VPC in
a new zone where I added KVM hypervisor, and I have this issue.
2021-09-13 11:36:18,621 DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:null) (logid:42d715f2) Trying to connect to
169.254.52.234
2021-09-13 11:36:21,684 DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:null) (logid:42d715f2) Could not connect to
169.254.52.234
Here is the full log : https://pastebin.com/0NwsccUc
root@usa-nyc-kvm-001:~# telnet 169.254.156.179 3922
Trying 169.254.156.179...
telnet: Unable to connect to remote host: Connection refused ( before
connection refused, I get no route to host, for sure the nic is not up )
root@usa-nyc-kvm-001:~# virsh list
Id Name State
---------------------------
2 v-4048-VM running
4 s-4050-VM running
7 r-4092-VM running
root@usa-nyc-kvm-001:~#
If I check the log inside the router, I see this (the router is also
responding very slow, waiting a lot to connect on it via virsh console):
r-4092-VM login: root
Password:
Login timed out after 60 seconds.
__?.o/ Apache CloudStack SystemVM 4.15.1
( )# https://cloudstack.apache.org
(___(_) Debian GNU/Linux 10 r-4093-VM ttyS0
r-4093-VM login: root
Password:
Linux r-4093-VM 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64
root@r-4093-VM:~# cat /var/log/cloud.log
Mon 13 Sep 2021 11:50:55 AM UTC Executing cloud-early-config
Mon 13 Sep 2021 11:50:55 AM UTC Bootstrapping systemvm appliance
Mon 13 Sep 2021 11:50:57 AM UTC Starting guest services for kvm
Mon 13 Sep 2021 11:51:04 AM UTC acpiphp and pci_hotplug module already
compiled in
Mon 13 Sep 2021 11:52:59 AM UTC Received a new non-empty cmdline file from
qemu-guest-agent
Mon 13 Sep 2021 11:52:59 AM UTC Booting from cloudstack, remove old
configuration files in /etc/cloudstack/
Mon 13 Sep 2021 11:54:30 AM UTC Configuring systemvm type=vpcrouter
Mon 13 Sep 2021 11:54:30 AM UTC Setting up VPC virtual router system vm
Mon 13 Sep 2021 11:54:31 AM UTC Setting up apache web server for VPC
Mon 13 Sep 2021 11:55:05 AM UTC Processors = 1 Enable service = 0
Mon 13 Sep 2021 11:55:05 AM UTC cloud: disable rp_filter
Mon 13 Sep 2021 11:55:05 AM UTC disable rpfilter
Mon 13 Sep 2021 11:55:05 AM UTC cloud: enable_fwding = 1
Mon 13 Sep 2021 11:55:05 AM UTC enable_fwding = 1
Mon 13 Sep 2021 11:55:05 AM UTC cloud: enabling passive FTP for guest VMs
Mon 13 Sep 2021 11:55:06 AM UTC Finished setting up systemvm
Mon 13 Sep 2021 11:55:06 AM UTC Finished setting up systemvm
2021-09-13 11:55:13,823 INFO update_config.py :: Processing incoming
file => cmd_line.json
2021-09-13 11:55:13,856 INFO Processing JSON file cmd_line.json
2021-09-13 11:55:13,858 INFO Continuing with the processing of file
'/var/cache/cloud/cmd_line.json'
2021-09-13 11:55:13,877 INFO Command of type cmdline received
2021-09-13 11:55:13,897 INFO Command of type ips received
2021-09-13 11:55:13,923 INFO Executing: ip addr show dev eth0
2021-09-13 11:55:13,975 INFO Executing: ip addr show dev eth0
2021-09-13 11:55:13,979 INFO Address found in DataBag ==> {u'public_ip':
u'169.254.41.129', u'one_to_one_nat': False, u'nic_dev_id': u'0',
u'network': u'169.254.0.0/16', u'netmask': u'255.255.0.0', u'source_nat':
False, u'broadcast': u'169.254.255.255', u'add': True, u'nw_type':
u'control', u'device': u'eth0', u'cidr': u'169.254.41.129/16', u'gateway':
u'', u'size': u'16'}
2021-09-13 11:55:13,979 INFO Address 169.254.41.129/16 on device eth0
already configured
2021-09-13 11:55:13,984 INFO Executing: ip addr show |grep -v secondary
2021-09-13 11:55:14,010 INFO Wrote edited file /etc/dnsmasq.d/cloud.conf
2021-09-13 11:55:14,010 INFO Updated file in-cache configuration
2021-09-13 11:55:14,010 INFO Nothing to commit. The /etc/dhcphosts.txt
file did not change
2021-09-13 11:55:14,010 INFO Nothing to commit. The
/var/lib/misc/dnsmasq.leases file did not change
2021-09-13 11:55:14,010 INFO Nothing to commit. The /etc/dhcpopts.txt
file did not change
2021-09-13 11:55:14,010 INFO Attempting to delete entries from
dnsmasq.leases file for VMs which are not on dhcphosts file
2021-09-13 11:55:14,010 ERROR Caught error while trying to delete entries
from dnsmasq.leases file: [Errno 2] No such file or directory:
'/etc/dhcphosts.txt'
2021-09-13 11:55:14,034 INFO Wrote edited file /etc/hosts
2021-09-13 11:55:14,044 INFO Updated file in-cache configuration
2021-09-13 11:55:14,044 INFO Updated hosts file
2021-09-13 11:55:14,046 INFO Executing: systemctl restart dnsmasq
2021-09-13 11:55:15,409 INFO Service dnsmasq restart
2021-09-13 11:55:15,453 INFO Executing: ip addr show |grep -v secondary
2021-09-13 11:55:15,832 INFO Nothing to commit. The
/etc/dnsmasq.d/cloud.conf file did not change
2021-09-13 11:55:15,957 INFO Nothing to commit. The /etc/dhcphosts.txt
file did not change
2021-09-13 11:55:15,963 INFO Nothing to commit. The
/var/lib/misc/dnsmasq.leases file did not change
2021-09-13 11:55:15,963 INFO Nothing to commit. The /etc/dhcpopts.txt
file did not change
2021-09-13 11:55:15,967 INFO Executing: systemctl is-active dnsmasq
2021-09-13 11:55:16,179 INFO Executing: systemctl reload dnsmasq
2021-09-13 11:55:16,665 INFO Service dnsmasq reload
2021-09-13 11:55:16,909 INFO Wrote edited file /etc/cron.d/process
2021-09-13 11:55:17,040 INFO Updated file in-cache configuration
2021-09-13 11:55:17,063 INFO Executing: iptables-save | grep
'^:FW_EGRESS_RULES' || iptables -t filter -N FW_EGRESS_RULES
2021-09-13 11:55:17,323 INFO Executing: iptables-save | grep '^-A
FW_EGRESS_RULES -j ACCEPT$' | sed 's/^-A/iptables -t filter -D/g' | bash
2021-09-13 11:55:17,697 INFO Executing: iptables -F FW_EGRESS_RULES
2021-09-13 11:55:17,990 INFO Executing: ipset -L | grep Name: | awk
{'print $2'} | ipset flush
2021-09-13 11:55:19,272 INFO Executing: ipset -L | grep Name: | awk
{'print $2'} | ipset destroy
2021-09-13 11:55:19,833 INFO Executing: iptables-save
2021-09-13 11:55:20,168 INFO Executing: iptables-save
2021-09-13 11:55:20,505 INFO Executing: ip6tables-save
2021-09-13 11:55:20,800 INFO Executing: systemctl stop conntrackd
2021-09-13 11:55:21,201 INFO Service conntrackd stop
2021-09-13 11:55:21,230 INFO Executing: systemctl stop keepalived
2021-09-13 11:55:21,578 INFO Service keepalived stop
2021-09-13 11:55:21,609 INFO Executing: mount
root@r-4093-VM:~#
Please ignore the router name, I did multiple tests to extract these logs, I
always see the same.
Regards,
Cristian
