Virtual Router /etc/hosts File
Hi all, I would like to know how to make my added IPs in the /etc/hosts file permanent. It seems every time cloudstack is restarted for some reason (power outage, ...) the IPs I added in the /etc/hosts file are lost. This is not related to the already existing configurations that exist in that file but instead, there are other IPs I add to that file that helps the NGINX proxy server I installed in the virtual router. Any thoughts or comments would be appreciated! Best regards,
Re: Virtual Router Domain Name Handling
Hi Andrija, The reverse proxy also was my first instinct, I already did it and it was successful, I used Nginx as a reverse proxy inside the VR and redirected the traffic to instances. I was hesitant too but I have to keep that in mind that they might get destroyed and I would have to reconfigure the proxy. Thanks. Regarding multiple emails, Sorry it is my frustration, sometimes I just remember a detail that I should have shared and then I end up sending another email, I’m going to avoid that in the future. Thank you so much you have been a helpful and wonderful community. Best Regards, On Wed, May 26, 2021 at 9:51 PM Andrija Panic wrote: > (and I don't know what is happening, but my Gmail sees multiple emails with > a same subject/duplicates coming from your side - please avoid sending > duplicate emails (if you are doing so), or it may be my Gmail after all...) > > > On Wed, 26 May 2021 at 21:48, Andrija Panic > wrote: > > > > > Based on your exact drawing (assumes a single public IP, and a single VR > > IP that receives all traffic on ports 80 and 443) - this can NOT be > > achieved by ACS itself. > > > > You need a reverse proxy inside VR - and you are risking having a mess - > > but I'm not saying you can't do it. > > All ACS port forwarding rules are in iptables (iptables-save | less) or > > loadbalancer implemented via HAPROXY can pass all traffic to a dedicated > > internal reverse proxy VM - so you forward 80/443 from home public IP > --> > > VR port forwarding --> to an internal single VM (that has nginx/varnish > > installed) > > If you opt to modify VR itself, that would be a problem to manage in the > > long run - as soon as you restart network with "clean up" - the existing > VR > > is destroyed and a brand new one is created for that network (or for VPC, > > if you are working with VPCs) > > So you would need to automate such thing in some way. > > > > Hope that helps, > > > > > > On Wed, 26 May 2021 at 16:39, Serge Byishimo > > wrote: > > > >> Centos8 > >> Cloudstack 4.15 > >> Hypervisor KVM > >> > >> This is a drawing of what I’m trying to do > >> > >> https://ibb.co/pKvzhHj > >> > >> I have different domain names that I want assigned to instances IP ports > >> > >> In the DNS, I put one public IP address that I have from ISP > 209.150.X.X, > >> > >> The cloudstack Virtual Router is at 192.168.1.86 > >> > >> Then in my home router I forward traffic from 209.150.X.X port 80 and > 443 > >> to the cloudstack virtual router at 192.168.1.86 also port 80 and 443 > >> > >> from 209.150.X.X :80 to 192.168.1.86 :80 > >> from 209.150.X.X :443 to 192.168.1.86 :443 > >> > >> How do I engage the virtual router to forward that traffic at respective > >> instances IP Ports? > >> > >> ACS Virtual Router Details: > >> https://ibb.co/1Jr2DZT > >> > >> > >> In other words, I need to know how to do port forwarding in the virtual > >> router > >> > >> I have Basic Network with Security Groups, I use Ingress Rules to allow > >> incoming traffic in the Instances > >> > >> For example: > >> > >> example.com to instance IP : 192.168.1.85:8080 > >> another.com to instance IP : 192.168.1.91:8443 > >> > >> I can ssh inside the Virtual Router VM, I just need to know what I > should > >> do! > >> > >> I was thinking on installing NGINX as a proxy on the ACS virtual router > VM > >> to forward the traffic it is receiving to Instances, but i’m not sure if > >> that will work. > >> > >> Thank you! > >> > > > > > > -- > > > > Andrija Panić > > > > > -- > > Andrija Panić > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Virtual Router Domain Name Handling
Centos8 Cloudstack 4.15 Hypervisor KVM This is a drawing of what I’m trying to do https://ibb.co/pKvzhHj I have different domain names that I want assigned to instances IP ports In the DNS, I put one public IP address that I have from ISP 209.150.X.X, The cloudstack Virtual Router is at 192.168.1.86 Then in my home router I forward traffic from 209.150.X.X port 80 and 443 to the cloudstack virtual router at 192.168.1.86 also port 80 and 443 from 209.150.X.X :80 to 192.168.1.86 :80 from 209.150.X.X :443 to 192.168.1.86 :443 How do I engage the virtual router to forward that traffic at respective instances IP Ports? ACS Virtual Router Details: https://ibb.co/1Jr2DZT In other words, I need to know how to do port forwarding in the virtual router I have Basic Network with Security Groups, I use Ingress Rules to allow incoming traffic in the Instances For example: example.com to instance IP : 192.168.1.85:8080 another.com to instance IP : 192.168.1.91:8443 I can ssh inside the Virtual Router VM, I just need to know what I should do! I was thinking on installing NGINX as a proxy on the ACS virtual router VM to forward the traffic it is receiving to Instances, but i’m not sure if that will work. Thank you!
Re: Virtual Router Domain Name Handling
This is a drawing of what I’m trying to do https://ibb.co/pKvzhHj I was thinking on installing NGINX as a proxy on the ACS virtual router VM to forward the traffic it is receiving to Instances, but i’m not sure if that will work. Regards, On Wed, May 26, 2021 at 10:38 AM Serge Byishimo wrote: > Virtual Router Details Image : > > https://ibb.co/1Jr2DZT > > On Wed, May 26, 2021 at 10:06 AM Serge Byishimo > wrote: > >> Centos8 >> Cloudstack: 4.15 >> Hypervisor: KVM >> >> I have different domain names that I want to be assigned to instances IP >> ports >> >> In the DNS, I put one public IP address that I have from the ISP >> 209.X.X.X, >> >> The cloudstack Virtual Router is at 192.168.1.86 >> >> Then in my home router, I forward traffic from 209.X.X.X port 80 and 443 >> to the cloudstack virtual router at 192.168.1.86 also port 80 and 443 >> >> from 209.X.X.X :80 to 192.168.1.86 :80 >> from 209.X.X.X :443 to 192.168.1.86 :443 >> >> How do I engage the virtual router to forward that traffic at respective >> instances IP Ports? >> >> In other words, I need to know how to do port forwarding in the virtual >> router >> >> I have Basic Network with Security Groups, I use Ingress Rules to allow >> incoming traffic in the Instances >> >> For example: >> >> example.com to instance IP : 192.168.1.85:8080 >> another.com to instance IP : 192.168.1.91:8443 >> >> I can ssh inside the Virtual Router VM, I just need to know what I should >> do! >> >> Thank you! >> >> Regards >> > > > -- > Byishimo Shema Serge > Kigali, Rwanda > Tel: +250 786 076 106 > www.serge.works > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Re: Virtual Router Domain Name Handling
Virtual Router Details Image : https://ibb.co/1Jr2DZT On Wed, May 26, 2021 at 10:06 AM Serge Byishimo wrote: > Centos8 > Cloudstack: 4.15 > Hypervisor: KVM > > I have different domain names that I want to be assigned to instances IP > ports > > In the DNS, I put one public IP address that I have from the ISP > 209.X.X.X, > > The cloudstack Virtual Router is at 192.168.1.86 > > Then in my home router, I forward traffic from 209.X.X.X port 80 and 443 > to the cloudstack virtual router at 192.168.1.86 also port 80 and 443 > > from 209.X.X.X :80 to 192.168.1.86 :80 > from 209.X.X.X :443 to 192.168.1.86 :443 > > How do I engage the virtual router to forward that traffic at respective > instances IP Ports? > > In other words, I need to know how to do port forwarding in the virtual > router > > I have Basic Network with Security Groups, I use Ingress Rules to allow > incoming traffic in the Instances > > For example: > > example.com to instance IP : 192.168.1.85:8080 > another.com to instance IP : 192.168.1.91:8443 > > I can ssh inside the Virtual Router VM, I just need to know what I should > do! > > Thank you! > > Regards > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Re: Virtual Router Domain Name Handling
Hi William, Sorry I didn't get what you meant but maybe it's because I didn't explain well, you can check my new email with the same subject. Kind Regards, On Tue, May 25, 2021 at 11:13 PM William Alvarez wrote: > Assign another public IP to the router and do port forwarding. Then have > your home router send the traffic to that ip > > Get Outlook for Android<https://aka.ms/AAb9ysg> > ____ > From: Serge Byishimo > Sent: Tuesday, May 25, 2021 11:10:07 PM > To: users@cloudstack.apache.org > Subject: Re: Virtual Router Domain Name Handling > > I have Basic Network with Security Groups, I use Ingress Rules to allow > incoming traffic in the Instances > > Thanks > > On Tue, May 25, 2021 at 10:02 PM Serge Byishimo > wrote: > > > In other words, I need to know how to do port forwarding in the virtual > > router > > > > Thanks, > > > > > > On Tue, May 25, 2021 at 9:52 PM Serge Byishimo > > wrote: > > > >> I have different domain names that I want to be assigned to instances IP > >> ports > >> > >> In the DNS, I put one public IP address that I have 209.X.X.X, > >> > >> Then in my home router, I forward traffic from port 80 and 443 to the > >> cloudstack virtual router at 192.168.1.86 > >> > >> How do I engage the router to forward that traffic at respective > >> instances IP ports? > >> > >> For example: > >> > >> example.com to instance IP port: 192.168.1.85:8080 > >> another.com to instance IP port: 192.168.1.91:8443 > >> > >> I can ssh inside the Virtual Router VM, I just need to know what I > should > >> do! > >> > >> Thank you! > >> > >> Regards, > >> > > > > > > -- > > Byishimo Shema Serge > > Kigali, Rwanda > > Tel: +250 786 076 106 > > www.serge.works<http://www.serge.works> > > > > > -- > Byishimo Shema Serge > Kigali, Rwanda > Tel: +250 786 076 106 > www.serge.works<http://www.serge.works> > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Virtual Router Domain Name Handling
Centos8 Cloudstack: 4.15 Hypervisor: KVM I have different domain names that I want to be assigned to instances IP ports In the DNS, I put one public IP address that I have from the ISP 209.X.X.X, The cloudstack Virtual Router is at 192.168.1.86 Then in my home router, I forward traffic from 209.X.X.X port 80 and 443 to the cloudstack virtual router at 192.168.1.86 also port 80 and 443 from 209.X.X.X :80 to 192.168.1.86 :80 from 209.X.X.X :443 to 192.168.1.86 :443 How do I engage the virtual router to forward that traffic at respective instances IP Ports? In other words, I need to know how to do port forwarding in the virtual router I have Basic Network with Security Groups, I use Ingress Rules to allow incoming traffic in the Instances For example: example.com to instance IP : 192.168.1.85:8080 another.com to instance IP : 192.168.1.91:8443 I can ssh inside the Virtual Router VM, I just need to know what I should do! Thank you! Regards
Re: Virtual Router Domain Name Handling
I have Basic Network with Security Groups, I use Ingress Rules to allow incoming traffic in the Instances Thanks On Tue, May 25, 2021 at 10:02 PM Serge Byishimo wrote: > In other words, I need to know how to do port forwarding in the virtual > router > > Thanks, > > > On Tue, May 25, 2021 at 9:52 PM Serge Byishimo > wrote: > >> I have different domain names that I want to be assigned to instances IP >> ports >> >> In the DNS, I put one public IP address that I have 209.X.X.X, >> >> Then in my home router, I forward traffic from port 80 and 443 to the >> cloudstack virtual router at 192.168.1.86 >> >> How do I engage the router to forward that traffic at respective >> instances IP ports? >> >> For example: >> >> example.com to instance IP port: 192.168.1.85:8080 >> another.com to instance IP port: 192.168.1.91:8443 >> >> I can ssh inside the Virtual Router VM, I just need to know what I should >> do! >> >> Thank you! >> >> Regards, >> > > > -- > Byishimo Shema Serge > Kigali, Rwanda > Tel: +250 786 076 106 > www.serge.works > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Re: Virtual Router Domain Name Handling
In other words, I need to know how to do port forwarding in the virtual router Thanks, On Tue, May 25, 2021 at 9:52 PM Serge Byishimo wrote: > I have different domain names that I want to be assigned to instances IP > ports > > In the DNS, I put one public IP address that I have 209.X.X.X, > > Then in my home router, I forward traffic from port 80 and 443 to the > cloudstack virtual router at 192.168.1.86 > > How do I engage the router to forward that traffic at respective instances > IP ports? > > For example: > > example.com to instance IP port: 192.168.1.85:8080 > another.com to instance IP port: 192.168.1.91:8443 > > I can ssh inside the Virtual Router VM, I just need to know what I should > do! > > Thank you! > > Regards, > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Virtual Router Domain Name Handling
I have different domain names that I want to be assigned to instances IP ports In the DNS, I put one public IP address that I have 209.X.X.X, Then in my home router, I forward traffic from port 80 and 443 to the cloudstack virtual router at 192.168.1.86 How do I engage the router to forward that traffic at respective instances IP ports? For example: example.com to instance IP port: 192.168.1.85:8080 another.com to instance IP port: 192.168.1.91:8443 I can ssh inside the Virtual Router VM, I just need to know what I should do! Thank you! Regards,
Re: Please Help 🙏🏼🙏🏼
to carry, what kind of ACS traffic?
> 4. Do you have an understanding of what happens when i.e. ACS creates a
> network for a tenant (say vlan 150) - what does ACS does, how it reads
> traffic label to see which bridge to look for, extract its child interface,
> then create VLAN 150 interface on top of that main interface, then create a
> new ("empty") bridge and joins this VLAN 150 interface to this new bridge,
> later joins VM's "vnet" interfaces to the same bridge, etc.
>
> Please do research a bit more, you seem to be missing a lot in network,
> and this is the usual place where people fail with ACS (where I also failed
> many years ago - so nothing uncommon...) - see this blog page which should
> explain a lot:
> - https://www.shapeblue.com/a-beginners-guide-to-cloudstack-networking/
> -
>
> https://www.shapeblue.com/understanding-cloudstacks-physical-networking-architecture/
>
> I really hope that helps - please invest some time into understanding this
> low-level, otherwise, your would be playing Russian roulette - it may to
> may not work...
>
> Cheers,
>
> On Mon, 24 May 2021 at 11:28, Serge Byishimo
> wrote:
>
> > I changed the traffic Label in the old UI for management, guest, storage
> > and public network to cloudbr0 as this bridge exists in the host (I have
> > one Host), as you can see in below images
> >
> > and then I restarted the System VMs, but I can’t telnet the Console proxy
> > ports 80, 443 from the management server on Console Proxy Public IP
> address
> >
> >
> > Do I have to reinstall the whole set up using the old UI?
> >
> > [root@srvr1 ~]#
> > [root@srvr1 ~]# telnet 192.168.1.66 80
> > Trying 192.168.1.66...
> > telnet: connect to address 192.168.1.66: No route to host
> > [root@srvr1 ~]#
> > [root@srvr1 ~]#
> > [root@srvr1 ~]# telnet 192.168.1.66 80
> > Trying 192.168.1.66...
> > telnet: connect to address 192.168.1.66: No route to host
> > [root@srvr1 ~]#
> > [root@srvr1 ~]# telnet 192.168.1.66 443
> > Trying 192.168.1.66...
> > telnet: connect to address 192.168.1.66: No route to host
> > [root@srvr1 ~]#
> > [root@srvr1 ~]# telnet 192.168.1.66 443
> > Trying 192.168.1.66...
> > telnet: connect to address 192.168.1.66: No route to host
> > [root@srvr1 ~]#
> >
> >
> >
> > Management Traffic Labels
> >
> > https://ibb.co/R6M861X
> >
> > Guest Traffic Labels
> >
> > https://ibb.co/9ynMBm0
> >
> >
> > Public Traffic Labels
> >
> > https://ibb.co/6YvjxgC
> >
> > Storage Traffic Labels
> >
> > https://ibb.co/99SMsT9
> >
> > Physical Network
> >
> > https://ibb.co/TL6T5yZ
> >
> > ==
> >
> > Host IFCONFIG
> >
> >
> > [root@srvr2 ~]#
> > [root@srvr2 ~]# ifconfig
> > br-200: flags=4099 mtu 1500
> > ether c2:0e:f3:be:8b:14 txqueuelen 1000 (Ethernet)
> > RX packets 0 bytes 0 (0.0 B)
> > RX errors 0 dropped 0 overruns 0 frame 0
> > TX packets 0 bytes 0 (0.0 B)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
> >
> > brenp0s25-200: flags=4163 mtu 1500
> > inet6 fe80::fc9c:5dff:fe6d:cd99 prefixlen 64 scopeid 0x20
> > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet)
> > RX packets 153429 bytes 5240417 (4.9 MiB)
> > RX errors 0 dropped 0 overruns 0 frame 0
> > TX packets 81 bytes 5926 (5.7 KiB)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
> >
> > cloud0: flags=4163 mtu 1500
> > inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0
> > inet6 fe80::c77:fbff:fe37:94cf prefixlen 64 scopeid 0x20
> > ether fe:00:a9:fe:4a:77 txqueuelen 1000 (Ethernet)
> > RX packets 51071 bytes 8524065 (8.1 MiB)
> > RX errors 0 dropped 0 overruns 0 frame 0
> > TX packets 59248 bytes 8380459 (7.9 MiB)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
> >
> > cloudbr0: flags=4163 mtu 1500
> > inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255
> > inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid 0x20
> > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet)
> > RX packets 1720903 bytes 13132311620 (12.2 GiB)
> > RX errors 0 dropped 67104 overruns 0 frame 0
> > TX packets 1468695 bytes 6965166448 (6.4 GiB)
> > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
> >
> &
Re: Please Help 🙏🏼🙏🏼
Dear Andrija, I replied, I'm ready to do whatever is necessary as I'm new to apache cloudstack but I've learned a lot. Thank you. On Mon, May 24, 2021 at 11:04 AM Andrija Panic wrote: > Read my reply to your latest email - you have not set (based on your > screenshots) traffic labels on each guest type - so you can expect that > NOTHING will work as VLAN interfaces that ACS provisions dynamically will > be created on wrong bridges if any, etc > > Best, > > On Mon, 24 May 2021 at 10:58, Serge Byishimo > wrote: > > > Hi Andrija, > > > > So how is Console Proxy SSVM supposed to open the web console if it is > > unreachable? > > > > Through open ports? > > > > Coz I tried to telnet open ports in the SSVM from the management server > and > > it doesn't work? > > > > Regards, > > > > > > On Mon, May 24, 2021 at 10:55 AM Andrija Panic > > wrote: > > > > > SSVM and CPVM do not respond to pings on its Public IP address (VRs > do) - > > > so yes, that is expected. > > > > > > Check and temp-turn-off all firewalls, to rule out this, as this is a > > usual > > > thing to start troubleshooting with. > > > > > > Best, > > > > > > On Sun, 23 May 2021 at 19:37, Serge Byishimo > > > wrote: > > > > > > > Centos8 > > > > Cloudstack 4.15 > > > > > > > > Here is my scenario > > > > > > > > I have two servers, one for the host and another for the management > > > server > > > > > > > > I used Advanced Networking Configuration and I can SSH Inside the > > System > > > > VMs, > > > > > > > > Inside console proxy VM I can Ping the Management Server > > > > > > > > but the problem is I can not ping the console proxy from the > management > > > > server, > > > > > > > > Is that how it is supposed to work? > > > > > > > > because I can't open the web console, it is timing out trying to > > connect > > > to > > > > it's public IP! > > > > > > > > > > > > > > > > root@v-2-VM:~# > > > > root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh > > > > > > > > First DNS server is 8.8.8.8 > > > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > > > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=32.198 ms > > > > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=33.279 ms > > > > --- 8.8.8.8 ping statistics --- > > > > 2 packets transmitted, 2 packets received, 0% packet loss > > > > round-trip min/avg/max/stddev = 32.198/32.739/33.279/0.541 ms > > > > Good: Can ping DNS server > > > > > > > > Good: DNS resolves cloudstack.apache.org > > > > > > > > nfs is currently mounted > > > > Mount point is /mnt/SecStorage/3636e78b-77d4-3562-995e-deb81e266bce > > > > Good: Can write to mount point > > > > > > > > Management server is 192.168.1.8. Checking connectivity. > > > > Good: Can connect to management server 192.168.1.8 port 8250 > > > > > > > > Good: Java process is running > > > > > > > > Tests Complete. Look for ERROR or WARNING above. > > > > root@v-2-VM:~# > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind Regards, > > > > > > > > > > > > > -- > > > > > > Andrija Panić > > > > > > > > > -- > > Byishimo Shema Serge > > Kigali, Rwanda > > Tel: +250 786 076 106 > > www.serge.works > > > > > -- > > Andrija Panić > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Re: Please Help 🙏🏼🙏🏼
16 bytes 1208 (1.1 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet1: flags=4163 mtu 1500 inet6 fe80::fc00:67ff:fe00:6 prefixlen 64 scopeid 0x20 ether fe:00:67:00:00:06 txqueuelen 1000 (Ethernet) RX packets 833 bytes 155599 (151.9 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1500 bytes 221292 (216.1 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet2: flags=4163 mtu 1500 inet6 fe80::fc00:d6ff:fe00:11 prefixlen 64 scopeid 0x20 ether fe:00:d6:00:00:11 txqueuelen 1000 (Ethernet) RX packets 66 bytes 2772 (2.7 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 64 bytes 3148 (3.0 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet3: flags=4163 mtu 1500 inet6 fe80::fc00:a9ff:fefe:e245 prefixlen 64 scopeid 0x20 ether fe:00:a9:fe:e2:45 txqueuelen 1000 (Ethernet) RX packets 161 bytes 19311 (18.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 206 bytes 16257 (15.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet4: flags=4163 mtu 1500 inet6 fe80::fc00:48ff:fe00:7 prefixlen 64 scopeid 0x20 ether fe:00:48:00:00:07 txqueuelen 1000 (Ethernet) RX packets 352 bytes 39104 (38.1 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1029 bytes 75691 (73.9 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet5: flags=4163 mtu 1500 inet6 fe80::fc00:dbff:fe00:12 prefixlen 64 scopeid 0x20 ether fe:00:db:00:00:12 txqueuelen 1000 (Ethernet) RX packets 51 bytes 2142 (2.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 64 bytes 3148 (3.0 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@srvr2 ~]# Bridge Configuration [root@srvr2 network-scripts]# [root@srvr2 network-scripts]# cat ifcfg-enp0s25 TYPE="Ethernet" BOOTPROTO="none" NAME="enp0s25" IPADDR="192.168.1.7" NETMASK="255.255.255.0" GATEWAY=192.168.1.1 DNS=192.168.1.1 DEVICE="enp0s25" ONBOOT="yes" BRIDGE=cloudbr0 HOTPLUG=no [root@srvr2 network-scripts]# [root@srvr2 network-scripts]# [root@srvr2 network-scripts]# [root@srvr2 network-scripts]# cat ifcfg-cloudbr0 DEVICE=cloudbr0 TYPE=Bridge ONBOOT=yes BOOTPROTO=none IPV6INIT=no IPV6_AUTOCONF=no DELAY=5 IPADDR=192.168.1.7 GATEWAY=192.168.1.1 NETMASK=255.255.255.0 STP=yes [root@srvr2 network-scripts]# [root@srvr2 network-scripts]# [root@srvr2 network-scripts]# cat ifcfg-enp0s25.200 DEVICE=enp0s25.200 ONBOOT=yes HOTPLUG=no BOOTPROTO=none TYPE=Ethernet VLAN=yes BRIDGE=cloudbr1 [root@srvr2 network-scripts]# On Mon, May 24, 2021 at 11:02 AM Andrija Panic wrote: > Based on your screenshots, you have NOT set "Traffic Label" for each > traffic type - this "traffic lab" should be == name of the bridge - so > CloudStack knows which bridge to use for specific VLAN interface > provisioning etc - this is the reason why nothing works in your env, > probably. > > Re the firewall, I guess reading the 2 following rules should tell you that > Console traffic to port 80 and 443 is allowed: > > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:443 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:80 > > Go to old UI, and set traffic labels for Management, Storage, Guest and > Public traffic types to be exactly the same as the name of the bridge that > you want to use for each traffic type (cloudb0 or cloudbr1 in your case - > but you need to understand what you want to do, via which bridge, and the > underlying VLAN physical interfaces on your hypervisors, and if your > switches support trunking mode (to be able to send all VLANs between > different hypervisor hosts etc) > > Best, > > On Mon, 24 May 2021 at 09:58, Serge Byishimo > wrote: > > > Hi Rohit, > > > > Please let me know how I should reset the network coz I did it multiple > > times, > > > > The public traffic type VLAN/VNI is vlan://untagged as you can see in the > > Public Traffic Type Image (https://ibb.co/pZyJ74x) , but while > configuring > > the Advanced Network Configuration they asked me for a VLAN / VNI range > and > > I Put 150-200 as you can see on the physical network 1 image ( > > https://ibb.co/BKnD79B) > > > > > > Zone Details > > > > https://ibb.co/RpHz3pG > > > > > > Physical Network 1 > > > > https://ibb.co/BKnD79B > > > > > > Guest Traffic Type > > > > https://ibb.co/JC7MRW9 > > > > Management Traffic Type >
Re: Please Help 🙏🏼🙏🏼
Hi Andrija, So how is Console Proxy SSVM supposed to open the web console if it is unreachable? Through open ports? Coz I tried to telnet open ports in the SSVM from the management server and it doesn't work? Regards, On Mon, May 24, 2021 at 10:55 AM Andrija Panic wrote: > SSVM and CPVM do not respond to pings on its Public IP address (VRs do) - > so yes, that is expected. > > Check and temp-turn-off all firewalls, to rule out this, as this is a usual > thing to start troubleshooting with. > > Best, > > On Sun, 23 May 2021 at 19:37, Serge Byishimo > wrote: > > > Centos8 > > Cloudstack 4.15 > > > > Here is my scenario > > > > I have two servers, one for the host and another for the management > server > > > > I used Advanced Networking Configuration and I can SSH Inside the System > > VMs, > > > > Inside console proxy VM I can Ping the Management Server > > > > but the problem is I can not ping the console proxy from the management > > server, > > > > Is that how it is supposed to work? > > > > because I can't open the web console, it is timing out trying to connect > to > > it's public IP! > > > > > > > > root@v-2-VM:~# > > root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh > > > > First DNS server is 8.8.8.8 > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=32.198 ms > > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=33.279 ms > > --- 8.8.8.8 ping statistics --- > > 2 packets transmitted, 2 packets received, 0% packet loss > > round-trip min/avg/max/stddev = 32.198/32.739/33.279/0.541 ms > > Good: Can ping DNS server > > > > Good: DNS resolves cloudstack.apache.org > > > > nfs is currently mounted > > Mount point is /mnt/SecStorage/3636e78b-77d4-3562-995e-deb81e266bce > > Good: Can write to mount point > > > > Management server is 192.168.1.8. Checking connectivity. > > Good: Can connect to management server 192.168.1.8 port 8250 > > > > Good: Java process is running > > > > Tests Complete. Look for ERROR or WARNING above. > > root@v-2-VM:~# > > > > > > > > > > > > > > > > Kind Regards, > > > > > -- > > Andrija Panić > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Re: Please Help 🙏🏼🙏🏼
Hi Rohit, Please let me know how I should reset the network coz I did it multiple times, The public traffic type VLAN/VNI is vlan://untagged as you can see in the Public Traffic Type Image (https://ibb.co/pZyJ74x) , but while configuring the Advanced Network Configuration they asked me for a VLAN / VNI range and I Put 150-200 as you can see on the physical network 1 image ( https://ibb.co/BKnD79B) Zone Details https://ibb.co/RpHz3pG Physical Network 1 https://ibb.co/BKnD79B Guest Traffic Type https://ibb.co/JC7MRW9 Management Traffic Type https://ibb.co/2Pg67D8 Public Traffic Type https://ibb.co/pZyJ74x https://ibb.co/X5cn2qm Storage Traffic Type https://ibb.co/PrxJBfN Console Proxy Details https://ibb.co/qkHFS4D Thank you, Kind Regards, On Mon, May 24, 2021 at 9:36 AM Rohit Yadav wrote: > Hi Serge, > > Are you using VLAN 200 for your public IP range? You either need to fix > your switches or remove the VLAN. Go to UI -> Zone -> physical network -> > and check the IP range for public management traffic type. If that's the > case, disable the zone and then destroy your systemvms and then remove the > IP range and readd the range with the VLAN untagged or blank. > > Regards. > > Regards, > Rohit Yadav > > > From: Serge Byishimo > Sent: Monday, May 24, 2021 12:58:42 PM > To: users@cloudstack.apache.org > Subject: Please Help 🙏🏼🙏🏼 > > Centos8 > Cloudstack 4.15 > > System VMs are running and their Agent is UP, > > System VMs can ping the management server but the management server can not > ping any of the IPs of the System VMs, thus the web console is not > accessible, > > I’m thinking it is firewall rules on the System VM side that is causing it, > > Any reasons why this is happening? > > > > Console Proxy > > > root@v-2-VM:~# > root@v-2-VM:~# > root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh > > First DNS server is 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=29.581 ms > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=30.140 ms > --- 8.8.8.8 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max/stddev = 29.581/29.861/30.140/0.280 ms > Good: Can ping DNS server > > Good: DNS resolves cloudstack.apache.org > > ERROR: NFS is not currently mounted > Try manually mounting from inside the VM > NFS server is 255.255.255.0 > PING 255.255.255.0 (255.255.255.0): 56 data bytes > --- 255.255.255.0 ping statistics --- > 2 packets transmitted, 0 packets received, 100% packet loss > WARNING: cannot ping nfs server > routing table follows > Kernel IP routing table > Destination Gateway Genmask Flags Metric RefUse > Iface > 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 > eth2 > 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 > eth0 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth1 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth2 > > Management server is 192.168.1.8. Checking connectivity. > Good: Can connect to management server 192.168.1.8 port 8250 > > Good: Java process is running > > Tests Complete. Look for ERROR or WARNING above. > root@v-2-VM:~# > > > > > > root@v-2-VM:~# > root@v-2-VM:~# iptables -nL > Chain INPUT (policy DROP) > target prot opt source destination > ACCEPT all -- 0.0.0.0/00.0.0.0/0 > ACCEPT icmp -- 0.0.0.0/00.0.0.0/0icmptype 8 > ACCEPT all -- 0.0.0.0/00.0.0.0/0state > RELATED,ESTABLISHED > ACCEPT all -- 0.0.0.0/00.0.0.0/0state > RELATED,ESTABLISHED > ACCEPT all -- 0.0.0.0/00.0.0.0/0state > RELATED,ESTABLISHED > ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:3922 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:8001 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:8001 > ACCE
Please Help 🙏🏼🙏🏼
Centos8 Cloudstack 4.15 System VMs are running and their Agent is UP, System VMs can ping the management server but the management server can not ping any of the IPs of the System VMs, thus the web console is not accessible, I’m thinking it is firewall rules on the System VM side that is causing it, Any reasons why this is happening? Console Proxy root@v-2-VM:~# root@v-2-VM:~# root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=29.581 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=30.140 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 29.581/29.861/30.140/0.280 ms Good: Can ping DNS server Good: DNS resolves cloudstack.apache.org ERROR: NFS is not currently mounted Try manually mounting from inside the VM NFS server is 255.255.255.0 PING 255.255.255.0 (255.255.255.0): 56 data bytes --- 255.255.255.0 ping statistics --- 2 packets transmitted, 0 packets received, 100% packet loss WARNING: cannot ping nfs server routing table follows Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 eth2 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 eth1 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 eth0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 Management server is 192.168.1.8. Checking connectivity. Good: Can connect to management server 192.168.1.8 port 8250 Good: Java process is running Tests Complete. Look for ERROR or WARNING above. root@v-2-VM:~# root@v-2-VM:~# root@v-2-VM:~# iptables -nL Chain INPUT (policy DROP) target prot opt source destination ACCEPT all -- 0.0.0.0/00.0.0.0/0 ACCEPT icmp -- 0.0.0.0/00.0.0.0/0icmptype 8 ACCEPT all -- 0.0.0.0/00.0.0.0/0state RELATED,ESTABLISHED ACCEPT all -- 0.0.0.0/00.0.0.0/0state RELATED,ESTABLISHED ACCEPT all -- 0.0.0.0/00.0.0.0/0state RELATED,ESTABLISHED ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:3922 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:8001 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:8001 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:443 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:80 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:8080 ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 Chain FORWARD (policy DROP) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination root@v-2-VM:~# root@v-2-VM:~# Host Network Configuration [root@srvr2 byishimo]# [root@srvr2 byishimo]# ifconfig br-200: flags=4099 mtu 1500 ether c2:0e:f3:be:8b:14 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 brenp0s25-200: flags=4163 mtu 1500 inet6 fe80::fc9c:5dff:fe6d:cd99 prefixlen 64 scopeid 0x20 ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) RX packets 151060 bytes 5159307 (4.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 77 bytes 5606 (5.4 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 cloud0: flags=4163 mtu 1500 inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 inet6 fe80::c77:fbff:fe37:94cf prefixlen 64 scopeid 0x20 ether fe:00:a9:fe:10:72 txqueuelen 1000 (Ethernet) RX packets 50364 bytes 8410715 (8.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 58434 bytes 8258165 (7.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 cloudbr0: flags=4163 mtu 1500 inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid 0x20 ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet)
Please Help 🙏🏼🙏🏼
Centos8 Cloudstack 4.15 Here is my scenario I have two servers, one for the host and another for the management server I used Advanced Networking Configuration and I can SSH Inside the System VMs, Inside console proxy VM I can Ping the Management Server but the problem is I can not ping the console proxy from the management server, Is that how it is supposed to work? because I can't open the web console, it is timing out trying to connect to it's public IP! root@v-2-VM:~# root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=32.198 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=33.279 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 32.198/32.739/33.279/0.541 ms Good: Can ping DNS server Good: DNS resolves cloudstack.apache.org nfs is currently mounted Mount point is /mnt/SecStorage/3636e78b-77d4-3562-995e-deb81e266bce Good: Can write to mount point Management server is 192.168.1.8. Checking connectivity. Good: Can connect to management server 192.168.1.8 port 8250 Good: Java process is running Tests Complete. Look for ERROR or WARNING above. root@v-2-VM:~# Kind Regards,
Re: Console IP is unreachable, but it’s Private IP is reachable
Thank you Rohit, the issue was resolved, I have another issue where when you open the console proxy on the instance, it times out !! and the management server is not able to ping the console proxy VM I can SSH inside it and It has this error : root@v-2-VM:~# root@v-2-VM:~# root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=29.581 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=30.140 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 29.581/29.861/30.140/0.280 ms Good: Can ping DNS server Good: DNS resolves cloudstack.apache.org ERROR: NFS is not currently mounted Try manually mounting from inside the VM NFS server is 255.255.255.0 PING 255.255.255.0 (255.255.255.0): 56 data bytes --- 255.255.255.0 ping statistics --- 2 packets transmitted, 0 packets received, 100% packet loss WARNING: cannot ping nfs server routing table follows Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 eth2 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 eth1 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 eth0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 Management server is 192.168.1.8. Checking connectivity. Good: Can connect to management server 192.168.1.8 port 8250 Good: Java process is running Tests Complete. Look for ERROR or WARNING above. root@v-2-VM:~# Kind Regards, On Sun, May 23, 2021 at 6:08 PM Rohit Yadav wrote: > Hi Serge, > > Have you added the public IP range with a VLAN (say while deploying the > zone)? You can ping your CPVM IP and use tcpdump on protocol icmp for > example "tcpdump -i icmp" across all hosts (start with the > KVM/CentOS host where the cpvm is running) and debug where the packet is > being dropped. > > > Regards. > > ____ > From: Serge Byishimo > Sent: Sunday, May 23, 2021 12:59 > To: users@cloudstack.apache.org > Subject: Console IP is unreachable, but it’s Private IP is reachable > > Centos8 > Cloudstack 4.15 > > My console IP address 192.168.1.86 is unreachable from anywhere > But it’s Private IP 192.168.1.68 is reachable EVERYWHERE > > The console also has access to the Internet. > > Network Setup : > > 2 servers : they are on a home Local Network router with internet > Home Router Gateway : 192.168.1.1 > Home Router Netmask : 255.255.255.0 > Home Route IP Range : 192.168.1.2 - .61 > > 1st server : management server > IP : 192.168.1.8 > > > == > > 2nd server : Host > IP : 192.168.1.7 > > > [root@srvr2 ~]# > [root@srvr2 ~]# ifconfig > br-200: flags=4099 mtu 1500 > ether c2:0e:f3:be:8b:14 txqueuelen 1000 (Ethernet) > RX packets 0 bytes 0 (0.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 0 bytes 0 (0.0 B) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > brenp0s25-200: flags=4163 mtu 1500 > inet6 fe80::fc9c:5dff:fe6d:cd99 prefixlen 64 scopeid 0x20 > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > RX packets 80947 bytes 2821053 (2.6 MiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 50 bytes 3636 (3.5 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > cloud0: flags=4163 mtu 1500 > inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 > inet6 fe80::c77:fbff:fe37:94cf prefixlen 64 scopeid 0x20 > ether fe:00:a9:fe:17:7d txqueuelen 1000 (Ethernet) > RX packets 27169 bytes 4740900 (4.5 MiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 30778 bytes 4633136 (4.4 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > cloudbr0: flags=4163 mtu 1500 > inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid 0x20 > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > RX packets 701422 bytes 1846775312 (1.7 GiB) > RX errors 0
Re: Help : Console Proxy SSVM Check NFS Error
I can Ping NFS Server from Console Proxy SSVM root@v-2-VM:~# root@v-2-VM:~# root@v-2-VM:~# ping 192.168.1.7 PING 192.168.1.7 (192.168.1.7): 56 data bytes 64 bytes from 192.168.1.7: icmp_seq=0 ttl=64 time=0.215 ms 64 bytes from 192.168.1.7: icmp_seq=1 ttl=64 time=0.207 ms 64 bytes from 192.168.1.7: icmp_seq=2 ttl=64 time=0.173 ms 64 bytes from 192.168.1.7: icmp_seq=3 ttl=64 time=0.158 ms 64 bytes from 192.168.1.7: icmp_seq=4 ttl=64 time=0.197 ms 64 bytes from 192.168.1.7: icmp_seq=5 ttl=64 time=0.196 ms 64 bytes from 192.168.1.7: icmp_seq=6 ttl=64 time=0.149 ms 64 bytes from 192.168.1.7: icmp_seq=7 ttl=64 time=0.216 ms ^C--- 192.168.1.7 ping statistics --- 8 packets transmitted, 8 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.149/0.189/0.216/0.024 ms root@v-2-VM:~# On Sun, May 23, 2021 at 5:29 PM Serge Byishimo wrote: > Route Table > > root@v-2-VM:~# route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric RefUse > Iface > 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 > eth2 > 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 > eth0 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth1 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth2 > root@v-2-VM:~# > > The NFS Server is at 192.168.1.7 > > > > On Sun, May 23, 2021 at 5:27 PM Serge Byishimo > wrote: > >> Here are some log files >> >> /var/log/cloud.log >> >> >> root@v-2-VM:/var/log# cat cloud.log >> Sun 23 May 2021 03:17:36 PM UTC Executing cloud-early-config >> Sun 23 May 2021 03:17:36 PM UTC Scripts checksum detected: >> oldmd5=4fc9a03fc8f196d863d30885e9a0c50a >> newmd5=032ab7551bf9db6662808252305f4ddf >> Sun 23 May 2021 03:17:36 PM UTC Patched scripts using >> /media/cdrom/cloud-scripts.tgz >> Sun 23 May 2021 03:17:36 PM UTC Bootstrapping systemvm appliance >> Sun 23 May 2021 03:17:36 PM UTC Starting guest services for kvm >> Sun 23 May 2021 03:17:37 PM UTC acpiphp and pci_hotplug module already >> compiled in >> Sun 23 May 2021 03:17:37 PM UTC Received a new non-empty cmdline file >> from qemu-guest-agent >> Sun 23 May 2021 03:17:37 PM UTC Booting from cloudstack, remove old >> configuration files in /etc/cloudstack/ >> Sun 23 May 2021 03:17:39 PM UTC Configuring systemvm type=consoleproxy >> Sun 23 May 2021 03:17:39 PM UTC Setting up console proxy system vm >> Sun 23 May 2021 03:17:39 PM UTC Not setting up route of RFC1918 space to >> 192.168.1.1 befause 192.168.1.66 is RFC1918. >> Sun 23 May 2021 03:17:39 PM UTC Setting up entry in hosts >> Sun 23 May 2021 03:17:39 PM UTC Applying iptables rules >> Sun 23 May 2021 03:17:39 PM UTC Configuring sshd >> Sun 23 May 2021 03:17:39 PM UTC cloud: disable rp_filter >> Sun 23 May 2021 03:17:39 PM UTC disable rpfilter >> Sun 23 May 2021 03:17:39 PM UTC cloud: enable_fwding = 0 >> Sun 23 May 2021 03:17:39 PM UTC enable_fwding = 0 >> Sun 23 May 2021 03:17:39 PM UTC Processors = 1 Enable service = 0 >> Sun 23 May 2021 03:17:39 PM UTC Finished setting up systemvm >> Sun 23 May 2021 03:17:39 PM UTC Finished setting up systemvm >> 2021-05-23 15:17:44,467 INFO [cloud.agent.AgentShell] (main:null) Agent >> started >> 2021-05-23 15:17:44,484 INFO [cloud.agent.AgentShell] (main:null) >> Implementation Version is 4.15.0.0 >> 2021-05-23 15:17:44,489 INFO [cloud.agent.AgentShell] (main:null) >> agent.properties found at /usr/local/cloud/systemvm/conf/agent.properties >> 2021-05-23 15:17:44,523 INFO [cloud.agent.AgentShell] (main:null) >> Defaulting to using properties file for storage >> 2021-05-23 15:17:44,524 INFO [cloud.agent.AgentShell] (main:null) >> Defaulting to the constant time backoff algorithm >> 2021-05-23 15:17:44,535 INFO [cloud.utils.LogUtils] (main:null) log4j >> configuration found at /usr/local/cloud/systemvm/conf/log4j-cloud.xml >> 2021-05-23 15:17:44,571 INFO [cloud.agent.AgentShell] (main:null) Using >> default Java settings for IPv6 preference for agent connection >> 2021-05-23 15:17:44,763 INFO [cloud.agent.Agent] (main:null) id is >> 2021-05-23 15:17:44,884 INFO >> [resource.consoleproxy.ConsoleProxyResource] (main:null) Receive proxyVmId >> in ConsoleProxyResource configuration as 2 >> 2021-05-23 15:17:44,928 INFO [cloud.agent.Agent] (main:null) Agent [id = >> new : type = ConsoleProxyResource : zone = 1 : pod = 1 : workers = 5 : host >> = 192.168.
Re: Help : Console Proxy SSVM Check NFS Error
Route Table root@v-2-VM:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 eth2 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 eth1 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 eth0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 root@v-2-VM:~# The NFS Server is at 192.168.1.7 On Sun, May 23, 2021 at 5:27 PM Serge Byishimo wrote: > Here are some log files > > /var/log/cloud.log > > > root@v-2-VM:/var/log# cat cloud.log > Sun 23 May 2021 03:17:36 PM UTC Executing cloud-early-config > Sun 23 May 2021 03:17:36 PM UTC Scripts checksum detected: > oldmd5=4fc9a03fc8f196d863d30885e9a0c50a > newmd5=032ab7551bf9db6662808252305f4ddf > Sun 23 May 2021 03:17:36 PM UTC Patched scripts using > /media/cdrom/cloud-scripts.tgz > Sun 23 May 2021 03:17:36 PM UTC Bootstrapping systemvm appliance > Sun 23 May 2021 03:17:36 PM UTC Starting guest services for kvm > Sun 23 May 2021 03:17:37 PM UTC acpiphp and pci_hotplug module already > compiled in > Sun 23 May 2021 03:17:37 PM UTC Received a new non-empty cmdline file from > qemu-guest-agent > Sun 23 May 2021 03:17:37 PM UTC Booting from cloudstack, remove old > configuration files in /etc/cloudstack/ > Sun 23 May 2021 03:17:39 PM UTC Configuring systemvm type=consoleproxy > Sun 23 May 2021 03:17:39 PM UTC Setting up console proxy system vm > Sun 23 May 2021 03:17:39 PM UTC Not setting up route of RFC1918 space to > 192.168.1.1 befause 192.168.1.66 is RFC1918. > Sun 23 May 2021 03:17:39 PM UTC Setting up entry in hosts > Sun 23 May 2021 03:17:39 PM UTC Applying iptables rules > Sun 23 May 2021 03:17:39 PM UTC Configuring sshd > Sun 23 May 2021 03:17:39 PM UTC cloud: disable rp_filter > Sun 23 May 2021 03:17:39 PM UTC disable rpfilter > Sun 23 May 2021 03:17:39 PM UTC cloud: enable_fwding = 0 > Sun 23 May 2021 03:17:39 PM UTC enable_fwding = 0 > Sun 23 May 2021 03:17:39 PM UTC Processors = 1 Enable service = 0 > Sun 23 May 2021 03:17:39 PM UTC Finished setting up systemvm > Sun 23 May 2021 03:17:39 PM UTC Finished setting up systemvm > 2021-05-23 15:17:44,467 INFO [cloud.agent.AgentShell] (main:null) Agent > started > 2021-05-23 15:17:44,484 INFO [cloud.agent.AgentShell] (main:null) > Implementation Version is 4.15.0.0 > 2021-05-23 15:17:44,489 INFO [cloud.agent.AgentShell] (main:null) > agent.properties found at /usr/local/cloud/systemvm/conf/agent.properties > 2021-05-23 15:17:44,523 INFO [cloud.agent.AgentShell] (main:null) > Defaulting to using properties file for storage > 2021-05-23 15:17:44,524 INFO [cloud.agent.AgentShell] (main:null) > Defaulting to the constant time backoff algorithm > 2021-05-23 15:17:44,535 INFO [cloud.utils.LogUtils] (main:null) log4j > configuration found at /usr/local/cloud/systemvm/conf/log4j-cloud.xml > 2021-05-23 15:17:44,571 INFO [cloud.agent.AgentShell] (main:null) Using > default Java settings for IPv6 preference for agent connection > 2021-05-23 15:17:44,763 INFO [cloud.agent.Agent] (main:null) id is > 2021-05-23 15:17:44,884 INFO [resource.consoleproxy.ConsoleProxyResource] > (main:null) Receive proxyVmId in ConsoleProxyResource configuration as 2 > 2021-05-23 15:17:44,928 INFO [cloud.agent.Agent] (main:null) Agent [id = > new : type = ConsoleProxyResource : zone = 1 : pod = 1 : workers = 5 : host > = 192.168.1.8 : port = 8250 > 2021-05-23 15:17:44,964 INFO [utils.nio.NioClient] (main:null) Connecting > to 192.168.1.8:8250 > 2021-05-23 15:17:44,974 INFO [utils.nio.Link] (main:null) Conf file > found: /usr/local/cloud/systemvm/conf/agent.properties > 2021-05-23 15:17:45,041 WARN [utils.nio.Link] (main:null) Failed to load > keystore, using trust all manager > 2021-05-23 15:17:45,689 ERROR [utils.nio.Link] (main:null) SSL error > caught during unwrap data: Received fatal alert: bad_certificate, for local > address=/192.168.1.96:55878, remote address=/192.168.1.8:8250. The client > may have invalid ca-certificates. > 2021-05-23 15:17:45,690 ERROR [utils.nio.NioClient] (main:null) SSL > Handshake failed while connecting to host: 192.168.1.8 port: 8250 > 2021-05-23 15:17:45,697 ERROR [utils.nio.NioConnection] (main:null) Unable > to initialize the threads. > java.io.IOException: SSL Handshake failed while connecting to host: > 192.168.1.8 port: 8250 > at com.cloud.utils.nio.NioClient.init(NioClient.java:67) > at com.cloud.utils.nio.NioConnection.start(NioConnection.java:95) > at com.cloud.agent.Agent.start(Agent.java:2
Re: Help : Console Proxy SSVM Check NFS Error
proxy_vm: 2 2021-05-23 15:17:57,511 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property eth2mask: 255.255.255.0 2021-05-23 15:17:57,512 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property localgw: 192.168.1.1 2021-05-23 15:17:57,513 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property keystore.passphrase: 6ujFeUTQdrRCw3c8 2021-05-23 15:17:57,513 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property eth0ip: 169.254.16.114 2021-05-23 15:17:57,514 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property eth1ip: 192.168.1.96 2021-05-23 15:17:57,515 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property port: 8250 2021-05-23 15:17:57,515 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property name: v-2-VM 2021-05-23 15:17:57,515 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property guid: Proxy.2 2021-05-23 15:17:57,515 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Property gateway: 192.168.1.1 2021-05-23 15:17:57,516 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Setting httpListenPort=80 2021-05-23 15:17:57,517 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Setting httpCmdListenPort=8001 2021-05-23 15:17:57,527 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Setting reconnectMaxRetry=5 2021-05-23 15:17:57,528 INFO [cloud.consoleproxy.ConsoleProxyBaseServerFactoryImpl] (Console-Proxy-Main:null) create HTTP server instance at port: 80 2021-05-23 15:17:57,543 INFO [cloud.agent.Agent] (agentRequest-Handler-3:null) Processing agent ready command, agent id = 2 2021-05-23 15:17:57,550 INFO [cloud.agent.Agent] (agentRequest-Handler-3:null) Set agent id 2 2021-05-23 15:17:57,554 INFO [cloud.agent.Agent] (agentRequest-Handler-3:null) Ready command is processed for agent id = 2 2021-05-23 15:17:57,563 INFO [resource.consoleproxy.ConsoleProxyResource] (agentRequest-Handler-3:null) Receive ReadyCommand, response with ReadyAnswer 2021-05-23 15:17:57,689 INFO [cloud.agent.Agent] (agentRequest-Handler-4:null) Processing agent ready command, agent id = 2 2021-05-23 15:17:57,690 INFO [cloud.agent.Agent] (agentRequest-Handler-4:null) Set agent id 2 2021-05-23 15:17:57,698 INFO [cloud.agent.Agent] (agentRequest-Handler-4:null) Processed new management server list: 192.168.1.8@static 2021-05-23 15:17:57,699 INFO [cloud.agent.Agent] (agentRequest-Handler-4:null) Ready command is processed for agent id = 2 2021-05-23 15:17:57,702 INFO [resource.consoleproxy.ConsoleProxyResource] (agentRequest-Handler-4:null) Receive ReadyCommand, response with ReadyAnswer 2021-05-23 15:17:57,837 INFO [cloud.consoleproxy.ConsoleProxy] (Console-Proxy-Main:null) Listening for HTTP CMDs on port 8001 root@v-2-VM:/var/log# On Sun, May 23, 2021 at 5:24 PM Serge Byishimo wrote: > Cloudstack 4.15 > > Console Proxy SSVM check has NFS error trying to reach 255.255.255.0, > which is not the configured secondary storage IP address > > I re installed the set up and still the problem is there > > root@v-2-VM:~# > root@v-2-VM:~# > root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh > > First DNS server is 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=31.999 ms > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=32.849 ms > --- 8.8.8.8 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max/stddev = 31.999/32.424/32.849/0.425 ms > Good: Can ping DNS server > > Good: DNS resolves cloudstack.apache.org > > ERROR: NFS is not currently mounted > Try manually mounting from inside the VM > NFS server is 255.255.255.0 > PING 255.255.255.0 (255.255.255.0): 56 data bytes > --- 255.255.255.0 ping statistics --- > 2 packets transmitted, 0 packets received, 100% packet loss > WARNING: cannot ping nfs server > routing table follows > Kernel IP routing table > Destination Gateway Genmask Flags Metric RefUse > Iface > 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 > eth2 > 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 > eth0 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth1 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth2 > > Management server is 192.168.1.8. Checking connectivity. > Good: Can connect to manage
Help : Console Proxy SSVM Check NFS Error
Cloudstack 4.15 Console Proxy SSVM check has NFS error trying to reach 255.255.255.0, which is not the configured secondary storage IP address I re installed the set up and still the problem is there root@v-2-VM:~# root@v-2-VM:~# root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=31.999 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=32.849 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 31.999/32.424/32.849/0.425 ms Good: Can ping DNS server Good: DNS resolves cloudstack.apache.org ERROR: NFS is not currently mounted Try manually mounting from inside the VM NFS server is 255.255.255.0 PING 255.255.255.0 (255.255.255.0): 56 data bytes --- 255.255.255.0 ping statistics --- 2 packets transmitted, 0 packets received, 100% packet loss WARNING: cannot ping nfs server routing table follows Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 eth2 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 eth1 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 eth0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 Management server is 192.168.1.8. Checking connectivity. Good: Can connect to management server 192.168.1.8 port 8250 Good: Java process is running Tests Complete. Look for ERROR or WARNING above. root@v-2-VM:~# root@v-2-VM:~# Secondary Storage SSVM Check is Successfuly root@s-1-VM:~# root@s-1-VM:~# root@s-1-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=32.518 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=32.773 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 32.518/32.645/32.773/0.128 ms Good: Can ping DNS server Good: DNS resolves cloudstack.apache.org nfs is currently mounted Mount point is /mnt/SecStorage/3636e78b-77d4-3562-995e-deb81e266bce Good: Can write to mount point Management server is 192.168.1.8. Checking connectivity. Good: Can connect to management server 192.168.1.8 port 8250 Good: Java process is running Tests Complete. Look for ERROR or WARNING above. root@s-1-VM:~# Kind Regards,
Re: I need Help with Networking
Hi Vash, Yes the System VMs are able to start and I can SSH into both of them, That NFS error is in the console proxy SSVM, thus the Web Console is timing out But in there is no error in Secondary Storage VM : ssvm-check.sh root@s-1-VM:~# root@s-1-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=30.151 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=29.075 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 29.075/29.613/30.151/0.538 ms Good: Can ping DNS server Good: DNS resolves cloudstack.apache.org nfs is currently mounted Mount point is /mnt/SecStorage/3636e78b-77d4-3562-995e-deb81e266bce Good: Can write to mount point Management server is 192.168.1.8. Checking connectivity. Good: Can connect to management server 192.168.1.8 port 8250 Good: Java process is running Tests Complete. Look for ERROR or WARNING above. Why does it say this in console proxy SSVM ? Do I have to redo the whole set up? ERROR: NFS is not currently mounted Try manually mounting from inside the VM NFS server is 255.255.255.0 PING 255.255.255.0 (255.255.255.0): 56 data bytes --- 255.255.255.0 ping statistics --- 2 packets transmitted, 0 packets received, 100% packet loss WARNING: cannot ping nfs server Regards, On Sun, May 23, 2021 at 4:41 PM vas...@gmx.de wrote: > Hi Serge, > > to be honest, it's hard for me to understand you current network setup > / deployment at the moment. > Taking a look at the log you've provided, there seems to be an > adressing issue with the nfs-server-address (as long as 255.255.255.0 > isn't the correct address of your nfs server ;-) ). > Regarding the questioned connectivitiy: > The connection of the ssvm to the management server seems to be > working. I am kind of troubled if the console proxy is able to start / > work correctly as long as the ssvm is having trouble with the provided > nfs-shares. > > with regards, > chris > > Am So., 23. Mai 2021 um 15:01 Uhr schrieb Serge Byishimo > : > > > > Management Server can not ping System VM Console Proxy IP address > > > > root@v-2-VM:~# > > root@v-2-VM:~# > > root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh > > > > First DNS server is 8.8.8.8 > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=29.581 ms > > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=30.140 ms > > --- 8.8.8.8 ping statistics --- > > 2 packets transmitted, 2 packets received, 0% packet loss > > round-trip min/avg/max/stddev = 29.581/29.861/30.140/0.280 ms > > Good: Can ping DNS server > > > > Good: DNS resolves cloudstack.apache.org > > > > ERROR: NFS is not currently mounted > > Try manually mounting from inside the VM > > NFS server is 255.255.255.0 > > PING 255.255.255.0 (255.255.255.0): 56 data bytes > > --- 255.255.255.0 ping statistics --- > > 2 packets transmitted, 0 packets received, 100% packet loss > > WARNING: cannot ping nfs server > > routing table follows > > Kernel IP routing table > > Destination Gateway Genmask Flags Metric RefUse > > Iface > > 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 > eth2 > > 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > > 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 > eth0 > > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth1 > > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth2 > > > > Management server is 192.168.1.8. Checking connectivity. > > Good: Can connect to management server 192.168.1.8 port 8250 > > > > Good: Java process is running > > > > Tests Complete. Look for ERROR or WARNING above. > > root@v-2-VM:~# > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
Re: Web Console Timing Out
Thank you Sergey How am I supposed to delete the secondary storage with running System VMs? It gives this Error : Cannot delete image store with active templates backup! Regards, On Sun, May 23, 2021 at 4:29 PM Sergey Levitskiy wrote: > The error is right here in the log. SSVM is supposed to have secondary > storage mounted over NFS. 255.255.255.0 is not the correct IP. Likely, the > cause is garbled record for secondary storage. Try to delete it and re-add. > > ERROR: NFS is not currently mounted > Try manually mounting from inside the VM > NFS server is 255.255.255.0 > PING 255.255.255.0 (255.255.255.0): 56 data bytes > --- 255.255.255.0 ping statistics --- > 2 packets transmitted, 0 packets received, 100% packet loss > WARNING: cannot ping nfs server > > > From: Serge Byishimo > Date: Sunday, May 23, 2021 at 5:29 AM > To: users@cloudstack.apache.org > Subject: Web Console Timing Out > Centos8 > Cloudstack 4.15 > > System VMs are running and their Agent is UP, > > System VMs can ping the management server but the management server can not > ping any of the IPs of the System VMs, thus the web console is not > accessible, > > I’m thinking it is firewall rules on the System VM side that is causing it, > > Any reasons why this is happening? > > > root@v-2-VM:~# > root@v-2-VM:~# > root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh > > First DNS server is 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=29.581 ms > 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=30.140 ms > --- 8.8.8.8 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max/stddev = 29.581/29.861/30.140/0.280 ms > Good: Can ping DNS server > > Good: DNS resolves cloudstack.apache.org > > ERROR: NFS is not currently mounted > Try manually mounting from inside the VM > NFS server is 255.255.255.0 > PING 255.255.255.0 (255.255.255.0): 56 data bytes > --- 255.255.255.0 ping statistics --- > 2 packets transmitted, 0 packets received, 100% packet loss > WARNING: cannot ping nfs server > routing table follows > Kernel IP routing table > Destination Gateway Genmask Flags Metric RefUse > Iface > 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 > eth2 > 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 > eth1 > 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 > eth0 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth1 > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth2 > > Management server is 192.168.1.8. Checking connectivity. > Good: Can connect to management server 192.168.1.8 port 8250 > > Good: Java process is running > > Tests Complete. Look for ERROR or WARNING above. > root@v-2-VM:~# > > > > > > root@v-2-VM:~# > root@v-2-VM:~# iptables -nL > Chain INPUT (policy DROP) > target prot opt source destination > ACCEPT all -- 0.0.0.0/00.0.0.0/0 > ACCEPT icmp -- 0.0.0.0/00.0.0.0/0icmptype 8 > ACCEPT all -- 0.0.0.0/00.0.0.0/0state > RELATED,ESTABLISHED > ACCEPT all -- 0.0.0.0/00.0.0.0/0state > RELATED,ESTABLISHED > ACCEPT all -- 0.0.0.0/00.0.0.0/0state > RELATED,ESTABLISHED > ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:3922 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:8001 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:8001 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:443 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:80 > ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW > tcp > dpt:8080 > ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 > > Chain FORWARD (policy DROP) > target prot opt source destination > > Chain OUTPUT (policy ACCEPT) > target prot opt source destination > root@v-2-VM:~# > root@v-2-VM:~# > > Thanks > -- Byishimo Shema Serge Kigali, Rwanda Tel: +250 786 076 106 www.serge.works
I need Help with Networking
Management Server can not ping System VM Console Proxy IP address root@v-2-VM:~# root@v-2-VM:~# root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=29.581 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=30.140 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 29.581/29.861/30.140/0.280 ms Good: Can ping DNS server Good: DNS resolves cloudstack.apache.org ERROR: NFS is not currently mounted Try manually mounting from inside the VM NFS server is 255.255.255.0 PING 255.255.255.0 (255.255.255.0): 56 data bytes --- 255.255.255.0 ping statistics --- 2 packets transmitted, 0 packets received, 100% packet loss WARNING: cannot ping nfs server routing table follows Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 eth2 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 eth1 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 eth0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 Management server is 192.168.1.8. Checking connectivity. Good: Can connect to management server 192.168.1.8 port 8250 Good: Java process is running Tests Complete. Look for ERROR or WARNING above. root@v-2-VM:~#
Web Console Timing Out
Centos8 Cloudstack 4.15 System VMs are running and their Agent is UP, System VMs can ping the management server but the management server can not ping any of the IPs of the System VMs, thus the web console is not accessible, I’m thinking it is firewall rules on the System VM side that is causing it, Any reasons why this is happening? root@v-2-VM:~# root@v-2-VM:~# root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=113 time=29.581 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=30.140 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 29.581/29.861/30.140/0.280 ms Good: Can ping DNS server Good: DNS resolves cloudstack.apache.org ERROR: NFS is not currently mounted Try manually mounting from inside the VM NFS server is 255.255.255.0 PING 255.255.255.0 (255.255.255.0): 56 data bytes --- 255.255.255.0 ping statistics --- 2 packets transmitted, 0 packets received, 100% packet loss WARNING: cannot ping nfs server routing table follows Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 eth2 8.8.4.4 192.168.1.1 255.255.255.255 UGH 0 00 eth1 8.8.8.8 192.168.1.1 255.255.255.255 UGH 0 00 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 eth0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 Management server is 192.168.1.8. Checking connectivity. Good: Can connect to management server 192.168.1.8 port 8250 Good: Java process is running Tests Complete. Look for ERROR or WARNING above. root@v-2-VM:~# root@v-2-VM:~# root@v-2-VM:~# iptables -nL Chain INPUT (policy DROP) target prot opt source destination ACCEPT all -- 0.0.0.0/00.0.0.0/0 ACCEPT icmp -- 0.0.0.0/00.0.0.0/0icmptype 8 ACCEPT all -- 0.0.0.0/00.0.0.0/0state RELATED,ESTABLISHED ACCEPT all -- 0.0.0.0/00.0.0.0/0state RELATED,ESTABLISHED ACCEPT all -- 0.0.0.0/00.0.0.0/0state RELATED,ESTABLISHED ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:3922 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:8001 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:8001 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:443 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:80 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0state NEW tcp dpt:8080 ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 Chain FORWARD (policy DROP) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination root@v-2-VM:~# root@v-2-VM:~# Thanks
Console IP is unreachable, but it’s Private IP is reachable
Centos8 Cloudstack 4.15 My console IP address 192.168.1.86 is unreachable from anywhere But it’s Private IP 192.168.1.68 is reachable EVERYWHERE The console also has access to the Internet. Network Setup : 2 servers : they are on a home Local Network router with internet Home Router Gateway : 192.168.1.1 Home Router Netmask : 255.255.255.0 Home Route IP Range : 192.168.1.2 - .61 1st server : management server IP : 192.168.1.8 == 2nd server : Host IP : 192.168.1.7 [root@srvr2 ~]# [root@srvr2 ~]# ifconfig br-200: flags=4099 mtu 1500 ether c2:0e:f3:be:8b:14 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 brenp0s25-200: flags=4163 mtu 1500 inet6 fe80::fc9c:5dff:fe6d:cd99 prefixlen 64 scopeid 0x20 ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) RX packets 80947 bytes 2821053 (2.6 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 50 bytes 3636 (3.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 cloud0: flags=4163 mtu 1500 inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 inet6 fe80::c77:fbff:fe37:94cf prefixlen 64 scopeid 0x20 ether fe:00:a9:fe:17:7d txqueuelen 1000 (Ethernet) RX packets 27169 bytes 4740900 (4.5 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 30778 bytes 4633136 (4.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 cloudbr0: flags=4163 mtu 1500 inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid 0x20 ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) RX packets 701422 bytes 1846775312 (1.7 GiB) RX errors 0 dropped 38954 overruns 0 frame 0 TX packets 566835 bytes 1064377633 (1015.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 cloudbr1: flags=4099 mtu 1500 ether 66:99:75:20:97:e0 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp0s25: flags=4163 mtu 1500 ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) RX packets 1737418 bytes 1938354995 (1.8 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 947393 bytes 100900024 (96.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 20 memory 0xfb20-fb22 enp0s25.200: flags=4163 mtu 1500 inet6 fe80::d250:99ff:fe81:89f5 prefixlen 64 scopeid 0x20 ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 79122 bytes 3326072 (3.1 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp3s0: flags=4163 mtu 1500 inet6 fe80::6a05:caff:fec6:6e9d prefixlen 64 scopeid 0x20 ether 68:05:ca:c6:6e:9d txqueuelen 1000 (Ethernet) RX packets 159042 bytes 21914322 (20.8 MiB) RX errors 0 dropped 38972 overruns 0 frame 0 TX packets 1096 bytes 102120 (99.7 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 18 memory 0xfb1c-fb1e lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (Local Loopback) RX packets 1655743 bytes 9484529950 (8.8 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1655743 bytes 9484529950 (8.8 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet0: flags=4163 mtu 1500 inet6 fe80::fc00:a9ff:fefe:3dc9 prefixlen 64 scopeid 0x20 ether fe:00:a9:fe:3d:c9 txqueuelen 1000 (Ethernet) RX packets 1823 bytes 236454 (230.9 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 2239 bytes 172684 (168.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet1: flags=4163 mtu 1500 inet6 fe80::fc00:43ff:fe00:16 prefixlen 64 scopeid 0x20 ether fe:00:43:00:00:16 txqueuelen 1000 (Ethernet) RX packets 249253 bytes 1008634858 (961.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 608622 bytes 2041552296 (1.9 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet2: flags=4163 mtu 1500 inet6 fe80::fc00:e9ff:fe00:1 prefixlen 64 scopeid 0x20 ether fe:00:e9:00:00:01 txqueuelen 1000 (Ethernet) RX packets 36 bytes 1512 (1.4 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX pack
Re: Agent Error
Hi Edward, Yes, the problem I'm seeing is that it is checking for "br-200" when in the docs they tell us to create "cloudbr0", Even when I create that bridge called "br-200", the next command in that scripts throws this error : RTNETLINK answers: Device or resource busy Even when the bridge is in state DOWN Regards, On Fri, May 21, 2021 at 4:01 PM Edward St Pierre wrote: > Hi, > > Did you create the bridge interfaces on the hypervisor? > > com.cloud.exception.InternalErrorException: Failed to create vnet 200: ls: > cannot access '/sys/class/net//brif/': No such file or directoryCommand > > > > On Fri, 21 May 2021 at 10:29, Serge Byishimo > wrote: > > > Centos 8 > > Cloudstack 4.15 > > > > > > 2021-05-21 05:25:13,312 INFO [kvm.storage.LibvirtStorageAdaptor] > > (agentRequest-Handler-5:null) (logid:5fb6ba1e) Trying to fetch storage > pool > > 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt > > 2021-05-21 05:25:13,314 INFO [kvm.storage.LibvirtStorageAdaptor] > > (agentRequest-Handler-1:null) (logid:e0f54056) Trying to fetch storage > pool > > 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt > > 2021-05-21 05:25:13,321 INFO [kvm.storage.LibvirtStorageAdaptor] > > (agentRequest-Handler-1:null) (logid:e0f54056) Creating volume > > 398d3b32-c842-44c7-9feb-5b14e33df9fb from template > > 30cf8eed-df4b-45fd-a73c-4719769dfd3e in pool > > 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d (NetworkFilesystem) with size (0 > > bytes) 0 > > 2021-05-21 05:25:13,322 INFO [kvm.storage.LibvirtStorageAdaptor] > > (agentRequest-Handler-1:null) (logid:e0f54056) Attempting to create > volume > > 398d3b32-c842-44c7-9feb-5b14e33df9fb (NetworkFilesystem) in pool > > 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d with size (2.44 GB) 262144 > > 2021-05-21 05:25:13,634 INFO [kvm.storage.LibvirtStorageAdaptor] > > (agentRequest-Handler-3:null) (logid:e0f54056) Trying to fetch storage > pool > > 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt > > 2021-05-21 05:25:13,643 INFO [kvm.storage.LibvirtStorageAdaptor] > > (agentRequest-Handler-3:null) (logid:e0f54056) Trying to fetch storage > pool > > 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt > > 2021-05-21 05:25:13,660 WARN > [resource.wrapper.LibvirtStartCommandWrapper] > > (agentRequest-Handler-3:null) (logid:e0f54056) InternalErrorException > > com.cloud.exception.InternalErrorException: Failed to create vnet 200: > ls: > > cannot access '/sys/class/net//brif/': No such file or directoryCommand > > line is not complete. Try option "help"ls: cannot access > > '/sys/class/net//brif/': No such file or directoryFailed to add vlan: > > br-200.200 to > > at > > > > > com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createVnet(BridgeVifDriver.java:325) > > at > > > > > com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createVnetBr(BridgeVifDriver.java:307) > > at > > > > > com.cloud.hypervisor.kvm.resource.BridgeVifDriver.plug(BridgeVifDriver.java:227) > > at > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVif(LibvirtComputingResource.java:2726) > > at > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVifs(LibvirtComputingResource.java:2454) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:80) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:45) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > at > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1643) > > at com.cloud.agent.Agent.processRequest(Agent.java:661) > > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079) > > at com.cloud.utils.nio.Task.call(Task.java:83) > > at com.cloud.utils.nio.Task.call(Task.java:29) > > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > at > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > at > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > at java.base/java.lang.Thread.run(Thread.java:829) > > 2021-05-21 05:25:13,671 INFO [kvm.storage.LibvirtStorageAdaptor] > > (agentRequest-Handler-3:null) (logid:e0f54056) Trying to fe
Agent Error
Centos 8 Cloudstack 4.15 2021-05-21 05:25:13,312 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-5:null) (logid:5fb6ba1e) Trying to fetch storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt 2021-05-21 05:25:13,314 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:e0f54056) Trying to fetch storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt 2021-05-21 05:25:13,321 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:e0f54056) Creating volume 398d3b32-c842-44c7-9feb-5b14e33df9fb from template 30cf8eed-df4b-45fd-a73c-4719769dfd3e in pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d (NetworkFilesystem) with size (0 bytes) 0 2021-05-21 05:25:13,322 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:e0f54056) Attempting to create volume 398d3b32-c842-44c7-9feb-5b14e33df9fb (NetworkFilesystem) in pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d with size (2.44 GB) 262144 2021-05-21 05:25:13,634 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:e0f54056) Trying to fetch storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt 2021-05-21 05:25:13,643 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:e0f54056) Trying to fetch storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt 2021-05-21 05:25:13,660 WARN [resource.wrapper.LibvirtStartCommandWrapper] (agentRequest-Handler-3:null) (logid:e0f54056) InternalErrorException com.cloud.exception.InternalErrorException: Failed to create vnet 200: ls: cannot access '/sys/class/net//brif/': No such file or directoryCommand line is not complete. Try option "help"ls: cannot access '/sys/class/net//brif/': No such file or directoryFailed to add vlan: br-200.200 to at com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createVnet(BridgeVifDriver.java:325) at com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createVnetBr(BridgeVifDriver.java:307) at com.cloud.hypervisor.kvm.resource.BridgeVifDriver.plug(BridgeVifDriver.java:227) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVif(LibvirtComputingResource.java:2726) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVifs(LibvirtComputingResource.java:2454) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:80) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:45) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1643) at com.cloud.agent.Agent.processRequest(Agent.java:661) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2021-05-21 05:25:13,671 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:e0f54056) Trying to fetch storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt 2021-05-21 05:25:16,542 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:5fb6ba1e) No existing libvirtd connection found. Opening a new one 2021-05-21 05:25:16,545 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:5fb6ba1e) Can not find a connection for Instance v-20-VM. Assuming the default connection. 2021-05-21 05:25:16,738 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-2:null) (logid:5fb6ba1e) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied. 2021-05-21 05:25:16,738 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-2:null) (logid:5fb6ba1e) Groovy scripting engine is not initialized. Data transformation skipped. 2021-05-21 05:25:16,784 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:e0f54056) No existing libvirtd connection found. Opening a new one 2021-05-21 05:25:16,785 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null) (logid:e0f54056) Can not find a connection for Instance s-19-VM. Assuming the default connection. 2021-05-21 05:25:16,858 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-4:null) (logid:e0f54056) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied.
Re: I Need Help 🙏🏼
The Host agent is starting well, you can see : [root@srvr2 agent]# systemctl status cloudstack-agent ● cloudstack-agent.service - CloudStack Agent Loaded: loaded (/usr/lib/systemd/system/cloudstack-agent.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2021-05-19 12:42:38 EDT; 16h ago Docs: http://www.cloudstack.org/ Main PID: 23371 (java) Tasks: 57 (limit: 409824) Memory: 334.4M CGroup: /system.slice/cloudstack-agent.service └─23371 /usr/bin/java -Djava.io.tmpdir=/usr/share/cloudstack-agent/tmp -Xms256m -Xmx2048m -cp /usr/share/cloudstack-agent/lib/*:/usr/share/cloudstack-agent/plu> May 20 04:53:44 srvr2.cloud.priv java[23371]: INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:) (logid:17b8f397) Trying to fetch storage pool 26a9efbf-f> May 20 04:53:44 srvr2.cloud.priv java[23371]: libvirt: Storage Driver error : Storage volume not found: no storage vol with matching name '7ed2a89e-9cac-492c-b252-3bffd23> May 20 04:53:44 srvr2.cloud.priv java[23371]: libvirt: Storage Driver error : Storage volume not found: no storage vol with matching name '7ed2a89e-9cac-492c-b252-3bffd23> May 20 04:54:14 srvr2.cloud.priv java[23371]: INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:) (logid:17b8f397) Trying to fetch storage pool 26a9efbf-f> May 20 04:54:14 srvr2.cloud.priv java[23371]: libvirt: Storage Driver error : Storage volume not found: no storage vol with matching name '7ed2a89e-9cac-492c-b252-3bffd23> May 20 04:54:14 srvr2.cloud.priv java[23371]: libvirt: Storage Driver error : Storage volume not found: no storage vol with matching name '7ed2a89e-9cac-492c-b252-3bffd23> May 20 04:54:22 srvr2.cloud.priv java[23371]: INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:) (logid:bb6c5137) Trying to fetch storage pool 26a9efbf-f> May 20 04:54:22 srvr2.cloud.priv java[23371]: INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:) (logid:bb6c5137) Asking libvirt to refresh storage pool > May 20 04:54:22 srvr2.cloud.priv java[23371]: INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:) (logid:bb6c5137) Trying to fetch storage pool 39e1d532-5> May 20 04:54:22 srvr2.cloud.priv java[23371]: INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:) (logid:bb6c5137) Asking libvirt to refresh storage pool On Thu, May 20, 2021 at 10:52 AM Serge Byishimo wrote: > On the host 192.168.1.7, There is no /var/log/cloud directory, > > Instead, there is /var/log/cloudstack/agent/agent.log with these logs > > 2021-05-20 04:44:20,995 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-5:null) (logid:eccdbd04) Asking libvirt to refresh > storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d > 2021-05-20 04:44:21,051 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-3:null) (logid:eccdbd04) Trying to fetch storage pool > 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt > 2021-05-20 04:44:21,052 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-3:null) (logid:eccdbd04) Asking libvirt to refresh > storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 > 2021-05-20 04:44:44,086 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-4:null) (logid:0934a268) Trying to fetch storage pool > 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt > 2021-05-20 04:44:44,101 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-4:null) (logid:0934a268) Trying to fetch storage pool > 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt > 2021-05-20 04:44:47,244 INFO [kvm.resource.LibvirtConnection] > (agentRequest-Handler-2:null) (logid:17b8f397) No existing libvirtd > connection found. Opening a new one > 2021-05-20 04:44:47,246 WARN [kvm.resource.LibvirtConnection] > (agentRequest-Handler-2:null) (logid:17b8f397) Can not find a connection > for Instance s-561-VM. Assuming the default connection. > 2021-05-20 04:44:47,329 WARN [kvm.resource.LibvirtKvmAgentHook] > (agentRequest-Handler-2:null) (logid:17b8f397) Groovy script > '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not > available. Transformations will not be applied. > 2021-05-20 04:44:47,329 WARN [kvm.resource.LibvirtKvmAgentHook] > (agentRequest-Handler-2:null) (logid:17b8f397) Groovy scripting engine is > not initialized. Data transformation skipped. > 2021-05-20 04:45:14,105 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-4:null) (logid:0934a268) Trying to fetch storage pool > 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt > 2021-05-20 04:45:21,135 INFO [kvm.storage.LibvirtStorageAdaptor] > (agentRequest-Handler-3:null) (logid:58b4761b) Trying to fetch storage pool > 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt > 2021-05-20 04:45:21,149 INFO [kvm.storage.Lib
Re: I Need Help 🙏🏼
On the host 192.168.1.7, There is no /var/log/cloud directory, Instead, there is /var/log/cloudstack/agent/agent.log with these logs 2021-05-20 04:44:20,995 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-5:null) (logid:eccdbd04) Asking libvirt to refresh storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d 2021-05-20 04:44:21,051 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:eccdbd04) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 04:44:21,052 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:eccdbd04) Asking libvirt to refresh storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 2021-05-20 04:44:44,086 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:null) (logid:0934a268) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 04:44:44,101 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:null) (logid:0934a268) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 04:44:47,244 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:17b8f397) No existing libvirtd connection found. Opening a new one 2021-05-20 04:44:47,246 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:17b8f397) Can not find a connection for Instance s-561-VM. Assuming the default connection. 2021-05-20 04:44:47,329 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-2:null) (logid:17b8f397) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied. 2021-05-20 04:44:47,329 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-2:null) (logid:17b8f397) Groovy scripting engine is not initialized. Data transformation skipped. 2021-05-20 04:45:14,105 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:null) (logid:0934a268) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 04:45:21,135 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:58b4761b) Trying to fetch storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt 2021-05-20 04:45:21,149 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:58b4761b) Asking libvirt to refresh storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d 2021-05-20 04:45:21,206 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:58b4761b) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 04:45:21,208 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:58b4761b) Asking libvirt to refresh storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 2021-05-20 04:45:44,122 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:null) (logid:0934a268) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt This is /etc/cloudstack/agent/agent.properties [root@srvr2 agent]# cat agent.properties #Storage #Wed May 19 16:43:09 EDT 2021 cluster=Cluster1 pod=Pod1 resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource domr.scripts.dir=scripts/network/domr/kvm router.aggregation.command.each.timeout=600 keystore.passphrase=passphrase hypervisor.type=kvm port=8250 zone=Zone1 local.storage.uuid=d520c74e-10f3-4fd0-868f-8eee666edb81 host=192.168.1.8@static guid=28b9a02d-a2a2-3ba1-9ef5-6dc969cb4427 LibvirtComputingResource.id=1 workers=5 iscsi.session.cleanup.enabled=false [root@srvr2 agent]# Regards, On Thu, May 20, 2021 at 10:42 AM Yordan Kostov wrote: > That is correct. > Actually I don’t see any mistake in SSVM config. > > Check hypervisor host logs /var/log/cloud/cloud.cfg > > Is there anything of relevance there? > > Regards, > Jordan > > -Original Message- > From: Serge Byishimo > Sent: Thursday, May 20, 2021 11:40 AM > To: users@cloudstack.apache.org > Subject: Re: I Need Help 🙏🏼 > > > [X] This message came from outside your organization > > > The fact that SSVMs need a public network to access the internet I > understand, but at least I think the System VMs should start and be able to > ssh into them before we reach that step. > > Isn't that the case Yordan? > > On Thu, May 20, 2021 at 10:29 AM Serge Byishimo > wrote: > > > I disabled the firewall on both servers > > > > If you check the physical network configuration in the UI and the network > > configuration of the servers I shared, you can immediately see the > mistake > > I might have made, I don't think it is related to KVM specifically. > > > > Thank you again, Yordan > > > > > > On Thu, May 20, 2021 at 10:21 AM Serge Byishimo > > > wrote: > > > >> Other Image
Re: I Need Help 🙏🏼
The fact that SSVMs need a public network to access the internet I understand, but at least I think the System VMs should start and be able to ssh into them before we reach that step. Isn't that the case Yordan? On Thu, May 20, 2021 at 10:29 AM Serge Byishimo wrote: > I disabled the firewall on both servers > > If you check the physical network configuration in the UI and the network > configuration of the servers I shared, you can immediately see the mistake > I might have made, I don't think it is related to KVM specifically. > > Thank you again, Yordan > > > On Thu, May 20, 2021 at 10:21 AM Serge Byishimo > wrote: > >> Other Image Links including the Host status, System VMs >> >> https://ibb.co/ggyY4Yj >> https://ibb.co/XCBdy6D >> https://ibb.co/LCmpj01 >> https://ibb.co/FJbXJCk >> >> Thank you Yordan >> >> >> On Thu, May 20, 2021 at 10:18 AM Serge Byishimo >> wrote: >> >>> 1st computer : >>> >>> ifconfig : >>> >>> [root@srvr1 ~]# ifconfig >>> eno1: flags=4163 mtu 1500 >>> inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255 >>> inet6 fe80::7a31:44f5:6cbd:f97e prefixlen 64 scopeid 0x20 >>> ether 68:1d:ef:24:d2:bb txqueuelen 1000 (Ethernet) >>> RX packets 683346 bytes 450075162 (429.2 MiB) >>> RX errors 0 dropped 31958 overruns 0 frame 0 >>> TX packets 483540 bytes 464984616 (443.4 MiB) >>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >>> >>> lo: flags=73 mtu 65536 >>> inet 127.0.0.1 netmask 255.0.0.0 >>> inet6 ::1 prefixlen 128 scopeid 0x10 >>> loop txqueuelen 1000 (Local Loopback) >>> RX packets 11164675 bytes 2804166400 (2.6 GiB) >>> RX errors 0 dropped 0 overruns 0 frame 0 >>> TX packets 11164675 bytes 2804166400 (2.6 GiB) >>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >>> >>> [root@srvr1 ~]# >>> >>> >>> [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-cloudbr0 >>> DEVICE=cloudbr0 >>> TYPE=Bridge >>> ONBOOT=yes >>> BOOTPROTO=static >>> IPV6INIT=no >>> IPV6_AUTOCONF=no >>> DELAY=5 >>> IPADDR=192.168.1.8 >>> GATEWAY=192.168.1.1 >>> NETMASK=255.255.255.0 >>> DNS1=8.8.8.8 >>> DNS2=8.8.4.4 >>> STP=yes >>> USERCTL=no >>> NM_CONTROLLED=no >>> [root@srvr1 ~]# >>> >>> >>> [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno1 >>> TYPE=Ethernet >>> BOOTPRO=static >>> NAME=eno1 >>> UUID=7dad60c5-e259-4fb9-9f5a-aac0ec29435f >>> DEVICE=eno1 >>> ONBOOT=yes >>> IPADDR-192.168.1.8 >>> PREFIX=24 >>> GATEWAY=192.168.1.1 >>> USRCTL=no >>> BRIDGE=cloudbr0 >>> [root@srvr1 ~]# >>> >>> >>> 2nd computer : >>> >>> The 2nd computer has 2 NICs : >>> >>> [root@srvr2 ~]# ifconfig >>> cloud0: flags=4163 mtu 1500 >>> inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 >>> inet6 fe80::f89d:edff:fe9c:e90b prefixlen 64 scopeid 0x20 >>> ether fa:9d:ed:9c:e9:0b txqueuelen 1000 (Ethernet) >>> RX packets 0 bytes 0 (0.0 B) >>> RX errors 0 dropped 0 overruns 0 frame 0 >>> TX packets 1401 bytes 59862 (58.4 KiB) >>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >>> >>> cloudbr0: flags=4163 mtu 1500 >>> inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255 >>> inet6 fe80::8040:8dff:fe57:4eb8 prefixlen 64 scopeid 0x20 >>> ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) >>> RX packets 220463 bytes 19759124 (18.8 MiB) >>> RX errors 0 dropped 22054 overruns 0 frame 0 >>> TX packets 620 bytes 55228 (53.9 KiB) >>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >>> >>> cloudbr1: flags=4099 mtu 1500 >>> ether 32:7c:82:bc:0f:ce txqueuelen 1000 (Ethernet) >>> RX packets 0 bytes 0 (0.0 B) >>> RX errors 0 dropped 0 overruns 0 frame 0 >>> TX packets 0 bytes 0 (0.0 B) >>> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >>> >>> enp0s25: flags=4163 mtu 1500 >>> ether d0:50:99:81:89:f5
Re: I Need Help 🙏🏼
I disabled the firewall on both servers If you check the physical network configuration in the UI and the network configuration of the servers I shared, you can immediately see the mistake I might have made, I don't think it is related to KVM specifically. Thank you again, Yordan On Thu, May 20, 2021 at 10:21 AM Serge Byishimo wrote: > Other Image Links including the Host status, System VMs > > https://ibb.co/ggyY4Yj > https://ibb.co/XCBdy6D > https://ibb.co/LCmpj01 > https://ibb.co/FJbXJCk > > Thank you Yordan > > > On Thu, May 20, 2021 at 10:18 AM Serge Byishimo > wrote: > >> 1st computer : >> >> ifconfig : >> >> [root@srvr1 ~]# ifconfig >> eno1: flags=4163 mtu 1500 >> inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255 >> inet6 fe80::7a31:44f5:6cbd:f97e prefixlen 64 scopeid 0x20 >> ether 68:1d:ef:24:d2:bb txqueuelen 1000 (Ethernet) >> RX packets 683346 bytes 450075162 (429.2 MiB) >> RX errors 0 dropped 31958 overruns 0 frame 0 >> TX packets 483540 bytes 464984616 (443.4 MiB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> lo: flags=73 mtu 65536 >> inet 127.0.0.1 netmask 255.0.0.0 >> inet6 ::1 prefixlen 128 scopeid 0x10 >> loop txqueuelen 1000 (Local Loopback) >> RX packets 11164675 bytes 2804166400 (2.6 GiB) >> RX errors 0 dropped 0 overruns 0 frame 0 >> TX packets 11164675 bytes 2804166400 (2.6 GiB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> [root@srvr1 ~]# >> >> >> [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-cloudbr0 >> DEVICE=cloudbr0 >> TYPE=Bridge >> ONBOOT=yes >> BOOTPROTO=static >> IPV6INIT=no >> IPV6_AUTOCONF=no >> DELAY=5 >> IPADDR=192.168.1.8 >> GATEWAY=192.168.1.1 >> NETMASK=255.255.255.0 >> DNS1=8.8.8.8 >> DNS2=8.8.4.4 >> STP=yes >> USERCTL=no >> NM_CONTROLLED=no >> [root@srvr1 ~]# >> >> >> [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno1 >> TYPE=Ethernet >> BOOTPRO=static >> NAME=eno1 >> UUID=7dad60c5-e259-4fb9-9f5a-aac0ec29435f >> DEVICE=eno1 >> ONBOOT=yes >> IPADDR-192.168.1.8 >> PREFIX=24 >> GATEWAY=192.168.1.1 >> USRCTL=no >> BRIDGE=cloudbr0 >> [root@srvr1 ~]# >> >> >> 2nd computer : >> >> The 2nd computer has 2 NICs : >> >> [root@srvr2 ~]# ifconfig >> cloud0: flags=4163 mtu 1500 >> inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 >> inet6 fe80::f89d:edff:fe9c:e90b prefixlen 64 scopeid 0x20 >> ether fa:9d:ed:9c:e9:0b txqueuelen 1000 (Ethernet) >> RX packets 0 bytes 0 (0.0 B) >> RX errors 0 dropped 0 overruns 0 frame 0 >> TX packets 1401 bytes 59862 (58.4 KiB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> cloudbr0: flags=4163 mtu 1500 >> inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255 >> inet6 fe80::8040:8dff:fe57:4eb8 prefixlen 64 scopeid 0x20 >> ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) >> RX packets 220463 bytes 19759124 (18.8 MiB) >> RX errors 0 dropped 22054 overruns 0 frame 0 >> TX packets 620 bytes 55228 (53.9 KiB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> cloudbr1: flags=4099 mtu 1500 >> ether 32:7c:82:bc:0f:ce txqueuelen 1000 (Ethernet) >> RX packets 0 bytes 0 (0.0 B) >> RX errors 0 dropped 0 overruns 0 frame 0 >> TX packets 0 bytes 0 (0.0 B) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> >> enp0s25: flags=4163 mtu 1500 >> ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) >> RX packets 250164 bytes 33642066 (32.0 MiB) >> RX errors 0 dropped 0 overruns 0 frame 0 >> TX packets 37371 bytes 2409704 (2.2 MiB) >> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 >> device interrupt 20 memory 0xfb20-fb22 >> >> enp3s0: flags=4163 mtu 1500 >> inet 192.168.1.60 netmask 255.255.255.0 broadcast 192.168.1.255 >> inet6 fe80::6a1a:9d26:8975:480c prefixlen 64 scopeid 0x20 >> ether 68:05:ca:c6:6e:9d txqueuelen 1000 (Ethernet) >> RX packets 618625 bytes 444641389 (424.0 MiB) >> RX errors 0 dropped 22070 overruns 0 frame
Re: I Need Help 🙏🏼
Other Image Links including the Host status, System VMs https://ibb.co/ggyY4Yj https://ibb.co/XCBdy6D https://ibb.co/LCmpj01 https://ibb.co/FJbXJCk Thank you Yordan On Thu, May 20, 2021 at 10:18 AM Serge Byishimo wrote: > 1st computer : > > ifconfig : > > [root@srvr1 ~]# ifconfig > eno1: flags=4163 mtu 1500 > inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::7a31:44f5:6cbd:f97e prefixlen 64 scopeid 0x20 > ether 68:1d:ef:24:d2:bb txqueuelen 1000 (Ethernet) > RX packets 683346 bytes 450075162 (429.2 MiB) > RX errors 0 dropped 31958 overruns 0 frame 0 > TX packets 483540 bytes 464984616 (443.4 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > lo: flags=73 mtu 65536 > inet 127.0.0.1 netmask 255.0.0.0 > inet6 ::1 prefixlen 128 scopeid 0x10 > loop txqueuelen 1000 (Local Loopback) > RX packets 11164675 bytes 2804166400 (2.6 GiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 11164675 bytes 2804166400 (2.6 GiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > [root@srvr1 ~]# > > > [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-cloudbr0 > DEVICE=cloudbr0 > TYPE=Bridge > ONBOOT=yes > BOOTPROTO=static > IPV6INIT=no > IPV6_AUTOCONF=no > DELAY=5 > IPADDR=192.168.1.8 > GATEWAY=192.168.1.1 > NETMASK=255.255.255.0 > DNS1=8.8.8.8 > DNS2=8.8.4.4 > STP=yes > USERCTL=no > NM_CONTROLLED=no > [root@srvr1 ~]# > > > [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno1 > TYPE=Ethernet > BOOTPRO=static > NAME=eno1 > UUID=7dad60c5-e259-4fb9-9f5a-aac0ec29435f > DEVICE=eno1 > ONBOOT=yes > IPADDR-192.168.1.8 > PREFIX=24 > GATEWAY=192.168.1.1 > USRCTL=no > BRIDGE=cloudbr0 > [root@srvr1 ~]# > > > 2nd computer : > > The 2nd computer has 2 NICs : > > [root@srvr2 ~]# ifconfig > cloud0: flags=4163 mtu 1500 > inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 > inet6 fe80::f89d:edff:fe9c:e90b prefixlen 64 scopeid 0x20 > ether fa:9d:ed:9c:e9:0b txqueuelen 1000 (Ethernet) > RX packets 0 bytes 0 (0.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 1401 bytes 59862 (58.4 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > cloudbr0: flags=4163 mtu 1500 > inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::8040:8dff:fe57:4eb8 prefixlen 64 scopeid 0x20 > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > RX packets 220463 bytes 19759124 (18.8 MiB) > RX errors 0 dropped 22054 overruns 0 frame 0 > TX packets 620 bytes 55228 (53.9 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > cloudbr1: flags=4099 mtu 1500 > ether 32:7c:82:bc:0f:ce txqueuelen 1000 (Ethernet) > RX packets 0 bytes 0 (0.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 0 bytes 0 (0.0 B) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > enp0s25: flags=4163 mtu 1500 > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > RX packets 250164 bytes 33642066 (32.0 MiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 37371 bytes 2409704 (2.2 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > device interrupt 20 memory 0xfb20-fb22 > > enp3s0: flags=4163 mtu 1500 > inet 192.168.1.60 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::6a1a:9d26:8975:480c prefixlen 64 scopeid 0x20 > ether 68:05:ca:c6:6e:9d txqueuelen 1000 (Ethernet) > RX packets 618625 bytes 444641389 (424.0 MiB) > RX errors 0 dropped 22070 overruns 0 frame 0 > TX packets 178295 bytes 28986719 (27.6 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > device interrupt 18 memory 0xfb1c-fb1e > > lo: flags=73 mtu 65536 > inet 127.0.0.1 netmask 255.0.0.0 > inet6 ::1 prefixlen 128 scopeid 0x10 > loop txqueuelen 1000 (Local Loopback) > RX packets 88970 bytes 20756782 (19.7 MiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 88970 bytes 20756782 (19.7 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > [root@srvr2 ~]# > > > > > > > > [root@srvr2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-cloudbr0 > DEVICE=cloudbr0 > TYPE=Bridge > ONBOOT=yes > BOOTPROTO
Re: I Need Help 🙏🏼
t 10:17 AM Serge Byishimo wrote: > Sorry Yordan, I didn't know that > > Here are all the Images I was sharing > > https://ibb.co/8rwfC16 > https://ibb.co/G3XmwDn > https://ibb.co/tXNGnN5 > https://ibb.co/NtcFqJ5 > https://ibb.co/j6SnXFf > https://ibb.co/sQFpKVF > https://ibb.co/7CbQwcz > https://ibb.co/mvPkQSG > > > They contain network configurations you can compare with the network > configuration on both servers below > > 1st computer : > > ifconfig : > > [root@srvr1 ~]# ifconfig > eno1: flags=4163 mtu 1500 > inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::7a31:44f5:6cbd:f97e prefixlen 64 scopeid 0x20 > ether 68:1d:ef:24:d2:bb txqueuelen 1000 (Ethernet) > RX packets 683346 bytes 450075162 (429.2 MiB) > RX errors 0 dropped 31958 overruns 0 frame 0 > TX packets 483540 bytes 464984616 (443.4 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > lo: flags=73 mtu 65536 > inet 127.0.0.1 netmask 255.0.0.0 > inet6 ::1 prefixlen 128 scopeid 0x10 > loop txqueuelen 1000 (Local Loopback) > RX packets 11164675 bytes 2804166400 (2.6 GiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 11164675 bytes 2804166400 (2.6 GiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > [root@srvr1 ~]# > > > [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-cloudbr0 > DEVICE=cloudbr0 > TYPE=Bridge > ONBOOT=yes > BOOTPROTO=static > IPV6INIT=no > IPV6_AUTOCONF=no > DELAY=5 > IPADDR=192.168.1.8 > GATEWAY=192.168.1.1 > NETMASK=255.255.255.0 > DNS1=8.8.8.8 > DNS2=8.8.4.4 > STP=yes > USERCTL=no > NM_CONTROLLED=no > [root@srvr1 ~]# > > > [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno1 > TYPE=Ethernet > BOOTPRO=static > NAME=eno1 > UUID=7dad60c5-e259-4fb9-9f5a-aac0ec29435f > DEVICE=eno1 > ONBOOT=yes > IPADDR-192.168.1.8 > PREFIX=24 > GATEWAY=192.168.1.1 > USRCTL=no > BRIDGE=cloudbr0 > [root@srvr1 ~]# > > > 2nd computer : > > The 2nd computer has 2 NICs : > > [root@srvr2 ~]# ifconfig > cloud0: flags=4163 mtu 1500 > inet 169.254.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 > inet6 fe80::f89d:edff:fe9c:e90b prefixlen 64 scopeid 0x20 > ether fa:9d:ed:9c:e9:0b txqueuelen 1000 (Ethernet) > RX packets 0 bytes 0 (0.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 1401 bytes 59862 (58.4 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > cloudbr0: flags=4163 mtu 1500 > inet 192.168.1.7 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::8040:8dff:fe57:4eb8 prefixlen 64 scopeid 0x20 > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > RX packets 220463 bytes 19759124 (18.8 MiB) > RX errors 0 dropped 22054 overruns 0 frame 0 > TX packets 620 bytes 55228 (53.9 KiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > cloudbr1: flags=4099 mtu 1500 > ether 32:7c:82:bc:0f:ce txqueuelen 1000 (Ethernet) > RX packets 0 bytes 0 (0.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 0 bytes 0 (0.0 B) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > enp0s25: flags=4163 mtu 1500 > ether d0:50:99:81:89:f5 txqueuelen 1000 (Ethernet) > RX packets 250164 bytes 33642066 (32.0 MiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 37371 bytes 2409704 (2.2 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > device interrupt 20 memory 0xfb20-fb22 > > enp3s0: flags=4163 mtu 1500 > inet 192.168.1.60 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::6a1a:9d26:8975:480c prefixlen 64 scopeid 0x20 > ether 68:05:ca:c6:6e:9d txqueuelen 1000 (Ethernet) > RX packets 618625 bytes 444641389 (424.0 MiB) > RX errors 0 dropped 22070 overruns 0 frame 0 > TX packets 178295 bytes 28986719 (27.6 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > device interrupt 18 memory 0xfb1c-fb1e > > lo: flags=73 mtu 65536 > inet 127.0.0.1 netmask 255.0.0.0 > inet6 ::1 prefixlen 128 scopeid 0x10 > loop txqueuelen 1000 (Local Loopback) > RX packets 88970 bytes 20756782 (19.7 MiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 88970 bytes 20756782 (19.7 MiB) > TX er
Re: I Need Help 🙏🏼
yes [root@srvr2 ~]# [root@srvr2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s25 TYPE=Ethernet NAME=enp0s25 UUID=93380612-3b97-4a34-ab9f-042b751cc468 DEVICE=enp0s25 ONBOOT=yes BRIDGE=cloudbr0 # cat /etc/sysconfig/network-scripts/enp0s25.200 DEVICE=enp0s25.200 ONBOOT=yes HOTPLUG=no BOOTPROTO=none TYPE=Ethernet VLAN=yes BRIDGE=cloudbr1 On Thu, May 20, 2021 at 10:12 AM Yordan Kostov wrote: > Hey Serge, > > We cannot see the screenshots because the mailing group > protection strips them. > However you can provide a link to a hosted image. > If you doubt networking disable firewalls/iptables. > Cloudstack VM communicates with the hypervisor host > through management. > Public network are required by SSVMs to access external > sources such internet or as a entry/exit point for NAT-ed user networks. > > Next go to ACS GUI and check if Zone is enabled? Then go > to pod -> clusters -> hosts and check if any of those has any red alert > instead of a green dot. > If such exists around cluster level and below – make sure > you configured the network traffic labels correct. I have not played with > ACS + KVM so I cannot give specifics here. > > Best regards, > Jordan > > From: Serge Byishimo > Sent: Thursday, May 20, 2021 11:02 AM > To: users@cloudstack.apache.org > Subject: Re: I Need Help 🙏🏼 > > > [X]This message came from outside your organization > > This is the Host Information > > [Screenshot 2021-05-20 at 10-01-32 Hosts - CloudStack.png] > > On Thu, May 20, 2021 at 9:48 AM Serge Byishimo <mailto:sergebyish...@gmail.com>> wrote: > I mounted the secondary storage on the management server /export/secondary > from 192.168.1.7:/export/secondary > > [root@srvr1 secondary]# ll > total 20 > drwx--. 2 root root 16384 May 18 16:18 lost+found > drwxr-xr-x. 3 root root 4096 May 19 11:56 template > [root@srvr1 secondary]# > > there is a template directory with template files > > [root@srvr1 secondary]# ll template/tmpl/1/3 > total 365536 > -rw-r--r--. 1 root root 374303744 May 19 11:58 > 1e6e90ea-c7e8-421f-b403-7a5e4bce848c.qcow2 > -rw-r--r--. 1 root root 287 May 19 11:58 template.properties > [root@srvr1 secondary]# > > It is downloaded as you can see on the attached screenshots > I think it might be a networking issue coz I got confused with the public > IP range and I put one IP that I have 209.150.129—, but it doesn’t have > anything to do with this set up, except that I use it to access resources > on the outside by port forwarding them > > Attached also is the console proxy and secondary storage VM, > > they are all stuck starting as shown in another screenshot > > > [Screenshot 2021-05-20 at 09-39-43 Templates - CloudStack.png] > [Screenshot 2021-05-20 at 09-41-25 System VMs - CloudStack.png] > [Screenshot 2021-05-20 at 09-41-41 System VMs - CloudStack.png] > [Screen Shot 2021-05-20 at 09.47.20.png] > > > On Thu, May 20, 2021 at 9:31 AM Yordan Kostov <mailto:yord...@nsogroup.com>> wrote: > Did you by any chance download and update the system vm templates? > > http://docs.cloudstack.apache.org/en/latest/installguide/management-server/index.html#prepare-the-system-vm-template > < > https://urldefense.com/v3/__http:/docs.cloudstack.apache.org/en/latest/installguide/management-server/index.html*prepare-the-system-vm-template__;Iw!!A6UyJA!xdMOpDgufOBL1WXIgUcUxgTcbsdp8WmZnpUB-DBK4ReGRx8n17VhUmhYJTvKy6me5hqQCYLdQ3Ow$ > > > That is another reason for the SSVM not starting. > > Regards, > Jordan > > From: Serge Byishimo sergebyish...@gmail.com>> > Sent: Thursday, May 20, 2021 10:14 AM > To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> > Subject: Re: I Need Help 🙏🏼 > > > [X]This message came from outside your organization > > Another thing I forgot to mention is : > > On the dashboard, it shows 0.00GB of Secondary Storage > > I mounted a partition on the Host and used NFS to connect it with the > management server : > > [root@srvr1 ~]# showmount -e 192.168.1.7 > Export list for 192.168.1.7< > https://urldefense.com/v3/__http:/192.168.1.7__;!!A6UyJA!0q19bVFm1wSBf0d9m2u4EiU0ugEkSxsNy6ecVs3JXSIhmsTYAZ3IpQrFxJf0SHRqggbVrSdyvvvt$ > >: > /export/primary * > /export/primary2 * > /export/secondary * > > and then added it on the UI > > > [Screen Shot 2021-05-20 at 09.05.30.png] > > On Thu, May 20, 2021 at 9:12 AM Serge Byishimo <mailto:sergebyish...@gmail.com><mailto:sergebyish...@gmail.com sergebyish...@gmail.com>>> wrote: > > This is the message I get when the Host 192.168.1.7 is try
Re: I Need Help 🙏🏼
entRequest-Handler-2:null) (logid:17b8f397) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 03:09:40,935 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:17b8f397) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 03:09:44,076 INFO [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:0934a268) No existing libvirtd connection found. Opening a new one 2021-05-20 03:09:44,078 WARN [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:0934a268) Can not find a connection for Instance v-550-VM. Assuming the default connection. 2021-05-20 03:09:44,161 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-5:null) (logid:0934a268) Groovy script '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is not available. Transformations will not be applied. 2021-05-20 03:09:44,161 WARN [kvm.resource.LibvirtKvmAgentHook] (agentRequest-Handler-5:null) (logid:0934a268) Groovy scripting engine is not initialized. Data transformation skipped. 2021-05-20 03:10:07,275 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:0e198ea0) Trying to fetch storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d from libvirt 2021-05-20 03:10:07,277 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:0e198ea0) Asking libvirt to refresh storage pool 26a9efbf-fb80-3f0d-a292-43bd0a3eec9d 2021-05-20 03:10:07,371 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:null) (logid:0e198ea0) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 03:10:07,372 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:null) (logid:0e198ea0) Asking libvirt to refresh storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 2021-05-20 03:10:10,939 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:17b8f397) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt 2021-05-20 03:10:40,955 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:17b8f397) Trying to fetch storage pool 39e1d532-590f-3d17-9b5a-10c1644e09f6 from libvirt Thank you Regards, On Thu, May 20, 2021 at 9:05 AM Yordan Kostov wrote: > System VMs SSH available only through the hypervisor. > You can read more about it here -> > http://docs.cloudstack.apache.org/en/latest/adminguide/systemvm.html#accessing-system-vms > On the hypervisor host 192.168.1.7 what do you see in > /var/log/cloud/cloud.log > > Any issues reported? > > Regards, > Jordan > > From: Serge Byishimo > Sent: Thursday, May 20, 2021 9:55 AM > To: users@cloudstack.apache.org > Subject: Re: I Need Help 🙏🏼 > > > [X]This message came from outside your organization > > Thank you Yordan, > > I have 6TB of primary storage available on the dashboard, > > But one of the problem I have is I can’t SSH through the SystemVMs to see > that log, I’m thinking It might a networking Issue, so let me show you my > configuration : > > I have 2 computers : 1st management server, 2nd Host > > > > They are on a private network, they don’t have public IP addresses, so I > was confused when cloudstack kept asking me about public IP addresses > > 1st computer : > > ifconfig : > > [root@srvr1 ~]# ifconfig > eno1: flags=4163 mtu 1500 > inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255 > inet6 fe80::7a31:44f5:6cbd:f97e prefixlen 64 scopeid 0x20 > ether 68:1d:ef:24:d2:bb txqueuelen 1000 (Ethernet) > RX packets 683346 bytes 450075162 (429.2 MiB) > RX errors 0 dropped 31958 overruns 0 frame 0 > TX packets 483540 bytes 464984616 (443.4 MiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > lo: flags=73 mtu 65536 > inet 127.0.0.1 netmask 255.0.0.0 > inet6 ::1 prefixlen 128 scopeid 0x10 > loop txqueuelen 1000 (Local Loopback) > RX packets 11164675 bytes 2804166400 (2.6 GiB) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 11164675 bytes 2804166400 (2.6 GiB) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > > [root@srvr1 ~]# > > > [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-cloudbr0 > DEVICE=cloudbr0 > TYPE=Bridge > ONBOOT=yes > BOOTPROTO=static > IPV6INIT=no > IPV6_AUTOCONF=no > DELAY=5 > IPADDR=192.168.1.8 > GATEWAY=192.168.1.1 > NETMASK=255.255.255.0 > DNS1=8.8.8.8 > DNS2=8.8.4.4 > STP=yes > USERCTL=no > NM_CONTROLLED=no > [root@srvr1 ~]# > > > [root@srvr1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno1 > TYPE=Ethernet > BOOTPRO=static > NAME=eno1 > UUID=7dad60
Re: I Need Help 🙏🏼
ONBOOT=yes
BOOTPROTO=none
IPV6INIT=no
IPV6_AUTOCONF=no
DELAY=5
STP=yes
[root@srvr2 ~]#
[root@srvr2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s25
TYPE=Ethernet
NAME=enp0s25
UUID=93380612-3b97-4a34-ab9f-042b751cc468
DEVICE=enp0s25
ONBOOT=yes
BRIDGE=cloudbr0
# cat /etc/sysconfig/network-scripts/enp0s25.200
DEVICE=enp0s25.200
ONBOOT=yes
HOTPLUG=no
BOOTPROTO=none
TYPE=Ethernet
VLAN=yes
BRIDGE=cloudbr1
[image: Screen Shot 2021-05-20 at 08.52.21.png]
[image: Screen Shot 2021-05-20 at 08.52.47.png]
[image: Screen Shot 2021-05-20 at 08.53.12.png]
[image: Screen Shot 2021-05-20 at 08.53.18.png]
I need help figuring out what I did wrong!
On Thu, May 20, 2021 at 8:30 AM Yordan Kostov wrote:
> Hello Serge,
>
> Check these snippets from your logs.
> Line 61:
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a
> deployment for VM[SecondaryStorageVm|s-454-VM]Scope=interface
> Line 66: com.cloud.exception.InsufficientServerCapacityException:
> Unable to create a deployment for
> VM[SecondaryStorageVm|s-454-VM]Scope=interface
> Line 72: (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115)
> (logid:a9f8cbaf) Unable to complete AsyncJobVO {id:1115, userId: 1,
> accountId: 1,
> Line 80: com.cloud.exception.InsufficientServerCapacityException:
> Unable to create a deployment for
> VM[SecondaryStorageVm|s-454-VM]Scope=interface
> Line 134: com.cloud.exception.InsufficientServerCapacityException:
> Unable to create a deployment for
> VM[SecondaryStorageVm|s-454-VM]Scope=interface
> Line 175: (secstorage-1:ctx-694f5c71) (logid:a34366a8) Unable to
> start secondary storage vm for standby capacity, vm id : 454, will recycle
> it and start a new one
>
> It seems the system cannot create the secondary VM thus the
> zone/pods/clusters are not functional.
> Check the hypervisor logs at "/var/log/cloud/cloud.log and let us
> know what you see there.
>
> Usually the cause of this issue is that primary storage cannot
> utilized (either not mounted or something else) on the hypervisor for usage.
>
> Best regards,
> Jordan
>
> -Original Message-
> From: Serge Byishimo
> Sent: Wednesday, May 19, 2021 6:49 PM
> To: users@cloudstack.apache.org
> Subject: I Need Help 🙏🏼
>
>
> [X] This message came from outside your organization
>
>
> Cloudstack: 4.15
> OS: Centos 8
>
> Two computers: one for the Management Server and the other for the Host
>
> Primary and Secondary storage is on the host and connected to the
> management through NFS,
>
>
> 2021-05-19 11:20:39,924 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8
> FirstFitRoutingAllocator) (logid:a9f8cbaf) Found 1 hosts for allocation
> after prioritization: [Host[-1-Routing]]
> 2021-05-19 11:20:39,924 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8
> FirstFitRoutingAllocator) (logid:a9f8cbaf) Looking for speed=500Mhz,
> Ram=512 MB
> 2021-05-19 11:20:39,924 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8
> FirstFitRoutingAllocator) (logid:a9f8cbaf) Host name: srvr2.cloud.priv,
> hostId: 1 is in avoid set, skipping this and trying other available hosts
> 2021-05-19 11:20:39,924 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8
> FirstFitRoutingAllocator) (logid:a9f8cbaf) Host Allocator returning 0
> suitable hosts
> 2021-05-19 11:20:39,924 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8)
> (logid:a9f8cbaf) No suitable hosts found
> 2021-05-19 11:20:39,924 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8)
> (logid:a9f8cbaf) No suitable hosts found under this Cluster: 1
> 2021-05-19 11:20:39,927 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8)
> (logid:a9f8cbaf) Could not find suitable Deployment Destination for this
> VM under any clusters, returning.
> 2021-05-19 11:20:39,930 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8)
> (logid:a9f8cbaf) Searching resources only under specified Pod: 1
> 2021-05-19 11:20:39,934 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8)
> (logid:a9f8cbaf) Listing clusters in order of aggregate capacity, that
> have (at least one host with) enough CPU and RAM capacity under this Pod: 1
> 2021-05-19 11:20:39,942 DEBUG [c.c.d.FirstFitPlanner]
> (Work-Job-Executor-8:ctx
I Need Help 🙏🏼
Cloudstack: 4.15 OS: Centos 8 Two computers: one for the Management Server and the other for the Host Primary and Secondary storage is on the host and connected to the management through NFS, 2021-05-19 11:20:39,924 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8 FirstFitRoutingAllocator) (logid:a9f8cbaf) Found 1 hosts for allocation after prioritization: [Host[-1-Routing]] 2021-05-19 11:20:39,924 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8 FirstFitRoutingAllocator) (logid:a9f8cbaf) Looking for speed=500Mhz, Ram=512 MB 2021-05-19 11:20:39,924 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8 FirstFitRoutingAllocator) (logid:a9f8cbaf) Host name: srvr2.cloud.priv, hostId: 1 is in avoid set, skipping this and trying other available hosts 2021-05-19 11:20:39,924 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8 FirstFitRoutingAllocator) (logid:a9f8cbaf) Host Allocator returning 0 suitable hosts 2021-05-19 11:20:39,924 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) No suitable hosts found 2021-05-19 11:20:39,924 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) No suitable hosts found under this Cluster: 1 2021-05-19 11:20:39,927 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) Could not find suitable Deployment Destination for this VM under any clusters, returning. 2021-05-19 11:20:39,930 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) Searching resources only under specified Pod: 1 2021-05-19 11:20:39,934 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Pod: 1 2021-05-19 11:20:39,942 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) Removing from the clusterId list these clusters from avoid set: [1] 2021-05-19 11:20:39,947 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) No clusters found after removing disabled clusters and clusters in avoid list, returning. 2021-05-19 11:20:39,974 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) VM state transitted from :Starting to Stopped with event: OperationFailedvm's original host id: null new host id: null host id before state transition: 1 2021-05-19 11:20:39,997 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) Hosts's actual total CPU: 43200 and CPU after applying overprovisioning: 43200 2021-05-19 11:20:39,997 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) Hosts's actual total RAM: (61.57 GB) 66114236416 and RAM after applying overprovisioning: (61.57 GB) 66114236416 2021-05-19 11:20:39,997 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) release cpu from host: 1, old used: 1000,reserved: 0, actual total: 43200, total with overprovisioning: 43200; new used: 500,reserved:0; movedfromreserved: false,moveToReserveredfalse 2021-05-19 11:20:39,998 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) release mem from host: 1, old used: (1.50 GB) 1610612736,reserved: (0 bytes) 0, total: (61.57 GB) 66114236416; new used: (1.00 GB) 1073741824,reserved:(0 bytes) 0; movedfromreserved: false,moveToReserveredfalse 2021-05-19 11:20:40,011 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-454-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2021-05-19 11:20:40,012 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115 ctx-9c8114c8) (logid:a9f8cbaf) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-454-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2021-05-19 11:20:40,012 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-8:ctx-e9ad7a77 job-1107/job-1115) (logid:a9f8cbaf) Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 454, job origin: 1107 2021-05-19 11:20:40,012 ERROR [c.c.v.VmWork
