RE: Fields are blank while creating guest network

[Paul Angus]

Hi Victor,

For us to look, is this a clean install of 4.11 or an upgrade?

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: victor  
Sent: 27 February 2018 23:21
To: users@cloudstack.apache.org
Subject: Fields are blank while creating guest network

Hello Guys,

I am using cloudstack version 4.11. While creating a guest network, I can see 
that the zone dropdown and network offering dropdown are all blank. Some bug 
has been reported related to this. Is it really a bug or any configuration 
issues. If it is really a bug, how can we fix it. Any thoughts

Regards

Victor

Fields are blank while creating guest network

[victor]

Hello Guys,

I am using cloudstack version 4.11. While creating a guest network, I 
can see that the zone dropdown and network offering dropdown are all 
blank. Some bug has been reported related to this. Is it really a bug or 
any configuration issues. If it is really a bug, how can we fix it. Any 
thoughts


Regards

Victor

Re: VR routing issues in Advanced Mode

[Dag Sonstebo]

Hi Andrei,

Sorry lost you - are you saying it's now all working when you allow icmp in 
your ACLs?

If not can you look at the tcpdumps on source and destination VRs as per my 
previous post? You may obviously have to run these on different interfaces 
depending on which VPC tier you are pinging from.

Regards, 
Dag Sonstebo
Cloud Architect
ShapeBlue
 S: +44 20 3603 0540  | dag.sonst...@shapeblue.com | http://www.shapeblue.com 
 | Twitter:@ShapeBlue 



On 27/02/2018, 10:27, "Andrei Mikhailovsky"  wrote:

Hi Dag,

Thanks, for your reply which I've missed earlier.

I have done some more digging around and would like to make some 
corrections to the problem at hand.

1. It seems that the problem only effects VPC networks within cloudstack. 
2. Networks which use non-VPC networking can talk to each other. 
3. VPC to VPC traffic is not working. 
4. VPC traffic can reach non-VPC network
5. non-VPC traffic can't reach VPC network

In all cases, if the icmp is allowed in the ACLs, all networks can ping 
each other. So, the traffic is being routed and reaches the virtual router.

Any advice? 

Thanks




dag.sonst...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

- Original Message -
> From: "Dag Sonstebo" 
> To: "users" 
> Sent: Friday, 23 February, 2018 16:45:04
> Subject: Re: VR routing issues in Advanced Mode

> Hi Andrei,
> 
> Next step is to do some tcpdumping on the two VRs. Set some ping’s going 
and
> check:
> 
> Private NIC: tcpdump -i eth0 icmp
> Public NIC: tcpdump -i eth2 icmp
> 
> That way you should be able to see how far your traffic reaches.
> 
> 
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
> 
> On 23/02/2018, 16:17, "Andrei Mikhailovsky"  
wrote:
> 
>Bump.
>
>Any ideas anyone? This issue is really annoying.
>
>Thanks
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>  
> 
> 
> - Original Message -
>> From: "Andrei Mikhailovsky" 
>> To: "users" , "users" 

>> Sent: Wednesday, 21 February, 2018 22:10:25
>> Subject: Re: VR routing issues in Advanced Mode
>
>> Dag,
>> 
>> 
>> 
>> 
>> Yeah, we have egress traffic enabled. We also use VPCs on some of 
the networks
>> and they are also effected by this issue along with None VPC 
networks.
>> 
>> 
>> 
>> 
>> Any thoughts?
>> 
>> 
>> 
>> 
>> Andrei Mikhailovsky
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> From: Dag Sonstebo
>> 
>> 
>> Sent: Wednesday 21 February, 18:30
>> 
>> 
>> Subject: Re: VR routing issues in Advanced Mode
>> 
>> 
>> To: users@cloudstack.apache.org
>> 
>> 
>> 
>> 
>> 
>> 
>> Hi Andrei,
>> 
>> 
>> 
>> 
>> Understand. To get all the obvious things out the way – have you 
allowed egress
>> traffic on the two networks (you mention ACLs which we only use on 
VPCs and
>> basic networks)?
>> 
>> 
>> 
>> 
>> Regards,
>> 
>> 
>> Dag Sonstebo
>> 
>> 
>> Cloud Architect
>> 
>> 
>> ShapeBlue
>> 
>> 
>> 
>> 
>> On 21/02/2018, 14:51, "Andrei Mikhailovsky" 
 wrote:
>> 
>> 
>> 
>> 
>> Hi Dag,
>> 
>> 
>> 
>> 
>> Please see my comments below:
>> 
>> 
>> 
>> 
>>> Hi Andrei,
>> 
>> 
>>> 
>> 
>> 
>>> You’re confusing the matter with your masking of public IP ranges. 
You said you
>> 
>> 
>>> have “2 x Public IP ranges with /26 netmask” – but since you are 
masking them
>> 
>> 
>>> out with X’s your email doesn’t make sense. If all the X’s are the 
same then a
>> 
>> 
>>> .10 and a .20 IP address would be on the same /26 network.
>> 
>> 
>>> 
>> 
>> 
>>> I will assume that you do in fact have 2 x 26-bit networks, e.g.:
>> 
 

Re: CPVM/SSVM agent state disconnects

[Dag Sonstebo]

What is your global setting for "host" set to?

Regards, 
Dag Sonstebo
Cloud Architect
ShapeBlue
 S: +44 20 3603 0540  | dag.sonst...@shapeblue.com | http://www.shapeblue.com 
 | Twitter:@ShapeBlue 



On 27/02/2018, 16:40, "Chen Zhang"  wrote:

Hi Dag,

Yes the VR is online all the time. I checked the cloud.log inside the
system VM, here is the problem:

2018-02-27 09:01:23,345 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-5:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:02:23,650 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-2:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:03:23,915 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-4:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:04:24,240 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-3:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:05:24,507 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-1:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:06:24,773 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-5:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:07:25,296 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-2:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:09:57,210 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
Lost connection to the server. Dealing with the remaining commands...
2018-02-27 09:09:57,218 INFO  [utils.nio.NioClient] (Agent-Handler-2:null)
NioClient connection closed
2018-02-27 09:09:57,218 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
Reconnecting to host:129.*.*.*
2018-02-27 09:09:57,219 INFO  [utils.nio.NioClient] (Agent-Handler-2:null)
Connecting to 129.*.*.*:8250
2018-02-27 09:09:57,228 ERROR [utils.nio.NioConnection]
(Agent-Handler-2:null) Unable to initialize the threads.
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:454)
at sun.nio.ch.Net.connect(Net.java:446)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
at com.cloud.utils.nio.NioClient.init(NioClient.java:56)
at com.cloud.utils.nio.NioConnection.start(NioConnection.java:95)
at com.cloud.agent.Agent.reconnect(Agent.java:442)
at com.cloud.agent.Agent$ServerHandler.doTask(Agent.java:1014)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-02-27 09:09:57,249 INFO  [utils.exception.CSExceptionErrorCode]
(Agent-Handler-2:null) Could not find exception:
com.cloud.utils.exception.NioConnectionException in error code list for
exceptions
2018-02-27 09:09:57,250 WARN  [cloud.agent.Agent] (Agent-Handler-2:null)
NIO Connection Exception  com.cloud.utils.exception.NioConnectionException:
No route to host
2018-02-27 09:09:57,250 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
Attempted to connect to the server, but received an unexpected exception,
trying again...
2018-02-27 09:09:57,250 INFO  [utils.nio.NioClient] (Agent-Handler-2:null)
NioClient connection closed

The management server has two IPs, the public IP (129.*.*.*) and a local IP
(192.168.1.101). The 8250 port is restricted by the public IP so it cannot
be accessed. I use the local IP as the cluster node ip and host ip in all
agents, so I do not understand why the system VM always suddenly
disconnected with the local ip and started connecting to the public IP. Is
there any way to fix the IP to local IP?

Thanks!
Chen

On Fri, Feb 23, 2018 at 11:01 AM, Dag Sonstebo 
wrote:

> Do VRs stay online and connected?
>
> What you need to do next is check your cloud.log on the system VMs,
> possibly also up the verbosity level in the logs to catch why they are
> dropping comms.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> On 23/02/2018, 15:25, "Chen Zhang"  wrote:
>
> Hi Dag,
>
> Yes I did recreate the new system VMs. The version is "Cloudstack
> 

Re: CPVM/SSVM agent state disconnects

[Chen Zhang]

Hi Dag,

Yes the VR is online all the time. I checked the cloud.log inside the
system VM, here is the problem:

2018-02-27 09:01:23,345 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-5:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:02:23,650 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-2:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:03:23,915 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-4:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:04:24,240 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-3:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:05:24,507 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-1:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:06:24,773 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-5:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:07:25,296 INFO
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-2:null) Determined host 192.168.1.101 corresponds to
IP 192.168.1.101
2018-02-27 09:09:57,210 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
Lost connection to the server. Dealing with the remaining commands...
2018-02-27 09:09:57,218 INFO  [utils.nio.NioClient] (Agent-Handler-2:null)
NioClient connection closed
2018-02-27 09:09:57,218 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
Reconnecting to host:129.*.*.*
2018-02-27 09:09:57,219 INFO  [utils.nio.NioClient] (Agent-Handler-2:null)
Connecting to 129.*.*.*:8250
2018-02-27 09:09:57,228 ERROR [utils.nio.NioConnection]
(Agent-Handler-2:null) Unable to initialize the threads.
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:454)
at sun.nio.ch.Net.connect(Net.java:446)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
at com.cloud.utils.nio.NioClient.init(NioClient.java:56)
at com.cloud.utils.nio.NioConnection.start(NioConnection.java:95)
at com.cloud.agent.Agent.reconnect(Agent.java:442)
at com.cloud.agent.Agent$ServerHandler.doTask(Agent.java:1014)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-02-27 09:09:57,249 INFO  [utils.exception.CSExceptionErrorCode]
(Agent-Handler-2:null) Could not find exception:
com.cloud.utils.exception.NioConnectionException in error code list for
exceptions
2018-02-27 09:09:57,250 WARN  [cloud.agent.Agent] (Agent-Handler-2:null)
NIO Connection Exception  com.cloud.utils.exception.NioConnectionException:
No route to host
2018-02-27 09:09:57,250 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
Attempted to connect to the server, but received an unexpected exception,
trying again...
2018-02-27 09:09:57,250 INFO  [utils.nio.NioClient] (Agent-Handler-2:null)
NioClient connection closed

The management server has two IPs, the public IP (129.*.*.*) and a local IP
(192.168.1.101). The 8250 port is restricted by the public IP so it cannot
be accessed. I use the local IP as the cluster node ip and host ip in all
agents, so I do not understand why the system VM always suddenly
disconnected with the local ip and started connecting to the public IP. Is
there any way to fix the IP to local IP?

Thanks!
Chen

On Fri, Feb 23, 2018 at 11:01 AM, Dag Sonstebo 
wrote:

> Do VRs stay online and connected?
>
> What you need to do next is check your cloud.log on the system VMs,
> possibly also up the verbosity level in the logs to catch why they are
> dropping comms.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> On 23/02/2018, 15:25, "Chen Zhang"  wrote:
>
> Hi Dag,
>
> Yes I did recreate the new system VMs. The version is "Cloudstack
> release
> 4.11.0".
>
> Thanks!
> Chen
>
> On Fri, Feb 23, 2018 at 9:27 AM, Dag Sonstebo <
> dag.sonst...@shapeblue.com>
> wrote:
>
> > Hi Chen,
> >
> > You say you just upgraded to 4.11 – did you destroy your system VMs
> and
> > let them recreate after the upgrade?
> >
> > Can you also check what version a “cat /etc/cloudstack-release”
> shows up
> > with on your SSVM/CPVM?
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> > On 23/02/2018, 14:00, "Chen Zhang"  wrote:
> >
> > Hello,
> >
> >
> > I am new in the list and I am stuck with a very annoying issue on
> > CPVM/SSVM.
> >
>   

Ceph and CloudStack day, London, April 19

[Steve Roles]

Hi all!

We have now published an agenda for the day and have lots of great talks, on 
Ceph, CloudStack, and 'Ceph & CloudStack'!


  *   Morning: Mix of Ceph & CloudStack with talks applying to both projects
  *   Afternoon: Split into separate tracks: Ceph and CloudStack



Registration is open, and you can also check out the agenda here: 
https://www.eventbrite.co.uk/e/cloudstack-european-user-group-ceph-day-tickets-42670526694


Hopefully see some of you on the day!

Best regards,


steve.ro...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

Question: Domain filed on the SSL upload form

[Andrija Panic]

Hi all,

I got confused about the domain fields/API parameter that is used when
uploading new SSL, to be used on CPVM and SSVM copy process (this is
domain_suffix in cloud.keystore table)

Due to some automation, I came across the following scenarios, which WORKS
FINE, but I'm confused as how and why it works.

New SSL that was issued for " *.domain1.com " was uploaded via API (CA,
intermediate, server cert, and the key in pkcs8) - but doman specified
during this SSL upload process was " domain2.com " (so NOT matching domain
of the certificate)

This causes the cloud.keystore table/rows to have this domain2.com in the
last column next to CA/intermediate/server/key... (this is domain_suffix
column)

But in global config we define " *.domain1.com " as the CERT to be used for
CPVM and for securing/encrypting secondary storage copy process between
zones
Same SSL is also used to i.e. download templates etc...

So it all works fine, but...how ?, when "domain1.com" (instead of "*.
domain2.com") was defined in uploadCertificate GUI/API - i.e. what is the
use of this domain_suffix field at all ?

Thx,

-- 

Andrija Panić

Re: VR routing issues in Advanced Mode

[Andrei Mikhailovsky]

Hi Dag,

Thanks, for your reply which I've missed earlier.

I have done some more digging around and would like to make some corrections to 
the problem at hand.

1. It seems that the problem only effects VPC networks within cloudstack. 
2. Networks which use non-VPC networking can talk to each other. 
3. VPC to VPC traffic is not working. 
4. VPC traffic can reach non-VPC network
5. non-VPC traffic can't reach VPC network

In all cases, if the icmp is allowed in the ACLs, all networks can ping each 
other. So, the traffic is being routed and reaches the virtual router.

Any advice? 

Thanks



- Original Message -
> From: "Dag Sonstebo" 
> To: "users" 
> Sent: Friday, 23 February, 2018 16:45:04
> Subject: Re: VR routing issues in Advanced Mode

> Hi Andrei,
> 
> Next step is to do some tcpdumping on the two VRs. Set some ping’s going and
> check:
> 
> Private NIC: tcpdump -i eth0 icmp
> Public NIC: tcpdump -i eth2 icmp
> 
> That way you should be able to see how far your traffic reaches.
> 
> 
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
> 
> On 23/02/2018, 16:17, "Andrei Mikhailovsky"  wrote:
> 
>Bump.
>
>Any ideas anyone? This issue is really annoying.
>
>Thanks
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>  
> 
> 
> - Original Message -
>> From: "Andrei Mikhailovsky" 
>> To: "users" , "users" 
> 
>> Sent: Wednesday, 21 February, 2018 22:10:25
>> Subject: Re: VR routing issues in Advanced Mode
>
>> Dag,
>> 
>> 
>> 
>> 
>> Yeah, we have egress traffic enabled. We also use VPCs on some of the 
> networks
>> and they are also effected by this issue along with None VPC networks.
>> 
>> 
>> 
>> 
>> Any thoughts?
>> 
>> 
>> 
>> 
>> Andrei Mikhailovsky
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> From: Dag Sonstebo
>> 
>> 
>> Sent: Wednesday 21 February, 18:30
>> 
>> 
>> Subject: Re: VR routing issues in Advanced Mode
>> 
>> 
>> To: users@cloudstack.apache.org
>> 
>> 
>> 
>> 
>> 
>> 
>> Hi Andrei,
>> 
>> 
>> 
>> 
>> Understand. To get all the obvious things out the way – have you allowed 
> egress
>> traffic on the two networks (you mention ACLs which we only use on VPCs 
> and
>> basic networks)?
>> 
>> 
>> 
>> 
>> Regards,
>> 
>> 
>> Dag Sonstebo
>> 
>> 
>> Cloud Architect
>> 
>> 
>> ShapeBlue
>> 
>> 
>> 
>> 
>> On 21/02/2018, 14:51, "Andrei Mikhailovsky"  
> wrote:
>> 
>> 
>> 
>> 
>> Hi Dag,
>> 
>> 
>> 
>> 
>> Please see my comments below:
>> 
>> 
>> 
>> 
>>> Hi Andrei,
>> 
>> 
>>> 
>> 
>> 
>>> You’re confusing the matter with your masking of public IP ranges. You 
> said you
>> 
>> 
>>> have “2 x Public IP ranges with /26 netmask” – but since you are 
> masking them
>> 
>> 
>>> out with X’s your email doesn’t make sense. If all the X’s are the same 
> then a
>> 
>> 
>>> .10 and a .20 IP address would be on the same /26 network.
>> 
>> 
>>> 
>> 
>> 
>>> I will assume that you do in fact have 2 x 26-bit networks, e.g.:
>> 
>> 
>>> 
>> 
>> 
>>> 192.168.0.0/26 – with default gateway 192.168.0.1
>> 
>> 
>>> 192.168.0.64/26 – with default gateway 192.168.0.65
>> 
>> 
>>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> That is correct. I do have two separate /26 networks similar to what 
> you've
>> described above. However, one /26 is used for direct public IP service
>> offering, where VRs are not involved in networking at all and the second 
> /26 is
>> used for the service offering where VRs are used to provide the 
> networking
>> function.
>> 
>> 
>> 
>> 
>> 
>> 
>>> If your two guest networks have VRs on separate public IP ranges you 
> will have
>> 
>> 
>>> e.g.
>> 
>> 
>>> 
>> 
>> 
>>> VR1: public IP 192.168.0.10
>> 
>> 
>>> VR2: public IP 192.168.0.70
>> 
>> 
>>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> Nope, the guest vms with VRs that can't talk to each other are on the 
> same /26
>> network. (in your example that would be on the same 192.168.0.0/26)
>> 
>> 
>> 
>> 
>> 
>> 
>>> For a VM hosted behind VR1 to reach a service NAT’ed on VR2 you need to 
> set up
>> 
>> 
>>> routing and possibly firewalling on the data centre device which 
> handles the
>