Re: ipv6 capability of shared networks

[Stephan Seitz]

Am Sonntag, den 28.03.2021, 20:33 +0200 schrieb Wido den Hollander:
> 
> On 26/03/2021 20:56, Stephan Seitz wrote:
> > Wido, thank's a lot!
> > 
> > I just had to look into the db. The correctly calculated SLAAC is
> > already there.
> > 
> 
> Double-check: The API and UI do show an IPv6 address for the NIC?
> 
> It's then up to you to make sure the Routers in  the (shared)
> network 
> send out the proper Router Advertisements.
> 
> Also check on the hypervisor with 'ip6tables-save' and ipset to see
> if 
> all the IPs have been programmed properly into the security groups.
> 
> Should just work. We have been using this code for years now.
> 
> Wido

I was a little puzzled due to the new UI. Indeed, it is shown in the
UI. I didn't check UI and API at first because of the outdated 4.11
docs which mentioned dhcp6. My fault and poor media literacy :)

To summarize: Your code works well and everything is configured (and
shown) as it should, I just tried the wrong approach with dhcp and
didn't look out of the box.

Anyway, thanks for pointing me to SLAAC!

Stephan

> > Sorry for the noise!
> > 
> > Stephan
> > 
> > Am Freitag, den 26.03.2021, 20:28 +0100 schrieb Wido den Hollander:
> > > On 26/03/2021 20:23, Stephan Seitz wrote:
> > > > Hi!
> > > > 
> > > > I've recently deployed 4.15.0 Advanced Zone with CentOS 8 kvm
> > > > hosts
> > > > and
> > > > classic linux bridges. I do know that CentOS 7 is preferred,
> > > > but
> > > > with
> > > > some initial tweaks here and there, i'ld say it's working quite
> > > > well.
> > > > 
> > > 
> > > VLAN or VXLAN?
> > 
> > small scale, so VLAN fits very well (just for the record)
> > > > Currently, I'm trying to use IPv6 on shared networks. I'd
> > > > learned
> > > > that
> > > > IPv6 only does not work, so I switched to IPv6 plus RFC 1918
> > > > IPv4
> > > > natted at the outer gateway. IPv4 is not a requirement, but if
> > > > it's
> > > > necessary to add, it doesn't harm.
> > > > 
> > > 
> > > Yes. IPv4 is still needed and RFC1918 is just fine. Cloud-init
> > > works
> > > over IPv4. It's a lot of work to get rid of IPv4 in CloudStack.
> > > 
> > > I'm a big IPv6 fan (wrote a lot of the code in CS), but I didn't
> > > bother
> > > getting rid of IPv4. Not a real use-case for v6-only just yet.
> > > 
> > > > The IPv4 addresses of the deployed hosts are provided by the
> > > > virtual
> > > > router as expected.
> > > > 
> > > > My problem is: I don't get any dhcp6 lease out of the VR. I dug
> > > > with
> > > > tcpdump on the host and VR. I see the solicit message arriving,
> > > > but
> > > > no
> > > > answering advertise message. I've tried almost everything at
> > > > the
> > > > host:
> > > > accepting RA, Autoconf, selectively disabling these. Also
> > > > modifying
> > > > the
> > > > dhcpv6 duid as seen on some 4.11 docs didn't change anything.
> > > > 
> > > 
> > > IPv6 does not work with DHCPv6. You should see that when the IPv6
> > > CIDR
> > > is set properly for the shared network in the database that
> > > CloudStack
> > > calculates/generates the IPv6 address the Instance should obtain
> > > through
> > > SLAAC (without privacy addresses!)
> > > 
> > > When that works you have security grouping also working. It then
> > > filters
> > > on source addresses from VMs and such.
> > > 
> > > We have thousands of VMs connected with IPv6 this way.
> > > 
> > > Wido
> > > 
> > > > Best case is, that I'm stuck with hosts correctly configured by
> > > > the
> > > > router advertisement, but ACS doesn't know about it. So
> > > > subsequently i
> > > > can't add records to the respective DNS Zones.
> > > > 
> > > > Alternatively, I could skip ACS and add the provable eui-64
> > > > addresses
> > > > to the zone, but I'ld like to avoid that.
> > > > 
> > > > After a few uneducated peeks into the VR's dnsmasq
> > > > configuration, I
> > > > cannot spot any setting for providing dhcp6 leases.
> > > > 
> > > > Initially I've deployed the 4.15.0 systemvmtemplate downloaded
> > > > from
> > > > http://download.cloudstack.org/systemvm/4.15/
> > > > Right now, I've switched to the 4.15.1 from the same location,
> > > > but
> > > > that
> > > > didn't change anything.
> > > > 
> > > > I've also tried switching the Zone from internal DNS to
> > > > external
> > > > DNS
> > > > and vice versa (these are identical, except the internal DNS is
> > > > also
> > > > equipped with the respective IPv6 addresses, which obviously
> > > > cannot
> > > > be
> > > > added to the external DNS). That didn't change anything either.
> > > > 
> > > > So, I'ld like to ask for any advise.
> > > > 
> > > > Thanks in advance!
> > > > 
> > > > Stephan
> > > > 
> > > > 

Re: ipv6 capability of shared networks

[Stephan Seitz]

Wido, thank's a lot!

I just had to look into the db. The correctly calculated SLAAC is
already there.

Sorry for the noise!

Stephan

Am Freitag, den 26.03.2021, 20:28 +0100 schrieb Wido den Hollander:
> 
> On 26/03/2021 20:23, Stephan Seitz wrote:
> > Hi!
> > 
> > I've recently deployed 4.15.0 Advanced Zone with CentOS 8 kvm hosts
> > and
> > classic linux bridges. I do know that CentOS 7 is preferred, but
> > with
> > some initial tweaks here and there, i'ld say it's working quite
> > well.
> > 
> 
> VLAN or VXLAN?

small scale, so VLAN fits very well (just for the record)
> 
> > Currently, I'm trying to use IPv6 on shared networks. I'd learned
> > that
> > IPv6 only does not work, so I switched to IPv6 plus RFC 1918 IPv4
> > natted at the outer gateway. IPv4 is not a requirement, but if it's
> > necessary to add, it doesn't harm.
> > 
> 
> Yes. IPv4 is still needed and RFC1918 is just fine. Cloud-init works 
> over IPv4. It's a lot of work to get rid of IPv4 in CloudStack.
> 
> I'm a big IPv6 fan (wrote a lot of the code in CS), but I didn't
> bother 
> getting rid of IPv4. Not a real use-case for v6-only just yet.
> 
> > The IPv4 addresses of the deployed hosts are provided by the
> > virtual
> > router as expected.
> > 
> > My problem is: I don't get any dhcp6 lease out of the VR. I dug
> > with
> > tcpdump on the host and VR. I see the solicit message arriving, but
> > no
> > answering advertise message. I've tried almost everything at the
> > host:
> > accepting RA, Autoconf, selectively disabling these. Also modifying
> > the
> > dhcpv6 duid as seen on some 4.11 docs didn't change anything.
> > 
> 
> IPv6 does not work with DHCPv6. You should see that when the IPv6
> CIDR 
> is set properly for the shared network in the database that
> CloudStack 
> calculates/generates the IPv6 address the Instance should obtain
> through 
> SLAAC (without privacy addresses!)
> 
> When that works you have security grouping also working. It then
> filters 
> on source addresses from VMs and such.
> 
> We have thousands of VMs connected with IPv6 this way.
> 
> Wido
> 
> > Best case is, that I'm stuck with hosts correctly configured by the
> > router advertisement, but ACS doesn't know about it. So
> > subsequently i
> > can't add records to the respective DNS Zones.
> > 
> > Alternatively, I could skip ACS and add the provable eui-64
> > addresses
> > to the zone, but I'ld like to avoid that.
> > 
> > After a few uneducated peeks into the VR's dnsmasq configuration, I
> > cannot spot any setting for providing dhcp6 leases.
> > 
> > Initially I've deployed the 4.15.0 systemvmtemplate downloaded from
> > http://download.cloudstack.org/systemvm/4.15/
> > Right now, I've switched to the 4.15.1 from the same location, but
> > that
> > didn't change anything.
> > 
> > I've also tried switching the Zone from internal DNS to external
> > DNS
> > and vice versa (these are identical, except the internal DNS is
> > also
> > equipped with the respective IPv6 addresses, which obviously cannot
> > be
> > added to the external DNS). That didn't change anything either.
> > 
> > So, I'ld like to ask for any advise.
> > 
> > Thanks in advance!
> > 
> > Stephan
> > 
> > 

ipv6 capability of shared networks

[Stephan Seitz]

Hi!

I've recently deployed 4.15.0 Advanced Zone with CentOS 8 kvm hosts and
classic linux bridges. I do know that CentOS 7 is preferred, but with
some initial tweaks here and there, i'ld say it's working quite well.

Currently, I'm trying to use IPv6 on shared networks. I'd learned that
IPv6 only does not work, so I switched to IPv6 plus RFC 1918 IPv4
natted at the outer gateway. IPv4 is not a requirement, but if it's
necessary to add, it doesn't harm.

The IPv4 addresses of the deployed hosts are provided by the virtual
router as expected.

My problem is: I don't get any dhcp6 lease out of the VR. I dug with
tcpdump on the host and VR. I see the solicit message arriving, but no
answering advertise message. I've tried almost everything at the host:
accepting RA, Autoconf, selectively disabling these. Also modifying the
dhcpv6 duid as seen on some 4.11 docs didn't change anything.

Best case is, that I'm stuck with hosts correctly configured by the
router advertisement, but ACS doesn't know about it. So subsequently i
can't add records to the respective DNS Zones.

Alternatively, I could skip ACS and add the provable eui-64 addresses
to the zone, but I'ld like to avoid that.

After a few uneducated peeks into the VR's dnsmasq configuration, I
cannot spot any setting for providing dhcp6 leases. 

Initially I've deployed the 4.15.0 systemvmtemplate downloaded from 
http://download.cloudstack.org/systemvm/4.15/
Right now, I've switched to the 4.15.1 from the same location, but that
didn't change anything.

I've also tried switching the Zone from internal DNS to external DNS
and vice versa (these are identical, except the internal DNS is also
equipped with the respective IPv6 addresses, which obviously cannot be
added to the external DNS). That didn't change anything either.

So, I'ld like to ask for any advise.

Thanks in advance!

Stephan

Re: _configDao.isPremium() please help where this information comes from.

[Stephan Seitz]

Hey Boris!

This isn't the problem, just setting

mvn install -P deps -Dnoredist -Dnonoss; # this target doesn't work

export ACS_BUILD_OPTS="-Dnoredist -Dnonoss"; dpkg-buildpackage

I don't know what's the difference between noredist and nonoss.


Anyway if I do build the packages with noredist (which is mentioned way more 
often than nonoss in different documentations / Install.txt's), 
I do not know how to include the respective *jar's getting baked into the 
systemvmtemplate. Otherwise the cloud service won't start and no further
configuration of the systemvm will take place. Say: it won't connect to the 
management.

I'm currently using tools/appliance -> build.sh systemvmtemplate, but afaik, 
these scripts doesn't honor noredist?




Am Mittwoch, den 11.07.2018, 07:03 + schrieb Boris Stoyanov:
> Hi Stephan,
> 
> have you tried:
> mvn clean install -P developer,systemvm -D noredis
> 
> here’s more info 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/How+to+build+CloudStack
> 
> Bobby.
> 
> 
> boris.stoya...@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>   
>  
> 
> On 10 Jul 2018, at 18:19, Stephan Seitz 
> mailto:s.se...@heinlein-support.de>> wrote:
> 
> Thanks for your feedback! Do you know how to build the systemvmtemplate w/ 
> noredist? I didn't find
> anything regarding that in tools/appliance, also cwiki keeps quiet about that.
> 
> Thanks!
> 
> Stepnan
> 
> Am Dienstag, den 10.07.2018, 07:38 -0700 schrieb Frank Maximus:
> That setting is part of a property file of
> cloudstack-plugin-hypervisor-vmware.
> Most likely you are using a system build with noredist.
> In that case you also need a systemvm build that way.
> 
> Kind Regards,
> Frank
> 
> On Tue, Jul 10, 2018 at 12:16 PM Stephan Seitz 
> mailto:s.se...@heinlein-support.de>>
> wrote:
> 
> 
> Hi there,
> 
> Upgrading 4.11.0 to 4.11.1 we found an interesting problem in our (well
> played) staging infrastructure.
> 
> During SSVN provisioning, a somewhat "premium" configuration is detected
> (well we're using noredist since ... ever?).
> So the SSVM is configured with
>   resource=com.cloud.storage.resource.PremiumSecondaryStorageResource
> instead of
> 
> resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
> 
> In fact this particular setup always used (and should further use) simple
> NFS as Secondary Storage.
> 
> Sorry, but I can't find the real source
> of com.cloud.configuration.dao.ConfigurationDao#isPremium() and how to set
> this to false.
> 
> If anyone could shed some light?
> 
> 
> Thank You!
> 
> 
> 
> 
> Mit freundlichen Grüßen,
> 
> Stephan Seitz
> 
> 
> --
> Heinlein Support GmbH
> Schwedter Str. 8/9b, 10119 Berlin
> 
> https://www.heinlein-support.de
> 
> Tel: 030 / 405051-44
> Fax: 030 / 405051-19
> 
> Amtsgericht Berlin-Charlottenburg - HRB 93818 B
> Geschäftsführer: Peer Heinlein - Sitz: Berlin
> 
> 
> Mit freundlichen Grüßen,
> 
> Stephan Seitz
> 
> 
> --
> Heinlein Support GmbH
> Schwedter Str. 8/9b, 10119 Berlin
> 
> https://www.heinlein-support.de
> 
> Tel: 030 / 405051-44
> Fax: 030 / 405051-19
> 
> Amtsgericht Berlin-Charlottenburg - HRB 93818 B
> Geschäftsführer: Peer Heinlein - Sitz: Berlin
> 
> 
Mit freundlichen Grüßen,

Stephan Seitz


--
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

https://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Amtsgericht Berlin-Charlottenburg - HRB 93818 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin



signature.asc
Description: This is a digitally signed message part

Re: _configDao.isPremium() please help where this information comes from.

[Stephan Seitz]

Thanks for your feedback! Do you know how to build the systemvmtemplate w/ 
noredist? I didn't find
anything regarding that in tools/appliance, also cwiki keeps quiet about that.

Thanks!

Stepnan

Am Dienstag, den 10.07.2018, 07:38 -0700 schrieb Frank Maximus:
> That setting is part of a property file of
> cloudstack-plugin-hypervisor-vmware.
> Most likely you are using a system build with noredist.
> In that case you also need a systemvm build that way.
> 
> Kind Regards,
> Frank
> 
> On Tue, Jul 10, 2018 at 12:16 PM Stephan Seitz 
> wrote:
> 
> > 
> > Hi there,
> > 
> > Upgrading 4.11.0 to 4.11.1 we found an interesting problem in our (well
> > played) staging infrastructure.
> > 
> > During SSVN provisioning, a somewhat "premium" configuration is detected
> > (well we're using noredist since ... ever?).
> > So the SSVM is configured with
> >   resource=com.cloud.storage.resource.PremiumSecondaryStorageResource
> > instead of
> > 
> > resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
> > 
> > In fact this particular setup always used (and should further use) simple
> > NFS as Secondary Storage.
> > 
> > Sorry, but I can't find the real source
> > of com.cloud.configuration.dao.ConfigurationDao#isPremium() and how to set
> > this to false.
> > 
> > If anyone could shed some light?
> > 
> > 
> > Thank You!
> > 
> > 
> > 
> > 
> > Mit freundlichen Grüßen,
> > 
> > Stephan Seitz
> > 
> > 
> > --
> > Heinlein Support GmbH
> > Schwedter Str. 8/9b, 10119 Berlin
> > 
> > https://www.heinlein-support.de
> > 
> > Tel: 030 / 405051-44
> > Fax: 030 / 405051-19
> > 
> > Amtsgericht Berlin-Charlottenburg - HRB 93818 B
> > Geschäftsführer: Peer Heinlein - Sitz: Berlin
> > 
> > 
Mit freundlichen Grüßen,

Stephan Seitz


--
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

https://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Amtsgericht Berlin-Charlottenburg - HRB 93818 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin



signature.asc
Description: This is a digitally signed message part

_configDao.isPremium() please help where this information comes from.

[Stephan Seitz]

Hi there,

Upgrading 4.11.0 to 4.11.1 we found an interesting problem in our (well played) 
staging infrastructure.

During SSVN provisioning, a somewhat "premium" configuration is detected (well 
we're using noredist since ... ever?).
So the SSVM is configured with
  resource=com.cloud.storage.resource.PremiumSecondaryStorageResource
instead of
  resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource

In fact this particular setup always used (and should further use) simple NFS 
as Secondary Storage.

Sorry, but I can't find the real source of 
com.cloud.configuration.dao.ConfigurationDao#isPremium() and how to set this to 
false.

If anyone could shed some light?


Thank You!




Mit freundlichen Grüßen,

Stephan Seitz


--
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

https://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Amtsgericht Berlin-Charlottenburg - HRB 93818 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin



signature.asc
Description: This is a digitally signed message part

suggestions for tiny changes in the systemvm templates

[Stephan Seitz]

Hi!

Having 4.11.1 at the horizon (btw. Thank You!), I've recently built packages 
and systemvm templates and wanted to share some thoughts about the systemvm.

Here a few things i came across (I'ld provide a PR, but wanted to discuss that 
in prior)

a) Entropy
SystemVM are usually VM and VM generally do have problems to gather entropy.
-> We could install rng-tools or (slightly better) haveged by default in the 
templates.
pro: having a decent entropy pool available. Would improve SSL at all.
con: well, cost's a few kB and a lightweight daemon running

b) NTP
At least for isolated networks (say VR / RVR) one usually needs to allow 
tcp/123 udp/123 for NTP to the VM behind.
-> We could provide broadcast and/or manycast and/or even unicast at the VR's 
NTP by just changing the /etc/ntp.conf
pro: easier setup of NTP (well, will add Stratum+1) for VM in isolated 
networks. Could also be announced via dhcp?
con: in case of multi- or manycast a few more packets on the wire

c) Monitoring
We're using check-mk for monitoring most parts of our infrastructure. Thank's 
to the Cloudstack API we collect indirect (and sometimes very abstract) health 
data of the systemvm running.
since there's already communication between systemvm and management, we thought 
that implementing the check-mk-agent (listening via xinetd) into the template 
could improve monitoring by
piggyback the metrics on the management node(s).
I'ld see that point different, since - even if the check-mk-agent wont do 
anything without getting queried - I don't know if it's feasible to add 
monitoring support for a solution which might be not
as wide spread as we think here. Anyhow, installation and usage would be very 
simple and (if unused) no impact.


cheers,

- Stephan







Mit freundlichen Grüßen,

Stephan Seitz


--
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

https://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Amtsgericht Berlin-Charlottenburg - HRB 93818 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin



signature.asc
Description: This is a digitally signed message part

Re: Current cloudstack prebuilt images wrong VR address

[Stephan Seitz]

Hi!

AFAIK, the password reset script always tries to connect to TCP/8080 on the
server that offered the dhcp lease. Which usually is the default gateway.
If you're running an isolated network with redundant VR, there was an iptables
rule on the VR blocking the host-IP, so that the request couldn't succeed.

That issue [1] has recently been fixed for 4.11.1.0.


[1] https://github.com/apache/cloudstack/issues/2544


cheers,

- Stephan



Am Montag, den 25.06.2018, 19:08 +0700 schrieb Ivan Kudryavtsev:
> Hello, Devs, Users.
> 
> Today I tried to deploy prebuilt centos image from
> 
> http://dl.openvm.eu/cloudstack/centos/
> 
> Previously I already used that images, so I just registered it and created
> VM. Unfortunately, I found that cloud-init tries network GW as a source for
> metadata when VM starts (to get the password, etc.). So, It doesn't use VR
> to get information but attempts to fetch it from the network default GW.
> So, It fails.
> 
> Next, I downloaded CentOS template which I have used for a year (also from
> http://dl.openvm.eu/cloudstack/centos/), created VM and it works nice. It
> fetches password and other information from correct VR endpoint.
> 
> I suppose there is the error in the current CentOS template. Maybe, someone
> who has built them assumes that default GW is always VR which it may be
> true sometimes (advanced zones, I suppose), but not in general (I use Basic
> Zone).
> 
> Have a good day.
> 
> 
> 
Mit freundlichen Grüßen,

Stephan Seitz

--

Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin




signature.asc
Description: This is a digitally signed message part

Re: Open Summit CFP anyone ?

[Stephan Seitz]

Hey!

I'm also interested :) Actually planning a talk abount attaching active/active 
iSCSI targets w/ TCQ and ALUA capability to a ceph cluster.

Am Donnerstag, den 21.06.2018, 15:43 + schrieb Tutkowski, Mike:
> I am interested.
> 
> > 
> > On Jun 21, 2018, at 2:38 AM, Giles Sirett  
> > wrote:
> > 
> > Hi Andrija
> > - yes I think it would be a great idea for Cloudstack to have some talks 
> > there.
> > 
> > Open source summit appears to be Linux Foundations replacement for 
> > Linuxcon/cloudopen - people from this community have spoken at these before 
> > and had good attendance
> > 
> > Lets coordinate on here some submissions.
> > 
> > First of all, anybody else fancy submitting for this ?
> > 
> > Kind regards
> > Giles
> > 
> > giles.sir...@shapeblue.com 
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> > 
> > 
> > 
> > 
> > -Original Message-
> > From: Andrija Panic  
> > Sent: 20 June 2018 21:52
> > To: dev 
> > Subject: Open Summit CFP anyone ?
> > 
> > Hi all,
> > 
> > Just wondering if anyone submitted CFP here:
> > 
> > https://events.linuxfoundation.org/events/open-source-summit-europe-2018/program/cfp/
> > 
> > Sounds like an interesting place to present the products (ACS) - if anyone 
> > interested, I'm happy to share the work(load) and present jointly, or 
> > similar...
> > 
> > Deadline for CFP is 1st July...
> > 
> > Anyone?
> > 
> > Cheers,
> > Andrija
Mit freundlichen Grüßen,

Stephan Seitz

--

Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin




signature.asc
Description: This is a digitally signed message part

Re: [DISCUSS] Blocking the creation of new Basic Networking zones

[Stephan Seitz]

Hi!


> > With that we would:
> > 
> > - Drop creation of new Basic Networking Zones
> > - Support IPv6 in shared IPv6 networks
> > - Java 9?
> > - Drop support for Ubuntu 12.04
> > - Other fancy stuff?
> - Versioned API: keep v1 API (< v5.0.0)  and create a v2 API >= v5.0.0
> where we fix all inconsistencies (ACL API generally, paging does not
> always work, returned keys sometime camel case (crossZone), a.s.o.)

- Usable Error Messages (including a reason why things failed). Nothing fancy
I think, following the respective Stacktrace in the Logfile, the top most 
exception
shows everything (in most cases), but looks like the last "generic" exception is
reported. 


> > 
> > - Support ConfigDrive in all scenarios properly




signature.asc
Description: This is a digitally signed message part

Re: Why does a VLAN and Network have IP information?

[Stephan Seitz]

Hi!

I just checked our own vlan table and noticed the (obviously dhcp?) range
of the L3 subnet is only present in the `vlan`.`description` field.

It looks like it's getting used if the corresponding 
`network`,`specify_ip_range` is 1.

cheers,

- Stephan


Am Dienstag, den 12.06.2018, 12:11 +0200 schrieb Rafael Weingärtner:
> In theory, the object (either in Java or a DB table) that represents a VLAN
> should not have IP information. However, it seems that someone “reused” the
> object. We would need to check if the IP data stored there is not really
> used before removing it.
> 
> 
> On Tue, Jun 12, 2018 at 11:32 AM, Daan Hoogland 
> wrote:
> 
> > 
> > Wido, I think we can remove ip data from the vlan table, though it is going
> > to require some hacking. Removing the vlan table seems not prudent to me,
> > especially since we now have l2 networks (without ip provisioned).
> > 
> > On Tue, Jun 12, 2018 at 11:12 AM, Wido den Hollander 
> > wrote:
> > 
> > > 
> > > Hi,
> > > 
> > > Looking at our design and tables in the database I'm wondering why both
> > > a VLAN and a Network has IP information.
> > > 
> > > A VLAN is a Layer 2 domain and shouldn't have any IP(4/6) information
> > > and we also seem to store redundant information in there.
> > > 
> > > Below is some information I have in a test database and I'm just trying
> > > to understand why both have IP information.
> > > 
> > > Imho this information should not be stored in the VLAN table as it's
> > > redundant anyway. But still, why is it there? And why do we actually use
> > > the VLAN table? Because even the VLAN tag is stored in the *networks*
> > > table.
> > > 
> > > Wido
> > > 
> > > mysql> select * from vlan limit 1 \G
> > > *** 1. row ***
> > >  id: 1
> > >    uuid: d14f30ab-072e-41b7-bfcf-0aadd156e01d
> > > vlan_id: 0
> > >    vlan_gateway: 192.168.200.1
> > >    vlan_netmask: 255.255.255.0
> > > description: 192.168.200.100-192.168.200.200
> > >   vlan_type: DirectAttached
> > >  data_center_id: 1
> > >  network_id: 203
> > > physical_network_id: 200
> > > ip6_gateway: 2001:db8:100::1
> > >    ip6_cidr: 2001:db8:100::/64
> > >   ip6_range: NULL
> > > removed: NULL
> > > created: 2018-06-09 18:53:26
> > > 1 row in set (0.00 sec)
> > > 
> > > mysql>
> > > 
> > > mysql> select * from networks where id = 203 \G
> > > *** 1. row ***
> > >    id: 203
> > >  name: GuestNetwork1
> > >  uuid: f1f7281d-bedd-422c-bd44-eae9be172157
> > >  display_text: GuestNetwork1
> > >  traffic_type: Guest
> > > broadcast_domain_type: Vlan
> > > broadcast_uri: vlan://untagged
> > >   gateway: 192.168.200.1
> > >  cidr: 192.168.200.0/24
> > >  mode: Dhcp
> > >   network_offering_id: 6
> > >   physical_network_id: 200
> > >    data_center_id: 1
> > > guru_name: DirectNetworkGuru
> > > state: Setup
> > >   related: 203
> > > domain_id: 1
> > >    account_id: 1
> > >  dns1: NULL
> > >  dns2: NULL
> > > guru_data: NULL
> > >    set_fields: 0
> > >  acl_type: Domain
> > >    network_domain: cs1cloud.internal
> > >    reservation_id: NULL
> > >    guest_type: Shared
> > >  restart_required: 0
> > >   created: 2018-06-09 18:53:26
> > >   removed: NULL
> > > specify_ip_ranges: 1
> > >    vpc_id: NULL
> > >   ip6_gateway: NULL
> > >  ip6_cidr: NULL
> > >  network_cidr: NULL
> > >   display_network: 1
> > >    network_acl_id: NULL
> > >   streched_l2: 0
> > > redundant: 0
> > >   external_id: NULL
> > > 1 row in set (0.01 sec)
> > > 
> > > mysql>
> > > 
> > 
> > 
> > --
> > Daan
> > 
> 
> 
Mit freundlichen Grüßen,

Stephan Seitz

--

Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin




signature.asc
Description: This is a digitally signed message part

VR SSL offloading

[Stephan Seitz]

Hi devs!

Thank's to Wei ZHOU who gave us nice codebase for implementing SSL offloading 
into the VR, we started working on a 4.11 and HEAD integration.

Since we'ld like to finally get this patches upstream, we'ld like to start with 
a design paper and therefore ask for access to cwiki.

Generally and AFAIK there's no API change necessary, since 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSL+Termination+Supporthttps://cwiki.apache.org/confluence/display/CLOUDSTACK/SSL+Term
ination+Support describes the current and (except for VR) working feature right 
now.

Additional and optional setup is currently done via TAGs and not integrated 
into the API. I don't know if that approach has any change to go upstream or if 
such configuration has to be moved into API
calls / API parameters.



Thanks!





Mit freundlichen Grüßen,

Stephan Seitz

--

Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin




signature.asc
Description: This is a digitally signed message part

Re: SSL offloading for Virtual Routers / Loadbalancer

[Stephan Seitz]

Hi Wei!

It would be very kind if you could provide some commits.

If it's ok for you, I'ld start a clone on github and try to port
your changes into 4.11 branch (if i find time also into master)

Thanks in advance!

cheers,

- Stephan




Am Donnerstag, den 12.04.2018, 11:36 +0200 schrieb Wei ZHOU:
> Hi Stephan,
> 
> It is done in our own fork based on cloudstack 4.7.1 . We are planning to
> port all our changes to 4.11 with pull requests.
> 
> If you need in urgently, I can share some commits with you (it might not
> work on 4.11).
> 
> -Wei
> 
> 2018-04-12 11:23 GMT+02:00 Stephan Seitz :
> 
> > 
> > Thank's for your feedback Wei!
> > 
> > I'll dscuss the configuration via tags/values with some collegues, but I
> > think that's a very practical way of configuring some LB specialities.
> > 
> > AFAIK there'll be some changes necessary to the codebase. Have you've done
> > that changes internally or do I live in an ideal world and it's available
> > maybe as pullrequest on github?
> > In short, may we use that work? :)
> > 
> > cheers,
> > 
> > - Stephan
> > 
> > Am Donnerstag, den 12.04.2018, 10:59 +0200 schrieb Wei ZHOU:
> > > 
> > > Hi Stephan,
> > > 
> > > We (Leaseweb in Netherlands) had some work on it. It is implemented by
> > > network tags and lb tags.
> > > Here is our KB:
> > > https://kb.leaseweb.com/display/KB/Network%3A+
> > CloudStack#Network:CloudStack-ConfiguringloadbalancerforanIP
> > AddressofanIsolatedNetwork
> > > 
> > > 
> > > -Wei
> > > 
> > > 2018-04-12 10:23 GMT+02:00 Stephan Seitz :
> > > 
> > > > 
> > > > 
> > > > Hi!
> > > > 
> > > > We've got some projects where it would be very reasonable to have SSL
> > > > offloading for https available at the loadbalancing component in the
> > VR.
> > > 
> > > > 
> > > > 
> > > > Since loadbalancing is done via haproxy, that wouldn't be impossible to
> > > > configure (at least for the haproxy.conf).
> > > > 
> > > > I wonder if there's some documentation for the management <-> VR
> > > > communication. IMHO we need to add
> > > > - upload/update of ssl certs from the management node to the
> > respective VR
> > > 
> > > > 
> > > > - configuring/updating SSL as additional LB method (besides the
> > > > tcp-oproxy, tcp and udp methods)
> > > > - some VR's feedback or canary code to inform the management node about
> > > > the LB capabilities(?)
> > > > 
> > > > It would be really nice if someone could share some information. How
> > would
> > > 
> > > > 
> > > > you start that?
> > > > 
> > > > 
> > > > Thanks!
> > > > 
> > > > - Stephan
> > > > 
> > Mit freundlichen Grüßen,
> > 
> > Stephan Seitz
> > 
> > --
> > 
> > Heinlein Support GmbH
> > Schwedter Str. 8/9b, 10119 Berlin
> > 
> > http://www.heinlein-support.de
> > 
> > Tel: 030 / 405051-44
> > Fax: 030 / 405051-19
> > 
> > Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
> > Berlin-Charlottenburg,
> > Geschäftsführer: Peer Heinlein -- Sitz: Berlin
> > 
> > 
> > 
Mit freundlichen Grüßen,

Stephan Seitz

--

Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin




signature.asc
Description: This is a digitally signed message part

Re: SSL offloading for Virtual Routers / Loadbalancer

[Stephan Seitz]

Thank's for your feedback Wei!

I'll dscuss the configuration via tags/values with some collegues, but I think 
that's a very practical way of configuring some LB specialities.

AFAIK there'll be some changes necessary to the codebase. Have you've done that 
changes internally or do I live in an ideal world and it's available maybe as 
pullrequest on github?
In short, may we use that work? :)

cheers,

- Stephan

Am Donnerstag, den 12.04.2018, 10:59 +0200 schrieb Wei ZHOU:
> Hi Stephan,
> 
> We (Leaseweb in Netherlands) had some work on it. It is implemented by
> network tags and lb tags.
> Here is our KB:
> https://kb.leaseweb.com/display/KB/Network%3A+CloudStack#Network:CloudStack-ConfiguringloadbalancerforanIPAddressofanIsolatedNetwork
> 
> -Wei
> 
> 2018-04-12 10:23 GMT+02:00 Stephan Seitz :
> 
> > 
> > Hi!
> > 
> > We've got some projects where it would be very reasonable to have SSL
> > offloading for https available at the loadbalancing component in the VR.
> > 
> > Since loadbalancing is done via haproxy, that wouldn't be impossible to
> > configure (at least for the haproxy.conf).
> > 
> > I wonder if there's some documentation for the management <-> VR
> > communication. IMHO we need to add
> > - upload/update of ssl certs from the management node to the respective VR
> > - configuring/updating SSL as additional LB method (besides the
> > tcp-oproxy, tcp and udp methods)
> > - some VR's feedback or canary code to inform the management node about
> > the LB capabilities(?)
> > 
> > It would be really nice if someone could share some information. How would
> > you start that?
> > 
> > 
> > Thanks!
> > 
> > - Stephan
> > 
Mit freundlichen Grüßen,

Stephan Seitz

--

Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-44
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin




signature.asc
Description: This is a digitally signed message part

SSL offloading for Virtual Routers / Loadbalancer

[Stephan Seitz]

Hi!

We've got some projects where it would be very reasonable to have SSL 
offloading for https available at the loadbalancing component in the VR.

Since loadbalancing is done via haproxy, that wouldn't be impossible to 
configure (at least for the haproxy.conf).

I wonder if there's some documentation for the management <-> VR communication. 
IMHO we need to add
- upload/update of ssl certs from the management node to the respective VR
- configuring/updating SSL as additional LB method (besides the tcp-oproxy, tcp 
and udp methods)
- some VR's feedback or canary code to inform the management node about the LB 
capabilities(?)

It would be really nice if someone could share some information. How would you 
start that?


Thanks!

- Stephan


signature.asc
Description: This is a digitally signed message part

Re: regarding CLOUDSTACK-9598

[Stephan Seitz]

Rafael,

we've noticed the change after upgrading 4.9.0.1 to 4.9.1. The particular
ACS was originally setup as 4.5 and upgraded to 4.6, 4.7, 4.8, 4.9, 4.9.0.1
and finally 4.9.1.

Obviously, we relied on a long existing bug ;) as the new behaviour does make
sense. But most of our vm's are intentionally equipped with multiple
routing tables and require the default-gateway announced from every attached
network.

By now, we've patched the CsDhcp.py inside systemvm.iso/cloud-scripts.tgz as
we didn't find any configuration option.

cheers,

- Stephan

Am Mittwoch, den 11.01.2017, 15:59 -0500 schrieb Rafael Weingärtner:
> Could you provide a little bit of more details of what happened?
> Was this during an upgrade? After an upgrade? what version of ACS?
> 
> On Mon, Jan 9, 2017 at 8:41 AM, Stephan Seitz <
> s.se...@secretresearchfacility.com> wrote:
> 
> > 
> > Hi dev's!
> > 
> > The bug fixed in CLOUDSTACK-9598 (VR offering a default route only on
> > "default" networks) did some harm to our deployed hosts.
> > Most of our hosts rely on the behaviour of getting a default route from
> > every network they're attached to.
> > 
> > Is there some quick fix or configuration setting, to get the old behaviour
> > back?
> > 
> > Thank's in advance!
> > 
> > - Stephan
> > 
> 
> 

regarding CLOUDSTACK-9598

[Stephan Seitz]

Hi dev's!

The bug fixed in CLOUDSTACK-9598 (VR offering a default route only on "default" 
networks) did some harm to our deployed hosts.
Most of our hosts rely on the behaviour of getting a default route from every 
network they're attached to.

Is there some quick fix or configuration setting, to get the old behaviour back?

Thank's in advance!

- Stephan

question about dnsmasq in vr template 4.6 (latest one)

[Stephan Seitz]

Hi!

We're recently hit by some gateway problems. In an 4.9 advanced zone,
shared network, a virtualrouter vm does dns and dhcp.
This router vm sometimes offers itself as default gateway.
I tried to trace this down, but I can't find the real problem.
Anyway, in /etc/dnsmasq.conf all entries are correct. In
/etc/dnsmasq.d/cloud.conf the option 3 shows 0.0.0.0 "sometimes".

As a quick fix, I commented /etc/default/dnsmasq  the
/etc/dnsmasq.d/,.dpkg-... Directory Variable.

I also changed this inside the template, but if I'm destroying this VR
my changes are not there. Is there any kind of caching of vr-templates? 
I'ld expect new VR deployed from that template...

Thanks for any suggestions.

Stephan

Re: trigger scripts by event

[Stephan Seitz]

Thanks Simon!

That was exactly what I was looking for!


Am Montag, den 19.09.2016, 15:44 + schrieb Simon Weller:
> Stephan,
> 
> 
> ACS has supported publishing events to RabbitMQ and Kafta for a few
> releases now.  See this doc: http://docs.cloudstack.apache.org/projec
> ts/cloudstack-administration/en/4.9/events.html
> 
> 
> 
> It's pretty easy to setup and the events make it very easy to trigger
> third party interactions.
> 
> 
> - Si
> 
> 
> 
> From: Stephan Seitz 
> Sent: Monday, September 19, 2016 10:02 AM
> To: dev@cloudstack.apache.org
> Subject: trigger scripts by event
> 
> Hi!
> 
> We'ld like to add some of the provisioned Networks, IPs and VM into
> our
> docu / inventory DB.
> By now, we're gathering infos from the ACS DB as well as from the API
> -
> filtering start/end-date.
> To get things in a more elegant way, I'ld like to run our skripts
> event-triggered.
> 
> So my question is:
> 
> Is there some kind of event-queue where we could identify actions
> like
> provision/destroy vm, add/remove IPs, create/remove an isolated or
> shared network, ...?
> 
> Thanks!
> 
> - Stephan
> 
> 

Re: [DISCUSS] Replacing the VR

[Stephan Seitz]

Hi!

Just to add my 2 cents to that thread:

I'ld really like to see something like vyatta or pfsense integrated as
"standard" VR.

We'd also talked internally about replacing the VR with some more
mature "appliance"-like router distro.

pfsense e.g. comes AFAIK with no defined API but instead has a very
nice GUI.
How would this fit into the concept of configuring the VR via ACS?
Would parts of the GUI - like IP configuration and basic firewall rules
 - hidden or greyed?
Where would one save the configuration, VPN certificates and so on?


- Stephan


Am Sonntag, den 18.09.2016, 15:19 + schrieb Marty Godsey:
> On this note I also mentioned pfsense earlier.
> 
> www.pfsense.org
> 
> 
> Regards,
> Marty Godsey
> 
> -Original Message-
> From: ilya [mailto:ilya.mailing.li...@gmail.com] 
> Sent: Sunday, September 18, 2016 1:09 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
> 
> Our options become much better if we consider BSD based routers.
> 
> Would that be on the table?
> 
> https://en.wikipedia.org/wiki/List_of_router_and_firewall_distributio
> ns
> 
> 
> On 9/16/16 12:04 PM, Will Stevens wrote:
> > 
> > Ya, your points are all valid Simon.  The lack of standard
> > libraries 
> > to handle a lot of the details is a problem.  I don't think it is
> > an 
> > unsolvable problem, but if we spend the time to do that, will we
> > have 
> > something that will work for us for the next 5 years?  This may be
> > the 
> > shortest path to getting us where we need to be for the time being.
> > 
> > What is the best case scenario for the VR going forward which will 
> > last us the next 5 years?  Maybe we just clean up what we have to
> > do a 
> > major restructuring of the pieces and how they are
> > implemented.  We 
> > need to keep in mind how maintainable this implementation is
> > because 
> > that is going to be key going forward IMO.
> > 
> > 
> > 
> > *Will STEVENS*
> > Lead Developer
> > 
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > tw 
> > @CloudOps_
> > 
> > On Fri, Sep 16, 2016 at 2:29 PM, Simon Weller 
> > wrote:
> > 
> > > 
> > > I think our other option is to take a real look at what it would
> > > take 
> > > to fix the VR. In my opinion, a lot of the problems are related
> > > to 
> > > the monolithic python code base and the fact nothing is actually
> > > separated.
> > > 
> > > Secondly, the python scripts (and bash scripts) don't use any 
> > > established libraries to complete tasks and instead shell out and
> > > run 
> > > commands that are both hard to track and hard to parse on return.
> > > 
> > > 
> > > If we daemonized this, used a real api for Agent to VR
> > > communication, 
> > > used common already existing libraries for the system service
> > > and 
> > > network interactions and spent a bit of time separating out code
> > > into 
> > > distinct modules, everything would behave a lot better.
> > > 
> > > 
> > > The pain and suffering is due to years and years of patches and 
> > > constant shelling out to complete tasks in my opinion. If we
> > > spend 
> > > time to rethink how we interact with the VR in general and we 
> > > abstract the systems and networking stuff and use well known and 
> > > stable libraries to do the work, the VR would be much easier to
> > > maintain.
> > > 
> > > 
> > > - Si
> > > 
> > > 
> > > 
> > > 
> > > 
> > > From: Marty Godsey 
> > > Sent: Friday, September 16, 2016 12:24 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: [DISCUSS] Replacing the VR
> > > 
> > > So based upon this discussion would it be prudent to wait on
> > > VyOS 
> > > 2.0? The current VR is giving us issues but would the time
> > > invested 
> > > in another "solution" be wasted especially if by the time
> > > another 
> > > option is chose, then coded, then tested, then implemented and
> > > right 
> > > as that time happened to be when VyOS 2.0 is released.  Of course
> > > you 
> > > said they are just in the scoping range so this could still be a
> > > year or more out.
> > > 
> > > Thoughts?
> > > 
> > > Regards,
> > > Marty Godsey
> > > nSource Solutions
> > > 
> > > -Original Message-
> > > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com]
> > > On 
> > > Behalf Of Will Stevens
> > > Sent: Friday, September 16, 2016 10:31 AM
> > > To: dev@cloudstack.apache.org
> > > Cc: dan...@baturin.org
> > > Subject: Re: [DISCUSS] Replacing the VR
> > > 
> > > I just had a quick chat with a couple of the guys over on the
> > > VyOS chat.
> > > I have CC'ed one of them in case we have more licensing
> > > questions.
> > > 
> > > So here is the status with the license "the code inherited from 
> > > Vyatta and our modifications from it is GPLv2 (strict, not v2+).
> > > The 
> > > config reading library is GPLv2 too, so anything that links to is
> > > is GPLv2.
> > > Some auxiliary components we made after the fork

trigger scripts by event

[Stephan Seitz]

Hi!

We'ld like to add some of the provisioned Networks, IPs and VM into our
docu / inventory DB.
By now, we're gathering infos from the ACS DB as well as from the API -
filtering start/end-date.
To get things in a more elegant way, I'ld like to run our skripts
event-triggered.

So my question is:

Is there some kind of event-queue where we could identify actions like
provision/destroy vm, add/remove IPs, create/remove an isolated or
shared network, ...?

Thanks!

- Stephan

Re: Consoleproxy FF Quicksearch

[Stephan Seitz]

As a quick fix, you could modify the systemvm.iso which is used to
initialize the console-proxy-vm.

Im not sure if it's 
/usr/share/cloudstack-common/vms/systemvm.iso
or
/usr/share/cloudstack-management/webapps/client/WEB-
INF/classes/vms/systemvm.iso

Inside the systemvm.iso you'll find a systemvm.zip which contains
js/ajaxviewer.js

Here's the patch https://issues.apache.org/jira/browse/CLOUDSTACK-9164

I think it's 10 minutes worth to modify the zip content and create a
new iso.

If you're not attempting to destroy your console-proxy-vm you could
also modify /usr/local/cloud/forgot-the-exact-path.../ajaxviewer.js 
inside the running instance.

Oh, and flushing the browsercache is also necessery.

cheers,

- Stephan



Am Montag, den 14.12.2015, 20:55 -0500 schrieb Pierre-Luc Dion:
> Yet that is very annoying, no chance it could make it in 4.7.0rc2 ?
> ...
> 
> Thanks Stephan!
> 
> 
> On Mon, Dec 14, 2015 at 10:39 AM, Nux!  wrote:
> 
> > 
> > Stephan,
> > 
> > Thanks for this. It was driving me nuts, I even complained to
> > Mozilla lol.
> > 
> > I ended up installing an extension that disabled the quicksearch on
> > "/"
> > press.
> > https://freeshell.de//~kaosmos/index-en.html#searchkeys
> > 
> > --
> > Sent from the Delta quadrant using Borg technology!
> > 
> > Nux!
> > www.nux.ro
> > 
> > - Original Message -
> > > 
> > > From: "Stephan Seitz" 
> > > To: dev@cloudstack.apache.org
> > > Sent: Monday, 14 December, 2015 15:36:33
> > > Subject: Consoleproxy FF Quicksearch
> > > 
> > > Hi devs!
> > > 
> > > Despite the fact I was too lazy that weekend cloning the repo and
> > > filing
> > > a pull request, I hope that small fix could make it upstream
> > > anyway.
> > > 
> > > --- ajaxviewer.js.orig 2015-12-13 11:10:47.851177577 +
> > > +++ ajaxviewer.js 2015-12-13 14:14:32.367382547 +
> > > @@ -649,6 +649,7 @@
> > > this.sendingEventInProgress = false;
> > > ajaxViewer.installMouseHook();
> > > ajaxViewer.installKeyboardHook();
> > > + ajaxViewer.panel.parent().focus();
> > > 
> > > $(window).bind("resize", function() {
> > > ajaxViewer.onWindowResize();
> > > @@ -1259,6 +1260,7 @@
> > > case 38 : // UP
> > > case 39 : // RIGHT
> > > case 40 : // DOWN
> > > + case 47 : // slash (prevent quick search)
> > > return false;
> > > }
> > > 
> > > 
> > > 
> > > I've already filed this as bug
> > > https://issues.apache.org/jira/browse/CLOUDSTACK-9164
> > > 
> > > These two lines prevent Firefox from starting quicksearch which
> > > is
> > > really annoying.
> > > 
> > > - Stephan

Re: cloudstack-usage no longer working / error saving account to cloud_usage db

[Stephan Seitz]

Hi Rohit,

thanks!

Having a recent DB backup at hand and only based on guesses, I did some
further experiments yesterday :)
- Rerun /usr/share/cloudstack-common/scripts/util/migrate-
dynamicroles.py
- Checked cloudstack-usage 4.8.0.1 instead of 4.9
- Updated back to cloudstack-usage 4.9
... and magically, it works after I changed the pid of the latest
cloud_usage.usage_job to the corresponding pid.

Though, I don't think thats the recommended way for a fix as I don't
know why it's working...

cloud.account shows role_id for every active account.
cloud_usage.account now shows role_id for these accounts also.
Only PrjAcct-$projectname-$id has role_id set to NULL, but I assume
this is correct since Projects are not assigned to roles.

Anyway, the metric/quota reports are working!

- Stephan

Am Donnerstag, den 18.08.2016, 09:03 + schrieb Rohit Yadav:
> Hi Stephan,
> 
> 
> In cloud_usage.account `role_id` can be NULL as there is no user of
> this field within the usage server. In cloud.account, the `role_id`
> should be automatically populated/migrated when you upgraded. From
> your shared db query result, I'm not sure if that's a select query on
> cloud.account or cloud_usage.account, can you confirm it?
> 
> 
> Based on the exception, we can only get that if the account being
> saved don't have any role_id defined. With a new account created and
> usage records generated, I could not reproduce your issue. It is
> likely caused by an account in cloud.account table whose role_id is
> NULL.
> 
> 
> Can you check (and share) that all of your accounts in cloud database
> (cloud.account) have non-NULL role_id? Please fix anything that is
> NULL. For root admin account type use set role_id=1, for resource
> admin set role_id=2, for domain admin set role_id=3 and for user
> account set role_type=4;
> 
> 
> Regards.
> 
> rohit.ya...@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>   
>  
> 

cloudstack-usage no longer working / error saving account to cloud_usage db

[Stephan Seitz]

Hi!

We've recently noticed cloudstack-usage is no longer able to write
usage data into mysql. I assume this happened after updateing to
4.8.0.1 or 4.9 - we missed that error in our testsetup...

Here's the usage.log: http://pastebin.com/Zpvnw4xX

Finally it fails with following message:

ERROR [usage.dao.UsageDaoImpl] (Usage-Job-1:null) (logid:) error saving
account to cloud_usage db


I've checked the changes between 4.8 and 4.9

https://fossies.org/diffs/apache-cloudstack/4.8.0.1-src_vs_4.9.0-src/en
gine/schema/src/com/cloud/usage/dao/UsageDaoImpl.java-diff.html

cloud_usage.account now shows role_id, but that table looks good to me:

mysql> show create table account \G
*** 1. row ***
   Table: account
Create Table: CREATE TABLE `account` (
  `id` bigint(20) unsigned NOT NULL,
  `account_name` varchar(100) DEFAULT NULL COMMENT 'an account name set
by the creator of the account, defaults to username for single
accounts',
  `uuid` varchar(40) DEFAULT NULL,
  `type` int(1) unsigned NOT NULL,
  `role_id` bigint(20) unsigned DEFAULT NULL,
  `domain_id` bigint(20) unsigned DEFAULT NULL,
  `state` varchar(10) NOT NULL DEFAULT 'enabled',
  `removed` datetime DEFAULT NULL COMMENT 'date removed',
  `cleanup_needed` tinyint(1) NOT NULL DEFAULT '0',
  `network_domain` varchar(100) DEFAULT NULL COMMENT 'Network domain
name of the Vms of the account',
  `default_zone_id` bigint(20) unsigned DEFAULT NULL,
  `default` int(1) unsigned NOT NULL DEFAULT '0' COMMENT '1 if account
is default',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uc_account__uuid` (`uuid`),
  KEY `i_account__removed` (`removed`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
1 row in set (0.00 sec)

Having a look at that data, I'm a bit confused since uuid,
network_domain and default_zone_id are NULL for every account.

As an example:

mysql> select * from account limit 1 \G
*** 1. row ***
 id: 1
   account_name: system
   uuid: NULL
   type: 1
role_id: NULL
  domain_id: 1
  state: enabled
removed: NULL
 cleanup_needed: 0
 network_domain: NULL
default_zone_id: NULL
default: 0
1 row in set (0.00 sec)


I've already looked at plain SQL statements logged by the mysqld, but
that's really messy and grepping for INSERT/UPDATE queries defined in
UsageDaoImpl didn't show anything. Finally grepping for 'statistics' in
the raw sql-log doesn't result a line. 

Could someone please shed some light how I could get the usage service
collecting and reporting data again?

Thanks in advance!

cheers,

- Stephan

Re: ./certs/realhostip.keystore in SSVN

[Stephan Seitz]

Sadhu,

thank you for your feedback. unfortunately, my problem is not using own
certificates on the SSVM/CPVM. This is already done.

We're missing some newer Root-CA certificates in the keystore, so
therefor some https-download-URL are not working since SSVM doesn't know
about that (even valid) root-CA.

My question is, how to I add root-CA to the keystore (say, an equivalent
to the system-wide "aptitude upgrade ca-certificates").

I think, I could also file a jira ticket but I want to understand the
mechanisms in prior.

Right now, we encounter Problems with D/L URL secured by LetsEncrypt and
some Comodo RSA Roots with SHA256 Intermediates.

I already fixed that by adding the respective certificates to the
keystore, but I assume it's better to get that persistent :)

Oh, and we're running 4.7 w/ 4.6 SSVM/CPVM-template.

cheers,

- Stephan

Am Mittwoch, den 16.03.2016, 09:22 + schrieb Suresh Sadhu: 
> Please check this link:
> http://sadhusuresh.blogspot.in/2015/01/t-hings-you-should-consider-while.html
> 
> 
> your uploaded certis loaded in the database in keystore table, after upload 
> ssl successful it recreate ssvm/cpvm with new key .
> 
> regards
> sadhu
> 
> 
> -Original Message-
> From: Stephan Seitz [mailto:s.se...@secretresearchfacility.com] 
> Sent: Wednesday, March 16, 2016 2:13 PM
> To: dev@cloudstack.apache.org
> Subject: ./certs/realhostip.keystore in SSVN
> 
> Hey devs!
> 
> I just added some recent root-CA certificates to running SSVM instances.
> I'ld like to persist this by updating the realhostip.keystore, and can't 
> locate that keystore file inside the template.vhd.
> Even after searching the git repo, I don't know where this file is deployed 
> from.
> 
> Could someone please shed some light where to find that keystore source?
> 
> Thanks in advance!
> 
> cheers,
> 
> - Stephan
> 
> 
> 
> 
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is the 
> property of Accelerite, a Persistent Systems business. It is intended only 
> for the use of the individual or entity to which it is addressed. If you are 
> not the intended recipient, you are not authorized to read, retain, copy, 
> print, distribute or use this message. If you have received this 
> communication in error, please notify the sender and delete all copies of 
> this message. Accelerite, a Persistent Systems business does not accept any 
> liability for virus infected mails.

Re: VmStatsCollector and memory utilization

[Stephan Seitz]

Thanks for this info!

> The pull request is still open. So the feature is still not available.

> >Memory utilisation of Vm has been introduced in 4.6.1 with CLOUDSTACK-8800
> >.
> >I think you are using lower version.

./certs/realhostip.keystore in SSVN

[Stephan Seitz]

Hey devs!

I just added some recent root-CA certificates to running SSVM instances.
I'ld like to persist this by updating the realhostip.keystore, and can't
locate that keystore file inside the template.vhd.
Even after searching the git repo, I don't know where this file is
deployed from.

Could someone please shed some light where to find that keystore source?

Thanks in advance!

cheers,

- Stephan

VmStatsCollector and memory utilization

[Stephan Seitz]

Hi Devs!

I'm just playing with graphite/grafana and stats.output.uri.

First, thanks! That's a great feature.
But, there's no memory information so far, right?

I looked into StatsCollector.java but even the VmStats interface shows
only ressources for cpu, net- and disk-I/O.

Are any memory statistics collected?

cheers,

- Stephan

signature.asc
Description: This is a digitally signed message part

Re: [update] ACS management unable to connect to xenserver hosts after reboot

[Stephan Seitz]

Paul,

thank you for your hint! That was the root cause of our problems:

https://bugs.launchpad.net/ubuntu/+source/ifenslave/+bug/1288196

We simply just didn't know that the msid is derived from the MAC.

Our services tend to be manageable again ;)

Thanks again guys!

cheers,

- Stephan


Am Mittwoch, den 17.02.2016, 19:16 + schrieb Paul Angus: 
> The msid is generated from the MAC address of the host when the service 
> starts, the two IDs are subtly different do you have some bonding in place 
> that is maybe miss-configured, which is generating the 2nd MAC?
> 
> 
> 
> Paul Angus
> VP Technology   ,   ShapeBlue



> 
> 
> t:  @cloudyangus
> 
> e:  paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com>|
>   w:  www.shapeblue.com<http://www.shapeblue.com>
> 
> 
> 
> 
> 
> -Original Message-
> From: Simon Weller [mailto:swel...@ena.com]
> Sent: Wednesday, February 17, 2016 6:11 PM
> To: dev@cloudstack.apache.org
> Cc: Glenn Wagner 
> Subject: Re: [update] ACS management unable to connect to xenserver hosts 
> after reboot
> 
> Stephan,
> 
> When you restart the management process, do you see any logs indicating it's 
> trying to peer with another management server?
> 
> - Si
> 
> 
> From: Stephan Seitz 
> Sent: Wednesday, February 17, 2016 9:28 AM
> To: dev@cloudstack.apache.org
> Cc: Glenn Wagner
> Subject: Re: [update] ACS management unable to connect to xenserver hosts 
> after reboot
> 
> Glenn,
> 
> thanks for your reply. Unfortunately the SSVM has been destroyed.
> 
> We don't have any firewall in between. ACS and XenServers are located in the 
> same /22. I've double checked every connection and there's no iptables or 
> similar in the way.
> Instead of the SSVM, I've just successfully checked if the consoleproxy VM is 
> able to connect to Port 8250.
> 
> To me it looks, like there's some strange "identity" problem.
> 
> mysql> select * from mshost;
> +++---+--+---+-++--+-+-+-+
> | id | msid | runid | name | state |
> version | service_ip | service_port | last_update | removed |
> alert_count |
> +++---+--+---+-++--+-+-+-+
> | 1 | 57177340185274 | 1455209855143 | acs-management-1 | Up | 4.7.1
> | 10.97.13.1 | 9090 | 2016-02-12 16:55:56 | NULL |
> 0 |
> | 3 | 57177340185273 | 1455639355379 | acs-management-1 | Up | 4.7.1
> | 10.97.13.1 | 9090 | 2016-02-17 11:31:50 | NULL |
> 0 |
> +++---+--+---+-++--+-+-+-+
> 2 rows in set (0.00 sec)
> 
> Indeed, there is (and always has been) only one management host in this 
> infrastructure.
> 
> With sqldumps at hand, we removed the second row and purged all the related 
> jobs to that id, but after restarting cloudstack-management, this entry wasi 
> created again.
> 
> Maybe, I'm completely wrong, but is it possible that our management host 
> "thinks" there's another management host responsible for our cluster?
> 
> Since we're fiddling at least two days without any success here, I'm willing 
> to get a few consulting hours thrown on that.
> 
> cheers,
> 
> - Stephan
> 
> btw. sorry, if this is a double post, but I think the list ate my last mail...
> 
> 
> Am Dienstag, den 16.02.2016, 20:39 + schrieb Glenn Wagner:
> > Hi Stephan,
> >
> > Check that you can telnet port 8250 on the management server from SSVM
> > , check that iptables has been setup correctly Looks like it’s a
> > firewall issue on the ACS Management server
> >
> > Thanks
> > Glenn
> >
> >
> >
> >
> >
> > ShapeBlue
> > Glenn Wagner
> > Senior
> > Consultant
> > ,
> > ShapeBlue
> > d:
> > | s: +27 21 527 0091
> > |
> > m:
> > +27 73 917 4111
> > e:
> > glenn.wag...@shapeblue.com | t:
> > |
> > w:
> > www.shapeblue.com
> > a:
> > 2nd Floor, Oudehuis Centre, 122 Main Rd, Somerset West Cape Town 7130
> > South Africa
> >
> > Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> > Services India LLP is a company incorporated in India and is operated
> > under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda
> > is a company incorporated

Re: [update] ACS management unable to connect to xenserver hosts after reboot

[Stephan Seitz]

Glenn,

thanks for your reply. Unfortunately the SSVM has been destroyed.

We don't have any firewall in between. ACS and XenServers are located in
the same /22. I've double checked every connection and there's no
iptables or similar in the way.
Instead of the SSVM, I've just successfully checked if the consoleproxy
VM is able to connect to Port 8250.

To me it looks, like there's some strange "identity" problem.

mysql> select * from mshost;
+++---+--+---+-++--+-+-+-+
| id | msid   | runid | name | state |
version | service_ip | service_port | last_update | removed |
alert_count |
+++---+--+---+-++--+-+-+-+
|  1 | 57177340185274 | 1455209855143 | acs-management-1 | Up| 4.7.1
| 10.97.13.1 | 9090 | 2016-02-12 16:55:56 | NULL|
0 |
|  3 | 57177340185273 | 1455639355379 | acs-management-1 | Up| 4.7.1
| 10.97.13.1 | 9090 | 2016-02-17 11:31:50 | NULL|
0 |
+++---+--+---+-++--+-+-+-+
2 rows in set (0.00 sec)

Indeed, there is (and always has been) only one management host in this
infrastructure.

With sqldumps at hand, we removed the second row and purged all the
related jobs to that id, but after restarting cloudstack-management,
this entry wasi created again.

Maybe, I'm completely wrong, but is it possible that our management host
"thinks" there's another management host responsible for our cluster?

Since we're fiddling at least two days without any success here, I'm
willing to get a few consulting hours thrown on that.

cheers,

- Stephan

btw. sorry, if this is a double post, but I think the list ate my last
mail...


Am Dienstag, den 16.02.2016, 20:39 + schrieb Glenn Wagner: 
> Hi Stephan,
> 
> Check that you can telnet port 8250 on the management server from
> SSVM , check that iptables has been setup correctly 
> Looks like it’s a firewall issue on the ACS Management server
> 
> Thanks
> Glenn
> 
> 
> 
> 
> 
> ShapeBlue
> Glenn Wagner
> Senior
> Consultant
> , 
> ShapeBlue
> d: 
>  | s: +27 21 527 0091
>  | 
> m: 
> +27 73 917 4111
> e: 
> glenn.wag...@shapeblue.com | t: 
>  | 
> w: 
> www.shapeblue.com
> a: 
> 2nd Floor, Oudehuis Centre, 122 Main Rd, Somerset West Cape Town 7130 South 
> Africa
> 
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated
> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda
> is a company incorporated in Brasil and is operated under license from
> Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The
> Republic of South Africa and is traded under license from Shape Blue
> Ltd. ShapeBlue is a registered trademark.
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related
> companies. If you are not the intended recipient of this email, you
> must neither take any action based upon its contents, nor copy or show
> it to anyone. Please contact the sender if you believe you have
> received this email in error.
> 
> 
> 
> 
> 
> -Original Message-
> From: Stephan Seitz [mailto:s.se...@secretresearchfacility.com] 
> Sent: Tuesday, 16 February 2016 5:19 PM
> To: us...@cloudstack.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: [update] ACS management unable to connect to xenserver hosts
> after reboot
> 
> Hi again!
> 
> I think we've found the root source, but are unable to mitigate that:
> 
> 2016-02-16 16:13:22,217 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-8:null) Seq 6--1: MgmtId 57177340185273: Req:
> Routing to peer
> 2016-02-16 16:13:22,217 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-9:null) Seq 6--1: MgmtId 57177340185273: Req:
> Cancel request received
> 2016-02-16 16:13:22,899 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-10:null) Seq 1-4458000681143369786: MgmtId
> 57177340185273: Req: Resource [Host:1] is unreachable: Host 1: Link is
> closed
> 2016-02-16 16:13:22,899 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-10:null) Seq 1--1: MgmtId 57177340185273: Req:
> Routing to peer
> 2016-02-16 16:1

Re: [update] ACS management unable to connect to xenserver hosts after reboot

[Stephan Seitz]

Glenn,

thanks for your reply. Unfortunately the SSVM has been destroyed.

We don't have any firewall in between. ACS and XenServers are located in
the same /22. I've double checked every connection and there's no
iptables or similar in the way.
Instead of the SSVM, I've just successfully checked if the consoleproxy
VM is able to connect to Port 8250.

To me it looks, like there's some strange "identity" problem.

mysql> select * from mshost;
+++---+--+---+-++--+-+-+-+
| id | msid   | runid | name | state |
version | service_ip | service_port | last_update | removed |
alert_count |
+++---+--+---+-++--+-+-+-+
|  1 | 57177340185274 | 1455209855143 | acs-management-1 | Up| 4.7.1
| 10.97.13.1 | 9090 | 2016-02-12 16:55:56 | NULL|
0 |
|  3 | 57177340185273 | 1455639355379 | acs-management-1 | Up| 4.7.1
| 10.97.13.1 | 9090 | 2016-02-17 11:31:50 | NULL|
0 |
+++---+--+---+-++--+-+-+-+
2 rows in set (0.00 sec)

Indeed, there is (and always has been) only one management host in this
infrastructure.

With sqldumps at hand, we removed the second row and purged all the
related jobs to that id, but after restarting cloudstack-management,
this entry wasi created again.

Maybe, I'm completely wrong, but is it possible that our management host
"thinks" there's another management host responsible for our cluster?

Since we're fiddling at least two days without any success here, I'm
willing to get a few consulting hours thrown on that.

cheers,

- Stephan

Am Dienstag, den 16.02.2016, 20:39 + schrieb Glenn Wagner: 
> Hi Stephan,
> 
> Check that you can telnet port 8250 on the management server from
> SSVM , check that iptables has been setup correctly 
> Looks like it’s a firewall issue on the ACS Management server
> 
> Thanks
> Glenn
> 
> 
> 
> 
> 
> ShapeBlue
> Glenn Wagner
> Senior
> Consultant
> , 
> ShapeBlue
> d: 
>  | s: +27 21 527 0091
>  | 
> m: 
> +27 73 917 4111
> e: 
> glenn.wag...@shapeblue.com | t: 
>  | 
> w: 
> www.shapeblue.com
> a: 
> 2nd Floor, Oudehuis Centre, 122 Main Rd, Somerset West Cape Town 7130 South 
> Africa
> 
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated
> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda
> is a company incorporated in Brasil and is operated under license from
> Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The
> Republic of South Africa and is traded under license from Shape Blue
> Ltd. ShapeBlue is a registered trademark.
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related
> companies. If you are not the intended recipient of this email, you
> must neither take any action based upon its contents, nor copy or show
> it to anyone. Please contact the sender if you believe you have
> received this email in error.
> 
> 
> 
> 
> 
> -Original Message-
> From: Stephan Seitz [mailto:s.se...@secretresearchfacility.com] 
> Sent: Tuesday, 16 February 2016 5:19 PM
> To: us...@cloudstack.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: [update] ACS management unable to connect to xenserver hosts
> after reboot
> 
> Hi again!
> 
> I think we've found the root source, but are unable to mitigate that:
> 
> 2016-02-16 16:13:22,217 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-8:null) Seq 6--1: MgmtId 57177340185273: Req:
> Routing to peer
> 2016-02-16 16:13:22,217 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-9:null) Seq 6--1: MgmtId 57177340185273: Req:
> Cancel request received
> 2016-02-16 16:13:22,899 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-10:null) Seq 1-4458000681143369786: MgmtId
> 57177340185273: Req: Resource [Host:1] is unreachable: Host 1: Link is
> closed
> 2016-02-16 16:13:22,899 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-10:null) Seq 1--1: MgmtId 57177340185273: Req:
> Routing to peer
> 2016-02-16 16:13:22,900 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentManager-Handler-1

[update] ACS management unable to connect to xenserver hosts after reboot

[Stephan Seitz]

Hi again!

I think we've found the root source, but are unable to mitigate that:

2016-02-16 16:13:22,217 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-8:null) Seq 6--1: MgmtId 57177340185273: Req:
Routing to peer
2016-02-16 16:13:22,217 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-9:null) Seq 6--1: MgmtId 57177340185273: Req:
Cancel request received
2016-02-16 16:13:22,899 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-10:null) Seq 1-4458000681143369786: MgmtId
57177340185273: Req: Resource [Host:1] is unreachable: Host 1: Link is
closed
2016-02-16 16:13:22,899 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-10:null) Seq 1--1: MgmtId 57177340185273: Req:
Routing to peer
2016-02-16 16:13:22,900 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-11:null) Seq 1--1: MgmtId 57177340185273: Req:
Cancel request received
2016-02-16 16:13:22,905 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentManager-Handler-12:null) Seq 3-2144839322535198778: MgmtId
57177340185273: Req: Resource [Host:3] is unreachable: Host 3: Link is
closed

Here's a longer excerpt from the logfile during startup:

http://pastebin.com/SftVJCs4

Maybe someone knows how to resolve this? To me it looks like our single
management-host has some kind of identity crisis? 


Am Dienstag, den 16.02.2016, 15:12 +0100 schrieb Stephan Seitz: 
> Hi acs gurus!
> 
> We're currently facing a really strange problem after two somewhat
> simple steps.
> 1. Reboot Management-Node (well there is also a 2nd. NFS-Storage
> located)
> 2. Upgrade 4.7.0 to 4.7.1
> 
> Both steps seemed successful and running, but after a few days I've
> noticed the SSVM in "running, not connected" state, so I decided to
> restart the SSVM. That's where all the trouble begun...
> 
> I've pasted a somewhat repetive log excerpt here
> http://pastebin.com/8MM6XUBk
> 
> If I try to (force) reconnect a host, we're getting huge repetive log
> entries like pasted here http://pastebin.com/cNR3TtkG
> 
> Cloudmonkey quits with following Response:
> 
> (local) 🐵 > reconnect host id=df4182f8-24a0-40ca-9ccc-6489f374cd4c
> Error Connection refused by server: ('Connection aborted.',
> BadStatusLine("''",))
> 
> 
> I've tcpdump'ed relevant traffic between management and xenservers and
> found simply nothing except some (i assume) unrelated NFS-Packets.
> 
> Could please someone shed some light, how to fix that?
> 
> Thanks in advance!
> 
> - Stephan

ACS management unable to connect to xenserver hosts after reboot

[Stephan Seitz]

Hi acs gurus!

We're currently facing a really strange problem after two somewhat
simple steps.
1. Reboot Management-Node (well there is also a 2nd. NFS-Storage
located)
2. Upgrade 4.7.0 to 4.7.1

Both steps seemed successful and running, but after a few days I've
noticed the SSVM in "running, not connected" state, so I decided to
restart the SSVM. That's where all the trouble begun...

I've pasted a somewhat repetive log excerpt here
http://pastebin.com/8MM6XUBk

If I try to (force) reconnect a host, we're getting huge repetive log
entries like pasted here http://pastebin.com/cNR3TtkG

Cloudmonkey quits with following Response:

(local) 🐵 > reconnect host id=df4182f8-24a0-40ca-9ccc-6489f374cd4c
Error Connection refused by server: ('Connection aborted.',
BadStatusLine("''",))


I've tcpdump'ed relevant traffic between management and xenservers and
found simply nothing except some (i assume) unrelated NFS-Packets.

Could please someone shed some light, how to fix that?

Thanks in advance!

- Stephan

Re: [VOTE] Apache CloudStack 4.7.0

[Stephan Seitz]

Hi devs,

could someone please test VR in advanced networking? In detail: adding
>=10 public IPs and adding >=5 firewall and portforwarding rules to
each of the IPs.
I've noticed that each single call into the VR updates and verifies the
whole set of rules. I don't know if this was introduced around 4.5 or
if I just hit an edgecase recently. Using more than a few IPs/rules is
not only really time consuming, in case of a network/VR restart it also
leads to a VR shutdown (maybe some watchdog kicks in after a timeout).

Thanks, and sorry that I don't have enough ressources this year to
check this on my own.

Stephan


Am Mittwoch, den 16.12.2015, 08:34 +0100 schrieb Boris Schrijver:
> Hi all,
> 
> Hereby my vote for the ACS 4.7.0 RC 1
> 
> Details:
> 
> Vote: +1
> 
> Besides the integration tests (which all ran fine) I've also tested
> the
> following:
> 
>  - S3 Integration (Secondary Storage) with NFS Staging store
>  - Ceph RBD storage (Primary Storage)
>  - Basic networking with security groups 
> 
> -- 
> 
> Met vriendelijke groet / Kind regards,
> 
> Boris Schrijver
> 
> PCextreme B.V.
> 
> http://www.pcextreme.nl/contact
> Tel direct: +31 (0) 118 700 215
> 
> > On December 16, 2015 at 12:34 AM Remi Bergsma <
> > rberg...@schubergphilis.com>
> > wrote:
> > 
> > 
> > +1 (binding)
> > 
> > This vote is based on testing on a real cloud.
> > 
> > At Schuberg Philis we built a new cloud based on ACS 4.7.0RC1
> > (upgraded from
> > 4.6). It runs XenServer 6.5 clusters, a CentOS 7 management
> > cluster, Galera DB
> > (also on CentOS 7), HA proxies (CentOS 7), NFS storage and
> > Nicira/NSX for
> > networking/SDN. Capacity to start with is about 12TB ram and 500+
> > cores.
> > Secondary storage is an S3 compatible solution (Cloudian) with NFS
> > staging
> > store. Configured LDAP for authentication.
> > 
> > Before a go-live we always do thorough testing and try to break the
> > setup
> > emulating crashes and problems.
> > 
> > We successful executed these CloudStack related tests:
> > 
> >   *   crashed a hypervisor which was poolmaster and saw recovery in
> > about 5
> > min (tested with/without returning of the hypervisor)
> >   *   crashed a hypervisor which was NOT poolmaster and saw
> > recovery in about
> > 5 min (tested with/without returning of the hypervisor)
> >   *   crashed overbooked hypervisor in a cluster with too many VMs
> > to run on
> > the remaining hypervisors. Saw it recovered fully when crashed
> > hypervisor
> > returned. (this you don’t want to happen, but at least the recovery
> > was
> > automatic)
> >   *   crashed one of the app servers; the other one continued and
> > took over.
> > No user impact.
> >   *   crashed the main Galera DB node, the two remaining nodes
> > survived and
> > kept working. No CloudStack impact.
> >   *   did performance tests and walked into the default 200mbps
> > limit on
> > tiers. When we removed it (aka configured it properly) we could use
> > full
> > 10gbps.
> >   *   crashed the NFS staging store, could not deploy VM from
> > template that
> > was not already on primary storage. Recovered automatically when
> > NFS returned
> > and VM was started.
> >   *   many functional tests, also covered In the integration tests
> > (spin many
> > VMs, migrate, make port forwardings etc).
> >   *   executed patch round (live migrating vms around), rebooting
> > all
> > hypervisors without user impact.
> > 
> > Conclusion:
> > It’s pretty solid, even with one management server and a degraded
> > database we
> > could still continue and operate existing VMs and start new ones.
> > When the
> > nodes returned recovery was automatic.
> > We feel confident running production with Apache CloudStack 4.7 and
> > will start
> > doing so later today!
> > 
> > Regards,
> > Remi
> > 
> > PS:
> > The integration tests we run in the dev/test environments were also
> > successful
> > (the same I executed on the PRs that were merged).
> > 
> > 
> > 
> > From: Remi Bergsma
> > mailto:rberg...@schubergphilis.com>>
> > Date: Sunday 13 December 2015 21:27
> > To: "dev@cloudstack.apache.org"
> > mailto:dev@cloudstack.apache.org>>
> > Subject: [VOTE] Apache CloudStack 4.7.0
> > 
> > Hi all,
> > 
> > Since our 4.6.0 release (on Nov 13th, exactly 1 month ago), we have
> > merged
> > 100+ pull requests [1] with lots of bug fixes, refactoring and of
> > course new
> > features. Time for a new release!
> > 
> > 
> > I've created a 4.7.0 release candidate, with the following
> > artifacts up for a
> > vote:
> > 
> > Git Branch and Commit SH:
> > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog
> > ;h=4.7.0-RC20151213T2109
> > 
> > Commit: 2f26a859a971a9852ed9f6f34fe35e52fe6028a9
> > 
> > Source release (checksums and signatures are available at the same
> > location):
> > https://dist.apache.org/repos/dist/dev/cloudstack/4.7.0/
> > 
> > PGP release keys (signed using A47DDC4F):
> > https://dist.apache.org/repos/dist/release/clouds

Re: additional keyboard languages in Consoleproxy

[Stephan Seitz]

Thank's for pointing to that PR.

Do you think it's useful to cut the keyboard selector?

As long as there's no wayu changing the keyboard setting after an
instance has been provisioned, the average user would need to fiddle
with API calls.

Additionally, the keyboard doesn't necessary have to be consistent
inside the instance, so a drop down is really handy.


Am Dienstag, den 15.12.2015, 04:53 + schrieb Anshul Gangwar: 
> There is already a PR requested some 4 months back addressing some of these 
> issues. With that patch one has to add only key mappings to support new 
> languages.
> 
> https://github.com/apache/cloudstack/pull/669

additional keyboard languages in Consoleproxy

[Stephan Seitz]

Hi!

I've recently experimented with different approaches to add languages to
the consoleproxy in a more modular way than the very hardcoded
ajaxviewer / ajaxkeys / index.jsp

I'ld like to ask if someone already did some work in that direction, so
I could join in?
Alternatively, I'll start on a somewhat bigger rewrite of that code.

My thoughts so far are:

- there's no sql based information which languages are supported
- it's in consoleproxy's index.jsp (very hardcoded for the dropdown)
additionally in ajaxkeys.js partly in array structures, partly in simple
suffixed variables/constants. ajaxviewer has a very own wording, also in
simple suffixed globals/constants.
- also the code for provisioning vms shows a completely independend
language selector. at least for 4.6.x there are different informations
(french in CP vs traditional chinese in the provisioning dialouge ...)
- default language can't be altered except via sql statements.

I'ld like to start with a complete rewrite of ajaxviewer.js and
keyboard-definitions for different languages which incorporate into
ajaxviewer if present. I'ld also move from static ul/dl languageselector
in index.jsp to a 100% jquery Set, which would build dynamically for
available languages.

As a sidekick - and If there's some consens on how to store available
languages in a more common way - I'ld like to add a language property to
instance-related api-calls (I didn't dug too deep, maybe there's already
something in place). The UI 'ld definitely need some rewrite to get an
up-to-date dropdown for available languages.

This'll be my personal "winter is boring"-project, so I'll target a PoC
in about two month or so.

Oh, if anyone already gathered and documented keycodes for different
keyboards? :)

I'ld aprpeciate any suggestions!

cheers,

Stephan 

Consoleproxy FF Quicksearch

[Stephan Seitz]

Hi devs!

Despite the fact I was too lazy that weekend cloning the repo and filing
a pull request, I hope that small fix could make it upstream anyway.

--- ajaxviewer.js.orig 2015-12-13 11:10:47.851177577 +
+++ ajaxviewer.js 2015-12-13 14:14:32.367382547 +
@@ -649,6 +649,7 @@
this.sendingEventInProgress = false;
ajaxViewer.installMouseHook();
ajaxViewer.installKeyboardHook();
+ ajaxViewer.panel.parent().focus();

$(window).bind("resize", function() {
ajaxViewer.onWindowResize();
@@ -1259,6 +1260,7 @@
case 38 : // UP
case 39 : // RIGHT
case 40 : // DOWN
+ case 47 : // slash (prevent quick search)
return false;
}



I've already filed this as bug
https://issues.apache.org/jira/browse/CLOUDSTACK-9164

These two lines prevent Firefox from starting quicksearch which is
really annoying.

- Stephan

Re: [poll] cloudstack exam

[Stephan Seitz]

> Quick poll: has anybody here taken the ACCEL cloudstack certification
> exam ? what did you think ? Too hard, too easy ? – about right ?

Well, I signed the usual NDA at pearson vue, so I shouldn't answer in
detail :)
The exam covered a lot of aspects around ACS, in my opinion well
balanced. I did it spontanously (but with (A)CS hands-on since 2.2) and
managed it.
It obviously shows some parallels to LPIC 304, but I assume this is
inevitable.

So, about right, I'ld say.

>  
> Also, by way of reminder: if you use the code  ACCELpromocodeASF when
> registering for the exam, 1/3 of the fee goes to the ACS project
>  
> Kind Regards
> Giles
>  
> Giles Sirett
> CEO


signature.asc
Description: This is a digitally signed message part

Re: [DISCUSS] ACS 4.5.3 release

[Stephan Seitz]

Hi Rohit,

I've reported one Bug[1], that affects the network.throttling.rate. I
assume, this is just some bad calculation.

Maybe this could be fixed for 4.5.3? :)

[1] https://issues.apache.org/jira/browse/CLOUDSTACK-8936


Am Freitag, den 20.11.2015, 06:27 + schrieb Rohit Yadav:
> Hi all, 
> 
> 
> I want to ask how happy people are with the last 4.5.2 release and if
> there are any issues they want to report or want to be fixed in a
> future minor release. If we’ve enough demand, we can work towards a
> last 4.5 minor release. Thanks.
> 
> 
> Rohit Yadav
> Software Architect
> 
> 
> 
> 
> 
> D: +44 20 3642 6102 | S: +44 20 3603 0540 | M: +91 88 262 30892
> 
>  
> rohit.ya...@shapeblue.com | www.shapeblue.com | Twitter:@ShapeBlue
>  
> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS
> 
> 
> Find out more about ShapeBlue and our range of CloudStack related
> services
> 
> IaaS Cloud Design & Build
> CSForge – rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Software Engineering
> CloudStack Infrastructure Support
> CloudStack Bootcamp Training Courses
> 
> 
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related
> companies. If you are not the intended recipient of this email, you
> must neither take any action based upon its contents, nor copy or show
> it to anyone. Please contact the sender if you believe you have
> received this email in error. Shape Blue Ltd is a company incorporated
> in England & Wales. ShapeBlue Services India LLP is a company
> incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA
> Pty Ltd is a company registered by The Republic of South Africa and is
> traded under license from Shape Blue Ltd. ShapeBlue is a registered
> trademark.

Re: [RFC] Metrics views for CloudStack UI

[Stephan Seitz]

This looks really great!
You've added the metrics view at the Infrastructure tab. This is nice
for the platform ops.
I assume the very same metrics would also be a benefit for domain-admins
(say: customers). I'ld suggest to add this view somewhere below the
Instances tab.

cheers,

- Stephan


Am Donnerstag, den 05.11.2015, 14:09 + schrieb Rohit Yadav: 
> Hi all,
> 
> 
> The present CloudStack UI hides most of the metrics data such as cpu,
> memory, disk, network usage in inner detail views. Such information is
> critical to find issues in one’s cloud, for example finding clusters
> where hosts are failing, or finding storage pools where disk space has
> depleted beyond configured global or cluster thresholds.
> 
> 
> The metrics views for CloudStack UI is an attempt to solve those
> problems that brings in several UI enhancements such as sortable
> tables, new status icons, methods to control breadcrumb navigation,
> making UI’s global list* API pagesize dynamic, a new table widget
> based on listView widget that is both horizontally and vertically
> scrollable, supports cell/threshold coloring, collapsible columns
> along with navigation from one view to another and quick-view actions.
> For example, currently support navigation are: Zone to Cluster to Host
> to Instance to Volumes, and Storage Pool to Volumes. 
> 
> 
> The current version implements six resource views for zone, cluster,
> host, instance, volume and storage pool (primary storage). The metrics
> framework (based on listView widget) would allow developers to write
> more such view where information can be densely packed.
> 
> 
> Please checkout the FS (with some screenshots) and the PR;
> 
> 
> FS: https://issues.apache.org/jira/browse/CLOUDSTACK-9020
> JIRA: https://issues.apache.org/jira/browse/CLOUDSTACK-9020
> PR: https://github.com/apache/cloudstack/pull/1038
> 
> 
> Comments and suggestions?
> 
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> 
> 
> 
> 
> 
> 
> 
> M. +91 88 262 30892 | rohit.ya...@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS
> 
> Find out more about ShapeBlue and our range of CloudStack related
> services
> 
> IaaS Cloud Design & Build
> CSForge – rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Software Engineering
> CloudStack Infrastructure Support
> CloudStack Bootcamp Training Courses
> 
> 
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related
> companies. If you are not the intended recipient of this email, you
> must neither take any action based upon its contents, nor copy or show
> it to anyone. Please contact the sender if you believe you have
> received this email in error. Shape Blue Ltd is a company incorporated
> in England & Wales. ShapeBlue Services India LLP is a company
> incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA
> Pty Ltd is a company registered by The Republic of South Africa and is
> traded under license from Shape Blue Ltd. ShapeBlue is a registered
> trademark.

Re: [ANNOUNCE] CloudStack Certification !!!!

[Stephan Seitz]

Today I did the exam at a pearson vue testcenter. I registered via
http://www.pearsonvue.com/accel/ 
The exam is - as far as I can tell - not visible on cs.lpi.org.



Am Freitag, den 30.10.2015, 21:51 +0100 schrieb Erik Weber:
> My local test center can't find the exam, has anyone else had any issues?
> They can see LPI examns, but not LPI-japan ones..
> 
> 



signature.asc
Description: This is a digitally signed message part

Re: Console keyboard improvements

[Stephan Seitz]

Hi Erik,

I found following link[1] useful do build a modified virtual router
systemvm.
I assume the very same process can be used for the console-vm.

You can upload your own template to ACS and define it in the settings
area via consoleproxy.system.offering


[1] http://bhaisaab.org/logs/building-systemvms/

cheers,

- Stephan


Am Dienstag, den 23.06.2015, 13:23 +0200 schrieb Erik Weber: 
> Hi Anshul,
> 
> Thank you for the update.
> 
> Do you have a rough estimate for when this refactor might hit the codebase?
> I don't expect my changes to hit 4.6 due to time constraints anyway, so if
> your changes are around the corner I might as well wait.
> 
> Do you happen to know what the easiest way to redeploy systemvm.iso for
> testing purposes is? Intended hypervisor is XenServer, but if it is easier
> with KVM I'm open to switching.
> 
> Regards,
> Erik
> 
> 
> On Tue, Jun 23, 2015 at 1:17 PM, Anshul Gangwar 
> wrote:
> 
> > Hi Erik,
> >
> > I will be working on console keyboard support refactoring. The main
> > purpose of this refactoring will be to enable end user to add their own
> > keyboard mappings without need to know CloudStack code much.
> >
> > As of now all  keyboard mappings are either into ajaxkeys.js or
> > ajaxviewer.js. These files goes into CPVM through systemvm.iso.
> >
> > Regards,
> > Anshul
> >
> > On 23-Jun-2015, at 3:54 pm, Erik Weber  > terbol...@gmail.com>> wrote:
> >
> > Hi,
> >
> > I am guessing there are more than me that's having trouble with keyboard
> > with non-us layouts.
> > I'd like to improve it, atleast for my native charset.
> >
> > Has anyone done something like this, and have some insight into the
> > process?
> >
> > I've found this [1], that to some degree explain the process.
> > If I'm testing locally, what are the steps required to update the files?
> > Do they rely on the mgmt server or the console proxy?
> >
> > If the latter, how do I update it (if not manually)?
> >
> > If anyone wanna co-work on fixing Norwegian charset, let me know :-)
> >
> > [1]
> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Non-US+Keyboard+Support+for+Console+Proxy
> > --
> > Erik
> >
> >

Re: Console keyboard improvements

[Stephan Seitz]

Hi Erik,

I'ld like to move in, but for the need of a german layout. Maybe we can
join forces :)

cheers,

- Stephan

Am Dienstag, den 23.06.2015, 12:24 +0200 schrieb Erik Weber: 
> Hi,
> 
> I am guessing there are more than me that's having trouble with keyboard
> with non-us layouts.
> I'd like to improve it, atleast for my native charset.
> 
> Has anyone done something like this, and have some insight into the process?
> 
> I've found this [1], that to some degree explain the process.
> If I'm testing locally, what are the steps required to update the files?
> Do they rely on the mgmt server or the console proxy?
> 
> If the latter, how do I update it (if not manually)?
> 
> If anyone wanna co-work on fixing Norwegian charset, let me know :-)
> 
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Non-US+Keyboard+Support+for+Console+Proxy