Re: CloudStack LTS EOL date?

[Ron Wheeler]

I am not sure that the world would end if the EOL was extended for a 
particularly popular LTS release if the subsequent releases were not as 
stable as some people liked or did not include sufficient new features 
to motivate a lot of happy users to upgrade.
If the pace of enhancements is slow and there are a small number of bug 
fixes requiring backporting, this could be manageable.


At the moment, would it be reasonable for a new installation to start 
with the latest 4.9.x if the EOL was 2020 and the EOL of 4.10.x was 2021?
Perhaps for many people the risk of starting with .1 release of 4.10 
today rather than a .3 release of 4.9 would outweigh the benefit of the 
extra year of support. That analysis would gradually change as 4.10 gets 
additional minor releases and 2020 gets closer.


One would hope that the number of critical enhancements and bug fixes 
will reduce as the product becomes more mature.


In addition, as the functionality required by an increasing percentage 
of users is incorporated into the product, the attractiveness of a new 
release will be reduced.  Users will want LTS to be extended for longer 
periods since upgrading does not buy them very much and possibly nothing 
at all in their particular use case as long as bugs are fixed.


This should be viewed as a sign of successful design and strong 
implementation rather than a loss of momentum.


Ron

On 22/11/2017 8:44 PM, Ivan Kudryavtsev wrote:

Hi, all. Previous reply to wrong thread. Copy here.
According to Paul, everything looks ok, but I still feel the website
content is lacking of the information. My belief that index should clearly
state:

Current LTS 4.9 | updated 2017.11.12 (4.9.3) | EOL=2018.X.Y
Previous LTS 4.X | updated 2017.04.01 (4.X.12) | EOL=2017.X.Y

Current 4.10 | updated 2017.11.20 (4.10.1) | EOL=2018.05.Y

The same is for download page. The reason is that some people don't need
new, they need very proven. Other need supported and third group needs
features.

E.g. Right now we updated our proxmox nodes to latest stable and found
windows 8 is no longer works as expected. Previous stable - ok. We rolled
back. I mean that it could be a good way for a lot of users to see and
realize what options they have. Even now, we still have 4.3 in production
and happy.

Right now, new person just downloads 4.10 and gets a lot of regressions and
unstable code. You might have seen last day e-mail threads. Even templates
created from snapshots are broken in 4.10 and it is critical/blocker bug.
The user can meet the situation, that after a months when ssvm is reloaded
all users lost tons of templates.

22 нояб. 2017 г. 11:49 ПП пользователь "Will Stevens" 
написал:

Paul, I thought a 'big mouth' was a prerequisite for the RM position.
Isn't that the only reason I was the 4.9 RM?  :P

*Will Stevens*
CTO



On Wed, Nov 22, 2017 at 11:39 AM, Paul Angus 
wrote:


HI All,

The current LTS cycle is based on having an LTS release twice a year (at
the time of design, ACS releases were coming out monthly).

So, twice a year (nominally, January and July) we take the then current
version of CloudStack, and declare that an LTS version, for which would we
would then backport fixes for a period of up to 2 years.  Thereby giving
end users a version of CloudStack which would receive bug fixes for an
extended period.

This year however, the current version in January was the same as the
current version in July, therefore 4.9 became the 'July' LTS as well as
January LTS and therefore 4.9 will be supported until summer 2019 (hence
the 4.9.3 release)

I and a number of my colleagues remain committed to continue to support
'LTS' releases in this fashion (there just wasn't anything really to
'announce' in July), which may be why people think that nothing is
happening.

With 2 LTS releases a year (6 months apart), 'next LTS +6 months' would
only be 12 months from release.  Which I think is really too short a

period

for the majority of enterprises.  Although we haven't written it this way,
the current scheme gives a EOL of 'next LTS + 18 months'.

So, I'm in favour of leaving things as they are.   The wiki page looks
like it needs updating to be clearer (I'm happy to do that)


But I DO think that we should start a new thread asking for a 4.11 RM
volunteer to get things going.   (I'm guessing y'all don't what my big
mouth in that position).


Kind regards,

Paul Angus


paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue




-Original Message-
From: Ivan Kudryavtsev [mailto:kudryavtsev...@bw-sw.com]
Sent: 21 November 2017 14:00
To: dev@cloudstack.apache.org
Subject: Re: CloudStack LTS EOL date?

Hello, it sounds very reasonable. The more lifecycle information the
better for adopters.

21 нояб. 2017 г. 8:56 ПП пользователь "Marc-Aurèle Brothier - Exoscale" <
ma...@exoscale.ch> написал:


It makes more sense to me too.

Re: CloudStack LTS EOL date?

[Ivan Kudryavtsev]

Hi, all. Previous reply to wrong thread. Copy here.
According to Paul, everything looks ok, but I still feel the website
content is lacking of the information. My belief that index should clearly
state:

Current LTS 4.9 | updated 2017.11.12 (4.9.3) | EOL=2018.X.Y
Previous LTS 4.X | updated 2017.04.01 (4.X.12) | EOL=2017.X.Y

Current 4.10 | updated 2017.11.20 (4.10.1) | EOL=2018.05.Y

The same is for download page. The reason is that some people don't need
new, they need very proven. Other need supported and third group needs
features.

E.g. Right now we updated our proxmox nodes to latest stable and found
windows 8 is no longer works as expected. Previous stable - ok. We rolled
back. I mean that it could be a good way for a lot of users to see and
realize what options they have. Even now, we still have 4.3 in production
and happy.

Right now, new person just downloads 4.10 and gets a lot of regressions and
unstable code. You might have seen last day e-mail threads. Even templates
created from snapshots are broken in 4.10 and it is critical/blocker bug.
The user can meet the situation, that after a months when ssvm is reloaded
all users lost tons of templates.

22 нояб. 2017 г. 11:49 ПП пользователь "Will Stevens" 
написал:

Paul, I thought a 'big mouth' was a prerequisite for the RM position.
Isn't that the only reason I was the 4.9 RM?  :P

*Will Stevens*
CTO



On Wed, Nov 22, 2017 at 11:39 AM, Paul Angus 
wrote:

> HI All,
>
> The current LTS cycle is based on having an LTS release twice a year (at
> the time of design, ACS releases were coming out monthly).
>
> So, twice a year (nominally, January and July) we take the then current
> version of CloudStack, and declare that an LTS version, for which would we
> would then backport fixes for a period of up to 2 years.  Thereby giving
> end users a version of CloudStack which would receive bug fixes for an
> extended period.
>
> This year however, the current version in January was the same as the
> current version in July, therefore 4.9 became the 'July' LTS as well as
> January LTS and therefore 4.9 will be supported until summer 2019 (hence
> the 4.9.3 release)
>
> I and a number of my colleagues remain committed to continue to support
> 'LTS' releases in this fashion (there just wasn't anything really to
> 'announce' in July), which may be why people think that nothing is
> happening.
>
> With 2 LTS releases a year (6 months apart), 'next LTS +6 months' would
> only be 12 months from release.  Which I think is really too short a
period
> for the majority of enterprises.  Although we haven't written it this way,
> the current scheme gives a EOL of 'next LTS + 18 months'.
>
> So, I'm in favour of leaving things as they are.   The wiki page looks
> like it needs updating to be clearer (I'm happy to do that)
>
>
> But I DO think that we should start a new thread asking for a 4.11 RM
> volunteer to get things going.   (I'm guessing y'all don't what my big
> mouth in that position).
>
>
> Kind regards,
>
> Paul Angus
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Ivan Kudryavtsev [mailto:kudryavtsev...@bw-sw.com]
> Sent: 21 November 2017 14:00
> To: dev@cloudstack.apache.org
> Subject: Re: CloudStack LTS EOL date?
>
> Hello, it sounds very reasonable. The more lifecycle information the
> better for adopters.
>
> 21 нояб. 2017 г. 8:56 ПП пользователь "Marc-Aurèle Brothier - Exoscale" <
> ma...@exoscale.ch> написал:
>
> > It makes more sense to me too.
> >
> >
> > On Tue, 2017-11-21 at 12:04 +0100, Rene Moser wrote:
> > > Hi all
> > >
> > > The current LTS release is 4.9 which is EOL in June 2018 according
> > > to https://cwiki.apache.org/confluence/display/CLOUDSTACK/LTS
> > >
> > > AFAIK there are no works planed for a new LTS. The release pace has
> > > slown down (the high pace and leaving users behind fixes was the
> > > reason for the LTS).
> > >
> > > I am still pro LTS but in my opinion we should have defined the EOL
> > > in relation of the successor LTS release date: "The EOL of the
> > > current LTS is +6 months after the next LTS release."
> > >
> > > Small example:
> > >
> > > Current LTS 4.9
> > > Next LTS 4.1x release on 01.04. --> LTS 4.9 is 01.10.
> > >
> > > Does this make sense? Other suggestions?
> > >
> > > Regards
> > > René
> >
>

Re: CloudStack LTS EOL date?

[Will Stevens]

Paul, I thought a 'big mouth' was a prerequisite for the RM position.
Isn't that the only reason I was the 4.9 RM?  :P

*Will Stevens*
CTO



On Wed, Nov 22, 2017 at 11:39 AM, Paul Angus 
wrote:

> HI All,
>
> The current LTS cycle is based on having an LTS release twice a year (at
> the time of design, ACS releases were coming out monthly).
>
> So, twice a year (nominally, January and July) we take the then current
> version of CloudStack, and declare that an LTS version, for which would we
> would then backport fixes for a period of up to 2 years.  Thereby giving
> end users a version of CloudStack which would receive bug fixes for an
> extended period.
>
> This year however, the current version in January was the same as the
> current version in July, therefore 4.9 became the 'July' LTS as well as
> January LTS and therefore 4.9 will be supported until summer 2019 (hence
> the 4.9.3 release)
>
> I and a number of my colleagues remain committed to continue to support
> 'LTS' releases in this fashion (there just wasn't anything really to
> 'announce' in July), which may be why people think that nothing is
> happening.
>
> With 2 LTS releases a year (6 months apart), 'next LTS +6 months' would
> only be 12 months from release.  Which I think is really too short a period
> for the majority of enterprises.  Although we haven't written it this way,
> the current scheme gives a EOL of 'next LTS + 18 months'.
>
> So, I'm in favour of leaving things as they are.   The wiki page looks
> like it needs updating to be clearer (I'm happy to do that)
>
>
> But I DO think that we should start a new thread asking for a 4.11 RM
> volunteer to get things going.   (I'm guessing y'all don't what my big
> mouth in that position).
>
>
> Kind regards,
>
> Paul Angus
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Ivan Kudryavtsev [mailto:kudryavtsev...@bw-sw.com]
> Sent: 21 November 2017 14:00
> To: dev@cloudstack.apache.org
> Subject: Re: CloudStack LTS EOL date?
>
> Hello, it sounds very reasonable. The more lifecycle information the
> better for adopters.
>
> 21 нояб. 2017 г. 8:56 ПП пользователь "Marc-Aurèle Brothier - Exoscale" <
> ma...@exoscale.ch> написал:
>
> > It makes more sense to me too.
> >
> >
> > On Tue, 2017-11-21 at 12:04 +0100, Rene Moser wrote:
> > > Hi all
> > >
> > > The current LTS release is 4.9 which is EOL in June 2018 according
> > > to https://cwiki.apache.org/confluence/display/CLOUDSTACK/LTS
> > >
> > > AFAIK there are no works planed for a new LTS. The release pace has
> > > slown down (the high pace and leaving users behind fixes was the
> > > reason for the LTS).
> > >
> > > I am still pro LTS but in my opinion we should have defined the EOL
> > > in relation of the successor LTS release date: "The EOL of the
> > > current LTS is +6 months after the next LTS release."
> > >
> > > Small example:
> > >
> > > Current LTS 4.9
> > > Next LTS 4.1x release on 01.04. --> LTS 4.9 is 01.10.
> > >
> > > Does this make sense? Other suggestions?
> > >
> > > Regards
> > > René
> >
>

Re: Fail with vpn customer gateway creation through terraform

[Nux!]

Ok, looking at the logs it looks like an encoding problem of sorts, when 
terraform is making the calls, the policy appears as:
sha1-aes256%3Bmodp2048

When cloudmonkey makes the calls (successfully) the policy looks like it should:
aes128-sha256;modp2048

Ideas?

https://paste.fedoraproject.org/paste/HpGdigqa33ZjTeIDrAwE9w

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Nux!" 
> To: "dev" 
> Sent: Wednesday, 22 November, 2017 09:11:28
> Subject: Re: Fail with vpn customer gateway creation through terraform

> Hi guys,
> 
> sha1-aes256;modp3072 works if I use the UI or cloudmonkey, that's why I am
> thinking it must be terraform or some weird encoding issues.
> 
> I tried replacing ; with - and also using modp2048, to no avail.
> 
> "* cloudstack_vpn_customer_gateway.default: Error creating VPN Customer 
> Gateway
> test-vpc: Undefined error: {"errorcode":431,"errortext":"The customer gateway
> IKE policy sha1-aes256-modp2048 is invalid!  Verify the required Diffie 
> Hellman
> (DH) group is specified."}"
> 
> 
> Logs here
> 
> https://paste.fedoraproject.org/paste/HpGdigqa33ZjTeIDrAwE9w
> 
> --
> Sent from the Delta quadrant using Borg technology!
> 
> Nux!
> www.nux.ro
> 
> - Original Message -
>> From: "Jayapal Uradi" 
>> To: "dev" 
>> Sent: Wednesday, 22 November, 2017 04:20:53
>> Subject: Re: Fail with vpn customer gateway creation through terraform
> 
>> Hi Lucian,
>> 
>> Try the the following in config, ‘-‘ instead of ‘;’ after the aes256 in the
>> config.
>> 
>> New: "sha1-aes256-modp3072”
>> Old: "sha1-aes256;modp3072”
>> 
>> Thanks,
>> Jayapal
>> On Nov 22, 2017, at 5:44 AM, Pierre-Luc Dion
>> > wrote:
>> 
>> Hi Nux,
>> 
>> Could it be your cloudstack version ?  modp3072 is recent I think in
>> CloudStack so if you run a older version maybe it's not there?
>> 
>> 
>> 
>> On Tue, Nov 21, 2017 at 6:55 PM, Nux! >
>> wrote:
>> 
>> Thanks Chiradeep,
>> 
>> Checked but brain says no. What should I have learned from there?
>> 
>> AFAIK this is a terraform fail.
>> 
>> Lucian
>> 
>> --
>> Sent from the Delta quadrant using Borg technology!
>> 
>> Nux!
>> www.nux.ro
>> 
>> - Original Message -
>> From: "Chiradeep Vittal" 
>> To: "dev" 
>> Sent: Tuesday, 21 November, 2017 19:14:16
>> Subject: Re: Fail with vpn customer gateway creation through terraform
>> 
>> Check
>> https://github.com/apache/cloudstack/blob/77864992fe8f80dbabd1240f6373d2
>> ba3e98713c/utils/src/main/java/com/cloud/utils/net/NetUtils.java#L1221
>> 
>> On Tue, Nov 21, 2017 at 10:11 AM, Nux!  wrote:
>> 
>> Hi,
>> 
>> I'm trying out terraform and had success so far, except for the vpn
>> customer gateway feature.
>> For some reason, terraform fails to create it, though I use the same
>> options as in UI/cloudmonkey where it works just fine.
>> 
>> The snippet for it is:
>> 
>> resource "cloudstack_vpn_customer_gateway" "default" {
>> name   = "test-vpc"
>> cidr   = "10.0.0.0/24"
>> esp_policy = "aes256-sha1"
>> gateway= "1.2.3.4"
>> ike_policy = "sha1-aes256;modp3072"
>> ipsec_psk  = "terraformxyz7"
>> }
>> 
>> It always complains about the ike_policy:
>> * cloudstack_vpn_customer_gateway.default: Error creating VPN Customer
>> Gateway test-vpc: Undefined error: {"errorcode":431,"errortext":"The
>> customer gateway IKE policy sha1-aes256;modp3072 is invalid!  Verify the
>> required Diffie Hellman (DH) group is specified."}
>> 
>> I tried all sorts of ways to write the ike_policy, escaped, web
>> encoded/decoded, nothing worked. What am I missing?
>> The example terraform docs provide suffers the same fate.
>> 
>> Lucian
>> 
>> --
>> Sent from the Delta quadrant using Borg technology!
>> 
>> Nux!
>> www.nux.ro
>> 
>> 
>> DISCLAIMER
>> ==
>> This e-mail may contain privileged and confidential information which is the
>> property of Accelerite, a Persistent Systems business. It is intended only 
>> for
>> the use of the individual or entity to which it is addressed. If you are not
>> the intended recipient, you are not authorized to read, retain, copy, print,
>> distribute or use this message. If you have received this communication in
>> error, please notify the sender and delete all copies of this message.
>> Accelerite, a Persistent Systems business does not accept any liability for
> > virus infected mails.

Re: Tag removal problem found. Need assistance how to fix

[Rafael Weingärtner]

That is what it seems

On Wed, Nov 22, 2017 at 10:14 AM, Ivan Kudryavtsev  wrote:

> So, basically I changed it to
>
> sb.and("resourceUuid", sb.entity().getResourceUuid(), SearchCriteria.Op.IN
> );
> sb.and("resourceType", sb.entity().getResourceType(),
> SearchCriteria.Op.EQ);
>
>
> And launched marvin tests. I suppose resourceId filtering is completely
> wrong here.
>
>
> 22 нояб. 2017 г. 7:12 ПП пользователь "Ivan Kudryavtsev" <
> kudryavtsev...@bw-sw.com> написал:
>
> Yes, you are right, but there are no other ids in the query, and rarely
> (like in my case) it leads to wrong results.
>
> 22 нояб. 2017 г. 7:07 ПП пользователь "Rafael Weingärtner" <
> rafaelweingart...@gmail.com> написал:
>
> The problem seems to be related to line 308 at
> > com.cloud.tags.TaggedResourceManagerImpl.deleteTags(List,
> > ResourceObjectType, Map).
> > It is being sent a list of resourceUUID as the filter for resourceId
> >
> > On Wed, Nov 22, 2017 at 10:03 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > The resourceId is not a real DB ID. It is the UUID converted to Long
> > :(
> > > This table has four "ID" like fields, ID, UUID, resourceID, and
> > > resourceUUID.
> > >
> > > On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev <
> > > kudryavtsev...@bw-sw.com> wrote:
> > >
> > >> Hi, I just enabled log all queries and copied actual query which I
> have
> > >> shown in the first mail. Also, I don't understand why the search
> should
> > >> look over inner ids... Is it a case when user can pass real db ids to
> an
> > >> api call?
> > >>
> > >> 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
> > >> rafaelweingart...@gmail.com> написал:
> > >>
> > >> > Yes, this I understood ;)
> > >> >
> > >> > However, I do not understand how the SQL that is being generated has
> > >> this
> > >> > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > >> > c1bc5440ea60'".
> > >> > The resourceId field in the entity is a long. So, even though that
> > long
> > >> > represents a String, in the final SQL that is generated it should
> be a
> > >> long
> > >> > value there.
> > >> >
> > >> >
> > >> >
> > >> > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> > >> > kudryavtsev...@bw-sw.com>
> > >> > wrote:
> > >> >
> > >> > > Take a look here:
> > >> > >
> > >> > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > >> > > resource_tags.value, resource_tags.domain_id,
> > >> resource_tags.account_id,
> > >> > > resource_tags.resource_id, resource_tags.resource_uuid,
> > >> > > resource_tags.resource_type, resource_tags.customer FROM
> > resource_tags
> > >> > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > >> > c1bc5440ea60'
> > >> > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > >> > > 9465-c1bc5440ea60'
> > >> > > )  AND resource_tags.resource_type = 'Account';
> > >> > >
> > >> > > ++--+---+---
> > >> > > +---++-+
> > >> > > --+---+--+
> > >> > > | id | uuid | key   | value |
> > >> domain_id |
> > >> > > account_id | resource_id | resource_uuid|
> > >> > > resource_type | customer |
> > >> > > ++--+---+---
> > >> > > +---++-+
> > >> > > --+---+--+
> > >> > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
> > >>  1
> > >> > |
> > >> > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
> > >> Account
> > >> > >   | NULL |
> > >> > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
> > >>  1
> > >> > |
> > >> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> > >> Account
> > >> > >   | NULL |
> > >> > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
> > >>  1
> > >> > |
> > >> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> > >> Account
> > >> > >   | NULL |
> > >> > > ++--+---+---
> > >> > > +---++-+
> > >> > > --+---+--+
> > >> > > 3 rows in set, 1 warning (0.01 sec)
> > >> > >
> > >> > > Try to figure out why id=7 is selected here?
> > >> > >
> > >> > > Because:
> > >> > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > >> > >
> > >> > > Matched unintentionally, because mysql converted uuid to int and
> > got 2
> > >> > > which is matched to resource_id of 2 (id=7).
> > >> > >
> > >> > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> > >> > > rafaelweingart...@gmail.com> написал:
> > >> > >
> > >> > > > Ah, ok now it makes sense the "IN", I thought you were only
> > talking
> > >> > 

Re: Tag removal problem found. Need assistance how to fix

[Ivan Kudryavtsev]

So, basically I changed it to

sb.and("resourceUuid", sb.entity().getResourceUuid(), SearchCriteria.Op.IN);
sb.and("resourceType", sb.entity().getResourceType(), SearchCriteria.Op.EQ);


And launched marvin tests. I suppose resourceId filtering is completely
wrong here.


22 нояб. 2017 г. 7:12 ПП пользователь "Ivan Kudryavtsev" <
kudryavtsev...@bw-sw.com> написал:

Yes, you are right, but there are no other ids in the query, and rarely
(like in my case) it leads to wrong results.

22 нояб. 2017 г. 7:07 ПП пользователь "Rafael Weingärtner" <
rafaelweingart...@gmail.com> написал:

The problem seems to be related to line 308 at
> com.cloud.tags.TaggedResourceManagerImpl.deleteTags(List,
> ResourceObjectType, Map).
> It is being sent a list of resourceUUID as the filter for resourceId
>
> On Wed, Nov 22, 2017 at 10:03 AM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > The resourceId is not a real DB ID. It is the UUID converted to Long
> :(
> > This table has four "ID" like fields, ID, UUID, resourceID, and
> > resourceUUID.
> >
> > On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com> wrote:
> >
> >> Hi, I just enabled log all queries and copied actual query which I have
> >> shown in the first mail. Also, I don't understand why the search should
> >> look over inner ids... Is it a case when user can pass real db ids to an
> >> api call?
> >>
> >> 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
> >> rafaelweingart...@gmail.com> написал:
> >>
> >> > Yes, this I understood ;)
> >> >
> >> > However, I do not understand how the SQL that is being generated has
> >> this
> >> > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> >> > c1bc5440ea60'".
> >> > The resourceId field in the entity is a long. So, even though that
> long
> >> > represents a String, in the final SQL that is generated it should be a
> >> long
> >> > value there.
> >> >
> >> >
> >> >
> >> > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> >> > kudryavtsev...@bw-sw.com>
> >> > wrote:
> >> >
> >> > > Take a look here:
> >> > >
> >> > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> >> > > resource_tags.value, resource_tags.domain_id,
> >> resource_tags.account_id,
> >> > > resource_tags.resource_id, resource_tags.resource_uuid,
> >> > > resource_tags.resource_type, resource_tags.customer FROM
> resource_tags
> >> > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> >> > c1bc5440ea60'
> >> > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> >> > > 9465-c1bc5440ea60'
> >> > > )  AND resource_tags.resource_type = 'Account';
> >> > >
> >> > > ++--+---+---
> >> > > +---++-+
> >> > > --+---+--+
> >> > > | id | uuid | key   | value |
> >> domain_id |
> >> > > account_id | resource_id | resource_uuid|
> >> > > resource_type | customer |
> >> > > ++--+---+---
> >> > > +---++-+
> >> > > --+---+--+
> >> > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
> >>  1
> >> > |
> >> > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
> >> Account
> >> > >   | NULL |
> >> > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
> >>  1
> >> > |
> >> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> >> Account
> >> > >   | NULL |
> >> > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
> >>  1
> >> > |
> >> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> >> Account
> >> > >   | NULL |
> >> > > ++--+---+---
> >> > > +---++-+
> >> > > --+---+--+
> >> > > 3 rows in set, 1 warning (0.01 sec)
> >> > >
> >> > > Try to figure out why id=7 is selected here?
> >> > >
> >> > > Because:
> >> > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> >> > >
> >> > > Matched unintentionally, because mysql converted uuid to int and
> got 2
> >> > > which is matched to resource_id of 2 (id=7).
> >> > >
> >> > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> >> > > rafaelweingart...@gmail.com> написал:
> >> > >
> >> > > > Ah, ok now it makes sense the "IN", I thought you were only
> talking
> >> > about
> >> > > > single values.
> >> > > >
> >> > > >
> >> > > >
> >> > > > I do not think that the UUID (resource UUID) is the representation
> >> of
> >> > ID
> >> > > > value in Hexadecimal, if it is we could simply get rid of one of
> >> them.
> >> > I
> >> > > > really dislike these search criteria...I am not seeing what you
> are
> >> > > saying.
> >> > 

Re: Tag removal problem found. Need assistance how to fix

[Rafael Weingärtner]

Then, I believe we can remove that filter in the search criteria as you
suggested and also remove the line 308.

On Wed, Nov 22, 2017 at 10:12 AM, Ivan Kudryavtsev  wrote:

> Yes, you are right, but there are no other ids in the query, and rarely
> (like in my case) it leads to wrong results.
>
> 22 нояб. 2017 г. 7:07 ПП пользователь "Rafael Weingärtner" <
> rafaelweingart...@gmail.com> написал:
>
> > The problem seems to be related to line 308 at
> > com.cloud.tags.TaggedResourceManagerImpl.deleteTags(List,
> > ResourceObjectType, Map).
> > It is being sent a list of resourceUUID as the filter for resourceId
> >
> > On Wed, Nov 22, 2017 at 10:03 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > The resourceId is not a real DB ID. It is the UUID converted to Long
> > :(
> > > This table has four "ID" like fields, ID, UUID, resourceID, and
> > > resourceUUID.
> > >
> > > On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev <
> > > kudryavtsev...@bw-sw.com> wrote:
> > >
> > >> Hi, I just enabled log all queries and copied actual query which I
> have
> > >> shown in the first mail. Also, I don't understand why the search
> should
> > >> look over inner ids... Is it a case when user can pass real db ids to
> an
> > >> api call?
> > >>
> > >> 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
> > >> rafaelweingart...@gmail.com> написал:
> > >>
> > >> > Yes, this I understood ;)
> > >> >
> > >> > However, I do not understand how the SQL that is being generated has
> > >> this
> > >> > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > >> > c1bc5440ea60'".
> > >> > The resourceId field in the entity is a long. So, even though that
> > long
> > >> > represents a String, in the final SQL that is generated it should
> be a
> > >> long
> > >> > value there.
> > >> >
> > >> >
> > >> >
> > >> > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> > >> > kudryavtsev...@bw-sw.com>
> > >> > wrote:
> > >> >
> > >> > > Take a look here:
> > >> > >
> > >> > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > >> > > resource_tags.value, resource_tags.domain_id,
> > >> resource_tags.account_id,
> > >> > > resource_tags.resource_id, resource_tags.resource_uuid,
> > >> > > resource_tags.resource_type, resource_tags.customer FROM
> > resource_tags
> > >> > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > >> > c1bc5440ea60'
> > >> > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > >> > > 9465-c1bc5440ea60'
> > >> > > )  AND resource_tags.resource_type = 'Account';
> > >> > >
> > >> > > ++--+---+---
> > >> > > +---++-+
> > >> > > --+---+--+
> > >> > > | id | uuid | key   | value |
> > >> domain_id |
> > >> > > account_id | resource_id | resource_uuid|
> > >> > > resource_type | customer |
> > >> > > ++--+---+---
> > >> > > +---++-+
> > >> > > --+---+--+
> > >> > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
> > >>  1
> > >> > |
> > >> > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
> > >> Account
> > >> > >   | NULL |
> > >> > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
> > >>  1
> > >> > |
> > >> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> > >> Account
> > >> > >   | NULL |
> > >> > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
> > >>  1
> > >> > |
> > >> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> > >> Account
> > >> > >   | NULL |
> > >> > > ++--+---+---
> > >> > > +---++-+
> > >> > > --+---+--+
> > >> > > 3 rows in set, 1 warning (0.01 sec)
> > >> > >
> > >> > > Try to figure out why id=7 is selected here?
> > >> > >
> > >> > > Because:
> > >> > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > >> > >
> > >> > > Matched unintentionally, because mysql converted uuid to int and
> > got 2
> > >> > > which is matched to resource_id of 2 (id=7).
> > >> > >
> > >> > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> > >> > > rafaelweingart...@gmail.com> написал:
> > >> > >
> > >> > > > Ah, ok now it makes sense the "IN", I thought you were only
> > talking
> > >> > about
> > >> > > > single values.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > I do not think that the UUID (resource UUID) is the
> representation
> > >> of
> > >> > ID
> > >> > > > value in Hexadecimal, if it is we could simply get rid of one of
> > >> them.
> > >> > I
> > >> > > > really 

Re: Tag removal problem found. Need assistance how to fix

[Ivan Kudryavtsev]

Yes, you are right, but there are no other ids in the query, and rarely
(like in my case) it leads to wrong results.

22 нояб. 2017 г. 7:07 ПП пользователь "Rafael Weingärtner" <
rafaelweingart...@gmail.com> написал:

> The problem seems to be related to line 308 at
> com.cloud.tags.TaggedResourceManagerImpl.deleteTags(List,
> ResourceObjectType, Map).
> It is being sent a list of resourceUUID as the filter for resourceId
>
> On Wed, Nov 22, 2017 at 10:03 AM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > The resourceId is not a real DB ID. It is the UUID converted to Long
> :(
> > This table has four "ID" like fields, ID, UUID, resourceID, and
> > resourceUUID.
> >
> > On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com> wrote:
> >
> >> Hi, I just enabled log all queries and copied actual query which I have
> >> shown in the first mail. Also, I don't understand why the search should
> >> look over inner ids... Is it a case when user can pass real db ids to an
> >> api call?
> >>
> >> 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
> >> rafaelweingart...@gmail.com> написал:
> >>
> >> > Yes, this I understood ;)
> >> >
> >> > However, I do not understand how the SQL that is being generated has
> >> this
> >> > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> >> > c1bc5440ea60'".
> >> > The resourceId field in the entity is a long. So, even though that
> long
> >> > represents a String, in the final SQL that is generated it should be a
> >> long
> >> > value there.
> >> >
> >> >
> >> >
> >> > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> >> > kudryavtsev...@bw-sw.com>
> >> > wrote:
> >> >
> >> > > Take a look here:
> >> > >
> >> > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> >> > > resource_tags.value, resource_tags.domain_id,
> >> resource_tags.account_id,
> >> > > resource_tags.resource_id, resource_tags.resource_uuid,
> >> > > resource_tags.resource_type, resource_tags.customer FROM
> resource_tags
> >> > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> >> > c1bc5440ea60'
> >> > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> >> > > 9465-c1bc5440ea60'
> >> > > )  AND resource_tags.resource_type = 'Account';
> >> > >
> >> > > ++--+---+---
> >> > > +---++-+
> >> > > --+---+--+
> >> > > | id | uuid | key   | value |
> >> domain_id |
> >> > > account_id | resource_id | resource_uuid|
> >> > > resource_type | customer |
> >> > > ++--+---+---
> >> > > +---++-+
> >> > > --+---+--+
> >> > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
> >>  1
> >> > |
> >> > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
> >> Account
> >> > >   | NULL |
> >> > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
> >>  1
> >> > |
> >> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> >> Account
> >> > >   | NULL |
> >> > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
> >>  1
> >> > |
> >> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> >> Account
> >> > >   | NULL |
> >> > > ++--+---+---
> >> > > +---++-+
> >> > > --+---+--+
> >> > > 3 rows in set, 1 warning (0.01 sec)
> >> > >
> >> > > Try to figure out why id=7 is selected here?
> >> > >
> >> > > Because:
> >> > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> >> > >
> >> > > Matched unintentionally, because mysql converted uuid to int and
> got 2
> >> > > which is matched to resource_id of 2 (id=7).
> >> > >
> >> > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> >> > > rafaelweingart...@gmail.com> написал:
> >> > >
> >> > > > Ah, ok now it makes sense the "IN", I thought you were only
> talking
> >> > about
> >> > > > single values.
> >> > > >
> >> > > >
> >> > > >
> >> > > > I do not think that the UUID (resource UUID) is the representation
> >> of
> >> > ID
> >> > > > value in Hexadecimal, if it is we could simply get rid of one of
> >> them.
> >> > I
> >> > > > really dislike these search criteria...I am not seeing what you
> are
> >> > > saying.
> >> > > > Let´s see this in SQL, so we can discuss.
> >> > > >
> >> > > > SELECT * FROM resource_tags
> >> > > > WHERE  (resource_id in () OR resource_uuid in (...))
> >> > > > AND resource_tags.resource_type = 'Account';
> >> > > >
> >> > > >
> >> > > > That is what the programmer who coded that Search criteria seemed
> to
> >> > > want,
> >> > > > right? I mean, 

Re: Tag removal problem found. Need assistance how to fix

[Rafael Weingärtner]

I did not understand what you said. Are you talking about the data used in
the SQL filters?

On Wed, Nov 22, 2017 at 10:10 AM, Ivan Kudryavtsev  wrote:

> So, I don't understand. As you can see there is no longs in sql dump. Just
> uuids. I run actual delete tags query against simulator and it leads to
> this sql search which filters wrong data. As I see, query is incorrect.
>
> 22 нояб. 2017 г. 7:03 ПП пользователь "Rafael Weingärtner" <
> rafaelweingart...@gmail.com> написал:
>
> > The resourceId is not a real DB ID. It is the UUID converted to Long
> :(
> > This table has four "ID" like fields, ID, UUID, resourceID, and
> > resourceUUID.
> >
> > On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com
> > > wrote:
> >
> > > Hi, I just enabled log all queries and copied actual query which I have
> > > shown in the first mail. Also, I don't understand why the search should
> > > look over inner ids... Is it a case when user can pass real db ids to
> an
> > > api call?
> > >
> > > 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
> > > rafaelweingart...@gmail.com> написал:
> > >
> > > > Yes, this I understood ;)
> > > >
> > > > However, I do not understand how the SQL that is being generated has
> > this
> > > > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > > > c1bc5440ea60'".
> > > > The resourceId field in the entity is a long. So, even though that
> long
> > > > represents a String, in the final SQL that is generated it should be
> a
> > > long
> > > > value there.
> > > >
> > > >
> > > >
> > > > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> > > > kudryavtsev...@bw-sw.com>
> > > > wrote:
> > > >
> > > > > Take a look here:
> > > > >
> > > > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > > > > resource_tags.value, resource_tags.domain_id,
> > resource_tags.account_id,
> > > > > resource_tags.resource_id, resource_tags.resource_uuid,
> > > > > resource_tags.resource_type, resource_tags.customer FROM
> > resource_tags
> > > > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > > > c1bc5440ea60'
> > > > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > > > > 9465-c1bc5440ea60'
> > > > > )  AND resource_tags.resource_type = 'Account';
> > > > >
> > > > > ++--+---+---
> > > > > +---++-+
> > > > > --+---+--+
> > > > > | id | uuid | key   | value |
> > > domain_id |
> > > > > account_id | resource_id | resource_uuid|
> > > > > resource_type | customer |
> > > > > ++--+---+---
> > > > > +---++-+
> > > > > --+---+--+
> > > > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
> > >  1
> > > > |
> > > > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
> > > Account
> > > > >   | NULL |
> > > > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
> > >  1
> > > > |
> > > > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> > > Account
> > > > >   | NULL |
> > > > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
> > >  1
> > > > |
> > > > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> > > Account
> > > > >   | NULL |
> > > > > ++--+---+---
> > > > > +---++-+
> > > > > --+---+--+
> > > > > 3 rows in set, 1 warning (0.01 sec)
> > > > >
> > > > > Try to figure out why id=7 is selected here?
> > > > >
> > > > > Because:
> > > > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > > > >
> > > > > Matched unintentionally, because mysql converted uuid to int and
> got
> > 2
> > > > > which is matched to resource_id of 2 (id=7).
> > > > >
> > > > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> > > > > rafaelweingart...@gmail.com> написал:
> > > > >
> > > > > > Ah, ok now it makes sense the "IN", I thought you were only
> talking
> > > > about
> > > > > > single values.
> > > > > >
> > > > > >
> > > > > >
> > > > > > I do not think that the UUID (resource UUID) is the
> representation
> > of
> > > > ID
> > > > > > value in Hexadecimal, if it is we could simply get rid of one of
> > > them.
> > > > I
> > > > > > really dislike these search criteria...I am not seeing what you
> are
> > > > > saying.
> > > > > > Let´s see this in SQL, so we can discuss.
> > > > > >
> > > > > > SELECT * FROM resource_tags
> > > > > > WHERE  (resource_id in () OR resource_uuid in (...))
> > > > > > AND resource_tags.resource_type = 'Account';
> > > > > >
> > > > > >
> > > > > > That is what the 

Re: Tag removal problem found. Need assistance how to fix

[Ivan Kudryavtsev]

So, I don't understand. As you can see there is no longs in sql dump. Just
uuids. I run actual delete tags query against simulator and it leads to
this sql search which filters wrong data. As I see, query is incorrect.

22 нояб. 2017 г. 7:03 ПП пользователь "Rafael Weingärtner" <
rafaelweingart...@gmail.com> написал:

> The resourceId is not a real DB ID. It is the UUID converted to Long :(
> This table has four "ID" like fields, ID, UUID, resourceID, and
> resourceUUID.
>
> On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com
> > wrote:
>
> > Hi, I just enabled log all queries and copied actual query which I have
> > shown in the first mail. Also, I don't understand why the search should
> > look over inner ids... Is it a case when user can pass real db ids to an
> > api call?
> >
> > 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
> > rafaelweingart...@gmail.com> написал:
> >
> > > Yes, this I understood ;)
> > >
> > > However, I do not understand how the SQL that is being generated has
> this
> > > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > > c1bc5440ea60'".
> > > The resourceId field in the entity is a long. So, even though that long
> > > represents a String, in the final SQL that is generated it should be a
> > long
> > > value there.
> > >
> > >
> > >
> > > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> > > kudryavtsev...@bw-sw.com>
> > > wrote:
> > >
> > > > Take a look here:
> > > >
> > > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > > > resource_tags.value, resource_tags.domain_id,
> resource_tags.account_id,
> > > > resource_tags.resource_id, resource_tags.resource_uuid,
> > > > resource_tags.resource_type, resource_tags.customer FROM
> resource_tags
> > > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > > c1bc5440ea60'
> > > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > > > 9465-c1bc5440ea60'
> > > > )  AND resource_tags.resource_type = 'Account';
> > > >
> > > > ++--+---+---
> > > > +---++-+
> > > > --+---+--+
> > > > | id | uuid | key   | value |
> > domain_id |
> > > > account_id | resource_id | resource_uuid|
> > > > resource_type | customer |
> > > > ++--+---+---
> > > > +---++-+
> > > > --+---+--+
> > > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
> >  1
> > > |
> > > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
> > Account
> > > >   | NULL |
> > > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
> >  1
> > > |
> > > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> > Account
> > > >   | NULL |
> > > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
> >  1
> > > |
> > > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> > Account
> > > >   | NULL |
> > > > ++--+---+---
> > > > +---++-+
> > > > --+---+--+
> > > > 3 rows in set, 1 warning (0.01 sec)
> > > >
> > > > Try to figure out why id=7 is selected here?
> > > >
> > > > Because:
> > > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > > >
> > > > Matched unintentionally, because mysql converted uuid to int and got
> 2
> > > > which is matched to resource_id of 2 (id=7).
> > > >
> > > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> > > > rafaelweingart...@gmail.com> написал:
> > > >
> > > > > Ah, ok now it makes sense the "IN", I thought you were only talking
> > > about
> > > > > single values.
> > > > >
> > > > >
> > > > >
> > > > > I do not think that the UUID (resource UUID) is the representation
> of
> > > ID
> > > > > value in Hexadecimal, if it is we could simply get rid of one of
> > them.
> > > I
> > > > > really dislike these search criteria...I am not seeing what you are
> > > > saying.
> > > > > Let´s see this in SQL, so we can discuss.
> > > > >
> > > > > SELECT * FROM resource_tags
> > > > > WHERE  (resource_id in () OR resource_uuid in (...))
> > > > > AND resource_tags.resource_type = 'Account';
> > > > >
> > > > >
> > > > > That is what the programmer who coded that Search criteria seemed
> to
> > > > want,
> > > > > right? I mean, the developer wanted to select resources that match
> > > either
> > > > > the ID or UUID field. Also, we may have more than a single value to
> > > > filter
> > > > > in both ID and UUID. I am also assuming that UUID does not
> > necessarily
> > > > > represents the ID as a hexadecimal.
> > > > >
> > > > > The problem seems to be when it is being 

Re: Tag removal problem found. Need assistance how to fix

[Rafael Weingärtner]

The problem seems to be related to line 308 at
com.cloud.tags.TaggedResourceManagerImpl.deleteTags(List,
ResourceObjectType, Map).
It is being sent a list of resourceUUID as the filter for resourceId

On Wed, Nov 22, 2017 at 10:03 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> The resourceId is not a real DB ID. It is the UUID converted to Long :(
> This table has four "ID" like fields, ID, UUID, resourceID, and
> resourceUUID.
>
> On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com> wrote:
>
>> Hi, I just enabled log all queries and copied actual query which I have
>> shown in the first mail. Also, I don't understand why the search should
>> look over inner ids... Is it a case when user can pass real db ids to an
>> api call?
>>
>> 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
>> rafaelweingart...@gmail.com> написал:
>>
>> > Yes, this I understood ;)
>> >
>> > However, I do not understand how the SQL that is being generated has
>> this
>> > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
>> > c1bc5440ea60'".
>> > The resourceId field in the entity is a long. So, even though that long
>> > represents a String, in the final SQL that is generated it should be a
>> long
>> > value there.
>> >
>> >
>> >
>> > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
>> > kudryavtsev...@bw-sw.com>
>> > wrote:
>> >
>> > > Take a look here:
>> > >
>> > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
>> > > resource_tags.value, resource_tags.domain_id,
>> resource_tags.account_id,
>> > > resource_tags.resource_id, resource_tags.resource_uuid,
>> > > resource_tags.resource_type, resource_tags.customer FROM resource_tags
>> > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
>> > c1bc5440ea60'
>> > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
>> > > 9465-c1bc5440ea60'
>> > > )  AND resource_tags.resource_type = 'Account';
>> > >
>> > > ++--+---+---
>> > > +---++-+
>> > > --+---+--+
>> > > | id | uuid | key   | value |
>> domain_id |
>> > > account_id | resource_id | resource_uuid|
>> > > resource_type | customer |
>> > > ++--+---+---
>> > > +---++-+
>> > > --+---+--+
>> > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
>>  1
>> > |
>> > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
>> Account
>> > >   | NULL |
>> > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
>>  1
>> > |
>> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
>> Account
>> > >   | NULL |
>> > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
>>  1
>> > |
>> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
>> Account
>> > >   | NULL |
>> > > ++--+---+---
>> > > +---++-+
>> > > --+---+--+
>> > > 3 rows in set, 1 warning (0.01 sec)
>> > >
>> > > Try to figure out why id=7 is selected here?
>> > >
>> > > Because:
>> > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
>> > >
>> > > Matched unintentionally, because mysql converted uuid to int and got 2
>> > > which is matched to resource_id of 2 (id=7).
>> > >
>> > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
>> > > rafaelweingart...@gmail.com> написал:
>> > >
>> > > > Ah, ok now it makes sense the "IN", I thought you were only talking
>> > about
>> > > > single values.
>> > > >
>> > > >
>> > > >
>> > > > I do not think that the UUID (resource UUID) is the representation
>> of
>> > ID
>> > > > value in Hexadecimal, if it is we could simply get rid of one of
>> them.
>> > I
>> > > > really dislike these search criteria...I am not seeing what you are
>> > > saying.
>> > > > Let´s see this in SQL, so we can discuss.
>> > > >
>> > > > SELECT * FROM resource_tags
>> > > > WHERE  (resource_id in () OR resource_uuid in (...))
>> > > > AND resource_tags.resource_type = 'Account';
>> > > >
>> > > >
>> > > > That is what the programmer who coded that Search criteria seemed to
>> > > want,
>> > > > right? I mean, the developer wanted to select resources that match
>> > either
>> > > > the ID or UUID field. Also, we may have more than a single value to
>> > > filter
>> > > > in both ID and UUID. I am also assuming that UUID does not
>> necessarily
>> > > > represents the ID as a hexadecimal.
>> > > >
>> > > > The problem seems to be when it is being translated:
>> > > >
>> > > > > ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440e
>> a60'
>> > > > > OR
>> > > > > 

Re: Tag removal problem found. Need assistance how to fix

[Rafael Weingärtner]

The resourceId is not a real DB ID. It is the UUID converted to Long :(
This table has four "ID" like fields, ID, UUID, resourceID, and
resourceUUID.

On Wed, Nov 22, 2017 at 10:00 AM, Ivan Kudryavtsev  wrote:

> Hi, I just enabled log all queries and copied actual query which I have
> shown in the first mail. Also, I don't understand why the search should
> look over inner ids... Is it a case when user can pass real db ids to an
> api call?
>
> 22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
> rafaelweingart...@gmail.com> написал:
>
> > Yes, this I understood ;)
> >
> > However, I do not understand how the SQL that is being generated has this
> > clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > c1bc5440ea60'".
> > The resourceId field in the entity is a long. So, even though that long
> > represents a String, in the final SQL that is generated it should be a
> long
> > value there.
> >
> >
> >
> > On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com>
> > wrote:
> >
> > > Take a look here:
> > >
> > > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > > resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
> > > resource_tags.resource_id, resource_tags.resource_uuid,
> > > resource_tags.resource_type, resource_tags.customer FROM resource_tags
> > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > c1bc5440ea60'
> > > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > > 9465-c1bc5440ea60'
> > > )  AND resource_tags.resource_type = 'Account';
> > >
> > > ++--+---+---
> > > +---++-+
> > > --+---+--+
> > > | id | uuid | key   | value |
> domain_id |
> > > account_id | resource_id | resource_uuid|
> > > resource_type | customer |
> > > ++--+---+---
> > > +---++-+
> > > --+---+--+
> > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
>  1
> > |
> > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
> Account
> > >   | NULL |
> > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
>  1
> > |
> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> Account
> > >   | NULL |
> > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
>  1
> > |
> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> Account
> > >   | NULL |
> > > ++--+---+---
> > > +---++-+
> > > --+---+--+
> > > 3 rows in set, 1 warning (0.01 sec)
> > >
> > > Try to figure out why id=7 is selected here?
> > >
> > > Because:
> > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > >
> > > Matched unintentionally, because mysql converted uuid to int and got 2
> > > which is matched to resource_id of 2 (id=7).
> > >
> > > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> > > rafaelweingart...@gmail.com> написал:
> > >
> > > > Ah, ok now it makes sense the "IN", I thought you were only talking
> > about
> > > > single values.
> > > >
> > > >
> > > >
> > > > I do not think that the UUID (resource UUID) is the representation of
> > ID
> > > > value in Hexadecimal, if it is we could simply get rid of one of
> them.
> > I
> > > > really dislike these search criteria...I am not seeing what you are
> > > saying.
> > > > Let´s see this in SQL, so we can discuss.
> > > >
> > > > SELECT * FROM resource_tags
> > > > WHERE  (resource_id in () OR resource_uuid in (...))
> > > > AND resource_tags.resource_type = 'Account';
> > > >
> > > >
> > > > That is what the programmer who coded that Search criteria seemed to
> > > want,
> > > > right? I mean, the developer wanted to select resources that match
> > either
> > > > the ID or UUID field. Also, we may have more than a single value to
> > > filter
> > > > in both ID and UUID. I am also assuming that UUID does not
> necessarily
> > > > represents the ID as a hexadecimal.
> > > >
> > > > The problem seems to be when it is being translated:
> > > >
> > > > > ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > > > > OR
> > > > > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > > > 9465-c1bc5440ea60'
> > > > > )
> > > > >
> > > >
> > > > It is not even using an “IN” structure in the SQL. Also, why is the
> > > > resource_id equals the UUID in the filter. Did you check the entity
> > that
> > > is
> > > > being sent as an example? Are the fields ID and UUID set with the
> same
> > > > values?
> > > >
> > > >
> > > >
> > > > P.S. Normally the ID field of entities 

Re: Tag removal problem found. Need assistance how to fix

[Ivan Kudryavtsev]

Hi, I just enabled log all queries and copied actual query which I have
shown in the first mail. Also, I don't understand why the search should
look over inner ids... Is it a case when user can pass real db ids to an
api call?

22 нояб. 2017 г. 6:55 ПП пользователь "Rafael Weingärtner" <
rafaelweingart...@gmail.com> написал:

> Yes, this I understood ;)
>
> However, I do not understand how the SQL that is being generated has this
> clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> c1bc5440ea60'".
> The resourceId field in the entity is a long. So, even though that long
> represents a String, in the final SQL that is generated it should be a long
> value there.
>
>
>
> On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com>
> wrote:
>
> > Take a look here:
> >
> > SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
> > resource_tags.resource_id, resource_tags.resource_uuid,
> > resource_tags.resource_type, resource_tags.customer FROM resource_tags
> > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> c1bc5440ea60'
> > OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > 9465-c1bc5440ea60'
> > )  AND resource_tags.resource_type = 'Account';
> >
> > ++--+---+---
> > +---++-+
> > --+---+--+
> > | id | uuid | key   | value | domain_id |
> > account_id | resource_id | resource_uuid|
> > resource_type | customer |
> > ++--+---+---
> > +---++-+
> > --+---+--+
> > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me| 1
> |
> >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f | Account
> >   | NULL |
> > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1
> |
> >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
> >   | NULL |
> > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   | 1
> |
> >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
> >   | NULL |
> > ++--+---+---
> > +---++-+
> > --+---+--+
> > 3 rows in set, 1 warning (0.01 sec)
> >
> > Try to figure out why id=7 is selected here?
> >
> > Because:
> > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> >
> > Matched unintentionally, because mysql converted uuid to int and got 2
> > which is matched to resource_id of 2 (id=7).
> >
> > 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> > rafaelweingart...@gmail.com> написал:
> >
> > > Ah, ok now it makes sense the "IN", I thought you were only talking
> about
> > > single values.
> > >
> > >
> > >
> > > I do not think that the UUID (resource UUID) is the representation of
> ID
> > > value in Hexadecimal, if it is we could simply get rid of one of them.
> I
> > > really dislike these search criteria...I am not seeing what you are
> > saying.
> > > Let´s see this in SQL, so we can discuss.
> > >
> > > SELECT * FROM resource_tags
> > > WHERE  (resource_id in () OR resource_uuid in (...))
> > > AND resource_tags.resource_type = 'Account';
> > >
> > >
> > > That is what the programmer who coded that Search criteria seemed to
> > want,
> > > right? I mean, the developer wanted to select resources that match
> either
> > > the ID or UUID field. Also, we may have more than a single value to
> > filter
> > > in both ID and UUID. I am also assuming that UUID does not necessarily
> > > represents the ID as a hexadecimal.
> > >
> > > The problem seems to be when it is being translated:
> > >
> > > > ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > > > OR
> > > > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > > 9465-c1bc5440ea60'
> > > > )
> > > >
> > >
> > > It is not even using an “IN” structure in the SQL. Also, why is the
> > > resource_id equals the UUID in the filter. Did you check the entity
> that
> > is
> > > being sent as an example? Are the fields ID and UUID set with the same
> > > values?
> > >
> > >
> > >
> > > P.S. Normally the ID field of entities is not exposed to users via API.
> > The
> > > field ID in the API is translated to UUID in ACS. The field ID in the
> > > database is intended as a dummy primary key for the table.
> > >
> > >
> > > On Wed, Nov 22, 2017 at 9:05 AM, Ivan Kudryavtsev <
> > > kudryavtsev...@bw-sw.com>
> > > wrote:
> > >
> > > > Hi, Rafael, 'IN" because API call assumes that several resourceIds
> can
> > be
> > > > provided, so IN solves it, EQ doesn't.
> > > >
> > > > But despite semantics 

Re: Tag removal problem found. Need assistance how to fix

[Rafael Weingärtner]

Yes, this I understood ;)

However, I do not understand how the SQL that is being generated has this
clause: " resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'".
The resourceId field in the entity is a long. So, even though that long
represents a String, in the final SQL that is generated it should be a long
value there.



On Wed, Nov 22, 2017 at 9:28 AM, Ivan Kudryavtsev 
wrote:

> Take a look here:
>
> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
> resource_tags.resource_id, resource_tags.resource_uuid,
> resource_tags.resource_type, resource_tags.customer FROM resource_tags
> WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> 9465-c1bc5440ea60'
> )  AND resource_tags.resource_type = 'Account';
>
> ++--+---+---
> +---++-+
> --+---+--+
> | id | uuid | key   | value | domain_id |
> account_id | resource_id | resource_uuid|
> resource_type | customer |
> ++--+---+---
> +---++-+
> --+---+--+
> |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me| 1 |
>2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f | Account
>   | NULL |
> | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1 |
>4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
>   | NULL |
> | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   | 1 |
>4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
>   | NULL |
> ++--+---+---
> +---++-+
> --+---+--+
> 3 rows in set, 1 warning (0.01 sec)
>
> Try to figure out why id=7 is selected here?
>
> Because:
> resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
>
> Matched unintentionally, because mysql converted uuid to int and got 2
> which is matched to resource_id of 2 (id=7).
>
> 22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
> rafaelweingart...@gmail.com> написал:
>
> > Ah, ok now it makes sense the "IN", I thought you were only talking about
> > single values.
> >
> >
> >
> > I do not think that the UUID (resource UUID) is the representation of ID
> > value in Hexadecimal, if it is we could simply get rid of one of them. I
> > really dislike these search criteria...I am not seeing what you are
> saying.
> > Let´s see this in SQL, so we can discuss.
> >
> > SELECT * FROM resource_tags
> > WHERE  (resource_id in () OR resource_uuid in (...))
> > AND resource_tags.resource_type = 'Account';
> >
> >
> > That is what the programmer who coded that Search criteria seemed to
> want,
> > right? I mean, the developer wanted to select resources that match either
> > the ID or UUID field. Also, we may have more than a single value to
> filter
> > in both ID and UUID. I am also assuming that UUID does not necessarily
> > represents the ID as a hexadecimal.
> >
> > The problem seems to be when it is being translated:
> >
> > > ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > > OR
> > > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > 9465-c1bc5440ea60'
> > > )
> > >
> >
> > It is not even using an “IN” structure in the SQL. Also, why is the
> > resource_id equals the UUID in the filter. Did you check the entity that
> is
> > being sent as an example? Are the fields ID and UUID set with the same
> > values?
> >
> >
> >
> > P.S. Normally the ID field of entities is not exposed to users via API.
> The
> > field ID in the API is translated to UUID in ACS. The field ID in the
> > database is intended as a dummy primary key for the table.
> >
> >
> > On Wed, Nov 22, 2017 at 9:05 AM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com>
> > wrote:
> >
> > > Hi, Rafael, 'IN" because API call assumes that several resourceIds can
> be
> > > provided, so IN solves it, EQ doesn't.
> > >
> > > But despite semantics ID/UUID you see that ID is integer and UUID is
> > string
> > > and that comparison does fault positive results, next when object
> access
> > > for caller is checked exception occured and no tag removal happen as a
> > > result because int(2) eq '2afcffdsfdsfds-... (UUID)".
> > >
> > > 2017-11-22 18:01 GMT+07:00 Rafael Weingärtner <
> > rafaelweingart...@gmail.com
> > > >:
> > >
> > > > Are ID and UUID set with the same values in that entity? If not, the
> > > > criteria seem correct. I mean, it is trying to filter for an ID if it
> > > > exists or by UUID if it exists in 

RE: cloudstack deployDataCentre with MariaDB

[Paul Angus]

Hi Henko,

As you have the error 'Execute cmd: addhost failed'.  I would look in the mgmt. 
server log first, to see it gives a reason why the CloudStack manager could not 
add the host.



Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Henko Holtzhausen [mailto:henko.holtzhau...@shapeblue.com] 
Sent: 22 November 2017 10:06
To: dev@cloudstack.apache.org
Subject: cloudstack deployDataCentre with MariaDB

Hi Everybody

My name is Henko, I am new to using cloudstack.

I have an issue deploying a datacentre in a simulated environment.

My command and response:

$ /usr/bin/python tools/marvin/marvin/deployDataCenter.py -i 
setup/dev/advanced.cfg

 Log Folder Path: 
/tmp//MarvinLogs//DeployDataCenter__Nov_22_2017_11_23_38_KCZF9L. All logs will 
be available here 

 Deploy DC Started 
Exception Occurred :['Traceback (most recent call last):\n', '  File 
"tools/marvin/marvin/deployDataCenter.py", line 142, in addHosts\nret = 
self.__apiClient.addHost(hostcmd)\n', '  File 
"/home/henko/.local/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py",
 line 2589, in addHost\nresponse = self.connection.marvinRequest(command, 
response_type=response, method=method)\n', '  File 
"/home/henko/.local/lib/python2.7/site-packages/marvin/cloudstackConnection.py",
 line 379, in marvinRequest\nraise e\n', 'CloudstackAPIException: Execute 
cmd: addhost failed, due to: errorCode: 530, errorText:Index: 0, Size: 0\n']
Exception Occurred :['Traceback (most recent call last):\n', '  File 
"tools/marvin/marvin/deployDataCenter.py", line 142, in addHosts\nret = 
self.__apiClient.addHost(hostcmd)\n', '  File 
"/home/henko/.local/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py",
 line 2589, in addHost\nresponse = self.connection.marvinRequest(command, 
response_type=response, method=method)\n', '  File 
"/home/henko/.local/lib/python2.7/site-packages/marvin/cloudstackConnection.py",
 line 379, in marvinRequest\nraise e\n', 'CloudstackAPIException: Execute 
cmd: addhost failed, due to: errorCode: 530, errorText:Index: 0, Size: 0\n']

===deploy dc failed, so cleaning the created entries===

DeployDC: CleanUp Started

Clean Up Entries=== {'PhysicalNetwork': 
[u'93ab6ef5-a886-44b7-8dd2-3e0d9f6b7ea2'], 'Cluster': 
[u'dd655d79-42ad-430c-b930-93b722c93684'], 'Pod': 
[u'6c883b9e-9787-4b06-bff7-bcafe749c7be'], 'order': ['Cluster', 'Pod', 
'PhysicalNetwork', 'Zone'], 'Zone': [u'81744ed7-ea3b-4282-bb5a-bcf7e8538a53']}

===Removing DataCenter Failed===


I am using the following versions:
MariaDB - mysql  Ver 15.1 Distrib 10.2.10-MariaDB, for debian-linux-gnu 
(x86_64) using readline 5.2 java version "1.8.0_151"
Python 2.7.12
Ubuntu 16.04.3 LTS

Some of my colleagues are using the same version, except instead of MariaDB 
they're using mysql, and it is working for them.

Regards

henko.holtzhau...@shapeblue.com
www.shapeblue.com
,   
@shapeblue
  
 

Re: Tag removal problem found. Need assistance how to fix

[Ivan Kudryavtsev]

Take a look here:

SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
resource_tags.resource_id, resource_tags.resource_uuid,
resource_tags.resource_type, resource_tags.customer FROM resource_tags
WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
OR resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
)  AND resource_tags.resource_type = 'Account';

++--+---+---
+---++-+
--+---+--+
| id | uuid | key   | value | domain_id |
account_id | resource_id | resource_uuid|
resource_type | customer |
++--+---+---
+---++-+
--+---+--+
|  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me| 1 |
   2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f | Account
  | NULL |
| 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1 |
   4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
  | NULL |
| 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   | 1 |
   4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
  | NULL |
++--+---+---
+---++-+
--+---+--+
3 rows in set, 1 warning (0.01 sec)

Try to figure out why id=7 is selected here?

Because:
resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'

Matched unintentionally, because mysql converted uuid to int and got 2
which is matched to resource_id of 2 (id=7).

22 нояб. 2017 г. 6:23 ПП пользователь "Rafael Weingärtner" <
rafaelweingart...@gmail.com> написал:

> Ah, ok now it makes sense the "IN", I thought you were only talking about
> single values.
>
>
>
> I do not think that the UUID (resource UUID) is the representation of ID
> value in Hexadecimal, if it is we could simply get rid of one of them. I
> really dislike these search criteria...I am not seeing what you are saying.
> Let´s see this in SQL, so we can discuss.
>
> SELECT * FROM resource_tags
> WHERE  (resource_id in () OR resource_uuid in (...))
> AND resource_tags.resource_type = 'Account';
>
>
> That is what the programmer who coded that Search criteria seemed to want,
> right? I mean, the developer wanted to select resources that match either
> the ID or UUID field. Also, we may have more than a single value to filter
> in both ID and UUID. I am also assuming that UUID does not necessarily
> represents the ID as a hexadecimal.
>
> The problem seems to be when it is being translated:
>
> > ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> > OR
> > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> 9465-c1bc5440ea60'
> > )
> >
>
> It is not even using an “IN” structure in the SQL. Also, why is the
> resource_id equals the UUID in the filter. Did you check the entity that is
> being sent as an example? Are the fields ID and UUID set with the same
> values?
>
>
>
> P.S. Normally the ID field of entities is not exposed to users via API. The
> field ID in the API is translated to UUID in ACS. The field ID in the
> database is intended as a dummy primary key for the table.
>
>
> On Wed, Nov 22, 2017 at 9:05 AM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com>
> wrote:
>
> > Hi, Rafael, 'IN" because API call assumes that several resourceIds can be
> > provided, so IN solves it, EQ doesn't.
> >
> > But despite semantics ID/UUID you see that ID is integer and UUID is
> string
> > and that comparison does fault positive results, next when object access
> > for caller is checked exception occured and no tag removal happen as a
> > result because int(2) eq '2afcffdsfdsfds-... (UUID)".
> >
> > 2017-11-22 18:01 GMT+07:00 Rafael Weingärtner <
> rafaelweingart...@gmail.com
> > >:
> >
> > > Are ID and UUID set with the same values in that entity? If not, the
> > > criteria seem correct. I mean, it is trying to filter for an ID if it
> > > exists or by UUID if it exists in the entity that is passed as an
> > example.
> > > What I do not understand is that they are using “SearchCriteria.Op.IN
> ”,
> > > but
> > > in my opinion, it should be “SearchCriteria.Op.EQ”.
> > >
> > > On Wed, Nov 22, 2017 at 7:58 AM, Ivan Kudryavtsev <
> > > kudryavtsev...@bw-sw.com>
> > > wrote:
> > >
> > > > Hi, I found interesting behaviour with tags:
> > > >
> > > > mysql> SELECT resource_tags.id, resource_tags.uuid,
> resource_tags.key,
> > > > resource_tags.value, resource_tags.domain_id,
> resource_tags.account_id,
> > > > resource_tags.resource_id, resource_tags.resource_uuid,
> > > > resource_tags.resource_type, 

Re: Tag removal problem found. Need assistance how to fix

[Rafael Weingärtner]

Ah, ok now it makes sense the "IN", I thought you were only talking about
single values.



I do not think that the UUID (resource UUID) is the representation of ID
value in Hexadecimal, if it is we could simply get rid of one of them. I
really dislike these search criteria...I am not seeing what you are saying.
Let´s see this in SQL, so we can discuss.

SELECT * FROM resource_tags
WHERE  (resource_id in () OR resource_uuid in (...))
AND resource_tags.resource_type = 'Account';


That is what the programmer who coded that Search criteria seemed to want,
right? I mean, the developer wanted to select resources that match either
the ID or UUID field. Also, we may have more than a single value to filter
in both ID and UUID. I am also assuming that UUID does not necessarily
represents the ID as a hexadecimal.

The problem seems to be when it is being translated:

> ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> OR
> resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> )
>

It is not even using an “IN” structure in the SQL. Also, why is the
resource_id equals the UUID in the filter. Did you check the entity that is
being sent as an example? Are the fields ID and UUID set with the same
values?



P.S. Normally the ID field of entities is not exposed to users via API. The
field ID in the API is translated to UUID in ACS. The field ID in the
database is intended as a dummy primary key for the table.


On Wed, Nov 22, 2017 at 9:05 AM, Ivan Kudryavtsev 
wrote:

> Hi, Rafael, 'IN" because API call assumes that several resourceIds can be
> provided, so IN solves it, EQ doesn't.
>
> But despite semantics ID/UUID you see that ID is integer and UUID is string
> and that comparison does fault positive results, next when object access
> for caller is checked exception occured and no tag removal happen as a
> result because int(2) eq '2afcffdsfdsfds-... (UUID)".
>
> 2017-11-22 18:01 GMT+07:00 Rafael Weingärtner  >:
>
> > Are ID and UUID set with the same values in that entity? If not, the
> > criteria seem correct. I mean, it is trying to filter for an ID if it
> > exists or by UUID if it exists in the entity that is passed as an
> example.
> > What I do not understand is that they are using “SearchCriteria.Op.IN”,
> > but
> > in my opinion, it should be “SearchCriteria.Op.EQ”.
> >
> > On Wed, Nov 22, 2017 at 7:58 AM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com>
> > wrote:
> >
> > > Hi, I found interesting behaviour with tags:
> > >
> > > mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > > resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
> > > resource_tags.resource_id, resource_tags.resource_uuid,
> > > resource_tags.resource_type, resource_tags.customer FROM resource_tags
> > > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> > c1bc5440ea60'
> > > OR
> > > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> > 9465-c1bc5440ea60'
> > > )
> > >  AND resource_tags.resource_type = 'Account';
> > >
> > > ++--+---+---
> > > +---++-+
> > > --+---+--+
> > > | id | uuid | key   | value |
> domain_id |
> > > account_id | resource_id | resource_uuid|
> > > resource_type | customer |
> > > ++--+---+---
> > > +---++-+
> > > --+---+--+
> > > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
>  1
> > |
> > >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f |
> Account
> > >   | NULL |
> > > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
>  1
> > |
> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> Account
> > >   | NULL |
> > > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
>  1
> > |
> > >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 |
> Account
> > >   | NULL |
> > > ++--+---+---
> > > +---++-+
> > > --+---+--+
> > > 3 rows in set, 1 warning (0.01 sec)
> > >
> > > Don't see that "resource_type" is "account". I just play with it.
> > >
> > > Take a look at ID=7. This row is found because:
> > >
> > > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60' when
> > > right
> > > part is converted to int. Corresponding code is here:
> > >
> > > https://github.com/apache/cloudstack/blob/87ef8137534fa79810
> 1f65c6691fcf
> > > 71513ac978/server/src/com/cloud/tags/TaggedResourceManagerIm
> pl.java#L301
> > >
> > > sb.and().op("resourceId", sb.entity().getResourceId(),
> > > SearchCriteria.Op.IN);
> > 

Re: Tag removal problem found. Need assistance how to fix

[Ivan Kudryavtsev]

I mean that it's like SQL injection. API call expects UUID for resourceID,
but in code that API value is checked against resourceID which is integer
and internal to DB and against resourceUuid which is known for API... This
looks weird even without keeping in mind it grabs extra irrelevant results
during the search.

22 нояб. 2017 г. 6:05 ПП пользователь "Ivan Kudryavtsev" <
kudryavtsev...@bw-sw.com> написал:

> Hi, Rafael, 'IN" because API call assumes that several resourceIds can be
> provided, so IN solves it, EQ doesn't.
>
> But despite semantics ID/UUID you see that ID is integer and UUID is
> string and that comparison does fault positive results, next when object
> access for caller is checked exception occured and no tag removal happen as
> a result because int(2) eq '2afcffdsfdsfds-... (UUID)".
>
> 2017-11-22 18:01 GMT+07:00 Rafael Weingärtner  >:
>
>> Are ID and UUID set with the same values in that entity? If not, the
>> criteria seem correct. I mean, it is trying to filter for an ID if it
>> exists or by UUID if it exists in the entity that is passed as an example.
>> What I do not understand is that they are using “SearchCriteria.Op.IN”,
>> but
>> in my opinion, it should be “SearchCriteria.Op.EQ”.
>>
>> On Wed, Nov 22, 2017 at 7:58 AM, Ivan Kudryavtsev <
>> kudryavtsev...@bw-sw.com>
>> wrote:
>>
>> > Hi, I found interesting behaviour with tags:
>> >
>> > mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
>> > resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
>> > resource_tags.resource_id, resource_tags.resource_uuid,
>> > resource_tags.resource_type, resource_tags.customer FROM resource_tags
>> > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440e
>> a60'
>> > OR
>> > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-9465-
>> c1bc5440ea60'
>> > )
>> >  AND resource_tags.resource_type = 'Account';
>> >
>> > ++--+---+---
>> > +---++-+
>> > --+---+--+
>> > | id | uuid | key   | value | domain_id
>> |
>> > account_id | resource_id | resource_uuid|
>> > resource_type | customer |
>> > ++--+---+---
>> > +---++-+
>> > --+---+--+
>> > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me|
>>  1 |
>> >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f | Account
>> >   | NULL |
>> > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me|
>>  1 |
>> >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
>> >   | NULL |
>> > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   |
>>  1 |
>> >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
>> >   | NULL |
>> > ++--+---+---
>> > +---++-+
>> > --+---+--+
>> > 3 rows in set, 1 warning (0.01 sec)
>> >
>> > Don't see that "resource_type" is "account". I just play with it.
>> >
>> > Take a look at ID=7. This row is found because:
>> >
>> > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60' when
>> > right
>> > part is converted to int. Corresponding code is here:
>> >
>> > https://github.com/apache/cloudstack/blob/87ef8137534fa79810
>> 1f65c6691fcf
>> > 71513ac978/server/src/com/cloud/tags/TaggedResourceManagerIm
>> pl.java#L301
>> >
>> > sb.and().op("resourceId", sb.entity().getResourceId(),
>> > SearchCriteria.Op.IN);
>> > sb.or("resourceUuid", sb.entity().getResourceUuid(),
>> SearchCriteria.Op.IN
>> > );
>> > sb.cp();
>> > sb.and("resourceType", sb.entity().getResourceType(),
>> > SearchCriteria.Op.EQ);
>> >
>> > I don't know why the writer uses "resourceId" or "resourceUuid". I
>> suppose
>> > it's a bug and code should be transformed to:
>> >
>> > sb.and("resourceUuid", sb.entity().getResourceUuid(),
>> SearchCriteria.Op.IN
>> > );
>> > sb.and("resourceType", sb.entity().getResourceType(),
>> > SearchCriteria.Op.EQ);
>> >
>> > Or MySQL query should be transformed to:
>> >
>> > mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
>> > resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
>> > resource_tags.resource_id, resource_tags.resource_uuid,
>> > resource_tags.resource_type, resource_tags.customer FROM resource_tags
>> > WHERE  ( concat("%", resource_tags.resource_id) =
>> > '2a4264fb-9f63-4d4f-9465-c1bc5440ea60' OR
>> > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-9465-
>> c1bc5440ea60'
>> > )
>> >  AND resource_tags.resource_type = 'Account';
>> > ++--+---+---
>> > 

Re: Tag removal problem found. Need assistance how to fix

[Ivan Kudryavtsev]

Hi, Rafael, 'IN" because API call assumes that several resourceIds can be
provided, so IN solves it, EQ doesn't.

But despite semantics ID/UUID you see that ID is integer and UUID is string
and that comparison does fault positive results, next when object access
for caller is checked exception occured and no tag removal happen as a
result because int(2) eq '2afcffdsfdsfds-... (UUID)".

2017-11-22 18:01 GMT+07:00 Rafael Weingärtner :

> Are ID and UUID set with the same values in that entity? If not, the
> criteria seem correct. I mean, it is trying to filter for an ID if it
> exists or by UUID if it exists in the entity that is passed as an example.
> What I do not understand is that they are using “SearchCriteria.Op.IN”,
> but
> in my opinion, it should be “SearchCriteria.Op.EQ”.
>
> On Wed, Nov 22, 2017 at 7:58 AM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com>
> wrote:
>
> > Hi, I found interesting behaviour with tags:
> >
> > mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
> > resource_tags.resource_id, resource_tags.resource_uuid,
> > resource_tags.resource_type, resource_tags.customer FROM resource_tags
> > WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-
> c1bc5440ea60'
> > OR
> > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> 9465-c1bc5440ea60'
> > )
> >  AND resource_tags.resource_type = 'Account';
> >
> > ++--+---+---
> > +---++-+
> > --+---+--+
> > | id | uuid | key   | value | domain_id |
> > account_id | resource_id | resource_uuid|
> > resource_type | customer |
> > ++--+---+---
> > +---++-+
> > --+---+--+
> > |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me| 1
> |
> >2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f | Account
> >   | NULL |
> > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1
> |
> >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
> >   | NULL |
> > | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   | 1
> |
> >4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
> >   | NULL |
> > ++--+---+---
> > +---++-+
> > --+---+--+
> > 3 rows in set, 1 warning (0.01 sec)
> >
> > Don't see that "resource_type" is "account". I just play with it.
> >
> > Take a look at ID=7. This row is found because:
> >
> > resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60' when
> > right
> > part is converted to int. Corresponding code is here:
> >
> > https://github.com/apache/cloudstack/blob/87ef8137534fa798101f65c6691fcf
> > 71513ac978/server/src/com/cloud/tags/TaggedResourceManagerImpl.java#L301
> >
> > sb.and().op("resourceId", sb.entity().getResourceId(),
> > SearchCriteria.Op.IN);
> > sb.or("resourceUuid", sb.entity().getResourceUuid(),
> SearchCriteria.Op.IN
> > );
> > sb.cp();
> > sb.and("resourceType", sb.entity().getResourceType(),
> > SearchCriteria.Op.EQ);
> >
> > I don't know why the writer uses "resourceId" or "resourceUuid". I
> suppose
> > it's a bug and code should be transformed to:
> >
> > sb.and("resourceUuid", sb.entity().getResourceUuid(),
> SearchCriteria.Op.IN
> > );
> > sb.and("resourceType", sb.entity().getResourceType(),
> > SearchCriteria.Op.EQ);
> >
> > Or MySQL query should be transformed to:
> >
> > mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> > resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
> > resource_tags.resource_id, resource_tags.resource_uuid,
> > resource_tags.resource_type, resource_tags.customer FROM resource_tags
> > WHERE  ( concat("%", resource_tags.resource_id) =
> > '2a4264fb-9f63-4d4f-9465-c1bc5440ea60' OR
> > resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-
> 9465-c1bc5440ea60'
> > )
> >  AND resource_tags.resource_type = 'Account';
> > ++--+---+---
> > +---++-+
> > --+---+--+
> > | id | uuid | key   | value | domain_id |
> > account_id | resource_id | resource_uuid|
> > resource_type | customer |
> > ++--+---+---
> > +---++-+
> > --+---+--+
> > | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1
> |
> >   

Re: Tag removal problem found. Need assistance how to fix

[Rafael Weingärtner]

Are ID and UUID set with the same values in that entity? If not, the
criteria seem correct. I mean, it is trying to filter for an ID if it
exists or by UUID if it exists in the entity that is passed as an example.
What I do not understand is that they are using “SearchCriteria.Op.IN”, but
in my opinion, it should be “SearchCriteria.Op.EQ”.

On Wed, Nov 22, 2017 at 7:58 AM, Ivan Kudryavtsev 
wrote:

> Hi, I found interesting behaviour with tags:
>
> mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
> resource_tags.resource_id, resource_tags.resource_uuid,
> resource_tags.resource_type, resource_tags.customer FROM resource_tags
> WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> OR
> resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> )
>  AND resource_tags.resource_type = 'Account';
>
> ++--+---+---
> +---++-+
> --+---+--+
> | id | uuid | key   | value | domain_id |
> account_id | resource_id | resource_uuid|
> resource_type | customer |
> ++--+---+---
> +---++-+
> --+---+--+
> |  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me| 1 |
>2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f | Account
>   | NULL |
> | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1 |
>4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
>   | NULL |
> | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   | 1 |
>4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
>   | NULL |
> ++--+---+---
> +---++-+
> --+---+--+
> 3 rows in set, 1 warning (0.01 sec)
>
> Don't see that "resource_type" is "account". I just play with it.
>
> Take a look at ID=7. This row is found because:
>
> resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60' when
> right
> part is converted to int. Corresponding code is here:
>
> https://github.com/apache/cloudstack/blob/87ef8137534fa798101f65c6691fcf
> 71513ac978/server/src/com/cloud/tags/TaggedResourceManagerImpl.java#L301
>
> sb.and().op("resourceId", sb.entity().getResourceId(),
> SearchCriteria.Op.IN);
> sb.or("resourceUuid", sb.entity().getResourceUuid(), SearchCriteria.Op.IN
> );
> sb.cp();
> sb.and("resourceType", sb.entity().getResourceType(),
> SearchCriteria.Op.EQ);
>
> I don't know why the writer uses "resourceId" or "resourceUuid". I suppose
> it's a bug and code should be transformed to:
>
> sb.and("resourceUuid", sb.entity().getResourceUuid(), SearchCriteria.Op.IN
> );
> sb.and("resourceType", sb.entity().getResourceType(),
> SearchCriteria.Op.EQ);
>
> Or MySQL query should be transformed to:
>
> mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
> resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
> resource_tags.resource_id, resource_tags.resource_uuid,
> resource_tags.resource_type, resource_tags.customer FROM resource_tags
> WHERE  ( concat("%", resource_tags.resource_id) =
> '2a4264fb-9f63-4d4f-9465-c1bc5440ea60' OR
> resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
> )
>  AND resource_tags.resource_type = 'Account';
> ++--+---+---
> +---++-+
> --+---+--+
> | id | uuid | key   | value | domain_id |
> account_id | resource_id | resource_uuid|
> resource_type | customer |
> ++--+---+---
> +---++-+
> --+---+--+
> | 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1 |
>4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
>   | NULL |
> | 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   | 1 |
>4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
>   | NULL |
> ++--+---+---
> +---++-+
> --+---+--+
> 2 rows in set (0.00 sec)
>
> Let me your thoughts and I'll fix it. Right now, obviously it's a bug.
>
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ 

cloudstack deployDataCentre with MariaDB

[Henko Holtzhausen]

Hi Everybody

My name is Henko, I am new to using cloudstack.

I have an issue deploying a datacentre in a simulated environment.

My command and response:

$ /usr/bin/python tools/marvin/marvin/deployDataCenter.py -i 
setup/dev/advanced.cfg

 Log Folder Path: 
/tmp//MarvinLogs//DeployDataCenter__Nov_22_2017_11_23_38_KCZF9L. All logs will 
be available here 

 Deploy DC Started 
Exception Occurred :['Traceback (most recent call last):\n', '  File 
"tools/marvin/marvin/deployDataCenter.py", line 142, in addHosts\nret = 
self.__apiClient.addHost(hostcmd)\n', '  File 
"/home/henko/.local/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py",
 line 2589, in addHost\nresponse = self.connection.marvinRequest(command, 
response_type=response, method=method)\n', '  File 
"/home/henko/.local/lib/python2.7/site-packages/marvin/cloudstackConnection.py",
 line 379, in marvinRequest\nraise e\n', 'CloudstackAPIException: Execute 
cmd: addhost failed, due to: errorCode: 530, errorText:Index: 0, Size: 0\n']
Exception Occurred :['Traceback (most recent call last):\n', '  File 
"tools/marvin/marvin/deployDataCenter.py", line 142, in addHosts\nret = 
self.__apiClient.addHost(hostcmd)\n', '  File 
"/home/henko/.local/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py",
 line 2589, in addHost\nresponse = self.connection.marvinRequest(command, 
response_type=response, method=method)\n', '  File 
"/home/henko/.local/lib/python2.7/site-packages/marvin/cloudstackConnection.py",
 line 379, in marvinRequest\nraise e\n', 'CloudstackAPIException: Execute 
cmd: addhost failed, due to: errorCode: 530, errorText:Index: 0, Size: 0\n']

===deploy dc failed, so cleaning the created entries===

DeployDC: CleanUp Started

Clean Up Entries=== {'PhysicalNetwork': 
[u'93ab6ef5-a886-44b7-8dd2-3e0d9f6b7ea2'], 'Cluster': 
[u'dd655d79-42ad-430c-b930-93b722c93684'], 'Pod': 
[u'6c883b9e-9787-4b06-bff7-bcafe749c7be'], 'order': ['Cluster', 'Pod', 
'PhysicalNetwork', 'Zone'], 'Zone': [u'81744ed7-ea3b-4282-bb5a-bcf7e8538a53']}

===Removing DataCenter Failed===


I am using the following versions:
MariaDB - mysql  Ver 15.1 Distrib 10.2.10-MariaDB, for debian-linux-gnu 
(x86_64) using readline 5.2
java version "1.8.0_151"
Python 2.7.12
Ubuntu 16.04.3 LTS

Some of my colleagues are using the same version, except instead of MariaDB 
they're using mysql, and it is working for them.

Regards

henko.holtzhau...@shapeblue.com 
www.shapeblue.com
,   
@shapeblue
  
 

Tag removal problem found. Need assistance how to fix

[Ivan Kudryavtsev]

Hi, I found interesting behaviour with tags:

mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
resource_tags.resource_id, resource_tags.resource_uuid,
resource_tags.resource_type, resource_tags.customer FROM resource_tags
WHERE  ( resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60'
OR
resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-9465-c1bc5440ea60' )
 AND resource_tags.resource_type = 'Account';

++--+---+---+---++-+--+---+--+
| id | uuid | key   | value | domain_id |
account_id | resource_id | resource_uuid|
resource_type | customer |
++--+---+---+---++-+--+---+--+
|  7 | 95a1a314-2247-4622-a33f-b9b2680bc2e1 | test  | me| 1 |
   2 |   2 | 3199fc71-cf39-11e7-af5d-dc0ea16ecd7f | Account
  | NULL |
| 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1 |
   4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
  | NULL |
| 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   | 1 |
   4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
  | NULL |
++--+---+---+---++-+--+---+--+
3 rows in set, 1 warning (0.01 sec)

Don't see that "resource_type" is "account". I just play with it.

Take a look at ID=7. This row is found because:

resource_tags.resource_id='2a4264fb-9f63-4d4f-9465-c1bc5440ea60' when right
part is converted to int. Corresponding code is here:

https://github.com/apache/cloudstack/blob/87ef8137534fa798101f65c6691fcf71513ac978/server/src/com/cloud/tags/TaggedResourceManagerImpl.java#L301

sb.and().op("resourceId", sb.entity().getResourceId(), SearchCriteria.Op.IN);
sb.or("resourceUuid", sb.entity().getResourceUuid(), SearchCriteria.Op.IN);
sb.cp();
sb.and("resourceType", sb.entity().getResourceType(), SearchCriteria.Op.EQ);

I don't know why the writer uses "resourceId" or "resourceUuid". I suppose
it's a bug and code should be transformed to:

sb.and("resourceUuid", sb.entity().getResourceUuid(), SearchCriteria.Op.IN);
sb.and("resourceType", sb.entity().getResourceType(), SearchCriteria.Op.EQ);

Or MySQL query should be transformed to:

mysql> SELECT resource_tags.id, resource_tags.uuid, resource_tags.key,
resource_tags.value, resource_tags.domain_id, resource_tags.account_id,
resource_tags.resource_id, resource_tags.resource_uuid,
resource_tags.resource_type, resource_tags.customer FROM resource_tags
WHERE  ( concat("%", resource_tags.resource_id) =
'2a4264fb-9f63-4d4f-9465-c1bc5440ea60' OR
resource_tags.resource_uuid=_binary'2a4264fb-9f63-4d4f-9465-c1bc5440ea60' )
 AND resource_tags.resource_type = 'Account';
++--+---+---+---++-+--+---+--+
| id | uuid | key   | value | domain_id |
account_id | resource_id | resource_uuid|
resource_type | customer |
++--+---+---+---++-+--+---+--+
| 10 | 6c247aa1-5524-4910-9b5f-c6cfd9b3bdd9 | test3 | me| 1 |
   4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
  | NULL |
| 12 | 25fb7848-af34-42f7-855e-0f5909a4e979 | test5 | me2   | 1 |
   4 |   4 | 2a4264fb-9f63-4d4f-9465-c1bc5440ea60 | Account
  | NULL |
++--+---+---+---++-+--+---+--+
2 rows in set (0.00 sec)

Let me your thoughts and I'll fix it. Right now, obviously it's a bug.



-- 
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ 

Re:Re: CloudStack LTS EOL date?

[Haijiao]

Agree.  And I suppose 4.11 will be the next LTS ? 


I think community has lots of good initiatives in past years,  but most 
important is to 'insist' and make it happen.  LTS concept is one of good 
references. 


Regards,


在2017年11月22 14时13分, "Milamber"写道:


Sound good for me too

On 21/11/2017 11:04, Rene Moser wrote:
> Hi all
>
> The current LTS release is 4.9 which is EOL in June 2018 according to
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/LTS
>
> AFAIK there are no works planed for a new LTS. The release pace has
> slown down (the high pace and leaving users behind fixes was the reason
> for the LTS).
>
> I am still pro LTS but in my opinion we should have defined the EOL in
> relation of the successor LTS release date: "The EOL of the current LTS
> is +6 months after the next LTS release."
>
> Small example:
>
> Current LTS 4.9
> Next LTS 4.1x release on 01.04. --> LTS 4.9 is 01.10.
>
> Does this make sense? Other suggestions?
>
> Regards
> René
> .
>

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rohit Yadav]

Hi René,


Thanks for your comments, this will be a strictly maintenance project at the 
moment I think we should consider expanding its scope to include any new 
feature/integration. The current aim and scope of this work are to migrate the 
base image to Debian9 and ensure no regressions. However, feel free to 
discuss/work on the proposed features/integrations separately.


Regards.



From: Rene Moser 
Sent: Tuesday, November 21, 2017 8:37:49 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

Hi Rohit

First, thank you very much for the effort!

On 11/21/2017 01:07 PM, Rohit Yadav wrote:

> Please advise if you can collaborate with me on this, especially around 
> testing. Thanks!

I have some 2 "nice to have"s, AFAICS these haven't been addressed yet:

- SNMP readonly listen on "linklocalip"
- HAProxy stats (or even exporter
https://github.com/prometheus/haproxy_exporter) listen on linklocalip

Regards
René


rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

Re: Fail with vpn customer gateway creation through terraform

[Nux!]

Hi guys,

sha1-aes256;modp3072 works if I use the UI or cloudmonkey, that's why I am 
thinking it must be terraform or some weird encoding issues.

I tried replacing ; with - and also using modp2048, to no avail.

"* cloudstack_vpn_customer_gateway.default: Error creating VPN Customer Gateway 
test-vpc: Undefined error: {"errorcode":431,"errortext":"The customer gateway 
IKE policy sha1-aes256-modp2048 is invalid!  Verify the required Diffie Hellman 
(DH) group is specified."}"


Logs here

https://paste.fedoraproject.org/paste/HpGdigqa33ZjTeIDrAwE9w

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Jayapal Uradi" 
> To: "dev" 
> Sent: Wednesday, 22 November, 2017 04:20:53
> Subject: Re: Fail with vpn customer gateway creation through terraform

> Hi Lucian,
> 
> Try the the following in config, ‘-‘ instead of ‘;’ after the aes256 in the
> config.
> 
> New: "sha1-aes256-modp3072”
> Old: "sha1-aes256;modp3072”
> 
> Thanks,
> Jayapal
> On Nov 22, 2017, at 5:44 AM, Pierre-Luc Dion
> > wrote:
> 
> Hi Nux,
> 
> Could it be your cloudstack version ?  modp3072 is recent I think in
> CloudStack so if you run a older version maybe it's not there?
> 
> 
> 
> On Tue, Nov 21, 2017 at 6:55 PM, Nux! >
> wrote:
> 
> Thanks Chiradeep,
> 
> Checked but brain says no. What should I have learned from there?
> 
> AFAIK this is a terraform fail.
> 
> Lucian
> 
> --
> Sent from the Delta quadrant using Borg technology!
> 
> Nux!
> www.nux.ro
> 
> - Original Message -
> From: "Chiradeep Vittal" 
> To: "dev" 
> Sent: Tuesday, 21 November, 2017 19:14:16
> Subject: Re: Fail with vpn customer gateway creation through terraform
> 
> Check
> https://github.com/apache/cloudstack/blob/77864992fe8f80dbabd1240f6373d2
> ba3e98713c/utils/src/main/java/com/cloud/utils/net/NetUtils.java#L1221
> 
> On Tue, Nov 21, 2017 at 10:11 AM, Nux!  wrote:
> 
> Hi,
> 
> I'm trying out terraform and had success so far, except for the vpn
> customer gateway feature.
> For some reason, terraform fails to create it, though I use the same
> options as in UI/cloudmonkey where it works just fine.
> 
> The snippet for it is:
> 
> resource "cloudstack_vpn_customer_gateway" "default" {
> name   = "test-vpc"
> cidr   = "10.0.0.0/24"
> esp_policy = "aes256-sha1"
> gateway= "1.2.3.4"
> ike_policy = "sha1-aes256;modp3072"
> ipsec_psk  = "terraformxyz7"
> }
> 
> It always complains about the ike_policy:
> * cloudstack_vpn_customer_gateway.default: Error creating VPN Customer
> Gateway test-vpc: Undefined error: {"errorcode":431,"errortext":"The
> customer gateway IKE policy sha1-aes256;modp3072 is invalid!  Verify the
> required Diffie Hellman (DH) group is specified."}
> 
> I tried all sorts of ways to write the ike_policy, escaped, web
> encoded/decoded, nothing worked. What am I missing?
> The example terraform docs provide suffers the same fate.
> 
> Lucian
> 
> --
> Sent from the Delta quadrant using Borg technology!
> 
> Nux!
> www.nux.ro
> 
> 
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is the
> property of Accelerite, a Persistent Systems business. It is intended only for
> the use of the individual or entity to which it is addressed. If you are not
> the intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Accelerite, a Persistent Systems business does not accept any liability for
> virus infected mails.