[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944578#comment-15944578
 ] 

ASF subversion and git services commented on CLOUDSTACK-9827:
-

Commit 525c45c1e5f2fbf46e9379e05754ad15f2ff811d in cloudstack's branch 
refs/heads/master from [~rajanik]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=525c45c ]

Merge pull request #1994 from nvazquez/CLOUDSTACK-9827

CLOUDSTACK-9827: Storage tags stored in multiple placesIssue description: 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827

### Fixes
- Create Primary Storage: Persist tags into `storage_pool_tags` instead of 
`storage_pool_details`
- List Storage Tags: Queries `storage_pool_tags` table instead of 
`storage_tag_view`
- Find Storage Pools by Tags using proper DAO
- Remove storage tags after deleting Primary Storage
- Remove unused `StorageTagDao`, `StorageTagDaoImpl`, `StorageTagVO` and 
`storage_tag_view`

* pr/1994:
  CLOUDSTACK-9827: Storage tags stored in multiple places

Signed-off-by: Rajani Karuturi 


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944574#comment-15944574
 ] 

ASF subversion and git services commented on CLOUDSTACK-9827:
-

Commit 525c45c1e5f2fbf46e9379e05754ad15f2ff811d in cloudstack's branch 
refs/heads/master from [~rajanik]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=525c45c ]

Merge pull request #1994 from nvazquez/CLOUDSTACK-9827

CLOUDSTACK-9827: Storage tags stored in multiple placesIssue description: 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827

### Fixes
- Create Primary Storage: Persist tags into `storage_pool_tags` instead of 
`storage_pool_details`
- List Storage Tags: Queries `storage_pool_tags` table instead of 
`storage_tag_view`
- Find Storage Pools by Tags using proper DAO
- Remove storage tags after deleting Primary Storage
- Remove unused `StorageTagDao`, `StorageTagDaoImpl`, `StorageTagVO` and 
`storage_tag_view`

* pr/1994:
  CLOUDSTACK-9827: Storage tags stored in multiple places

Signed-off-by: Rajani Karuturi 


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944571#comment-15944571
 ] 

ASF subversion and git services commented on CLOUDSTACK-9827:
-

Commit 525c45c1e5f2fbf46e9379e05754ad15f2ff811d in cloudstack's branch 
refs/heads/master from [~rajanik]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=525c45c ]

Merge pull request #1994 from nvazquez/CLOUDSTACK-9827

CLOUDSTACK-9827: Storage tags stored in multiple placesIssue description: 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827

### Fixes
- Create Primary Storage: Persist tags into `storage_pool_tags` instead of 
`storage_pool_details`
- List Storage Tags: Queries `storage_pool_tags` table instead of 
`storage_tag_view`
- Find Storage Pools by Tags using proper DAO
- Remove storage tags after deleting Primary Storage
- Remove unused `StorageTagDao`, `StorageTagDaoImpl`, `StorageTagVO` and 
`storage_tag_view`

* pr/1994:
  CLOUDSTACK-9827: Storage tags stored in multiple places

Signed-off-by: Rajani Karuturi 


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944569#comment-15944569
 ] 

ASF subversion and git services commented on CLOUDSTACK-9827:
-

Commit 525c45c1e5f2fbf46e9379e05754ad15f2ff811d in cloudstack's branch 
refs/heads/master from [~rajanik]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=525c45c ]

Merge pull request #1994 from nvazquez/CLOUDSTACK-9827

CLOUDSTACK-9827: Storage tags stored in multiple placesIssue description: 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827

### Fixes
- Create Primary Storage: Persist tags into `storage_pool_tags` instead of 
`storage_pool_details`
- List Storage Tags: Queries `storage_pool_tags` table instead of 
`storage_tag_view`
- Find Storage Pools by Tags using proper DAO
- Remove storage tags after deleting Primary Storage
- Remove unused `StorageTagDao`, `StorageTagDaoImpl`, `StorageTagVO` and 
`storage_tag_view`

* pr/1994:
  CLOUDSTACK-9827: Storage tags stored in multiple places

Signed-off-by: Rajani Karuturi 


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944567#comment-15944567
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/1994


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944566#comment-15944566
 ] 

ASF subversion and git services commented on CLOUDSTACK-9827:
-

Commit edf0e2b26fefdca392d325879ebb5eb0afaaf75a in cloudstack's branch 
refs/heads/master from [~nicolas.vazquez]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=edf0e2b ]

CLOUDSTACK-9827: Storage tags stored in multiple places


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944562#comment-15944562
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user karuturi commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Thank you all. I will merge #1961 and #1994 now


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940905#comment-15940905
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@karuturi Travis is now failing as it doesn't find key "nfs2"

2017-03-24 17:33:45,621 - CRITICAL - EXCEPTION: 
test_03_migration_options_storage_tags: ['Traceback (most recent call 
last):\n', '  File "/opt/python/2.7.12/lib/python2.7/unittest/case.py", line 
329, in run\ntestMethod()\n', '  File 
"/home/travis/.local/lib/python2.7/site-packages/marvin/lib/decoratorGenerators.py",
 line 30, in test_wrapper\nreturn test(self, *args, **kwargs)\n', '  File 
"/home/travis/build/apache/cloudstack/test/integration/smoke/test_primary_storage.py",
 line 520, in test_03_migration_options_storage_tags\n
self.services["nfs2"],\n', "KeyError: 'nfs2'\n"]

It is introduced in PR #1961, can these two PRs be merged together?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940741#comment-15940741
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@karuturi I refactored last marvin test which was failing on Travis. These 
are results in our env:


[root@ussarlabcsmgt41 cloudstack]# cat /tmp//MarvinLogs//4GSNSY/results.txt
Test primary storage pools - XEN. Not Supported for kvm,hyperv,vmware ... 
SKIP: iscsi primary storage not supported on kvm, VMWare, Hyper-V, or LXC
Test primary storage pools - XEN, KVM, VMWare. Not Supported for hyperv ... 
=== TestName: test_01_primary_storage_nfs | Status : SUCCESS ===
ok
Test Deploy VMS using different Service Offerings with Storage Tags ... === 
TestName: test_01_deploy_vms_storage_tags | Status : SUCCESS ===
ok
Test Edit Storage Tags ... === TestName: test_02_edit_primary_storage_tags 
| Status : SUCCESS ===
ok
Test Volume migration options for Storage Pools with different Storage Tags 
... === TestName: test_03_migration_options_storage_tags | Status : SUCCESS ===
ok

--
Ran 5 tests in 442.216s

OK (SKIP=1)


@rhtyd @borisstoyanov @jburwell on `test_primary_storage.py` we are 
creating 2 primary storages, using urls in `"nfs"` and `"nfs2"` test data 
templates. As PR #1961 uses `"nfs2"` template on `test_snapshots.py`, should it 
be a problem for BlueOrangutan?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938215#comment-15938215
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Thanks @mike-tutkowski! I pushed force to kick off Travis again


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937596#comment-15937596
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user mike-tutkowski commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
OK, this LGTM (as long as Travis shows green before we merge). Thanks!


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937132#comment-15937132
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user mike-tutkowski commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
I should have a chance to look through this later tonight. In the 
meanwhile, perhaps we can re-push the code to kick off Travis again (since its 
most recent run has a failure). Thanks!


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15934895#comment-15934895
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Thanks @serg38, we are using `mgtSvr` details provided in .cfg file, should 
we use these for Marvin too?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933546#comment-15933546
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@nvazquez Looks like Travis cant SSH into management server during the test 
.

2017-03-17 19:26:58,672 - DEBUG - mount -t nfs 
nfs:/export/automation/1/testprimary 
/mnt/marvin-1b4190cb-b9f5-4441-9bdc-bd89e9b7567b
2017-03-17 19:27:13,468 - CRITICAL - EXCEPTION: 
test_03_migration_options_storage_tags: ['Traceback (most recent call 
last):\n', '  File "/opt/python/2.7.12/lib/python2.7/unittest/case.py", line 
329, in run\ntestMethod()\n', '  File 
"/home/travis/.local/lib/python2.7/site-packages/marvin/lib/decoratorGenerators.py",
 line 30, in test_wrapper\nreturn test(self, *args, **kwargs)\n', '  File 
"/home/travis/build/apache/cloudstack/test/integration/smoke/test_primary_storage.py",
 line 559, in test_03_migration_options_storage_tags\n
log_lvl=logging.INFO\n', '  File 
"/home/travis/.local/lib/python2.7/site-packages/marvin/sshClient.py", line 81, 
in __init__\nraise internalError("SSH Connection Failed")\n', 
'internalError: SSH Connection Failed\n']


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931823#comment-15931823
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user karuturi commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Thanks @nvazquez Waiting for LGTMs


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930527#comment-15930527
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@karuturi I added marvin tests to simulate tests performed by 
@mike-tutkowski.

This are results in our env:

[root@ussarlabcsmgt41 cloudstack]# cat /tmp//MarvinLogs//011CTF/results.txt
Test primary storage pools - XEN. Not Supported for kvm,hyperv,vmware ... 
SKIP: iscsi primary storage not supported on kvm, VMWare, Hyper-V, or LXC
Test primary storage pools - XEN, KVM, VMWare. Not Supported for hyperv ... 
=== TestName: test_01_primary_storage_nfs | Status : SUCCESS ===
ok
Test Deploy VMS using different Service Offerings with Storage Tags ... === 
TestName: test_01_deploy_vms_storage_tags | Status : SUCCESS ===
ok
Test edit Storage Tags ... === TestName: test_02_edit_primary_storage_tags 
| Status : SUCCESS ===
ok
Test Volume migration options for Storage Pools with different Storage Tags 
... SKIP: Skipping test as it is not running on simulator

--
Ran 5 tests in 295.996s

OK (SKIP=2)


@rafaelweingartner @mike-tutkowski @karuturi can you please review marvin 
tests added?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930508#comment-15930508
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r106724343
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +92,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
--- End diff --

Done! Thanks a lot @rafaelweingartner!


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929926#comment-15929926
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Hi @karuturi, I've been working on marvin tests, I hope posting them today


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929458#comment-15929458
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user karuturi commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@nvazquez any update? 


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927234#comment-15927234
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r106313430
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +92,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
--- End diff --

No problem ;)

It is awesome the changes you have done here. The code became cleaner, with 
good documentation and on top of that, we now have unit tests to guarantee that 
the methods are workings as expected.

Great job @nvazquez, well done!

PS: Sorry if I have been too picky with the code changes.



> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927220#comment-15927220
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r106311985
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +92,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
--- End diff --

Nice, I agree with you that deleting the line can be removed. Sorry for the 
confusion, I'll try adding some unit tests for this change. Thanks!


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927151#comment-15927151
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r106306374
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +92,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
--- End diff --

Ah, no this is not what I meant.
I meant only removing the `if` and letting the `while` directly there.
Just deleting line 103


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927135#comment-15927135
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r106303439
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +92,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
--- End diff --

Hi @rafaelweingartner, I assumed that you meant deleting the whole `if` 
block (lines 103-108), is it correct or you mean just deleting line 103 (and 
108)?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15926792#comment-15926792
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r106001103
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +92,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
--- End diff --

I mistyped the lines in my last sentence, it should have been 110-112.
Will it be loaded with a bigger batch size? From my reading, even without 
the `if ` at line 103, if `stIds.length` is smaller than the batch size, the 
`while` is not executed (not even once). Then, the flow proceeds the same as if 
the `if` had been executed. Then, at line 111 the `batch_size` is calculated 
using `stIds.length - curr_index`.



> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924827#comment-15924827
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r106002237
  
--- Diff: 
engine/schema/test/org/apache/cloudstack/storage/datastore/db/PrimaryDataStoreDaoImplTest.java
 ---
@@ -0,0 +1,151 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.storage.datastore.db;
+
+import static org.mockito.Mockito.doReturn;
+import static org.mockito.Mockito.verify;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import 
org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDaoImpl.ValueType;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Matchers;
+import org.mockito.Mock;
+import org.mockito.Spy;
+import org.powermock.modules.junit4.PowerMockRunner;
+
+import com.cloud.storage.ScopeType;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.StoragePoolTagsDao;
+
+import junit.framework.TestCase;
+
+@RunWith(PowerMockRunner.class)
+public class PrimaryDataStoreDaoImplTest extends TestCase {
+
+@Mock
+StoragePoolDetailsDao _detailsDao;
+@Mock
+StoragePoolHostDao _hostDao;
+@Mock
+StoragePoolTagsDao _tagsDao;
+
+@Spy
+@InjectMocks
+private static PrimaryDataStoreDaoImpl primaryDataStoreDao = new 
PrimaryDataStoreDaoImpl();
+
+@Mock
+static StoragePoolVO storagePoolVO;
--- End diff --

Thanks! I removed it


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924803#comment-15924803
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105998903
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +92,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
--- End diff --

I think that `if` is needed to control the max query size. I agree with the 
example you provided, but let's say for example that lengthOfStIds is greater 
that batchSize. If we remove that `if`, we will load pools on lines 100-112 but 
with a query size greater that batchSize (defined in line 111)


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924762#comment-15924762
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105993059
  
--- Diff: 
engine/schema/test/org/apache/cloudstack/storage/datastore/db/PrimaryDataStoreDaoImplTest.java
 ---
@@ -0,0 +1,151 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package org.apache.cloudstack.storage.datastore.db;
+
+import static org.mockito.Mockito.doReturn;
+import static org.mockito.Mockito.verify;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import 
org.apache.cloudstack.storage.datastore.db.PrimaryDataStoreDaoImpl.ValueType;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Matchers;
+import org.mockito.Mock;
+import org.mockito.Spy;
+import org.powermock.modules.junit4.PowerMockRunner;
+
+import com.cloud.storage.ScopeType;
+import com.cloud.storage.dao.StoragePoolHostDao;
+import com.cloud.storage.dao.StoragePoolTagsDao;
+
+import junit.framework.TestCase;
+
+@RunWith(PowerMockRunner.class)
+public class PrimaryDataStoreDaoImplTest extends TestCase {
+
+@Mock
+StoragePoolDetailsDao _detailsDao;
+@Mock
+StoragePoolHostDao _hostDao;
+@Mock
+StoragePoolTagsDao _tagsDao;
+
+@Spy
+@InjectMocks
+private static PrimaryDataStoreDaoImpl primaryDataStoreDao = new 
PrimaryDataStoreDaoImpl();
+
+@Mock
+static StoragePoolVO storagePoolVO;
--- End diff --

Here you have other static variable in a test case that does not need to be 
static.


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924758#comment-15924758
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105992671
  
--- Diff: 
engine/schema/test/com/cloud/storage/dao/StoragePoolTagsDaoImplTest.java ---
@@ -0,0 +1,105 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.storage.dao;
+
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Matchers;
+import org.mockito.Mock;
+import org.mockito.Spy;
+import org.powermock.modules.junit4.PowerMockRunner;
+
+import com.cloud.storage.StoragePoolTagVO;
+import com.cloud.utils.db.Filter;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+
+import static org.mockito.Mockito.when;
+import static org.mockito.Mockito.doReturn;
+import static org.mockito.Mockito.verify;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import junit.framework.TestCase;
+
+@RunWith(PowerMockRunner.class)
+public class StoragePoolTagsDaoImplTest extends TestCase {
+
+@Mock
+ConfigurationDao _configDao;
+@Mock
+SearchBuilder StoragePoolIdsSearch;
+
+@Spy
+@InjectMocks
+private StoragePoolTagsDaoImpl _storagePoolTagsDaoImpl = new 
StoragePoolTagsDaoImpl();
+
+@Mock
+static StoragePoolTagVO storagePoolTag1;
--- End diff --

Not really, thinking about it there's no why to make them static, I removed 
static declaration from them


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924759#comment-15924759
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105992559
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +92,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
--- End diff --

Correct me if I am wrong, but this `if` is not needed.
Let's assume the configuration:
```
batchSize=2000
current_index=0
lengthOfStIds=100
```

`curr_index + detailsBatchSize = 0 + 2000`, which is not less than the size 
of the array `(100)`. Therefore, the while is not executed. Then, the pools 
will be loaded at lines 100-112



> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924748#comment-15924748
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105990460
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,68 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
+while ((curr_index + detailsBatchSize) <= stIds.length) {
+searchForStoragePoolIdsInternal(curr_index, 
detailsBatchSize, stIds, uvList);
+curr_index += detailsBatchSize;
+}
+}
+
+if (curr_index < stIds.length) {
+int batch_size = (stIds.length - curr_index);
+searchForStoragePoolIdsInternal(curr_index, batch_size, stIds, 
uvList);
+}
+
+return uvList;
+}
+
+/**
+ * Search storage pools
+ * @param currIndex current index
+ * @param batchSize batch size
+ * @param stIds storage tags array
+ * @param uvList
+ */
+protected void searchForStoragePoolIdsInternal(int currIndex, int 
batchSize, Long[] stIds, List uvList) {
+Long[] ids = new Long[batchSize];
+for (int k = 0, j = currIndex; j < currIndex + batchSize; j++, 
k++) {
+ids[k] = stIds[j];
+}
+SearchCriteria sc = 
StoragePoolIdsSearch.create();
+sc.setParameters("idIN", (Object[])ids);
+List vms = searchIncludingRemoved(sc, null, 
null, false);
+if (vms != null) {
+uvList.addAll(vms);
+}
+}
+
+/**
+ * Retrieve {@code detail.batch.query.size} configuration value. If 
not available, return the default value (2000)
+ * @return detail.batch.query.size configuration value
+ */
+protected int getDetailsBatchSize() {
+String batchCfg = _configDao.getValue("detail.batch.query.size");
+return batchCfg != null ? Integer.parseInt(batchCfg) : 2000;
--- End diff --

Done, thanks!


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924741#comment-15924741
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105989477
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,68 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
+while ((curr_index + detailsBatchSize) <= stIds.length) {
+searchForStoragePoolIdsInternal(curr_index, 
detailsBatchSize, stIds, uvList);
+curr_index += detailsBatchSize;
+}
+}
+
+if (curr_index < stIds.length) {
+int batch_size = (stIds.length - curr_index);
+searchForStoragePoolIdsInternal(curr_index, batch_size, stIds, 
uvList);
+}
+
+return uvList;
+}
+
+/**
+ * Search storage pools
+ * @param currIndex current index
+ * @param batchSize batch size
+ * @param stIds storage tags array
+ * @param uvList
+ */
+protected void searchForStoragePoolIdsInternal(int currIndex, int 
batchSize, Long[] stIds, List uvList) {
--- End diff --

Excellent improvement, I copied almost exactly as you wrote it :) I also 
added some description on `uvList` parameter, which also renamed to `pools`


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924529#comment-15924529
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105959436
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,68 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
+while ((curr_index + detailsBatchSize) <= stIds.length) {
+searchForStoragePoolIdsInternal(curr_index, 
detailsBatchSize, stIds, uvList);
+curr_index += detailsBatchSize;
+}
+}
+
+if (curr_index < stIds.length) {
+int batch_size = (stIds.length - curr_index);
+searchForStoragePoolIdsInternal(curr_index, batch_size, stIds, 
uvList);
+}
+
+return uvList;
+}
+
+/**
+ * Search storage pools
+ * @param currIndex current index
+ * @param batchSize batch size
+ * @param stIds storage tags array
+ * @param uvList
+ */
+protected void searchForStoragePoolIdsInternal(int currIndex, int 
batchSize, Long[] stIds, List uvList) {
--- End diff --

What about improving this java doc here?
Something like:

>  Search for storage pools based on their IDs. The search is executed in 
batch, this means that we will load a batch of size #getDetailsBatchSize  
@link{StoragePoolTagVO} at each time. The loaded storage pools are added in the 
uvList parameter.

The links to classes, methods and parameters in the Java doc would have to 
be fixed (I wrote it without using the IDE and I did not know the proper markup)

What do you think about the "@param" notation? I normally prefer not to use 
them if the onyl thing I will write is the parameter name such as `@param 
uvList`



> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924531#comment-15924531
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105959564
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,68 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+final int detailsBatchSize = getDetailsBatchSize();
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
+while ((curr_index + detailsBatchSize) <= stIds.length) {
+searchForStoragePoolIdsInternal(curr_index, 
detailsBatchSize, stIds, uvList);
+curr_index += detailsBatchSize;
+}
+}
+
+if (curr_index < stIds.length) {
+int batch_size = (stIds.length - curr_index);
+searchForStoragePoolIdsInternal(curr_index, batch_size, stIds, 
uvList);
+}
+
+return uvList;
+}
+
+/**
+ * Search storage pools
+ * @param currIndex current index
+ * @param batchSize batch size
+ * @param stIds storage tags array
+ * @param uvList
+ */
+protected void searchForStoragePoolIdsInternal(int currIndex, int 
batchSize, Long[] stIds, List uvList) {
+Long[] ids = new Long[batchSize];
+for (int k = 0, j = currIndex; j < currIndex + batchSize; j++, 
k++) {
+ids[k] = stIds[j];
+}
+SearchCriteria sc = 
StoragePoolIdsSearch.create();
+sc.setParameters("idIN", (Object[])ids);
+List vms = searchIncludingRemoved(sc, null, 
null, false);
+if (vms != null) {
+uvList.addAll(vms);
+}
+}
+
+/**
+ * Retrieve {@code detail.batch.query.size} configuration value. If 
not available, return the default value (2000)
+ * @return detail.batch.query.size configuration value
+ */
+protected int getDetailsBatchSize() {
+String batchCfg = _configDao.getValue("detail.batch.query.size");
+return batchCfg != null ? Integer.parseInt(batchCfg) : 2000;
--- End diff --

What about extracting the `2000` to a constant in the class?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924530#comment-15924530
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105959827
  
--- Diff: 
engine/schema/test/com/cloud/storage/dao/StoragePoolTagsDaoImplTest.java ---
@@ -0,0 +1,105 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+package com.cloud.storage.dao;
+
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Matchers;
+import org.mockito.Mock;
+import org.mockito.Spy;
+import org.powermock.modules.junit4.PowerMockRunner;
+
+import com.cloud.storage.StoragePoolTagVO;
+import com.cloud.utils.db.Filter;
+import com.cloud.utils.db.SearchBuilder;
+import com.cloud.utils.db.SearchCriteria;
+
+import static org.mockito.Mockito.when;
+import static org.mockito.Mockito.doReturn;
+import static org.mockito.Mockito.verify;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import junit.framework.TestCase;
+
+@RunWith(PowerMockRunner.class)
+public class StoragePoolTagsDaoImplTest extends TestCase {
+
+@Mock
+ConfigurationDao _configDao;
+@Mock
+SearchBuilder StoragePoolIdsSearch;
+
+@Spy
+@InjectMocks
+private StoragePoolTagsDaoImpl _storagePoolTagsDaoImpl = new 
StoragePoolTagsDaoImpl();
+
+@Mock
+static StoragePoolTagVO storagePoolTag1;
--- End diff --

I am curious here, is there any reason to declare the variables here 
`static`?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923179#comment-15923179
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105797159
  
--- Diff: 
engine/schema/src/org/apache/cloudstack/storage/datastore/db/PrimaryDataStoreDaoImpl.java
 ---
@@ -409,15 +460,13 @@ public StoragePoolVO persist(StoragePoolVO pool, 
Map details) {
 sc.and(sc.entity().getScope(), Op.EQ, ScopeType.ZONE);
 return sc.list();
 } else {
-Map details = tagsToDetails(tags);
-
-StringBuilder sql = new 
StringBuilder(ZoneWideDetailsSqlPrefix);
+StringBuilder sql = new StringBuilder(ZoneWideTagsSqlPrefix);
--- End diff --

Thanks for pointing this out, I had missed it out. Created methods 
`getSqlPreparedStatement` and `searchStoragePoolsPreparedStatement` which are 
called from many methods and allow storage pool retrieval


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923174#comment-15923174
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105796872
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+String batchCfg = _configDao.getValue("detail.batch.query.size");
+
+final int detailsBatchSize = batchCfg != null ? 
Integer.parseInt(batchCfg) : 2000;
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
+while ((curr_index + detailsBatchSize) <= stIds.length) {
+Long[] ids = new Long[detailsBatchSize];
+
+for (int k = 0, j = curr_index; j < curr_index + 
detailsBatchSize; j++, k++) {
+ids[k] = stIds[j];
+}
+
+SearchCriteria sc = 
StoragePoolIdsSearch.create();
--- End diff --

Done, created method `searchForStoragePoolIdsInternal`


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923173#comment-15923173
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105796786
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+String batchCfg = _configDao.getValue("detail.batch.query.size");
--- End diff --

About number 2000, I assumed it was a default value for that configuration


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923167#comment-15923167
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105796076
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+String batchCfg = _configDao.getValue("detail.batch.query.size");
--- End diff --

Done, thanks!


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923166#comment-15923166
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@rafaelweingartner I pushed changes and squashed my commits as it could be 
easier to review. I also added unit tests for new methods


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907426#comment-15907426
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@mike-tutkowski awesome, thanks for testing this PR!

@rafaelweingartner thanks for reviewing, I'll work on changes proposed

@karuturi sure, I'll work on it, thanks!


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906844#comment-15906844
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user karuturi commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@nvazquez I think any of the existing marvin tests didnt catch this bug. Is 
it possible to add a marvin test for it?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906078#comment-15906078
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user mike-tutkowski commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
I have run the following use cases successfully:

PS = Primary Storage
ST = Storage Tag
CO = Compute Offering
DO = Disk Offering
V = Volume

Create PS_1 with ST NFS-A

Create CO_1 with ST NFS-A
Create CO_2 with ST NFS-B

Create VM_1 with CO_1 (VM starts)
Create VM_2 with CO_2 (VM fails to find storage)

Edit ST of PS_1 from NFS-A to NFS-B

Create VM_2 with CO_1 (VM fails to find storage)
Create VM_2 with CO_2 (VM starts)

Create DO_1 with ST NFS-A
Create DO_2 with ST NFS-B

Create V_1 with DO_1 and attach to VM_1 (fails to find storage)
Create V_2 with DO_2 and attach to VM_1

Edit ST of PS_1 from NFS-B to NFS-A

Create V_1 with DO_1 and attach to VM_1

Create PS_2 with no ST

Edit ST of PS_2 to NFS-B

Create V_3 with DO_1 and attach to VM_2 (lands on expected PS)
Create V_4 with DO_2 and attach to VM_2 (lands on expected PS)

Create PS_3 with ST NFS-B

Migrate V_4 from PS_2 to PS_3 (PS_1 is not an option)
Migrate V_4 from PS_3 to PS_2 (PS_1 is not an option)

Migrate V_3 from PS_1 to PS_2 (Neither PS_2 nor PS_3 is an option)

Migrate root V of VM_1 from PS_1 to PS_2 (Neither PS_2 nor PS_3 is an 
option)

Migrate root V of VM_2 from PS_1 to PS_3 (PS_2 and PS_3 are options)

listStorageTags API returned expected results (I ran this at several points 
along the way)


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905700#comment-15905700
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105488722
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+String batchCfg = _configDao.getValue("detail.batch.query.size");
+
+final int detailsBatchSize = batchCfg != null ? 
Integer.parseInt(batchCfg) : 2000;
+
+// query details by batches
+List uvList = new ArrayList();
+int curr_index = 0;
+
+if (stIds.length > detailsBatchSize) {
+while ((curr_index + detailsBatchSize) <= stIds.length) {
+Long[] ids = new Long[detailsBatchSize];
+
+for (int k = 0, j = curr_index; j < curr_index + 
detailsBatchSize; j++, k++) {
+ids[k] = stIds[j];
+}
+
+SearchCriteria sc = 
StoragePoolIdsSearch.create();
--- End diff --

@nvazquez lines 105-119 are the same as 128-142.
What about extracting them for a method?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905699#comment-15905699
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105488183
  
--- Diff: 
engine/schema/src/com/cloud/storage/dao/StoragePoolTagsDaoImpl.java ---
@@ -77,4 +90,71 @@ public void deleteTags(long poolId) {
 txn.commit();
 }
 
+@Override
+public List searchByIds(Long... stIds) {
+String batchCfg = _configDao.getValue("detail.batch.query.size");
--- End diff --

@nvazquez what about extracting lines 95-97 to a method?
Then, we can have test cases and some doc for it. Especially for that 2000 
magic number there.



> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905701#comment-15905701
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/1994#discussion_r105489528
  
--- Diff: 
engine/schema/src/org/apache/cloudstack/storage/datastore/db/PrimaryDataStoreDaoImpl.java
 ---
@@ -409,15 +460,13 @@ public StoragePoolVO persist(StoragePoolVO pool, 
Map details) {
 sc.and(sc.entity().getScope(), Op.EQ, ScopeType.ZONE);
 return sc.list();
 } else {
-Map details = tagsToDetails(tags);
-
-StringBuilder sql = new 
StringBuilder(ZoneWideDetailsSqlPrefix);
+StringBuilder sql = new StringBuilder(ZoneWideTagsSqlPrefix);
--- End diff --

@nvazquez what about extracting lines 463-469 to a method? Then, we can 
create doc and most of all test cases. This is a pretty tricky bit of code 
(Especially line 468).


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905536#comment-15905536
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user mike-tutkowski commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
I should be able to do so over the weekend.

On Mar 10, 2017, at 11:26 AM, serg38 
> wrote:


@mike-tutkowski Can you test this fix in 
your environment?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on 
GitHub, 
or mute the 
thread.



> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905511#comment-15905511
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@mike-tutkowski Can you test this fix in your environment?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905328#comment-15905328
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Trillian test result (tid-946)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 28653 seconds
Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr1994-t946-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
Test completed. 47 look ok, 2 have error(s)


Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_04_rvpc_privategw_static_routes | `Failure` | 350.78 | 
test_privategw_acl.py
test_02_list_snapshots_with_removed_data_store | `Error` | 0.04 | 
test_snapshots.py
test_01_vpc_site2site_vpn | Success | 155.07 | test_vpc_vpn.py
test_01_vpc_remote_access_vpn | Success | 61.14 | test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn | Success | 251.03 | test_vpc_vpn.py
test_02_VPC_default_routes | Success | 290.08 | test_vpc_router_nics.py
test_01_VPC_nics_after_destroy | Success | 525.27 | test_vpc_router_nics.py
test_05_rvpc_multi_tiers | Success | 527.95 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | Success | 1442.83 | 
test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | 
Success | 569.18 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | Success | 760.64 | 
test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Success | 1312.03 | 
test_vpc_redundant.py
test_09_delete_detached_volume | Success | 156.64 | test_volumes.py
test_08_resize_volume | Success | 156.42 | test_volumes.py
test_07_resize_fail | Success | 156.51 | test_volumes.py
test_06_download_detached_volume | Success | 156.40 | test_volumes.py
test_05_detach_volume | Success | 155.84 | test_volumes.py
test_04_delete_attached_volume | Success | 151.29 | test_volumes.py
test_03_download_attached_volume | Success | 151.28 | test_volumes.py
test_02_attach_volume | Success | 94.89 | test_volumes.py
test_01_create_volume | Success | 711.29 | test_volumes.py
test_03_delete_vm_snapshots | Success | 275.21 | test_vm_snapshots.py
test_02_revert_vm_snapshots | Success | 100.74 | test_vm_snapshots.py
test_01_create_vm_snapshots | Success | 163.86 | test_vm_snapshots.py
test_deploy_vm_multiple | Success | 262.94 | test_vm_life_cycle.py
test_deploy_vm | Success | 0.03 | test_vm_life_cycle.py
test_advZoneVirtualRouter | Success | 0.03 | test_vm_life_cycle.py
test_10_attachAndDetach_iso | Success | 26.70 | test_vm_life_cycle.py
test_09_expunge_vm | Success | 125.18 | test_vm_life_cycle.py
test_08_migrate_vm | Success | 30.98 | test_vm_life_cycle.py
test_07_restore_vm | Success | 0.13 | test_vm_life_cycle.py
test_06_destroy_vm | Success | 125.83 | test_vm_life_cycle.py
test_03_reboot_vm | Success | 125.88 | test_vm_life_cycle.py
test_02_start_vm | Success | 10.17 | test_vm_life_cycle.py
test_01_stop_vm | Success | 40.33 | test_vm_life_cycle.py
test_CreateTemplateWithDuplicateName | Success | 40.50 | test_templates.py
test_08_list_system_templates | Success | 0.03 | test_templates.py
test_07_list_public_templates | Success | 0.04 | test_templates.py
test_05_template_permissions | Success | 0.14 | test_templates.py
test_04_extract_template | Success | 5.16 | test_templates.py
test_03_delete_template | Success | 5.11 | test_templates.py
test_02_edit_template | Success | 90.17 | test_templates.py
test_01_create_template | Success | 40.43 | test_templates.py
test_10_destroy_cpvm | Success | 161.69 | test_ssvm.py
test_09_destroy_ssvm | Success | 168.76 | test_ssvm.py
test_08_reboot_cpvm | Success | 131.64 | test_ssvm.py
test_07_reboot_ssvm | Success | 103.62 | test_ssvm.py
test_06_stop_cpvm | Success | 131.88 | test_ssvm.py
test_05_stop_ssvm | Success | 163.87 | test_ssvm.py
test_04_cpvm_internals | Success | 1.24 | test_ssvm.py
test_03_ssvm_internals | Success | 3.48 | test_ssvm.py
test_02_list_cpvm_vm | Success | 0.15 | test_ssvm.py
test_01_list_sec_storage_vm | Success | 0.14 | test_ssvm.py
test_01_snapshot_root_disk | Success | 11.18 | test_snapshots.py
test_04_change_offering_small | Success | 239.97 | test_service_offerings.py
test_03_delete_service_offering | Success | 0.04 | test_service_offerings.py
test_02_edit_service_offering | Success | 0.06 | test_service_offerings.py
test_01_create_service_offering | Success | 0.11 | test_service_offerings.py

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904674#comment-15904674
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has 
been kicked to run smoke tests


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904672#comment-15904672
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user borisstoyanov commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@blueorangutan test


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904670#comment-15904670
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Packaging result: ✔centos6 ✔centos7 ✔debian. JID-585


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904630#comment-15904630
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep 
you posted as I make progress.


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904629#comment-15904629
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user borisstoyanov commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@blueorangutan package


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904238#comment-15904238
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@rhtyd @borisstoyanov Can you kick off new tests for this release blocker?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904209#comment-15904209
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@serg38 actually is not being used anymore, I'll add removalof the view on 
last commit


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904201#comment-15904201
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@nvazquez Is storage_tag_view still in use after this change that reference 
old way of retrieving the tags? If not should we remove it?  E.g. host tags 
don't have a separate view.


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904184#comment-15904184
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user nvazquez commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Hi @rafaelweingartner, you're right, it was basically that fix. 
I've pushed another commit due to issue reported by @mike-tutkowski in 
mailing list:

I have an NFS SR as primary storage under CloudStack with a storage tag of 
NFS-A. I have a compute offering with a matching storage tag. I can’t seem to 
get a VM to spin up, however: It says insufficient capacity. The CPU, MHz, and 
memory are all low (and what I typically use), so I think the problem is with 
matching the storage tag.


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903853#comment-15903853
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user rafaelweingartner commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@nvazquez I am assuming the change needed to fix the bug is line 135 at 
PrimaryDataStoreHelper.java?
The other changes are regarding code conformity



> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903391#comment-15903391
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Trillian test result (tid-940)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 28984 seconds
Marvin logs: 
https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr1994-t940-kvm-centos7.zip
Intermitten failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermitten failure detected: /marvin/tests/smoke/test_snapshots.py
Test completed. 47 look ok, 2 have error(s)


Test | Result | Time (s) | Test File
--- | --- | --- | ---
test_04_rvpc_privategw_static_routes | `Failure` | 366.22 | 
test_privategw_acl.py
test_02_list_snapshots_with_removed_data_store | `Error` | 0.04 | 
test_snapshots.py
test_01_vpc_site2site_vpn | Success | 161.27 | test_vpc_vpn.py
test_01_vpc_remote_access_vpn | Success | 71.26 | test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn | Success | 266.14 | test_vpc_vpn.py
test_02_VPC_default_routes | Success | 272.29 | test_vpc_router_nics.py
test_01_VPC_nics_after_destroy | Success | 573.38 | test_vpc_router_nics.py
test_05_rvpc_multi_tiers | Success | 524.87 | test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics | Success | 1296.05 | 
test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | 
Success | 558.76 | test_vpc_redundant.py
test_02_redundant_VPC_default_routes | Success | 761.27 | 
test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Success | 1310.90 | 
test_vpc_redundant.py
test_09_delete_detached_volume | Success | 156.60 | test_volumes.py
test_08_resize_volume | Success | 156.91 | test_volumes.py
test_07_resize_fail | Success | 161.50 | test_volumes.py
test_06_download_detached_volume | Success | 156.34 | test_volumes.py
test_05_detach_volume | Success | 155.76 | test_volumes.py
test_04_delete_attached_volume | Success | 151.23 | test_volumes.py
test_03_download_attached_volume | Success | 156.46 | test_volumes.py
test_02_attach_volume | Success | 124.66 | test_volumes.py
test_01_create_volume | Success | 713.58 | test_volumes.py
test_03_delete_vm_snapshots | Success | 275.17 | test_vm_snapshots.py
test_02_revert_vm_snapshots | Success | 95.75 | test_vm_snapshots.py
test_01_create_vm_snapshots | Success | 163.78 | test_vm_snapshots.py
test_deploy_vm_multiple | Success | 272.95 | test_vm_life_cycle.py
test_deploy_vm | Success | 0.03 | test_vm_life_cycle.py
test_advZoneVirtualRouter | Success | 0.03 | test_vm_life_cycle.py
test_10_attachAndDetach_iso | Success | 26.88 | test_vm_life_cycle.py
test_09_expunge_vm | Success | 125.25 | test_vm_life_cycle.py
test_08_migrate_vm | Success | 30.90 | test_vm_life_cycle.py
test_07_restore_vm | Success | 0.13 | test_vm_life_cycle.py
test_06_destroy_vm | Success | 130.98 | test_vm_life_cycle.py
test_03_reboot_vm | Success | 125.88 | test_vm_life_cycle.py
test_02_start_vm | Success | 10.17 | test_vm_life_cycle.py
test_01_stop_vm | Success | 40.33 | test_vm_life_cycle.py
test_CreateTemplateWithDuplicateName | Success | 50.75 | test_templates.py
test_08_list_system_templates | Success | 0.03 | test_templates.py
test_07_list_public_templates | Success | 0.04 | test_templates.py
test_05_template_permissions | Success | 0.06 | test_templates.py
test_04_extract_template | Success | 5.13 | test_templates.py
test_03_delete_template | Success | 5.11 | test_templates.py
test_02_edit_template | Success | 90.21 | test_templates.py
test_01_create_template | Success | 40.46 | test_templates.py
test_10_destroy_cpvm | Success | 166.67 | test_ssvm.py
test_09_destroy_ssvm | Success | 164.14 | test_ssvm.py
test_08_reboot_cpvm | Success | 131.56 | test_ssvm.py
test_07_reboot_ssvm | Success | 133.64 | test_ssvm.py
test_06_stop_cpvm | Success | 131.75 | test_ssvm.py
test_05_stop_ssvm | Success | 133.76 | test_ssvm.py
test_04_cpvm_internals | Success | 1.22 | test_ssvm.py
test_03_ssvm_internals | Success | 3.37 | test_ssvm.py
test_02_list_cpvm_vm | Success | 0.13 | test_ssvm.py
test_01_list_sec_storage_vm | Success | 0.17 | test_ssvm.py
test_01_snapshot_root_disk | Success | 11.11 | test_snapshots.py
test_04_change_offering_small | Success | 240.71 | test_service_offerings.py
test_03_delete_service_offering | Success | 0.04 | test_service_offerings.py
test_02_edit_service_offering | Success | 0.05 | test_service_offerings.py
test_01_create_service_offering | Success | 0.11 | test_service_offerings.py

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902704#comment-15902704
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has 
been kicked to run smoke tests


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902702#comment-15902702
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user borisstoyanov commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@blueorangutan test


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902695#comment-15902695
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
Packaging result: ✔centos6 ✔centos7 ✔debian. JID-578


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902656#comment-15902656
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user blueorangutan commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep 
you posted as I make progress.


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902655#comment-15902655
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user borisstoyanov commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@blueorangutan package


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Assignee: Nicolas Vazquez
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902038#comment-15902038
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@nvazquez Should we simply replace storage_tag_view instead of switching 
queries over to another view?


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902035#comment-15902035
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


Github user serg38 commented on the issue:

https://github.com/apache/cloudstack/pull/1994
  
@mike-tutkowski @karuturi @koushik-das @rafaelweingartner  Please review 
the fix for 4.10 blocker


> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (CLOUDSTACK-9827) Storage tags stored in multiple places

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/CLOUDSTACK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901936#comment-15901936
 ] 

ASF GitHub Bot commented on CLOUDSTACK-9827:


GitHub user nvazquez opened a pull request:

https://github.com/apache/cloudstack/pull/1994

CLOUDSTACK-9827: Storage tags stored in multiple places

Issue description: https://issues.apache.org/jira/browse/CLOUDSTACK-9827

### Fixes
- Create Primary Storage: Persist tags into `storage_pool_tags` instead of 
`storage_pool_details`
- List Storage Tags: Queries `storage_pool_tags` table instead of 
`storage_tag_view`

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nvazquez/cloudstack CLOUDSTACK-9827

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/1994.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1994


commit e106919443949f704d77cee3dde7b36100609a73
Author: nvazquez 
Date:   2017-03-08T18:19:55Z

CLOUDSTACK-9827: Storage tags stored in multiple places




> Storage tags stored in multiple places
> --
>
> Key: CLOUDSTACK-9827
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9827
> Project: CloudStack
>  Issue Type: Bug
>  Security Level: Public(Anyone can view this level - this is the 
> default.) 
>  Components: Management Server
>Affects Versions: 4.10.0.0
> Environment: N/A
>Reporter: Mike Tutkowski
>Priority: Blocker
> Fix For: 4.10.0.0
>
>
> I marked this as a Blocker because it concerns me that we are not handling 
> storage tags correctly in 4.10 and, as such, VM storage might get placed in 
> locations that users don't want.
> From e-mails I sent to dev@ (most recent to oldest):
> If I add a new primary storage and give it a storage tag, the tag ends up in 
> storage_pool_details.
> If I edit an existing storage pool’s storage tags, it places them in 
> storage_pool_tags.
> **
> I believe I have found another bug (one that we should either fix or examine 
> in detail before releasing 4.10).
> It looks like we have a new table: cloud.storage_pool_tags.
> The addition of this table seems to have broken the listStorageTags API 
> command. When this command runs, it doesn’t pick up any storage tags for me 
> (and I know I have one storage tag).
> This data used to be stored in the cloud.storage_pool_details table. It’s 
> good to put it in its own table, but will our upgrade process move the 
> existing tags from storage_pool_details to storage_pool_tags?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)