Hi Daan and Users,
I opened the issue on Github and found that this bug (#3181) was fixed on
4.13.0 release.
Il giorno lun 3 feb 2020 alle ore 10:25 Daan Hoogland <
daan.hoogl...@gmail.com> ha scritto:
> thanks Charlie, can you take your issue to a github issue on the project
> please? thanks
>
> On Mon, Feb 3, 2020 at 9:57 AM Charlie Holeowsky <
> charlie.holeow...@gmail.com> wrote:
>
>> Hi Daan and users,
>> Yes, I resolved the issue but it's only a work-around, I have found that
>> the problem is reproducible but I do not understand if I am the only one to
>> have noticed it and if there is another more elegant and definitive
>> solution.
>>
>>
>>
>> Il giorno ven 31 gen 2020 alle ore 13:31 Daan Hoogland <
>> daan.hoogl...@gmail.com> ha scritto:
>>
>>> sorry to not have any focus on this Charlie,
>>> Do I read correctly that you resolved your issue?
>>>
>>> On Tue, Jan 28, 2020 at 2:46 PM Charlie Holeowsky <
>>> charlie.holeow...@gmail.com> wrote:
>>>
>>>> In this period I have performed some testsand I found a workarount for
>>>> the metrics problem.
>>>> I created a test environment in the laboratory with the main
>>>> characteristics equal to that of production (acs 4.11.2.0, all Ubuntu 16.04
>>>> OS, KVM, NFS as shared storage and advanced network). Then I added a ubuntu
>>>> 18.04 new primary storage.
>>>>
>>>> I create a new VM in the new storage server and and after a while the
>>>> metrics appeared as on the first storage, so the storage is working.
>>>>
>>>> I destroyed this VM, I create a new one on the first (old) storage and
>>>> then I migrate it on the new storage.
>>>>
>>>> After migrating the disk from the first primary storage to the second
>>>> one i have encountered the same problem (same error on agent.log
>>>> com.cloud.utils.exception.CloudRuntimeException: Can't find
>>>> volume:5a521eb0...) and the volume metrics and metrics didn't appear or
>>>> update.
>>>>
>>>> In this test environment I created a symbolic link in the filesystem of
>>>> the storage server taking the name of the disk just migrated (the one with
>>>> the name path) and the uuid name (the one that appear in error message).
>>>>
>>>>
>>>> Here is an example to better explain me.
>>>>
>>>> I took the list of volumes where path is different from uuid which
>>>> returns the data of the migrated volumes:
>>>>
>>>> mysql> select id,uuid,path from volumes where uuid!=path;
>>>>
>>>> +----+--------------------------------------+--------------------------------------+
>>>> | id | uuid | path
>>>> |
>>>>
>>>> +----+--------------------------------------+--------------------------------------+
>>>> | 10 | 5a521eb0-266f-4353-b4b2-1d63a483e5b5 |
>>>> 165b92ba-68f1-4172-be35-bbe1d032cb7c |
>>>> | 12 | acb3bb29-9bac-4a2a-aefa-3c6ac1c2846b |
>>>> 56aa3961-dbc2-4f98-9246-a7497eef3214 |
>>>>
>>>> +----+--------------------------------------+--------------------------------------+
>>>>
>>>> In the storage server I make the symbolic links (ln -s <path> <uuid>):
>>>>
>>>> # ln
>>>> -s 5a521eb0-266f-4353-b4b2-1d63a483e5b5
>>>> 165b92ba-68f1-4172-be35-bbe1d032cb7c
>>>> # ln -s acb3bb29-9bac-4a2a-aefa-3c6ac1c2846b
>>>> 56aa3961-dbc2-4f98-9246-a7497eef3214
>>>>
>>>> After doing this, I waited some time and then I found that metrics were
>>>> updated and the message in agent.log no longer appeared.
>>>>
>>>>
>>>> Il giorno gio 23 gen 2020 alle ore 17:53 Charlie Holeowsky <
>>>> charlie.holeow...@gmail.com> ha scritto:
>>>>
>>>>> I still don't understand
>>>>> why com.cloud.hypervisor.kvm.storage.LibvirtStoragePool don't find the
>>>>> volume d93d3c0a-3859-4473-951d-9b5c5912c76 that exists as file
>>>>> 39148fe1-842b-433a-8a7f-85e90f316e04...
>>>>>
>>>>> It's the only anomaly I have found. Where can I look again?
>>>>>
>>>>> Il giorno lun 20 gen 2020 alle ore 16:27 Daan Hoogland <
>>>>> daan.hoogl...@gmail.com> ha scritto:
>>>>>
>>>>>> but the record you send earlier also says that is should be looking
>>>>>> for 39148fe1-842b-433a-8a7f-85e90f316e04, in the path field. the message
>>>>>> might be just that, a message.
>>>>>>
>>>>>> On Mon, Jan 20, 2020 at 3:35 PM Charlie Holeowsky <
>>>>>> charlie.holeow...@gmail.com> wrote:
>>>>>>
>>>>>>> I think that's the problem because in the logs that I have forwarded
>>>>>>> it reads:
>>>>>>> Can't find volume: d93d3c0a-3859-4473-951d-9b5c5912c767
>>>>>>>
>>>>>>> This is the volume ID of migrated file but it do not exist on
>>>>>>> primary storage (new or old one) but it exist
>>>>>>> as 39148fe1-842b-433a-8a7f-85e90f316e04.
>>>>>>>
>>>>>>> Il giorno lun 20 gen 2020 alle ore 12:35 Daan Hoogland <
>>>>>>> daan.hoogl...@gmail.com> ha scritto:
>>>>>>>
>>>>>>>> also, can you see the primary storage being mounted?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jan 20, 2020 at 12:33 PM Daan Hoogland <
>>>>>>>> daan.hoogl...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Why do you think that Charlie? Is it in the logs like that
>>>>>>>>> somewhere?
>>>>>>>>>
>>>>>>>>> On Mon, Jan 20, 2020 at 9:52 AM Charlie Holeowsky <
>>>>>>>>> charlie.holeow...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Daan,
>>>>>>>>>> in fact I find the volume file
>>>>>>>>>> (39148fe1-842b-433a-8a7f-85e90f316e04) in the repositry id = 3 (the
>>>>>>>>>> new
>>>>>>>>>> one) but it seems to me that the cloudstack system goes looking for
>>>>>>>>>> the
>>>>>>>>>> volume with its "old" name (path) that doesn't exist...
>>>>>>>>>>
>>>>>>>>>> Il giorno sab 18 gen 2020 alle ore 21:41 Daan Hoogland <
>>>>>>>>>> daan.hoogl...@gmail.com> ha scritto:
>>>>>>>>>>
>>>>>>>>>>> Charlie,
>>>>>>>>>>> forgive my not replying in a timely manner. This might happen if
>>>>>>>>>>> the disk was migrated. In this case probably from primary storage
>>>>>>>>>>> with id 1
>>>>>>>>>>> to the one with id 3. the second record (pool_id 1) is removed, so
>>>>>>>>>>> you can
>>>>>>>>>>> ignore that one. The first seems legit. You should be able to find
>>>>>>>>>>> that
>>>>>>>>>>> disks on your primary storage with id 3.
>>>>>>>>>>> hope this helps.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 16, 2020 at 2:07 PM Charlie Holeowsky <
>>>>>>>>>>> charlie.holeow...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Daan and users,
>>>>>>>>>>>> to better explain I show you the two records related to the
>>>>>>>>>>>> disk that generates the error message.
>>>>>>>>>>>>
>>>>>>>>>>>> In the first query there are the data of the disk currently in
>>>>>>>>>>>> use which has the "uuid" equal to the name searched by the
>>>>>>>>>>>> procedure
>>>>>>>>>>>> com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.getPhysicalDiskand
>>>>>>>>>>>> a
>>>>>>>>>>>> has a different "path" field.
>>>>>>>>>>>>
>>>>>>>>>>>> In the second one we note that the "path" field is equal to the
>>>>>>>>>>>> "uuid" of the volume in use but has null "uuid" (and
>>>>>>>>>>>> state=Expunged).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> mysql> select * from volumes where
>>>>>>>>>>>> uuid='d93d3c0a-3859-4473-951d-9b5c5912c767';
>>>>>>>>>>>>
>>>>>>>>>>>> +-----+------------+-----------+---------+--------------+-------------+-----------+----------+--------------------------------------+-------------+---------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+---------------+-------------------+
>>>>>>>>>>>> | id | account_id | domain_id | pool_id | last_pool_id |
>>>>>>>>>>>> instance_id | device_id | name | uuid
>>>>>>>>>>>> |
>>>>>>>>>>>> size | folder | path
>>>>>>>>>>>> |
>>>>>>>>>>>> pod_id | data_center_id | iscsi_name | host_ip | volume_type |
>>>>>>>>>>>> pool_type |
>>>>>>>>>>>> disk_offering_id | template_id | first_snapshot_backup_uuid |
>>>>>>>>>>>> recreatable |
>>>>>>>>>>>> created | attached | updated | removed |
>>>>>>>>>>>> state |
>>>>>>>>>>>> chain_info | update_count | disk_type | vm_snapshot_chain_size |
>>>>>>>>>>>> iso_id |
>>>>>>>>>>>> display_volume | format | min_iops | max_iops | hv_ss_reserve |
>>>>>>>>>>>> provisioning_type |
>>>>>>>>>>>>
>>>>>>>>>>>> +-----+------------+-----------+---------+--------------+-------------+-----------+----------+--------------------------------------+-------------+---------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+---------------+-------------------+
>>>>>>>>>>>> | 213 | 2 | 1 | 3 | 1 |
>>>>>>>>>>>> 148 | 1 | DATA-148 |
>>>>>>>>>>>> d93d3c0a-3859-4473-951d-9b5c5912c767 |
>>>>>>>>>>>> 53687091200 | /srv/primary | 39148fe1-842b-433a-8a7f-85e90f316e04
>>>>>>>>>>>> | NULL
>>>>>>>>>>>> | 1 | NULL | NULL | DATADISK | NULL |
>>>>>>>>>>>> 34 | NULL | NULL | 0 |
>>>>>>>>>>>> 2019-11-26 10:41:46 | NULL | 2019-11-26 10:41:50 | NULL |
>>>>>>>>>>>> Ready |
>>>>>>>>>>>> NULL | 2 | NULL | NULL |
>>>>>>>>>>>> NULL |
>>>>>>>>>>>> 1 | QCOW2 | NULL | NULL | NULL | thin
>>>>>>>>>>>> |
>>>>>>>>>>>>
>>>>>>>>>>>> +-----+------------+-----------+---------+--------------+-------------+-----------+----------+--------------------------------------+-------------+---------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+---------------+-------------------+
>>>>>>>>>>>> 1 row in set (0.00 sec)
>>>>>>>>>>>>
>>>>>>>>>>>> mysql> select * from volumes where
>>>>>>>>>>>> path='d93d3c0a-3859-4473-951d-9b5c5912c767';
>>>>>>>>>>>>
>>>>>>>>>>>> +-----+------------+-----------+---------+--------------+-------------+-----------+----------+------+-------------+--------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------------------+----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+---------------+-------------------+
>>>>>>>>>>>> | id | account_id | domain_id | pool_id | last_pool_id |
>>>>>>>>>>>> instance_id | device_id | name | uuid | size | folder |
>>>>>>>>>>>> path
>>>>>>>>>>>> | pod_id | data_center_id | iscsi_name
>>>>>>>>>>>> |
>>>>>>>>>>>> host_ip | volume_type | pool_type | disk_offering_id | template_id
>>>>>>>>>>>> |
>>>>>>>>>>>> first_snapshot_backup_uuid | recreatable | created |
>>>>>>>>>>>> attached |
>>>>>>>>>>>> updated | removed | state | chain_info |
>>>>>>>>>>>> update_count | disk_type | vm_snapshot_chain_size | iso_id |
>>>>>>>>>>>> display_volume
>>>>>>>>>>>> | format | min_iops | max_iops | hv_ss_reserve | provisioning_type
>>>>>>>>>>>> |
>>>>>>>>>>>>
>>>>>>>>>>>> +-----+------------+-----------+---------+--------------+-------------+-----------+----------+------+-------------+--------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------------------+----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+---------------+-------------------+
>>>>>>>>>>>> | 212 | 2 | 1 | 1 | NULL |
>>>>>>>>>>>> 148 | 1 | DATA-148 | NULL | 53687091200 | NULL |
>>>>>>>>>>>> d93d3c0a-3859-4473-951d-9b5c5912c767 | NULL | 1 |
>>>>>>>>>>>> NULL
>>>>>>>>>>>> | NULL | DATADISK | NULL | 34 |
>>>>>>>>>>>> NULL | NULL
>>>>>>>>>>>> | 0 | 2019-11-26 10:38:23 | NULL
>>>>>>>>>>>> |
>>>>>>>>>>>> 2019-11-26 10:41:50 | 2019-11-26 10:41:50 | Expunged | NULL |
>>>>>>>>>>>> 8 | NULL | NULL | NULL |
>>>>>>>>>>>> 1 | QCOW2
>>>>>>>>>>>> | NULL | NULL | NULL | thin |
>>>>>>>>>>>>
>>>>>>>>>>>> +-----+------------+-----------+---------+--------------+-------------+-----------+----------+------+-------------+--------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------------------+----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+---------------+-------------------+
>>>>>>>>>>>> 1 row in set (0.00 sec)
>>>>>>>>>>>>
>>>>>>>>>>>> Il giorno mar 14 gen 2020 alle ore 15:05 Daan Hoogland <
>>>>>>>>>>>> daan.hoogl...@gmail.com> ha scritto:
>>>>>>>>>>>>
>>>>>>>>>>>>> So Charlie,
>>>>>>>>>>>>> d93d3c0a-3859-4473-951d-9b5c5912c767 is actually a valid disk?
>>>>>>>>>>>>> does it exist on the backend nfs?
>>>>>>>>>>>>> and the pool 9af0d1c6-85f2-3c55-94af-6ac17cb4024c does it
>>>>>>>>>>>>> exist both in cloudstack and on the backend?
>>>>>>>>>>>>>
>>>>>>>>>>>>> if both are answered with yes, you probably have a permissions
>>>>>>>>>>>>> issue, which might be in the network.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jan 14, 2020 at 10:21 AM Charlie Holeowsky <
>>>>>>>>>>>>> charlie.holeow...@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Daan and users,
>>>>>>>>>>>>>> the infrastructure is based on the Linux environment. The
>>>>>>>>>>>>>> management server, hosts and storage are all Ubuntu 16.04 except
>>>>>>>>>>>>>> the new
>>>>>>>>>>>>>> storage server which is an Ubuntu 18.04. The hypervisor used is
>>>>>>>>>>>>>> Qemu-kvm
>>>>>>>>>>>>>> with NFS to share the storage.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We tried to add another primary storage and creating a VM
>>>>>>>>>>>>>> that would use it we found no problems, the statistics update
>>>>>>>>>>>>>> and no error
>>>>>>>>>>>>>> messages appear.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Gere is an excerpt of the logs of the most complete agent:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2020-01-14 09:01:45,749 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-2:null)
>>>>>>>>>>>>>> (logid:c3851d3a) Trying to fetch storage pool
>>>>>>>>>>>>>> 171e90f4-511e-3b10-9310-b9eec0094be6 from libvirt
>>>>>>>>>>>>>> 2020-01-14 09:01:45,752 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-2:null)
>>>>>>>>>>>>>> (logid:c3851d3a) Asking libvirt to refresh storage pool
>>>>>>>>>>>>>> 171e90f4-511e-3b10-9310-b9eec0094be6
>>>>>>>>>>>>>> 2020-01-14 09:01:46,641 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-4:null)
>>>>>>>>>>>>>> (logid:c3851d3a) Trying to fetch storage pool
>>>>>>>>>>>>>> 9af0d1c6-85f2-3c55-94af-6ac17cb4024c from libvirt
>>>>>>>>>>>>>> 2020-01-14 09:01:46,643 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-4:null)
>>>>>>>>>>>>>> (logid:c3851d3a) Asking libvirt to refresh storage pool
>>>>>>>>>>>>>> 9af0d1c6-85f2-3c55-94af-6ac17cb4024c
>>>>>>>>>>>>>> 2020-01-14 09:05:51,529 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-1:null)
>>>>>>>>>>>>>> (logid:2765ff88) Trying to fetch storage pool
>>>>>>>>>>>>>> 9af0d1c6-85f2-3c55-94af-6ac17cb4024c from libvirt
>>>>>>>>>>>>>> 2020-01-14 09:05:51,532 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-1:null)
>>>>>>>>>>>>>> (logid:2765ff88) Asking libvirt to refresh storage pool
>>>>>>>>>>>>>> 9af0d1c6-85f2-3c55-94af-6ac17cb4024c
>>>>>>>>>>>>>> 2020-01-14 09:10:47,286 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-3:null)
>>>>>>>>>>>>>> (logid:6d27b740) Trying to fetch storage pool
>>>>>>>>>>>>>> 9af0d1c6-85f2-3c55-94af-6ac17cb4024c from libvirt
>>>>>>>>>>>>>> 2020-01-14 09:10:47,419 WARN [cloud.agent.Agent]
>>>>>>>>>>>>>> (agentRequest-Handler-3:null) (logid:6d27b740) Caught:
>>>>>>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: Can't find
>>>>>>>>>>>>>> volume:d93d3c0a-3859-4473-951d-9b5c5912c767
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:149)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.getVolumeStat(LibvirtGetVolumeStatsCommandWrapper.java:63)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:52)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:40)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1450)
>>>>>>>>>>>>>> at com.cloud.agent.Agent.processRequest(Agent.java:645)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1083)
>>>>>>>>>>>>>> at com.cloud.utils.nio.Task.call(Task.java:83)
>>>>>>>>>>>>>> at com.cloud.utils.nio.Task.call(Task.java:29)
>>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>>>>>>>>>>> 2020-01-14 09:20:48,390 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-4:null)
>>>>>>>>>>>>>> (logid:ec72387b) Trying to fetch storage pool
>>>>>>>>>>>>>> 9af0d1c6-85f2-3c55-94af-6ac17cb4024c from libvirt
>>>>>>>>>>>>>> 2020-01-14 09:20:48,536 WARN [cloud.agent.Agent]
>>>>>>>>>>>>>> (agentRequest-Handler-4:null) (logid:ec72387b) Caught:
>>>>>>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: Can't find
>>>>>>>>>>>>>> volume:d93d3c0a-3859-4473-951d-9b5c5912c767
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:149)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.getVolumeStat(LibvirtGetVolumeStatsCommandWrapper.java:63)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:52)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:40)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1450)
>>>>>>>>>>>>>> at com.cloud.agent.Agent.processRequest(Agent.java:645)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1083)
>>>>>>>>>>>>>> at com.cloud.utils.nio.Task.call(Task.java:83)
>>>>>>>>>>>>>> at com.cloud.utils.nio.Task.call(Task.java:29)
>>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>>>>>>>>>>> 2020-01-14 09:25:15,259 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-5:null)
>>>>>>>>>>>>>> (logid:1a7e082e) Trying to fetch storage pool
>>>>>>>>>>>>>> 9af0d1c6-85f2-3c55-94af-6ac17cb4024c from libvirt
>>>>>>>>>>>>>> 2020-01-14 09:25:15,261 INFO
>>>>>>>>>>>>>> [kvm.storage.LibvirtStorageAdaptor]
>>>>>>>>>>>>>> (agentRequest-Handler-5:null)
>>>>>>>>>>>>>> (logid:1a7e082e) Asking libvirt to refresh storage pool
>>>>>>>>>>>>>> 9af0d1c6-85f2-3c55-94af-6ac17cb4024c
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And here the management server log:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2020-01-14 09:21:27,105 DEBUG [c.c.a.t.Request]
>>>>>>>>>>>>>> (AgentManager-Handler-2:null) (logid:) Seq 15-705657766613619075:
>>>>>>>>>>>>>> Processing: { Ans: , MgmtId: 220777304233416, via: 15, Ver: v1,
>>>>>>>>>>>>>> Flags: 10,
>>>>>>>>>>>>>> [{"com.cloud.agent.api.Answer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException:
>>>>>>>>>>>>>> Can't find volume:d93d3c0a-3859-4473-951d-9b5c5912c767\n\tat
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:149)\n\tat
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.getVolumeStat(LibvirtGetVolumeStatsCommandWrapper.java:63)\n\tat
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:52)\n\tat
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:40)\n\tat
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)\n\tat
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1450)\n\tat
>>>>>>>>>>>>>> com.cloud.agent.Agent.processRequest(Agent.java:645)\n\tat
>>>>>>>>>>>>>> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1083)\n\tat
>>>>>>>>>>>>>> com.cloud.utils.nio.Task.call(Task.java:83)\n\tat
>>>>>>>>>>>>>> com.cloud.utils.nio.Task.call(Task.java:29)\n\tat
>>>>>>>>>>>>>> java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat
>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat
>>>>>>>>>>>>>> java.lang.Thread.run(Thread.java:748)\n","wait":0}}] }
>>>>>>>>>>>>>> 2020-01-14 09:21:27,105 DEBUG [c.c.a.t.Request]
>>>>>>>>>>>>>> (StatsCollector-6:ctx-fd801d0a) (logid:ec72387b) Seq
>>>>>>>>>>>>>> 15-705657766613619075:
>>>>>>>>>>>>>> Received: { Ans: , MgmtId: 220777304233416, via: 15(csdell017),
>>>>>>>>>>>>>> Ver: v1,
>>>>>>>>>>>>>> Flags: 10, { Answer } }
>>>>>>>>>>>>>> 2020-01-14 09:21:27,105 DEBUG [c.c.a.m.AgentManagerImpl]
>>>>>>>>>>>>>> (StatsCollector-6:ctx-fd801d0a) (logid:ec72387b) Details from
>>>>>>>>>>>>>> executing
>>>>>>>>>>>>>> class com.cloud.agent.api.GetVolumeStatsCommand:
>>>>>>>>>>>>>> com.cloud.utils.exception.CloudRuntimeException: Can't find
>>>>>>>>>>>>>> volume:d93d3c0a-3859-4473-951d-9b5c5912c767
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.storage.LibvirtStoragePool.getPhysicalDisk(LibvirtStoragePool.java:149)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.getVolumeStat(LibvirtGetVolumeStatsCommandWrapper.java:63)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:52)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVolumeStatsCommandWrapper.execute(LibvirtGetVolumeStatsCommandWrapper.java:40)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1450)
>>>>>>>>>>>>>> at com.cloud.agent.Agent.processRequest(Agent.java:645)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1083)
>>>>>>>>>>>>>> at com.cloud.utils.nio.Task.call(Task.java:83)
>>>>>>>>>>>>>> at com.cloud.utils.nio.Task.call(Task.java:29)
>>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> on 09/01/20 12:58, Daan Hoogland wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Charlie, I think you'll have to explain a bit more about your
>>>>>>>>>>>>>> environment
>>>>>>>>>>>>>> to get an answer. what type of storage is it? Where did you
>>>>>>>>>>>>>> migrate the VM
>>>>>>>>>>>>>> from and to? What types() of hypervisors are you using? Though
>>>>>>>>>>>>>> saying **the**
>>>>>>>>>>>>>> agent logs suggests KVM, you are still leaving people guessing a
>>>>>>>>>>>>>> lot.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Daan
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Daan
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Daan
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Daan
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Daan
>>>>>>
>>>>>
>>>
>>> --
>>> Daan
>>>
>>
>
> --
> Daan
>
