mbertolina opened a new issue, #14448:
URL: https://github.com/apache/cloudstack/issues/14448
### problem
# Title
Duplicate Management Server Entries Created After Host and Management Server
Outage
## Description
Specially following an outage affecting the hosts and the Management Server,
CloudStack creates a new Management Server entry instead of reusing the
existing one. The previous entries remain in the database with `state=Down`,
resulting in multiple entries for the same Management Server, all sharing the
same hostname and IP address.
We have observed this behavior in CloudStack 4.23.0. So far, we have been
unable to reproduce it by simply restarting the Management Server or agent
services.
The issue appears to leave stale Management Server records in the database,
requiring manual cleanup.
## Steps to Reproduce
1. Experience an outage affecting the hosts and the Management Server.
2. Restore the infrastructure and allow the Management Server to start.
3. Run `cmk listmanagementservers` to inspect the registered Management
Servers.
**Observed result:** Multiple entries exist for the same Management Server,
with identical hostnames and IP addresses. Previous entries remain in the
`Down` state, while the latest entry is `Up`.
## Expected Behavior
CloudStack should identify and reuse the existing Management Server entry
when the Management Server restarts after an outage, rather than creating
duplicate entries for the same instance.
## Workaround
Manually remove the stale Management Server entries using the `cmk` tool.
These entries cannot currently be deleted through the CloudStack UI.
## Environment
- CloudStack version: 4.23.0
- OS: Red Hat Enterprise Linux 9.8
- Java: OpenJDK 17.0.20.1+1-LTS
## Additional Information
The output of `cmk listmanagementservers` shows five entries for the same
Management Server (`acs01v`, IP address `108.145.117.211`): four entries in the
`Down` state and one in the `Up` state.
The entries have different UUIDs and boot/start timestamps, suggesting that
CloudStack registers a new Management Server record after certain recovery
scenarios rather than updating the existing record.
The attached command output provides evidence of the duplicate entries.
```
# cmk listmanagementservers
{
"count": 5,
"managementserver": [
{
"agents": [],
"agentscount": 0,
"id": "2a4899aa-c94a-4a45-b175-6e382afd2908",
"ipaddress": "108.145.117.211",
"javadistribution": "Red Hat, Inc.",
"javaversion": "17.0.20.1+1-LTS",
"lastagents": [],
"lastboottime": "2026-08-04T17:11:55+0000",
"lastserverstart": "2026-09-07T14:28:10+0000",
"lastserverstop": "2026-09-11T11:54:00+0000",
"name": "acs01v",
"osdistribution": "Red Hat Enterprise Linux 9.8 (Plow)",
"pendingjobscount": 0,
"serviceip": "108.145.117.211",
"state": "Down",
"version": "4.23.0.0"
},
{
"agents": [],
"agentscount": 0,
"id": "b426d612-69cf-4076-b9f7-fd51f7f09848",
"ipaddress": "108.145.117.211",
"javadistribution": "Red Hat, Inc.",
"javaversion": "17.0.20.1+1-LTS",
"lastagents": [],
"lastboottime": "2026-09-11T15:00:02+0000",
"lastserverstart": "2026-09-11T15:03:10+0000",
"name": "acs01v",
"osdistribution": "Red Hat Enterprise Linux 9.8 (Plow)",
"pendingjobscount": 0,
"serviceip": "108.145.117.211",
"state": "Down",
"version": "4.23.0.0"
},
{
"agents": [],
"agentscount": 0,
"id": "36d2aea4-de92-4144-8092-b85e6b5a0bc6",
"ipaddress": "108.145.117.211",
"javadistribution": "Red Hat, Inc.",
"javaversion": "17.0.20.1+1-LTS",
"lastagents": [],
"lastboottime": "2026-09-22T13:09:55+0000",
"lastserverstart": "2026-09-22T13:12:20+0000",
"name": "acs01v",
"osdistribution": "Red Hat Enterprise Linux 9.8 (Plow)",
"pendingjobscount": 0,
"serviceip": "108.145.117.211",
"state": "Down",
"version": "4.23.0.0"
},
{
"agents": [],
"agentscount": 0,
"id": "f8106e4a-ae0d-4379-9455-1db3022406df",
"ipaddress": "108.145.117.211",
"javadistribution": "Red Hat, Inc.",
"javaversion": "17.0.20.1+1-LTS",
"lastagents": [],
"lastboottime": "2026-10-16T00:45:02+0000",
"lastserverstart": "2026-10-16T00:45:19+0000",
"name": "acs01v",
"osdistribution": "Red Hat Enterprise Linux 9.8 (Plow)",
"pendingjobscount": 0,
"serviceip": "108.145.117.211",
"state": "Down",
"version": "4.23.0.0"
},
{
"agents": [
"ddf6a499-b749-43d4-9bf2-b48750d7a60e",
"5991fc17-8524-47bd-aea8-08bd02992e21",
"d77e251f-e3f3-4287-beee-b54f2ec9de4e",
"29b219f5-4e3d-4df9-b4f1-a837c17b00d8",
"27c1f440-6c52-4868-9c80-65704c4e13a0",
"18cb67d7-e201-4a07-8f38-ab97282c197a",
"fa70337a-5adc-47f4-8911-f5e628aa9540",
"2e79a92d-a842-4713-aa68-3ab4050af8b9",
"8aa046f3-85a6-45ab-8a0f-39312899e84d",
"f300c7ab-ec0f-49c6-be68-e2167baa4ce0",
"7bbec373-a643-4dc7-ac8b-3dcc35381191",
"f9a87e6d-8afd-4837-bd06-7994f87e0868",
"bd2c3d29-2c73-482f-83d5-479ed18bbd0a",
"9ee31367-3f4b-496c-9f53-0f825ff65782",
"b344da4e-123f-4376-8e7d-ce6405647b59",
"4a208773-a87b-4757-b1f0-a0b777edbdb7",
"d3858ead-b752-48c6-8682-40527153456f",
"2782f1a7-3f03-4249-ba73-11afa118e960",
"f9d490cd-4718-4838-ada7-5cab656222aa",
"ec0b57af-cb03-41bd-9606-c0336697d145",
"9b804eb2-b3d9-4b84-89db-06fe217b029b",
"41814525-d029-45a5-939a-3e306a3fc3d0",
"5af03a0c-85c3-4e03-b9dc-37e4f2d330f1",
"26be2dcf-c4db-465c-a323-1729ba793d74",
"f2076754-6950-49c6-991f-4452d5ffbc83",
"ae987858-0d31-49d4-acfb-8ea4e9ebcdec",
"e2888a12-c26f-46f2-8366-70b9503b0a09",
"8011c492-42ce-4297-a142-97926abd7cef"
],
"agentscount": 28,
"id": "5ddba450-94bb-4013-9d76-bb0215ab91e7",
"ipaddress": "108.145.117.211",
"javadistribution": "Red Hat, Inc.",
"javaversion": "17.0.20.1+1-LTS",
"lastagents": [
"ddf6a499-b749-43d4-9bf2-b48750d7a60e",
"5991fc17-8524-47bd-aea8-08bd02992e21",
"d77e251f-e3f3-4287-beee-b54f2ec9de4e",
"29b219f5-4e3d-4df9-b4f1-a837c17b00d8",
"27c1f440-6c52-4868-9c80-65704c4e13a0",
"18cb67d7-e201-4a07-8f38-ab97282c197a",
"fa70337a-5adc-47f4-8911-f5e628aa9540",
"2e79a92d-a842-4713-aa68-3ab4050af8b9",
"8aa046f3-85a6-45ab-8a0f-39312899e84d",
"f300c7ab-ec0f-49c6-be68-e2167baa4ce0",
"7bbec373-a643-4dc7-ac8b-3dcc35381191",
"f9a87e6d-8afd-4837-bd06-7994f87e0868",
"bd2c3d29-2c73-482f-83d5-479ed18bbd0a",
"9ee31367-3f4b-496c-9f53-0f825ff65782",
"b344da4e-123f-4376-8e7d-ce6405647b59",
"4a208773-a87b-4757-b1f0-a0b777edbdb7",
"d3858ead-b752-48c6-8682-40527153456f",
"2782f1a7-3f03-4249-ba73-11afa118e960",
"f9d490cd-4718-4838-ada7-5cab656222aa",
"ec0b57af-cb03-41bd-9606-c0336697d145",
"9b804eb2-b3d9-4b84-89db-06fe217b029b",
"41814525-d029-45a5-939a-3e306a3fc3d0",
"5af03a0c-85c3-4e03-b9dc-37e4f2d330f1",
"26be2dcf-c4db-465c-a323-1729ba793d74",
"f2076754-6950-49c6-991f-4452d5ffbc83",
"ae987858-0d31-49d4-acfb-8ea4e9ebcdec"
],
"lastboottime": "2026-10-16T13:20:49+0000",
"lastserverstart": "2026-10-19T19:08:59+0000",
"lastserverstop": "2026-10-19T19:08:58+0000",
"name": "acs01v",
"osdistribution": "Red Hat Enterprise Linux 9.8 (Plow)",
"pendingjobscount": 0,
"serviceip": "108.145.117.211",
"state": "Up",
"version": "4.23.0.0"
}
]
}
#
```
### versions
The versions of ACS, hypervisors, storage, network etc..
### The steps to reproduce the bug
1.
2.
3.
...
### What to do about it?
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
