nagaboinaramgopal opened a new pull request, #14156:
URL: https://github.com/apache/cloudstack/pull/14156
### Description
**Depends on #14033.** The new table goes into `schema-42300to2400.sql`,
which #14033 adds, so this branch contains the #14033 commits. Only the last
commit, `metrics: add host usage history`, belongs to this PR. I will rebase
once #14033 is merged.
This adds a usage history for hosts, the same way it already exists for
instances, system VMs and volumes. The host stats collected by the management
server are only kept in memory today, so there is no way to see how the CPU,
memory and network usage of a host changed over time.
Every collected host stats sample is now stored in a new `host_stats` table,
and a new admin API `listHostsUsageHistory` returns them. It is the host
counterpart of `listVirtualMachinesUsageHistory`, `listSystemVmsUsageHistory`
and `listVolumesUsageHistory`, supports `id`, `ids`, `name` (substring),
`keyword`, `startdate`, `enddate` and paging, and returns the stats in the same
`StatsResponse` format. Only routing hosts are returned.
Old samples are removed by the new `host.stats.max.retention.time` setting
(minutes, default 720, same as `vm.stats.max.retention.time`). Setting it to 0
or less stops storing and pruning host stats. The setting is dynamic.
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
- [ ] test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [x] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
N/A
### How Has This Been Tested?
Unit tests added in `StatsCollectorTest` (persisting a sample, cleanup
enabled and disabled) and `MetricsServiceImplTest` (host search, date filters,
response mapping), and a smoke test `test_list_hosts_metrics_history` in
`test_metrics_api.py`. Both test classes pass (61 and 26 tests), with
checkstyle.
mvn -pl server,plugins/metrics test
-Dtest=StatsCollectorTest,MetricsServiceImplTest
Live tested on a KVM zone with two hosts and NFS storage, upgraded from 4.22
to packages built from this branch (on top of #14033). Management server and
hosts on Debian 13, MariaDB 11.8.
Upgrade, the new table is created by the 4.23.0.0 to 24.0.0 step:
21:33:47 INFO [DatabaseUpgradeChecker] Database upgrade must be
performed from 4.22.2.0 to 24.0.0
21:33:49 INFO [DatabaseUpgradeChecker] Cleanup upgrade
Upgrade42210to42300 to upgrade from 4.22.1.0-4.23.0.0 to 4.23.0.0
21:33:49 INFO [DatabaseUpgradeChecker] Cleanup upgrade
Upgrade42300to2400 to upgrade from 4.23.0.0-24.0.0 to 24.0.0
version table 4.22.2.0, 4.23.0.0, 24.0.0 (all Complete)
host_stats table present
both hosts Up, agent 24.0.0-SNAPSHOT
With the default settings (`host.stats.interval` 60000,
`host.stats.max.retention.time` 720), one sample per host per minute:
host 1 4 rows 21:35:22 to 21:38:23
host 2 4 rows 21:35:22 to 21:38:23
`listHostsUsageHistory` (system VM hosts are not returned):
listApis name=listHostsUsageHistory since 24.0.0, params id, ids,
name, keyword, startdate, enddate, page, pagesize
no filter count 2, host 1 = 4 samples, host
2 = 4 samples
id=<host 1> count 1, host 1 = 4 samples
name=<part of host 1 name> count 1, host 1 = 4 samples
keyword=<common name prefix> count 2
name=no-such-host empty list
startdate=now-2min count 2, 2 samples per host
enddate=now-2min count 2, 2 samples per host
startdate after enddate HTTP 431 startDate cannot be
after endDate.
page=1&pagesize=1 count 2, 1 host returned
A sample from the response:
{"timestamp": "2026-09-11T21:38:23+0000", "cpuused": "0.85%",
"memoryintfreekbs": 11305792, "memorykbs": 12253368, "networkkbsread": 2051148,
"networkkbswrite": 1959142, "networkread": "2003.07 MiB", "networkwrite":
"1913.22 MiB"}
Retention, changed at runtime with updateConfiguration:
host.stats.max.retention.time=1 before: 10 rows (21:35:22
to 21:39:23)
150s later 21:41:54 4 rows
(21:40:23 to 21:41:23), older rows removed
host.stats.max.retention.time=0 before: 4 rows, newest
21:41:23
150s later 4 rows, newest
21:41:23, nothing stored or removed
host.stats.max.retention.time=720 150s later 21:47:14 8 rows, newest
21:46:24, storing resumed
21:46:22 INFO [HostStatsDaoImpl] Removed a total of [0] host_stats rows
older than [Fri Sep 11 09:46:22 UTC 2026].
#### How did you try to break this feature and the system with this change?
A start date after the end date is rejected with 431, a name that matches no
host returns an empty list, and paging returns the total count. With the
retention set to 0 nothing is stored and nothing is removed, and changing it
back resumes storing without a restart. System VM hosts are not included, only
routing hosts. The table has no foreign key, so the history of a removed host
stays until it is pruned by the retention. The existing instance, system VM and
volume usage history APIs and the in-memory host stats used by listHostsMetrics
are unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
