This is an automated email from the ASF dual-hosted git repository.
capistrant pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 26301598f7e feat: Refactor names for tier segment cache usage to help
with observability and monitoring (#20199)
26301598f7e is described below
commit 26301598f7e280cd3497bafbdda8a52af0206dda
Author: Lucas Capistrant <[email protected]>
AuthorDate: Fri Sep 11 06:09:23 2026 -0500
feat: Refactor names for tier segment cache usage to help with
observability and monitoring (#20199)
Helps distinguish between actual disk usage and assignable cache usage. Now
that virtual storage historical can advertise larger max assignable size than
actual disk, the metrics needed some massaging to make sense in both worlds.
---
docs/operations/metrics.md | 8 +++-
.../embedded/query/QueryVirtualStorageTest.java | 8 ++--
.../src/main/resources/defaultMetrics.json | 8 +++-
.../main/resources/defaultMetricDimensions.json | 4 ++
.../resources/loggingEmitterAllowedMetrics.json | 4 ++
.../resources/loggingEmitterAllowedMetrics.json | 4 ++
.../duty/PrepareBalancerAndLoadQueues.java | 15 ++++--
.../loading/StrategicSegmentAssigner.java | 5 +-
.../druid/server/coordinator/stats/Stats.java | 44 ++++++++++++++++-
.../server/coordinator/duty/RunRulesTest.java | 4 +-
.../server/coordinator/rules/LoadRuleTest.java | 10 ++--
.../simulate/HistoricalTierAliasTest.java | 56 +++++++++++++++++++---
12 files changed, 143 insertions(+), 27 deletions(-)
diff --git a/docs/operations/metrics.md b/docs/operations/metrics.md
index 62d01316e6a..c30d34d14dc 100644
--- a/docs/operations/metrics.md
+++ b/docs/operations/metrics.md
@@ -474,8 +474,12 @@ These metrics are emitted by the Druid Coordinator in
every run of the correspon
|`tier/historical/clone/count`|Number of historical nodes in a tier which are
a clone of another historical in the same or different tier. The `tierAlias`
dimension is emitted only when the tier belongs to an alias configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration), and
can be used to aggregate metrics across the tiers in an alias.|`tier`,
`tierAlias`|Varies|
|`tier/historical/clone/synced`|Number of historical clones in a tier which
are currently in sync with their source server. |`server`, `tier`|1 if synced,
O if not synced|
|`tier/replication/factor`|Configured maximum replication factor in each tier.
The `tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
-|`tier/required/capacity`|Total capacity in bytes required in each tier. The
`tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
-|`tier/total/capacity`|Total capacity in bytes available in each tier. The
`tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
+|`tier/storage/required`|Segment storage in bytes that the load rules require
in each tier, counting each replica at its full segment size. Compare against
`tier/storage/capacity` to see how heavily the tier is subscribed. The
`tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
+|`tier/storage/capacity`|Total segment storage in bytes that each tier
advertises for segment assignment. Can be greater than the physical disk
reported by `tier/segmentCache/capacity` if using virtual storage, so use that
metric rather than this one to monitor physical capacity. The `tierAlias`
dimension is emitted only when the tier belongs to an alias configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
+|`tier/segmentCache/capacity`|Total physical size in bytes of the segment
cache locations configured on the historicals in each tier. The `tierAlias`
dimension is emitted only when the tier belongs to an alias configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
+|`tier/segmentCache/used`|Bytes occupied on disk in each tier by the segments
its historicals have announced. Partially loaded segments count only the bytes
the historical reported loading, not their full size. Compare against
`tier/segmentCache/capacity` to see how full the tier's disks are. The
`tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
+|`tier/required/capacity`|Deprecated. Use `tier/storage/required` instead,
which reports the same value under a name that groups it with the storage
capacity it should be compared against. This metric will be removed in a future
release.|`tier`, `tierAlias`|Varies|
+|`tier/total/capacity`|Deprecated. Use `tier/storage/capacity` instead, which
reports the same value under a name that groups it with the required storage it
should be compared against. This metric will be removed in a future
release.|`tier`, `tierAlias`|Varies|
|`compact/task/count`|Number of tasks issued in the auto compaction run.|
|Varies|
|`compactTask/maxSlot/count`|Maximum number of task slots available for auto
compaction tasks in the auto compaction run.| |Varies|
|`compactTask/availableSlot/count`|Number of currently vacant task slots out
of the total slots allocated for auto compaction tasks. This value is computed
as the difference between the total number of task slots allocated for auto
compaction and the estimated number of task slots currently occupied by running
compaction tasks. The number of sub-tasks of each compaction task is estimated
to be `maxNumConcurrentSubTasks`.| |Varies|
diff --git
a/embedded-tests/src/test/java/org/apache/druid/testing/embedded/query/QueryVirtualStorageTest.java
b/embedded-tests/src/test/java/org/apache/druid/testing/embedded/query/QueryVirtualStorageTest.java
index bc259097551..466f2cb9eb0 100644
---
a/embedded-tests/src/test/java/org/apache/druid/testing/embedded/query/QueryVirtualStorageTest.java
+++
b/embedded-tests/src/test/java/org/apache/druid/testing/embedded/query/QueryVirtualStorageTest.java
@@ -295,15 +295,15 @@ class QueryVirtualStorageTest extends
EmbeddedClusterTestBase
Assertions.assertEquals(0,
emitter.getMetricEventLongSum(StorageMonitor.VSF_REJECT_COUNT));
Assertions.assertTrue(emitter.getLatestMetricEventValue(StorageMonitor.VSF_USED_BYTES,
0).longValue() > 0);
- coordinatorEmitter.waitForEvent(event ->
event.hasMetricName(Stats.Tier.STORAGE_CAPACITY.getMetricName()));
+ coordinatorEmitter.waitForEvent(event ->
event.hasMetricName(Stats.Tier.CACHE_CAPACITY.getMetricName()));
Assertions.assertEquals(
CACHE_SIZE,
-
coordinatorEmitter.getLatestMetricEventValue(Stats.Tier.STORAGE_CAPACITY.getMetricName())
+
coordinatorEmitter.getLatestMetricEventValue(Stats.Tier.CACHE_CAPACITY.getMetricName())
);
- coordinatorEmitter.waitForEvent(event ->
event.hasMetricName(Stats.Tier.TOTAL_CAPACITY.getMetricName()));
+ coordinatorEmitter.waitForEvent(event ->
event.hasMetricName(Stats.Tier.STORAGE_CAPACITY.getMetricName()));
Assertions.assertEquals(
MAX_SIZE,
-
coordinatorEmitter.getLatestMetricEventValue(Stats.Tier.TOTAL_CAPACITY.getMetricName())
+
coordinatorEmitter.getLatestMetricEventValue(Stats.Tier.STORAGE_CAPACITY.getMetricName())
);
}
diff --git
a/extensions-contrib/prometheus-emitter/src/main/resources/defaultMetrics.json
b/extensions-contrib/prometheus-emitter/src/main/resources/defaultMetrics.json
index 53492716265..57e4b8c7e99 100644
---
a/extensions-contrib/prometheus-emitter/src/main/resources/defaultMetrics.json
+++
b/extensions-contrib/prometheus-emitter/src/main/resources/defaultMetrics.json
@@ -188,8 +188,12 @@
"segment/availableDeepStorageOnly/count" : { "dimensions" : ["dataSource",
"tier"], "type" : "gauge", "help": "Number of unique segments that are only
available for querying directly from deep storage."},
"tier/historical/count" : { "dimensions" : ["tier", "tierAlias"], "type" :
"count", "help": "Number of available historical nodes in each tier."},
"tier/replication/factor" : { "dimensions" : ["tier", "tierAlias"], "type" :
"count", "help": "Configured maximum replication factor in each tier."},
- "tier/required/capacity" : { "dimensions" : ["tier", "tierAlias"], "type" :
"count", "help": "Total capacity in bytes required in each tier."},
- "tier/total/capacity" : { "dimensions" : ["tier", "tierAlias"], "type" :
"count", "help": "Total capacity in bytes available in each tier."},
+ "tier/storage/required": { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge", "help": "Segment storage in bytes required in each tier."},
+ "tier/storage/capacity": { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge", "help": "Total segment storage in bytes available for assignment in
each tier."},
+ "tier/segmentCache/capacity": { "dimensions" : ["tier", "tierAlias"], "type"
: "gauge", "help": "Total physical capacity in bytes of the segment cache
locations in each tier."},
+ "tier/segmentCache/used": { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge", "help": "Total bytes occupied on disk in each tier."},
+ "tier/required/capacity" : { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge", "help": "Deprecated. Use tier/storage/required instead."},
+ "tier/total/capacity" : { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge", "help": "Deprecated. Use tier/storage/capacity instead."},
"compact/task/count" : { "dimensions" : [], "type" : "gauge", "help":
"Number of tasks issued in the auto compaction run."},
"compactTask/maxSlot/count" : { "dimensions" : [], "type" : "gauge", "help":
"Maximum number of task slots available for auto compaction tasks in the auto
compaction run."},
"compactTask/availableSlot/count" : { "dimensions" : [], "type" : "gauge",
"help": "Number of available task slots that can be used for auto compaction
tasks in the auto compaction run."},
diff --git
a/extensions-contrib/statsd-emitter/src/main/resources/defaultMetricDimensions.json
b/extensions-contrib/statsd-emitter/src/main/resources/defaultMetricDimensions.json
index 0b86e8efc5d..77f34f1da4e 100644
---
a/extensions-contrib/statsd-emitter/src/main/resources/defaultMetricDimensions.json
+++
b/extensions-contrib/statsd-emitter/src/main/resources/defaultMetricDimensions.json
@@ -182,6 +182,10 @@
"coordinator/time" : { "dimensions" : [], "type" : "timer"},
"coordinator/global/time" : { "dimensions" : ["dutyGroup"], "type" :
"timer"},
+ "tier/storage/required": { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge"},
+ "tier/storage/capacity": { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge"},
+ "tier/segmentCache/capacity": { "dimensions" : ["tier", "tierAlias"], "type"
: "gauge"},
+ "tier/segmentCache/used": { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge"},
"tier/required/capacity" : { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge" },
"tier/total/capacity" : { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge" },
"tier/replication/factor" : { "dimensions" : ["tier", "tierAlias"], "type" :
"gauge" },
diff --git a/processing/src/main/resources/loggingEmitterAllowedMetrics.json
b/processing/src/main/resources/loggingEmitterAllowedMetrics.json
index e0d89fac679..a421c96021f 100644
--- a/processing/src/main/resources/loggingEmitterAllowedMetrics.json
+++ b/processing/src/main/resources/loggingEmitterAllowedMetrics.json
@@ -249,6 +249,10 @@
"tier/replication/factor": [],
"tier/required/capacity": [],
"tier/total/capacity": [],
+ "tier/storage/required": [],
+ "tier/storage/capacity": [],
+ "tier/segmentCache/used": [],
+ "tier/segmentCache/capacity": [],
"zk/connected": [],
"zk/reconnect/time": []
}
diff --git a/processing/src/test/resources/loggingEmitterAllowedMetrics.json
b/processing/src/test/resources/loggingEmitterAllowedMetrics.json
index 5818ed2d36a..b9451f0fcf9 100644
--- a/processing/src/test/resources/loggingEmitterAllowedMetrics.json
+++ b/processing/src/test/resources/loggingEmitterAllowedMetrics.json
@@ -248,6 +248,10 @@
"tier/replication/factor": [],
"tier/required/capacity": [],
"tier/total/capacity": [],
+ "tier/storage/required": [],
+ "tier/storage/capacity": [],
+ "tier/segmentCache/used": [],
+ "tier/segmentCache/capacity": [],
"zk/connected": [],
"zk/reconnect/time": []
}
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/duty/PrepareBalancerAndLoadQueues.java
b/server/src/main/java/org/apache/druid/server/coordinator/duty/PrepareBalancerAndLoadQueues.java
index 0ad2c1cb199..02f428cbd48 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/duty/PrepareBalancerAndLoadQueues.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/duty/PrepareBalancerAndLoadQueues.java
@@ -184,20 +184,27 @@ public class PrepareBalancerAndLoadQueues implements
CoordinatorDuty
: RowKey.with(Dimension.TIER,
tier).and(Dimension.TIER_ALIAS, alias);
stats.add(Stats.Tier.HISTORICAL_COUNT, rowKey, historicals.size());
- long totalCapacity = 0;
long storageCapacity = 0;
+ long cacheCapacity = 0;
+ long cacheUsed = 0;
long cloneCount = 0;
for (ServerHolder holder : historicals) {
if (holder.isUnmanaged()) {
cloneCount += 1;
} else {
- totalCapacity += holder.getMaxSize();
- storageCapacity += holder.getStorageSize();
+ storageCapacity += holder.getMaxSize();
+ cacheCapacity += holder.getStorageSize();
+ // getCurrSize(), not getSizeUsed(): queued loads are counted at
full segment size, which would overstate
+ // disk usage on a tier serving partial loads for the duration of
every load
+ cacheUsed += holder.getServer().getCurrSize();
}
}
stats.add(Stats.Tier.CLONE_COUNT, rowKey, cloneCount);
- stats.add(Stats.Tier.TOTAL_CAPACITY, rowKey, totalCapacity);
stats.add(Stats.Tier.STORAGE_CAPACITY, rowKey, storageCapacity);
+ // Deprecated alias of tier/storage/capacity, emitted until the
deprecation period is over
+ stats.add(Stats.Tier.TOTAL_CAPACITY, rowKey, storageCapacity);
+ stats.add(Stats.Tier.CACHE_CAPACITY, rowKey, cacheCapacity);
+ stats.add(Stats.Tier.CACHE_USED, rowKey, cacheUsed);
});
}
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
b/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
index 42cb7d3ab2c..89e51f99660 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
@@ -780,8 +780,11 @@ public class StrategicSegmentAssigner implements
SegmentActionHandler
private void reportTierCapacityStats(DataSegment segment, int
requiredReplicas, String tier)
{
final RowKey rowKey = tierRowKey(tier);
+ final long requiredStorage = segment.getSize() * requiredReplicas;
stats.updateMax(Stats.Tier.REPLICATION_FACTOR, rowKey, requiredReplicas);
- stats.add(Stats.Tier.REQUIRED_CAPACITY, rowKey, segment.getSize() *
requiredReplicas);
+ stats.add(Stats.Tier.REQUIRED_STORAGE, rowKey, requiredStorage);
+ // Deprecated alias of tier/storage/required, emitted until the
deprecation period is over
+ stats.add(Stats.Tier.REQUIRED_CAPACITY, rowKey, requiredStorage);
}
/**
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/stats/Stats.java
b/server/src/main/java/org/apache/druid/server/coordinator/stats/Stats.java
index 138d7068189..c2f763dbb6f 100644
--- a/server/src/main/java/org/apache/druid/server/coordinator/stats/Stats.java
+++ b/server/src/main/java/org/apache/druid/server/coordinator/stats/Stats.java
@@ -108,12 +108,52 @@ public class Stats
public static class Tier
{
+ // Segment storage of a tier, denominated in full segment size. These are
the assignment ledger: REQUIRED_STORAGE
+ // is the demand the rules place on the tier, STORAGE_CAPACITY the supply
it advertises. Under virtual storage a
+ // tier can address more storage than it has disk, so both may exceed
CACHE_CAPACITY.
+
+ /**
+ * The demand the rules place on the tier to full load all segments.
+ */
+ public static final CoordinatorStat REQUIRED_STORAGE
+ = CoordinatorStat.toDebugAndEmit("reqdStorage",
"tier/storage/required");
+ /**
+ * The total storage the tier advertises for assignment, defined by
summing the configured max size of all servers
+ * in the tier.
+ */
+ public static final CoordinatorStat STORAGE_CAPACITY
+ = CoordinatorStat.toDebugAndEmit("storageCap",
"tier/storage/capacity");
+ /**
+ * Superseded by {@link #REQUIRED_STORAGE}, which carries the same value
under a name that groups it with the
+ * storage capacity it should be compared against. Still emitted so
existing dashboards keep working; remove
+ * after deprecation.
+ */
+ @Deprecated
public static final CoordinatorStat REQUIRED_CAPACITY
= CoordinatorStat.toDebugAndEmit("reqdCap", "tier/required/capacity");
+ /**
+ * Superseded by {@link #STORAGE_CAPACITY}, which carries the same value
under a name that groups it with the
+ * required storage it should be compared against. Still emitted so
existing dashboards keep working; remove after
+ * deprecation.
+ */
+ @Deprecated
public static final CoordinatorStat TOTAL_CAPACITY
= CoordinatorStat.toDebugAndEmit("totalCap", "tier/total/capacity");
- public static final CoordinatorStat STORAGE_CAPACITY
- = CoordinatorStat.toDebugAndEmit("storageCap",
"tier/storage/capacity");
+
+ // Physical segment cache of a tier: the size of the segment cache
locations configured on its historicals and the
+ // bytes actually occupied on them. Named for
druid.segmentCache.locations, which is what these measure.
+
+ /**
+ * Aggregate physical disk capacity of the tier's segment cache locations.
+ */
+ public static final CoordinatorStat CACHE_CAPACITY
+ = CoordinatorStat.toDebugAndEmit("cacheCap",
"tier/segmentCache/capacity");
+ /**
+ * Actual physical disk consumed by the tier. Only reports realized bytes
on disk, so partial loads when using
+ * virtual storage report their true loaded size, which can be less than
the full segment size.
+ */
+ public static final CoordinatorStat CACHE_USED
+ = CoordinatorStat.toDebugAndEmit("cacheUsed",
"tier/segmentCache/used");
public static final CoordinatorStat REPLICATION_FACTOR
= CoordinatorStat.toDebugAndEmit("maxRepFactor",
"tier/replication/factor");
public static final CoordinatorStat HISTORICAL_COUNT
diff --git
a/server/src/test/java/org/apache/druid/server/coordinator/duty/RunRulesTest.java
b/server/src/test/java/org/apache/druid/server/coordinator/duty/RunRulesTest.java
index 67ffe445343..8ba9cec03a5 100644
---
a/server/src/test/java/org/apache/druid/server/coordinator/duty/RunRulesTest.java
+++
b/server/src/test/java/org/apache/druid/server/coordinator/duty/RunRulesTest.java
@@ -1126,7 +1126,7 @@ public class RunRulesTest
final RowKey tierRowKey = RowKey.of(Dimension.TIER,
DruidServer.DEFAULT_TIER);
Assertions.assertEquals(
dataSegment.getSize() * numReplicants,
- stats.get(Stats.Tier.REQUIRED_CAPACITY, tierRowKey)
+ stats.get(Stats.Tier.REQUIRED_STORAGE, tierRowKey)
);
// Verify that primary assignment failed
@@ -1181,7 +1181,7 @@ public class RunRulesTest
final RowKey tierRowKey = RowKey.of(Dimension.TIER,
DruidServer.DEFAULT_TIER);
Assertions.assertEquals(
dataSegment.getSize() * numReplicants,
- stats.get(Stats.Tier.REQUIRED_CAPACITY, tierRowKey)
+ stats.get(Stats.Tier.REQUIRED_STORAGE, tierRowKey)
);
Assertions.assertEquals(0L, stats.getSegmentStat(Stats.Segments.ASSIGNED,
DruidServer.DEFAULT_TIER, DATASOURCE));
Assertions.assertFalse(stats.hasStat(Stats.Segments.DROPPED));
diff --git
a/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
b/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
index 344c140d844..cdb4f136070 100644
---
a/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
+++
b/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
@@ -588,15 +588,19 @@ public class LoadRuleTest
)
);
- // Required capacity is reported against the physical tier AND tagged with
the alias
+ // Required storage is reported against the physical tier AND tagged with
the alias
final RowKey t2WithAlias = RowKey.with(Dimension.TIER,
Tier.T2).and(Dimension.TIER_ALIAS, Tier.T1);
final RowKey t3WithAlias = RowKey.with(Dimension.TIER,
Tier.T3).and(Dimension.TIER_ALIAS, Tier.T1);
+ Assertions.assertEquals(segment.getSize(),
stats.get(Stats.Tier.REQUIRED_STORAGE, t2WithAlias));
+ Assertions.assertEquals(segment.getSize(),
stats.get(Stats.Tier.REQUIRED_STORAGE, t3WithAlias));
+
+ // The deprecated tier/required/capacity carries the same value while it
is still emitted
Assertions.assertEquals(segment.getSize(),
stats.get(Stats.Tier.REQUIRED_CAPACITY, t2WithAlias));
Assertions.assertEquals(segment.getSize(),
stats.get(Stats.Tier.REQUIRED_CAPACITY, t3WithAlias));
// The same stat without the alias dimension is a different row and must
be absent
- Assertions.assertEquals(0L, stats.get(Stats.Tier.REQUIRED_CAPACITY,
RowKey.of(Dimension.TIER, Tier.T2)));
- Assertions.assertEquals(0L, stats.get(Stats.Tier.REQUIRED_CAPACITY,
RowKey.of(Dimension.TIER, Tier.T3)));
+ Assertions.assertEquals(0L, stats.get(Stats.Tier.REQUIRED_STORAGE,
RowKey.of(Dimension.TIER, Tier.T2)));
+ Assertions.assertEquals(0L, stats.get(Stats.Tier.REQUIRED_STORAGE,
RowKey.of(Dimension.TIER, Tier.T3)));
}
/**
diff --git
a/server/src/test/java/org/apache/druid/server/coordinator/simulate/HistoricalTierAliasTest.java
b/server/src/test/java/org/apache/druid/server/coordinator/simulate/HistoricalTierAliasTest.java
index 33edcc1d032..ca33c5ee5e2 100644
---
a/server/src/test/java/org/apache/druid/server/coordinator/simulate/HistoricalTierAliasTest.java
+++
b/server/src/test/java/org/apache/druid/server/coordinator/simulate/HistoricalTierAliasTest.java
@@ -38,6 +38,7 @@ import java.util.Set;
public class HistoricalTierAliasTest extends CoordinatorSimulationBaseTest
{
private static final long SIZE_1TB = 1_000_000;
+ private static final long SEGMENT_SIZE = 500_000_000;
private static final String ALIAS = "hot";
private DruidServer historicalT1;
@@ -79,18 +80,22 @@ public class HistoricalTierAliasTest extends
CoordinatorSimulationBaseTest
final long expectedCapacity = SIZE_1TB << 20;
- // tier/total/capacity is emitted per physical tier AND tagged with the
alias
+ // tier/storage/capacity is emitted per physical tier AND tagged with the
alias
verifyValue(
- Stats.Tier.TOTAL_CAPACITY.getMetricName(),
- Map.of(Dimension.TIER.reportedName(), Tier.T1,
Dimension.TIER_ALIAS.reportedName(), ALIAS),
+ Stats.Tier.STORAGE_CAPACITY.getMetricName(),
+ aliasedTier(Tier.T1),
expectedCapacity
);
verifyValue(
- Stats.Tier.TOTAL_CAPACITY.getMetricName(),
- Map.of(Dimension.TIER.reportedName(), Tier.T2,
Dimension.TIER_ALIAS.reportedName(), ALIAS),
+ Stats.Tier.STORAGE_CAPACITY.getMetricName(),
+ aliasedTier(Tier.T2),
expectedCapacity
);
+ // The deprecated tier/total/capacity carries the same value while it is
still emitted
+ verifyValue(Stats.Tier.TOTAL_CAPACITY.getMetricName(),
aliasedTier(Tier.T1), expectedCapacity);
+ verifyValue(Stats.Tier.TOTAL_CAPACITY.getMetricName(),
aliasedTier(Tier.T2), expectedCapacity);
+
// tier/historical/count carries the alias too, so it can be summed across
the pair
verifyValue(
Stats.Tier.HISTORICAL_COUNT.getMetricName(),
@@ -104,6 +109,43 @@ public class HistoricalTierAliasTest extends
CoordinatorSimulationBaseTest
);
}
+ @Test
+ public void testUsedStorageReportsOnlyBytesLoadedAndAnnounced()
+ {
+ // tier/segmentCache/used does not report bytes consumed until historicals
have actually loaded and announced the segments. It only reports the actual
realized bytes of loaded segments.
+ final CoordinatorSimulation sim =
+ CoordinatorSimulation.builder()
+ .withSegments(Segments.WIKI_10X1D)
+ .withServers(historicalT1, historicalT2)
+ .withRules(datasource, Load.on(ALIAS,
1).forever())
+ .withDynamicConfig(
+ CoordinatorDynamicConfig.builder()
+
.withHistoricalTierAliases(
+ Map.of(ALIAS,
Set.of(Tier.T1, Tier.T2))
+ )
+
.withSmartSegmentLoading(true)
+ .build()
+ )
+ .withImmediateSegmentLoading(true)
+ .build();
+
+ startSimulation(sim);
+
+ runCoordinatorCycle();
+ verifyValue(Stats.Tier.CACHE_USED.getMetricName(), aliasedTier(Tier.T1),
0L);
+ verifyValue(Stats.Tier.CACHE_USED.getMetricName(), aliasedTier(Tier.T2),
0L);
+
+ runCoordinatorCycle();
+ final long loadedBytes = 10 * SEGMENT_SIZE;
+ verifyValue(Stats.Tier.CACHE_USED.getMetricName(), aliasedTier(Tier.T1),
loadedBytes);
+ verifyValue(Stats.Tier.CACHE_USED.getMetricName(), aliasedTier(Tier.T2),
loadedBytes);
+ }
+
+ private static Map<String, Object> aliasedTier(String tier)
+ {
+ return Map.of(Dimension.TIER.reportedName(), tier,
Dimension.TIER_ALIAS.reportedName(), ALIAS);
+ }
+
@Test
public void testCapacityMetricsHaveNoAliasWhenNotConfigured()
{
@@ -127,12 +169,12 @@ public class HistoricalTierAliasTest extends
CoordinatorSimulationBaseTest
// Without an alias configured, capacity is reported against the physical
tier only
verifyValue(
- Stats.Tier.TOTAL_CAPACITY.getMetricName(),
+ Stats.Tier.STORAGE_CAPACITY.getMetricName(),
filterByTier(Tier.T1),
expectedCapacity
);
verifyValue(
- Stats.Tier.TOTAL_CAPACITY.getMetricName(),
+ Stats.Tier.STORAGE_CAPACITY.getMetricName(),
filterByTier(Tier.T2),
expectedCapacity
);
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
