RocMarshal commented on code in PR #26985:
URL: https://github.com/apache/flink/pull/26985#discussion_r2358583200
##########
flink-core/src/main/java/org/apache/flink/configuration/TraceOptions.java:
##########
@@ -67,6 +82,31 @@ public static Configuration forReporter(Configuration
configuration, String repo
+ " any of the names in the list will be
started. Otherwise, all reporters that could be found in"
+ " the configuration will be started.");
+ /** The detail level for reporting checkpoint spans. */
+ public static final ConfigOption<TraceOptions.CheckpointSpanDetailLevel>
+ CHECKPOINT_SPAN_DETAIL_LEVEL =
+ key("traces.checkpoint.span-detail-level")
+
.enumType(TraceOptions.CheckpointSpanDetailLevel.class)
+
.defaultValue(CheckpointSpanDetailLevel.SPAN_PER_CHECKPOINT)
+ .withDescription(
+ String.format(
+ "Detail level for reporting
checkpoint spans. Possible values:\n"
+ + "- %s (default): Single
span per checkpoint. Aggregated sum/max for submetrics from all tasks and
subtasks per checkpoint\n"
Review Comment:
```suggestion
+ "- %s (default):
Single span per checkpoint. Aggregated sum/max for sub-metrics from all tasks
and subtasks per checkpoint\n"
```
##########
flink-metrics/flink-metrics-core/src/main/java/org/apache/flink/traces/SimpleSpan.java:
##########
@@ -73,6 +78,11 @@ public Map<String, Object> getAttributes() {
return attributes;
}
+ @Override
+ public List<Span> getChildren() {
+ return children;
Review Comment:
Did here need return a unmodifiableList ?
##########
flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/DefaultCheckpointStatsTrackerTest.java:
##########
@@ -434,6 +437,254 @@ private void assertCheckpointAttributes(
assertThat(attributes.get("isUnaligned")).isEqualTo(Boolean.toString(isUnaligned));
}
+ @Test
+ public void testSpanCreationBreakDownPerCheckpoint() {
+
testSpanCreationTemplate(TraceOptions.CheckpointSpanDetailLevel.SPAN_PER_CHECKPOINT);
+ }
+
+ @Test
+ public void testSpanCreationBreakDownPerCheckpointWithTasks() {
+ testSpanCreationTemplate(
+
TraceOptions.CheckpointSpanDetailLevel.SPAN_PER_CHECKPOINT_WITH_TASKS);
+ }
+
+ @Test
+ public void testSpanCreationBreakDownPerTask() {
+
testSpanCreationTemplate(TraceOptions.CheckpointSpanDetailLevel.CHILDREN_SPANS_PER_TASK);
+ }
+
+ @Test
+ public void testSpanCreationBreakDownPerSubTask() {
+
testSpanCreationTemplate(TraceOptions.CheckpointSpanDetailLevel.CHILDREN_SPANS_PER_SUBTASK);
+ }
+
+ public void
testSpanCreationTemplate(TraceOptions.CheckpointSpanDetailLevel detailLevel) {
+ JobVertexID jobVertexID0 = new JobVertexID();
+ JobVertexID jobVertexID1 = new JobVertexID();
+
+ final List<Span> reportedSpans = new ArrayList<>();
+ final List<Event> reportedEvents = new ArrayList<>();
+
+ produceTestSpans(jobVertexID0, jobVertexID1, detailLevel,
reportedSpans, reportedEvents);
+ assertThat(reportedSpans.size()).isEqualTo(1);
+ assertThat(reportedEvents.size()).isEqualTo(1);
+
+ Map<String, Object> expected = new HashMap<>();
+ expected.put("checkpointId", 42L);
Review Comment:
Could this constants be extracted as immutable strings?
- One advantage of doing so is that it avoids using them directly in
multiple places.
- Of course, such a change might come at the cost of some readability.
##########
flink-core/src/main/java/org/apache/flink/configuration/TraceOptions.java:
##########
@@ -67,6 +82,31 @@ public static Configuration forReporter(Configuration
configuration, String repo
+ " any of the names in the list will be
started. Otherwise, all reporters that could be found in"
+ " the configuration will be started.");
+ /** The detail level for reporting checkpoint spans. */
+ public static final ConfigOption<TraceOptions.CheckpointSpanDetailLevel>
+ CHECKPOINT_SPAN_DETAIL_LEVEL =
+ key("traces.checkpoint.span-detail-level")
+
.enumType(TraceOptions.CheckpointSpanDetailLevel.class)
+
.defaultValue(CheckpointSpanDetailLevel.SPAN_PER_CHECKPOINT)
+ .withDescription(
+ String.format(
+ "Detail level for reporting
checkpoint spans. Possible values:\n"
+ + "- %s (default): Single
span per checkpoint. Aggregated sum/max for submetrics from all tasks and
subtasks per checkpoint\n"
+ + "- %s: Single span per
checkpoint. Same as %s, plus arrays of aggregated values per task.\n"
+ + "- %s: Same as %s plus
children spans per each task. Each task span with aggregated sum/max submetrics
from subtasks.\n"
Review Comment:
```suggestion
+ "- %s: Same as %s plus
children spans per each task. Each task span with aggregated sum/max
sub-metrics from subtasks.\n"
```
##########
flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/DefaultCheckpointStatsTrackerTest.java:
##########
@@ -434,6 +437,254 @@ private void assertCheckpointAttributes(
assertThat(attributes.get("isUnaligned")).isEqualTo(Boolean.toString(isUnaligned));
}
+ @Test
+ public void testSpanCreationBreakDownPerCheckpoint() {
+
testSpanCreationTemplate(TraceOptions.CheckpointSpanDetailLevel.SPAN_PER_CHECKPOINT);
+ }
+
+ @Test
+ public void testSpanCreationBreakDownPerCheckpointWithTasks() {
+ testSpanCreationTemplate(
+
TraceOptions.CheckpointSpanDetailLevel.SPAN_PER_CHECKPOINT_WITH_TASKS);
+ }
+
+ @Test
+ public void testSpanCreationBreakDownPerTask() {
+
testSpanCreationTemplate(TraceOptions.CheckpointSpanDetailLevel.CHILDREN_SPANS_PER_TASK);
+ }
+
+ @Test
+ public void testSpanCreationBreakDownPerSubTask() {
+
testSpanCreationTemplate(TraceOptions.CheckpointSpanDetailLevel.CHILDREN_SPANS_PER_SUBTASK);
+ }
+
+ public void
testSpanCreationTemplate(TraceOptions.CheckpointSpanDetailLevel detailLevel) {
+ JobVertexID jobVertexID0 = new JobVertexID();
+ JobVertexID jobVertexID1 = new JobVertexID();
+
+ final List<Span> reportedSpans = new ArrayList<>();
+ final List<Event> reportedEvents = new ArrayList<>();
+
+ produceTestSpans(jobVertexID0, jobVertexID1, detailLevel,
reportedSpans, reportedEvents);
+ assertThat(reportedSpans.size()).isEqualTo(1);
+ assertThat(reportedEvents.size()).isEqualTo(1);
Review Comment:
How about
'assertThat(reportedSpans).hasSameSizeAs(reportedEvents).hasSize(1);' ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
