gyfora commented on code in PR #707:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/707#discussion_r1390470139
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobAutoScalerImpl.java:
##########
@@ -115,10 +123,39 @@ public void cleanup(KEY jobKey) {
stateStore.removeInfoFromCache(jobKey);
}
- private void clearParallelismOverrides(Context ctx) throws Exception {
- var parallelismOverrides = stateStore.getParallelismOverrides(ctx);
- if (parallelismOverrides.isPresent()) {
+ private void clearStatesAfterAutoscalerDisabled(Context ctx) throws
Exception {
+ var needFlush = false;
+ var parallelismOverridesOpt = stateStore.getParallelismOverrides(ctx);
+ if (parallelismOverridesOpt.isPresent()) {
+ needFlush = true;
stateStore.removeParallelismOverrides(ctx);
+ }
+
+ Optional<SortedMap<Instant, CollectedMetrics>> collectedMetricsOpt =
+ stateStore.getCollectedMetrics(ctx);
+ if (collectedMetricsOpt.isPresent()) {
+ needFlush = true;
+ stateStore.removeCollectedMetrics(ctx);
+ }
+
+ Optional<Map<JobVertexID, SortedMap<Instant, ScalingSummary>>>
scalingHistoryOpt =
+ stateStore.getScalingHistory(ctx);
+ if (scalingHistoryOpt.isPresent()) {
+ Map<JobVertexID, SortedMap<Instant, ScalingSummary>>
scalingHistory =
+ trimScalingHistory(
+ clock.instant(), ctx.getConfiguration(),
scalingHistoryOpt.get());
+ if (scalingHistory.isEmpty()) {
+ // All scaling histories are trimmed.
+ needFlush = true;
+ stateStore.removeScalingHistory(ctx);
+ } else if (!scalingHistoryOpt.get().equals(scalingHistory)) {
+ // Some scaling histories are trimmed.
+ needFlush = true;
+ stateStore.storeScalingHistory(ctx, scalingHistory);
+ }
+ }
Review Comment:
This looks strange. Why not simply call `
stateStore.removeScalingHistory(ctx);` all the time? Then we wouldn't need to
touch the trimming logic.
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobAutoScalerImpl.java:
##########
@@ -115,10 +123,39 @@ public void cleanup(KEY jobKey) {
stateStore.removeInfoFromCache(jobKey);
}
- private void clearParallelismOverrides(Context ctx) throws Exception {
- var parallelismOverrides = stateStore.getParallelismOverrides(ctx);
- if (parallelismOverrides.isPresent()) {
+ private void clearStatesAfterAutoscalerDisabled(Context ctx) throws
Exception {
+ var needFlush = false;
+ var parallelismOverridesOpt = stateStore.getParallelismOverrides(ctx);
+ if (parallelismOverridesOpt.isPresent()) {
+ needFlush = true;
stateStore.removeParallelismOverrides(ctx);
+ }
+
+ Optional<SortedMap<Instant, CollectedMetrics>> collectedMetricsOpt =
+ stateStore.getCollectedMetrics(ctx);
+ if (collectedMetricsOpt.isPresent()) {
+ needFlush = true;
+ stateStore.removeCollectedMetrics(ctx);
+ }
+
+ Optional<Map<JobVertexID, SortedMap<Instant, ScalingSummary>>>
scalingHistoryOpt =
Review Comment:
lets be consistent and use:
```
var scalingHistoryOpt = ...
...
var scalingHistory = ...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
