[
https://issues.apache.org/jira/browse/FLINK-39959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gyula Fora updated FLINK-39959:
-------------------------------
Fix Version/s: kubernetes-operator-1.16.0
> Improve the autoscaler context structure and propagation
> --------------------------------------------------------
>
> Key: FLINK-39959
> URL: https://issues.apache.org/jira/browse/FLINK-39959
> Project: Flink
> Issue Type: Improvement
> Components: Autoscaler, Kubernetes Operator
> Affects Versions: kubernetes-operator-1.15.0
> Reporter: Dennis-Mircea Ciupitu
> Priority: Major
> Fix For: kubernetes-operator-1.16.0
>
>
> h1. Background
> As part of the recent extensibility work, several plugin SPIs were introduced
> into the autoscaler: the custom metric evaluator (FLIP-514), the parallelism
> alignment mode (FLIP-586), and the scaling executor plugin (FLIP-575). Each
> SPI defined its own nested {{Context}} type to carry the inputs a plugin
> needs at its phase of the scaling process.
> h1. Problem
> Each plugin context independently wraps {{JobAutoScalerContext}} and
> re-exposes or re-derives overlapping data such as the configuration, the
> evaluated metrics, and the job topology. This has several downsides.
> * Multiple parallel context types carry duplicated fields and getters.
> * Configuration is exposed in more than one place. The autoscaler context
> already carries the job configuration, yet the plugin contexts add their own
> configuration views, which is confusing for plugin authors and easy to drift
> out of sync.
> * There is no single authoritative context flowing through the autoscaler
> backbone. Each stage (collector, evaluator, executor) and each plugin builds
> its own view, so propagation is ad hoc.
> h1. Proposed improvement
> Define a single main autoscaler context as the backbone, and have the plugin
> contexts extend or enrich it instead of wrapping and duplicating it.
> * One canonical context, built on {{JobAutoScalerContext}}, is created once
> per autoscaling cycle and threaded through the backbone components.
> * Each plugin phase enriches that context with only the additional,
> phase-specific information it needs, instead of re-declaring shared fields.
> * Per-instance and per-plugin configuration is layered on the single context
> rather than duplicated next to the job configuration.
> h2. Design notes
> The operator already uses a compositional enriched-context pattern, where
> {{FlinkResourceContext}} wraps the JOSDK {{Context}}. The autoscaler context
> model should adopt a consistent single-context approach. The exact mechanism
> (inheritance versus layered composition), and the way per-instance
> configuration is merged and exposed, should be settled during design, while
> keeping the concrete subtype such as {{KubernetesJobAutoScalerContext}}
> accessible to plugins.
> h1. Benefits
> * Removes duplicated context fields and getters across the plugin SPIs.
> * Gives plugin authors a single, consistent configuration surface.
> * Makes context propagation across the autoscaler backbone explicit and
> uniform, from the collector through the evaluator and executor to the plugins.
> * Lowers maintenance cost. New context data is added once and becomes
> available everywhere.
> h1. Scope
> This is a follow-up improvement agreed during the FLIP-575
> ({{ScalingExecutorPlugin}}) review, deferred to avoid blocking that PR. It
> consolidates the existing context types and does not change plugin behavior
> or introduce new SPIs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
