[
https://issues.apache.org/jira/browse/FLINK-40445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40445:
-----------------------------------
Labels: pull-request-available (was: )
> Autoscaler: integration test for Kafka/Pulsar source partition-metric
> detection
> -------------------------------------------------------------------------------
>
> Key: FLINK-40445
> URL: https://issues.apache.org/jira/browse/FLINK-40445
> Project: Flink
> Issue Type: Bug
> Components: Autoscaler, Kubernetes Operator
> Reporter: Purushottam Sinha
> Priority: Minor
> Labels: pull-request-available
>
> Problem
> ScalingMetricCollector derives a source's partition count by parsing
> Kafka/Pulsar metric names (PartitionMetricNameParser, added in FLINK-40443 /
> PR #1187). This path has no automated coverage against a real broker, so a
> regression in partition detection could go unnoticed until it affects source
> autoscaling.
> Evidence
> - flink-autoscaler/.../ScalingMetricCollector.java →
> updateKafkaPulsarSourceNumPartitions uses PartitionMetricNameParser; only
> unit/differential tests exist today.
> - FLINK-40443 was verified manually against Apache Kafka 3.9.1 and Pulsar
> 3.3.1 on minikube, not in CI.
> - Existing precedent: flink-autoscaler-plugin-jdbc uses Testcontainers for
> the JDBC state store.
> Proposed fix
> - Add a Testcontainers-based integration test (separate PR; not blocking
> 1.16.0) running a source job against real Kafka and Pulsar.
> - Assert derived numSourcePartitions for: multi-topic summing, dotted/hyphen
> topic names, source parallelism > 1, and multiple source vertices in one job.
> Acceptance
> - Test asserts detected partition count equals broker ground truth for each
> dimension above.
> - Runs in CI.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
