[ https://issues.apache.org/jira/browse/FLINK-26548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flink Jira Bot updated FLINK-26548: ----------------------------------- Labels: pull-request-available stale-assigned (was: pull-request-available) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issue is assigned but has not received an update in 30 days, so it has been labeled "stale-assigned". If you are still working on the issue, please remove the label and add a comment updating the community on your progress. If this issue is waiting on feedback, please consider this a reminder to the committer/reviewer. Flink is a very active project, and so we appreciate your patience. If you are no longer working on the issue, please unassign yourself so someone else may work on it. > the source parallelism is not set correctly with AdaptiveBatchScheduler > ----------------------------------------------------------------------- > > Key: FLINK-26548 > URL: https://issues.apache.org/jira/browse/FLINK-26548 > Project: Flink > Issue Type: Bug > Components: Connectors / FileSystem, Runtime / Coordination > Affects Versions: 1.15.0 > Reporter: zhouli > Assignee: zhouli > Priority: Critical > Labels: pull-request-available, stale-assigned > Attachments: image-2022-03-09-19-00-18-396.png > > > When running *_org.apache.flink.table.tpcds.TpcdsTestProgram_* with > {_}*AdaptiveBatchScheduler*{_}, I ran into a problem:the num of records sent > by the source operator is always 1, and the parallelism of source operator is > also 1 even I set > *_jobmanager.adaptive-batch-scheduler.default-source-parallelism_* to 8. > !image-2022-03-09-19-00-18-396.png! > After some research, I found that the operator A is not the actual file > reader, it just splits files and assigns splits to downstream tasks for > further processing, and the operator B is the actual file reader task. Here, > the parallelism of operator B is 64, and the records sent by operator A is 1, > this means, operator A assigned all splits to a task of operator B, {*}_the > other 63 tasks of operator B is idle_{*}, it is unreasonable. > In this case, the parallelism of operator B should be > *_jobmanager.adaptive-batch-scheduler.default-source-parallelism_* and the > num of records sent by operator A also should be > {*}_jobmanager.adaptive-batch-scheduler.default-source-parallelism_{*}. > -- This message was sent by Atlassian Jira (v8.20.10#820010)