[
https://issues.apache.org/jira/browse/FLINK-39494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuepeng Pan updated FLINK-39494:
--------------------------------
Description:
h1. 1 Prepare to build Flink binaries and test jobs
1.1. Environment requirements:
Standard JDK and Maven setups for building Flink from source will suffice.
h3.
1.2. Build flink binary file
{code:java}
# cd into you target directory
git clone https://github.com/apache/flink # $repo_path, the placeholder name
will be used later.
cd flink
git checkout ${target_tag or target_branch}
mvn clean package -DskipTests -T 4C {code}
h3.
1.3. Build an example job
{code:java}
mvn archetype:generate \
-DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-walkthrough-datastream-java \
-DarchetypeVersion=2.2.0 \
-DgroupId=frauddetection \
-DartifactId=frauddetection \
-Dversion=0.1 \
-Dpackage=frauddetection \
-DinteractiveMode=false
cd frauddetection
mvn clean package -DskipTests {code}
h1. 2 Testing cases for streaming jobs
h2. 2.1 For disabled adaptive scheduler and disabled max rescale history
2.1.1. run shell command:
{code:java}
cd $repo_path/build-target
./bin/start-cluster.sh {code}
2.1.2. Visit [http://localhost:8081/]
2.1.3. Use flink WEB UI to submit the frauddetection jar as a job.
2.1.4. Check if the UI shown logic is correct
The testing case will be expected if the there isn't Rescale tab page in UI
like follows,
If not , this test would be tagged failed.
!image-2026-04-29-22-45-23-546.png!
2.1.5. run shell command to reset the env
{code:java}
cd $repo_path/build-target
./bin/stop-cluster.sh {code}
h2. 2.2 For disabled adaptive scheduler and enabled rescale history
2.2.1. Set the rescale history option in
`$repo_path/build-target/conf/config.yaml` as follows:
{code:java}
web:
adaptive-scheduler:
rescale-history:
size: 2 {code}
2.2.2. run shell command:
{code:java}
cd $repo_path/build-target
./bin/start-cluster.sh {code}
2.2.3. then visit http://localhost:8081/
2.2.4. Use flink WEB UI to submit the frauddetection jar as a job.
2.2.5. Check that the testing case will be expected if the there isn't Rescales
tab page in UI like follows:
after that,
The testing case will be expected if the there isn't Rescale tab page in UI
like follows, If not , this test would be tagged failed.
!image-2026-04-29-22-49-25-250.png!
2.2.6. Delete the following changed config content in
`$repo_path/build-target/conf/config.yaml`
{code:java}
web:
adaptive-scheduler:
rescale-history:
size: 2 {code}
2.2.7 run shell command to stop flink cluster:
{code:java}
cd $repo_path/build-target
./bin/stop-cluster.sh {code}
h2. 2.3 For enabled adaptive scheduler and disabled rescale history
2.3.1. Add the rescale history option into
`$repo_path/build-target/conf/config.yaml` as follows:
{code:java}
jobmanager:
scheduler: adaptive {code}
2.3.2. run shell command:
{code:java}
cd $repo_path/build-target
./bin/start-cluster.sh {code}
2.3.3. Visit http://localhost:8081/
2.3.4. Use flink WEB UI to submit the frauddetection jar as a job.
2.3.5. Check UI
The testing case will be expected if the there isn't Rescale tab page in UI
like follows, if not , this test would be tagged failed：
- For Overview page
!image-2026-04-29-22-51-54-495.png!
- For History page
!image-2026-04-29-22-52-08-177.png!
- For Summary page
!image-2026-04-29-22-52-20-391.png!
- For Configuration page
Only the Configuration sub-page is not empty.
!image-2026-04-29-22-52-36-259.png!
2.3.6. Delete the config changed content in
`$repo_path/build-target/conf/config.yaml`
{code:java}
jobmanager:
scheduler: adaptive {code}
2.3.7. run shell command to stop cluster
{code:java}
cd $repo_path/build-target
./bin/stop-cluster.sh {code}
h2. 2.4 For enabled adaptive scheduler and enabled rescale history
2.4.1. Add the rescale history option in
`$repo_path/build-target/conf/config.yaml` as follows:
{code:java}
jobmanager:
scheduler: adaptive
web:
adaptive-scheduler:
rescale-history:
size: 3
taskmanager:
numberOfTaskSlots: 8 {code}
2.4.2. run shell command:
{code:java}
cd $repo_path/build-target
./bin/start-cluster.sh {code}
2.4.3. Visit http://localhost:8081/
2.4.4. Use flink WEB UI to submit the frauddetection jar as a job.
2.4.5. Manually adjust the job parallelism to trigger a rescale.
After the job is successfully scheduled for the first time, as shown in the
figure below, manually adjust the job parallelism twice via the subsequent
prompts to actively generate two new rescales.
!image-2026-04-29-22-59-13-102.png!
Please note that the interval between the two manual rescale triggers must be
very short. As shown in the figure below, immediately after sending the first
request to increase parallelism (a notification will appear in the upper-right
corner), initiate the second parallelism increase without delay. Then, verify
whether an 'ignored' type rescale is generated. Since the minimum interval
between two manual rescales can be difficult to gauge, it is recommended to
repeat the attempt several times until an 'ignored' rescale record appears.
This is primarily to test whether the generation of 'Ignored' type Rescales is
functioning correctly
!image-2026-04-29-22-59-27-876.png!
!image-2026-04-29-22-59-36-834.png!
2.4.6. Check the related UI pages
- Overview
!image-2026-04-29-23-04-24-498.png!
- Overview-View details and table header hints.
!image-2026-04-29-23-06-40-867.png!
- History
-- for brief rescale view
!image-2026-04-29-23-06-52-568.png!
-- View details and table header hints.
!image-2026-04-29-23-07-21-359.png!
-- for detailed rescale
!image-2026-04-29-23-07-44-964.png!
!image-2026-04-29-23-08-00-925.png!
-- View details and table header hints.
!image-2026-04-29-23-09-05-876.png!
- Summary
!image-2026-04-29-23-09-21-305.png!
!image-2026-04-29-23-09-31-426.png!
- Configuration
!image-2026-04-29-23-09-41-317.png!
2.4.7. Manually trigger another rescale and verify whether the History page
retains the most recent rescale records as specified by the
`web.adaptive-scheduler.rescale-history.size` configuration value.
!image-2026-04-29-23-09-53-011.png!
!image-2026-04-29-23-10-02-825.png!
2.4.8. Run shell command to stop cluster:
```shell
cd $repo_path/build-target
./bin/stop-cluster.sh
```
h1. 3 Please record anything unexpected by comments in the current ticket.
Thank you.
was:
h1. Prepare to build Flink binaries and test jobs
h3. 1. Environment requirements:
Standard JDK and Maven setups for building Flink from source will suffice.
h3.
2. Build flink binary file
{code:java}
# cd into you target directory
git clone https://github.com/apache/flink # $repo_path, the placeholder name
will be used later.
cd flink
git checkout ${target_tag or target_branch}
mvn clean package -DskipTests -T 4C {code}
h3.
3. Build an example job
{code:java}
mvn archetype:generate \
-DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-walkthrough-datastream-java \
-DarchetypeVersion=2.2.0 \
-DgroupId=frauddetection \
-DartifactId=frauddetection \
-Dversion=0.1 \
-Dpackage=frauddetection \
-DinteractiveMode=false
cd frauddetection
mvn clean package -DskipTests {code}
h1. Testing cases for streaming jobs
h2. For disabled adaptive scheduler and disabled max rescale history
1. run shell command:
{code:java}
cd $repo_path/build-target
./bin/start-cluster.sh {code}
2. Visit http://localhost:8081/
3. Use flink WEB UI to submit the frauddetection jar as a job.
4. Check if the UI shown logic is correct
The testing case will be expected if the there isn't Rescale tab page in UI
like follows,
If not , this test would be tagged failed.
!image-2026-04-29-22-45-23-546.png!
5. run shell command to reset the env
{code:java}
cd $repo_path/build-target
./bin/stop-cluster.sh {code}
> Release Testing: Verify FLIP-487: Show history of rescales in Web UI for
> AdaptiveScheduler
> ------------------------------------------------------------------------------------------
>
> Key: FLINK-39494
> URL: https://issues.apache.org/jira/browse/FLINK-39494
> Project: Flink
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 2.3.0
> Reporter: Yuepeng Pan
> Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: image-2026-04-29-22-45-23-546.png,
> image-2026-04-29-22-49-25-250.png, image-2026-04-29-22-51-54-495.png,
> image-2026-04-29-22-52-08-177.png, image-2026-04-29-22-52-20-391.png,
> image-2026-04-29-22-52-36-259.png, image-2026-04-29-22-59-13-102.png,
> image-2026-04-29-22-59-27-876.png, image-2026-04-29-22-59-36-834.png,
> image-2026-04-29-23-00-12-453.png, image-2026-04-29-23-00-23-748.png,
> image-2026-04-29-23-01-35-076.png, image-2026-04-29-23-04-24-498.png,
> image-2026-04-29-23-04-33-591.png, image-2026-04-29-23-06-40-867.png,
> image-2026-04-29-23-06-52-568.png, image-2026-04-29-23-07-21-359.png,
> image-2026-04-29-23-07-44-964.png, image-2026-04-29-23-08-00-925.png,
> image-2026-04-29-23-09-05-876.png, image-2026-04-29-23-09-21-305.png,
> image-2026-04-29-23-09-31-426.png, image-2026-04-29-23-09-41-317.png,
> image-2026-04-29-23-09-53-011.png, image-2026-04-29-23-10-02-825.png
>
>
> h1. 1 Prepare to build Flink binaries and test jobs
>
> 1.1. Environment requirements:
> Standard JDK and Maven setups for building Flink from source will suffice.
> h3.
> 1.2. Build flink binary file
> {code:java}
> # cd into you target directory
> git clone https://github.com/apache/flink # $repo_path, the placeholder name
> will be used later.
> cd flink
> git checkout ${target_tag or target_branch}
> mvn clean package -DskipTests -T 4C {code}
> h3.
> 1.3. Build an example job
> {code:java}
> mvn archetype:generate \
> -DarchetypeGroupId=org.apache.flink \
> -DarchetypeArtifactId=flink-walkthrough-datastream-java \
> -DarchetypeVersion=2.2.0 \
> -DgroupId=frauddetection \
> -DartifactId=frauddetection \
> -Dversion=0.1 \
> -Dpackage=frauddetection \
> -DinteractiveMode=false
> cd frauddetection
> mvn clean package -DskipTests {code}
> h1. 2 Testing cases for streaming jobs
> h2. 2.1 For disabled adaptive scheduler and disabled max rescale history
> 2.1.1. run shell command:
>
> {code:java}
> cd $repo_path/build-target
> ./bin/start-cluster.sh {code}
> 2.1.2. Visit [http://localhost:8081/]
> 2.1.3. Use flink WEB UI to submit the frauddetection jar as a job.
> 2.1.4. Check if the UI shown logic is correct
> The testing case will be expected if the there isn't Rescale tab page in UI
> like follows,
> If not , this test would be tagged failed.
>
> !image-2026-04-29-22-45-23-546.png!
> 2.1.5. run shell command to reset the env
> {code:java}
> cd $repo_path/build-target
> ./bin/stop-cluster.sh {code}
>
>
> h2. 2.2 For disabled adaptive scheduler and enabled rescale history
>
> 2.2.1. Set the rescale history option in
> `$repo_path/build-target/conf/config.yaml` as follows:
>
> {code:java}
> web:
> adaptive-scheduler:
> rescale-history:
> size: 2 {code}
> 2.2.2. run shell command:
>
> {code:java}
> cd $repo_path/build-target
> ./bin/start-cluster.sh {code}
> 2.2.3. then visit http://localhost:8081/
> 2.2.4. Use flink WEB UI to submit the frauddetection jar as a job.
> 2.2.5. Check that the testing case will be expected if the there isn't
> Rescales tab page in UI like follows:
> after that,
> The testing case will be expected if the there isn't Rescale tab page in UI
> like follows, If not , this test would be tagged failed.
> !image-2026-04-29-22-49-25-250.png!
> 2.2.6. Delete the following changed config content in
> `$repo_path/build-target/conf/config.yaml`
> {code:java}
> web:
> adaptive-scheduler:
> rescale-history:
> size: 2 {code}
> 2.2.7 run shell command to stop flink cluster:
> {code:java}
> cd $repo_path/build-target
> ./bin/stop-cluster.sh {code}
>
>
> h2. 2.3 For enabled adaptive scheduler and disabled rescale history
>
> 2.3.1. Add the rescale history option into
> `$repo_path/build-target/conf/config.yaml` as follows:
>
> {code:java}
> jobmanager:
> scheduler: adaptive {code}
> 2.3.2. run shell command:
>
> {code:java}
> cd $repo_path/build-target
> ./bin/start-cluster.sh {code}
> 2.3.3. Visit http://localhost:8081/
> 2.3.4. Use flink WEB UI to submit the frauddetection jar as a job.
> 2.3.5. Check UI
> The testing case will be expected if the there isn't Rescale tab page in UI
> like follows, if not , this test would be tagged failed：
> - For Overview page
>
> !image-2026-04-29-22-51-54-495.png!
> - For History page
>
> !image-2026-04-29-22-52-08-177.png!
> - For Summary page
>
> !image-2026-04-29-22-52-20-391.png!
> - For Configuration page
> Only the Configuration sub-page is not empty.
> !image-2026-04-29-22-52-36-259.png!
> 2.3.6. Delete the config changed content in
> `$repo_path/build-target/conf/config.yaml`
>
> {code:java}
> jobmanager:
> scheduler: adaptive {code}
> 2.3.7. run shell command to stop cluster
> {code:java}
> cd $repo_path/build-target
> ./bin/stop-cluster.sh {code}
> h2. 2.4 For enabled adaptive scheduler and enabled rescale history
>
> 2.4.1. Add the rescale history option in
> `$repo_path/build-target/conf/config.yaml` as follows:
>
> {code:java}
> jobmanager:
> scheduler: adaptive
> web:
> adaptive-scheduler:
> rescale-history:
> size: 3
> taskmanager:
> numberOfTaskSlots: 8 {code}
> 2.4.2. run shell command:
>
> {code:java}
> cd $repo_path/build-target
> ./bin/start-cluster.sh {code}
> 2.4.3. Visit http://localhost:8081/
> 2.4.4. Use flink WEB UI to submit the frauddetection jar as a job.
> 2.4.5. Manually adjust the job parallelism to trigger a rescale.
> After the job is successfully scheduled for the first time, as shown in the
> figure below, manually adjust the job parallelism twice via the subsequent
> prompts to actively generate two new rescales.
>
> !image-2026-04-29-22-59-13-102.png!
> Please note that the interval between the two manual rescale triggers must be
> very short. As shown in the figure below, immediately after sending the first
> request to increase parallelism (a notification will appear in the
> upper-right corner), initiate the second parallelism increase without delay.
> Then, verify whether an 'ignored' type rescale is generated. Since the
> minimum interval between two manual rescales can be difficult to gauge, it is
> recommended to repeat the attempt several times until an 'ignored' rescale
> record appears. This is primarily to test whether the generation of 'Ignored'
> type Rescales is functioning correctly
>
> !image-2026-04-29-22-59-27-876.png!
> !image-2026-04-29-22-59-36-834.png!
> 2.4.6. Check the related UI pages
> - Overview
> !image-2026-04-29-23-04-24-498.png!
> - Overview-View details and table header hints.
> !image-2026-04-29-23-06-40-867.png!
> - History
> -- for brief rescale view
> !image-2026-04-29-23-06-52-568.png!
> -- View details and table header hints.
>
> !image-2026-04-29-23-07-21-359.png!
> -- for detailed rescale
>
> !image-2026-04-29-23-07-44-964.png!
> !image-2026-04-29-23-08-00-925.png!
> -- View details and table header hints.
>
> !image-2026-04-29-23-09-05-876.png!
> - Summary
> !image-2026-04-29-23-09-21-305.png!
> !image-2026-04-29-23-09-31-426.png!
> - Configuration
>
> !image-2026-04-29-23-09-41-317.png!
> 2.4.7. Manually trigger another rescale and verify whether the History page
> retains the most recent rescale records as specified by the
> `web.adaptive-scheduler.rescale-history.size` configuration value.
> !image-2026-04-29-23-09-53-011.png!
> !image-2026-04-29-23-10-02-825.png!
> 2.4.8. Run shell command to stop cluster:
> ```shell
> cd $repo_path/build-target
> ./bin/stop-cluster.sh
> ```
>
>
> h1. 3 Please record anything unexpected by comments in the current ticket.
> Thank you.
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
