[
https://issues.apache.org/jira/browse/FLINK-28355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huang Xingbo reassigned FLINK-28355:
------------------------------------
Assignee: Huang Xingbo
> Python Bash e2e tests don't clean-up after they've ran, causing disk space
> issues
> ---------------------------------------------------------------------------------
>
> Key: FLINK-28355
> URL: https://issues.apache.org/jira/browse/FLINK-28355
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Test Infrastructure
> Affects Versions: 1.16.0, 1.15.2, 1.14.6
> Reporter: Martijn Visser
> Assignee: Huang Xingbo
> Priority: Critical
>
> The Bash based E2E tests that are used in Python aren't cleaned-up after
> they've ran. These cause disk space issues further downstream.
> See the CI run from https://github.com/apache/flink/pull/20114 for results,
> for example:
> -- When starting with the Bash e2e tests
> {code:java}
> 08:47:10 ##[group]Top 15 biggest directories in terms of used disk space
> Jul 01 08:47:12 3983560 .
> Jul 01 08:47:12 1266692 ./flink-end-to-end-tests
> Jul 01 08:47:12 624568 ./flink-dist
> Jul 01 08:47:12 624180 ./flink-dist/target
> Jul 01 08:47:12 500076 ./flink-dist/target/flink-1.16-SNAPSHOT-bin
> Jul 01 08:47:12 500072
> ./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT
> Jul 01 08:47:12 460812 ./flink-connectors
> Jul 01 08:47:12 392588 ./.git
> Jul 01 08:47:12 366396 ./.git/objects
> Jul 01 08:47:12 366388 ./.git/objects/pack
> Jul 01 08:47:12 349272 ./flink-table
> Jul 01 08:47:12 335592
> ./.git/objects/pack/pack-38d46915823ebec2bc660fd160e5cfca5bc3e567.pack
> Jul 01 08:47:12 293044
> ./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT/opt
> Jul 01 08:47:12 251272 ./flink-filesystems
> Jul 01 08:47:12 246596
> ./flink-end-to-end-tests/flink-streaming-kinesis-test
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37425&view=logs&j=ef799394-2d67-5ff4-b2e5-410b80c9c0af&t=860bfb5d-81b0-5968-f128-2a8b5362110d&l=664
> -- After completing all Bash bashed e2e tests:
> {code:java}
> 2022-07-01T10:20:17.3594718Z Jul 01 10:20:17 ##[group]Top 15 biggest
> directories in terms of used disk space
> 2022-07-01T10:20:18.7520631Z Jul 01 10:20:18 5425892 .
> 2022-07-01T10:20:18.7521823Z Jul 01 10:20:18 1521472 ./flink-end-to-end-tests
> 2022-07-01T10:20:18.7522566Z Jul 01 10:20:18 1242528 ./flink-python
> 2022-07-01T10:20:18.7523244Z Jul 01 10:20:18 952336 ./flink-python/dev
> 2022-07-01T10:20:18.7524159Z Jul 01 10:20:18 878764
> ./flink-python/dev/.conda
> 2022-07-01T10:20:18.7524870Z Jul 01 10:20:18 834200
> ./flink-python/dev/.conda/lib
> 2022-07-01T10:20:18.7525619Z Jul 01 10:20:18 726528
> ./flink-python/dev/.conda/lib/python3.7
> 2022-07-01T10:20:18.7526397Z Jul 01 10:20:18 683256
> ./flink-python/dev/.conda/lib/python3.7/site-packages
> 2022-07-01T10:20:18.7527101Z Jul 01 10:20:18 624568 ./flink-dist
> 2022-07-01T10:20:18.7527768Z Jul 01 10:20:18 624180 ./flink-dist/target
> 2022-07-01T10:20:18.7528494Z Jul 01 10:20:18 500076
> ./flink-dist/target/flink-1.16-SNAPSHOT-bin
> 2022-07-01T10:20:18.7529298Z Jul 01 10:20:18 500072
> ./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT
> 2022-07-01T10:20:18.7530046Z Jul 01 10:20:18 460812 ./flink-connectors
> 2022-07-01T10:20:18.7530546Z Jul 01 10:20:18 392588 ./.git
> 2022-07-01T10:20:18.7531014Z Jul 01 10:20:18 366396 ./.git/objects
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37425&view=logs&j=ef799394-2d67-5ff4-b2e5-410b80c9c0af&t=860bfb5d-81b0-5968-f128-2a8b5362110d&l=9631
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
