Martijn Visser created FLINK-28355:
--------------------------------------
Summary: Python Bash e2e tests don't clean-up after they've ran,
causing disk space issues
Key: FLINK-28355
URL: https://issues.apache.org/jira/browse/FLINK-28355
Project: Flink
Issue Type: Bug
Components: API / Python, Test Infrastructure
Affects Versions: 1.16.0, 1.15.2, 1.14.6
Reporter: Martijn Visser
The Bash based E2E tests that are used in Python aren't cleaned-up after
they've ran. These cause disk space issues further downstream.
See the CI run from https://github.com/apache/flink/pull/20114 for results, for
example:
-- When starting with the Bash e2e tests
{code:java}
08:47:10 ##[group]Top 15 biggest directories in terms of used disk space
Jul 01 08:47:12 3983560 .
Jul 01 08:47:12 1266692 ./flink-end-to-end-tests
Jul 01 08:47:12 624568 ./flink-dist
Jul 01 08:47:12 624180 ./flink-dist/target
Jul 01 08:47:12 500076 ./flink-dist/target/flink-1.16-SNAPSHOT-bin
Jul 01 08:47:12 500072
./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT
Jul 01 08:47:12 460812 ./flink-connectors
Jul 01 08:47:12 392588 ./.git
Jul 01 08:47:12 366396 ./.git/objects
Jul 01 08:47:12 366388 ./.git/objects/pack
Jul 01 08:47:12 349272 ./flink-table
Jul 01 08:47:12 335592
./.git/objects/pack/pack-38d46915823ebec2bc660fd160e5cfca5bc3e567.pack
Jul 01 08:47:12 293044
./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT/opt
Jul 01 08:47:12 251272 ./flink-filesystems
Jul 01 08:47:12 246596 ./flink-end-to-end-tests/flink-streaming-kinesis-test
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37425&view=logs&j=ef799394-2d67-5ff4-b2e5-410b80c9c0af&t=860bfb5d-81b0-5968-f128-2a8b5362110d&l=664
-- After completing all Bash bashed e2e tests:
{code:java}
2022-07-01T10:20:17.3594718Z Jul 01 10:20:17 ##[group]Top 15 biggest
directories in terms of used disk space
2022-07-01T10:20:18.7520631Z Jul 01 10:20:18 5425892 .
2022-07-01T10:20:18.7521823Z Jul 01 10:20:18 1521472 ./flink-end-to-end-tests
2022-07-01T10:20:18.7522566Z Jul 01 10:20:18 1242528 ./flink-python
2022-07-01T10:20:18.7523244Z Jul 01 10:20:18 952336 ./flink-python/dev
2022-07-01T10:20:18.7524159Z Jul 01 10:20:18 878764
./flink-python/dev/.conda
2022-07-01T10:20:18.7524870Z Jul 01 10:20:18 834200
./flink-python/dev/.conda/lib
2022-07-01T10:20:18.7525619Z Jul 01 10:20:18 726528
./flink-python/dev/.conda/lib/python3.7
2022-07-01T10:20:18.7526397Z Jul 01 10:20:18 683256
./flink-python/dev/.conda/lib/python3.7/site-packages
2022-07-01T10:20:18.7527101Z Jul 01 10:20:18 624568 ./flink-dist
2022-07-01T10:20:18.7527768Z Jul 01 10:20:18 624180 ./flink-dist/target
2022-07-01T10:20:18.7528494Z Jul 01 10:20:18 500076
./flink-dist/target/flink-1.16-SNAPSHOT-bin
2022-07-01T10:20:18.7529298Z Jul 01 10:20:18 500072
./flink-dist/target/flink-1.16-SNAPSHOT-bin/flink-1.16-SNAPSHOT
2022-07-01T10:20:18.7530046Z Jul 01 10:20:18 460812 ./flink-connectors
2022-07-01T10:20:18.7530546Z Jul 01 10:20:18 392588 ./.git
2022-07-01T10:20:18.7531014Z Jul 01 10:20:18 366396 ./.git/objects
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=37425&view=logs&j=ef799394-2d67-5ff4-b2e5-410b80c9c0af&t=860bfb5d-81b0-5968-f128-2a8b5362110d&l=9631
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
