Timo Walther created FLINK-9775:
-----------------------------------
Summary: BucketingSink E2E test fails in certain environments
Key: FLINK-9775
URL: https://issues.apache.org/jira/browse/FLINK-9775
Project: Flink
Issue Type: Improvement
Components: Tests
Reporter: Timo Walther
The BucketingSink E2E test currently fails in my environment:
{code}
Java version: 1.8.0_102, vendor: Oracle Corporation, runtime:
/Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.3", arch: "x86_64", family: "mac"
{code}
The output should be:
{code}
Flink dist directory: /Users/twalthr/flink/flink/build-target
TEST_DATA_DIR:
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N
Starting cluster.
Starting standalonesession daemon on host TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
Dispatcher/TaskManagers are not yet up
Waiting for dispatcher REST endpoint to come up...
Dispatcher/TaskManagers are not yet up
Waiting for dispatcher REST endpoint to come up...
Waiting for dispatcher REST endpoint to come up...
Dispatcher REST endpoint is up.
[INFO] 1 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
[INFO] 2 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
[INFO] 3 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
Job (d08473e9f26e7e3bf8c39d355862b9d3) is running.
Killing TM
TaskManager 78808 killed.
Starting TM
[INFO] 3 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
Killing 2 TMs
TaskManager 78508 killed.
TaskManager 79381 killed.
Starting 2 TMs and waiting for successful completion
[INFO] 2 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
[INFO] 3 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
Truncating buckets
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result9/part-1-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000448 secs (211777753 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result1/part-2-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000443 secs (214171366 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result0/part-2-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000842 secs (112664326 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result7/part-3-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000676 secs (140313963 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result5/part-3-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000811 secs (117003939 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result3/part-3-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000394 secs (240732243 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result2/part-3-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000927 secs (102322036 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result8/part-0-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000455 secs (208449658 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result6/part-0-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000488 secs (194396872 bytes/sec)
Truncating
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-14N/out/result4/part-0-0
to 94874
1+0 records in
1+0 records out
94874 bytes transferred in 0.000464 secs (204486330 bytes/sec)
pass Bucketing Sink
Stopping taskexecutor daemon (pid: 80558) on host TTMACBOOK.
Stopping standalonesession daemon (pid: 78216) on host TTMACBOOK.
No zookeeper daemon to stop on host TTMACBOOK.
Skipping taskexecutor daemon (pid: 78508), because it is not running anymore on
TTMACBOOK.
Skipping taskexecutor daemon (pid: 78808), because it is not running anymore on
TTMACBOOK.
Stopping taskexecutor daemon (pid: 79094) on host TTMACBOOK.
Skipping taskexecutor daemon (pid: 79381), because it is not running anymore on
TTMACBOOK.
Stopping taskexecutor daemon (pid: 79973) on host TTMACBOOK.
Stopping taskexecutor daemon (pid: 80272) on host TTMACBOOK.
No taskexecutor daemon to stop on host TTMACBOOK.
No standalonesession daemon to stop on host TTMACBOOK.
No zookeeper daemon to stop on host TTMACBOOK.
All tests PASS
{code}
But is:
{code}
TEST_DATA_DIR:
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-11N
Starting cluster.
[INFO] 1 instance(s) of standalonesession are already running on TTMACBOOK.
Starting standalonesession daemon on host TTMACBOOK.
[INFO] 1 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
Dispatcher REST endpoint is up.
[INFO] 2 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
[INFO] 3 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
[INFO] 4 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
Job () is running.
Killing TM
TaskManager 47360 killed.
Starting TM
[INFO] 4 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
Killing 2 TMs
TaskManager 57913 killed.
TaskManager 58784 killed.
Starting 2 TMs and waiting for successful completion
[INFO] 3 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
[INFO] 4 instance(s) of taskexecutor are already running on TTMACBOOK.
Starting taskexecutor daemon on host TTMACBOOK.
Truncating buckets
Truncating to
dd: no value specified for if
usage: rm [-f | -i] [-dPRrvW] file ...
unlink file
usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory
./test_streaming_bucketing.sh: line 93:
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-11N/complete_result:
No such file or directory
sort: No such file or directory
FAIL Bucketing Sink: Output hash mismatch. Got
d41d8cd98f00b204e9800998ecf8427e, expected 01aba5ff77a0ef5e5cf6a727c248bdc3.
head hexdump of actual:
head:
/Users/twalthr/flink/flink/flink-end-to-end-tests/test-scripts/temp-test-directory-11N/complete_result*:
No such file or directory
Stopping taskexecutor daemon (pid: 59368) on host TTMACBOOK.
No standalonesession daemon (pid: 57043) is running anymore on TTMACBOOK.
No zookeeper daemon to stop on host TTMACBOOK.
Skipping taskexecutor daemon (pid: 47360), because it is not running anymore on
TTMACBOOK.
Stopping taskexecutor daemon (pid: 57337) on host TTMACBOOK.
Stopping taskexecutor daemon (pid: 57627) on host TTMACBOOK.
Skipping taskexecutor daemon (pid: 57913), because it is not running anymore on
TTMACBOOK.
Stopping taskexecutor daemon (pid: 58200) on host TTMACBOOK.
Skipping taskexecutor daemon (pid: 58784), because it is not running anymore on
TTMACBOOK.
Stopping taskexecutor daemon (pid: 59082) on host TTMACBOOK.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
