Andrey created FLINK-6647:
-----------------------------
Summary: Fail-fast on invalid RocksDBStateBackend configuration
Key: FLINK-6647
URL: https://issues.apache.org/jira/browse/FLINK-6647
Project: Flink
Issue Type: Bug
Reporter: Andrey
Currently:
* setup "state.backend.rocksdb.checkpointdir=hdfs:///some/base/path/hdfs"
* setup backend: state.backend:
"org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory"
* rocksdb doesn't support hdfs backend so in logs:
{code}
2017-05-19 15:42:33,737 ERROR
org.apache.flink.contrib.streaming.state.RocksDBStateBackend - Local DB files
directory '/some/base/path/hdfs' does not exist and cannot be created.
{code}
* however job continue execution and IOManager temp directory will be picked up
for rocksdb files.
There are several issues with such approach:
* after "ERROR" message printed and before developer fixes configuration, /tmp
directory/partition might run out of disk space.
* if hdfs base path is the same as local path, then no errors in logs and
rocksdb files will be written into an incorrect location. For example:
"hdfs:///home/flink/data" will cause an issue.
Expected:
* validate URI and throw IllegalArgumentException like already implemented in
"RocksDBStateBackend.setDbStoragePaths" method.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
