using glusterfs
>> volume mount (thus file access protocol file:///) was working fine till
>> 1.4.2 for us. So we like to understand where the breakage happened in
>> 1.5.0.
>>
>> Can you please mention me the relevant source code files related to
>> rocksdb “custom
db “custom file path” parsing logic? We would be interested to
> investigate this.
>
>
>
> I also observed below in the log –
>
>
>
> Config uses deprecated configuration key
> 'state.backend.rocksdb.checkpointdir' instead of proper key
> 'state.backend.rocksdb.localdir'
>
ration key
'state.backend.rocksdb.checkpointdir' instead of proper key
'state.backend.rocksdb.localdir'
Regards,
Shaswata
*From:*Chesnay Schepler [mailto:ches...@apache.org]
*Sent:* Tuesday, July 03, 2018 5:52 PM
*To:* Data Engineer
*Cc:* user@flink.apache.org
*Subject:* Re: Checkpointing in Fli
:*Chesnay Schepler [mailto:ches...@apache.org]
*Sent:* Tuesday, July 03, 2018 5:52 PM
*To:* Data Engineer
*Cc:* user@flink.apache.org
*Subject:* Re: Checkpointing in Flink 1.5.0
The code appears to be working fine.
This may happen because you're using a GlusterFS volume.
The RocksDBStateBackend uses
Schepler [mailto:ches...@apache.org]
Sent: Tuesday, July 03, 2018 5:52 PM
To: Data Engineer
Cc: user@flink.apache.org
Subject: Re: Checkpointing in Flink 1.5.0
The code appears to be working fine.
This may happen because you're using a GlusterFS volume.
The RocksDBStateBackend uses java Files
The code appears to be working fine.
This may happen because you're using a GlusterFS volume.
The RocksDBStateBackend uses java Files internally (NOT nio Paths),
which AFAIK only work properly against the plain local file-system.
The GlusterFS nio FIleSystem implementation also explicitly
Thanks. Looks like RocksDBStateBackend.setDbStoragePaths has some custom
file path parsing logic, will probe it a bit to see what the issue is.
On 03.07.2018 13:45, Data Engineer wrote:
2018-07-03 11:30:35,703 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
2018-07-03 11:30:35,703 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
2018-07-03 11:30:35,705 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
Starting
Doesn't sound like intended behavior, can you give us the stacktrace?
On 03.07.2018 13:17, Data Engineer wrote:
The Flink documentation says that we need to specify the filesystem
type (file://, hdfs://) when configuring the rocksdb backend dir.
The Flink documentation says that we need to specify the filesystem type
(file://, hdfs://) when configuring the rocksdb backend dir.
https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/state/state_backends.html#the-rocksdbstatebackend
But when I do this, I get an error on job
10 matches
Mail list logo
