2. Yes. This is exactly what I mean. Storing the HA information relevant to
a specific component in a single ConfigMap and ensuring that “Get(check the
leader)-and-Update(write back to the ConfigMap)” is a transactional
operation. Since we only store the job graph stateHandler(not the real
data) in
Hey all,
Thanks for your patience. I've got a small repo that reproduces the issue
here: https://github.com/austince/flink-1.10-sql-windowing-error
Not sure what I'm doing wrong but it feels silly.
Thanks so much!
Austin
On Tue, Sep 29, 2020 at 3:48 PM Austin Cawley-Edwards <
austin.caw...@gmai
Hello folks, I'm seeing application failures where our Blobserver is refusing
connections mid application:
2020-09-30 13:56:06,227 INFO [flink-akka.actor.default-dispatcher-18]
org.apache.flink.runtime.taskexecutor.TaskExecutor- Un-registering
task and sending final execution state
Hi,
I’m trying to work through an example with Flink Stateful Functions in Python.
I have a series of custom protobuf messages that I’ve defined but I’m
struggling with how they should be provided to the runtime so the messages in
Kafka can be deserialized. I see,
type: statefun.kafka.io/rou
Thanks for the clarifications Yang Wang.
2. Keeping the HA information relevant for a component (Dispatcher,
JobManager, ResourceManager) in a single ConfigMap sounds good. We should
check that we don't exceed the 1 MB size limit with this approach though.
The Dispatcher's ConfigMap would then con
Thanks for your explanation. It would be fine if only checking leadership &
actually write information is atomic.
Best,
tison.
Yang Wang 于2020年9月30日周三 下午3:57写道：
> Thanks till and tison for your comments.
>
> @Till Rohrmann
> 1. I am afraid we could not do this if we are going to use fabric8
>
Thanks till and tison for your comments.
@Till Rohrmann
1. I am afraid we could not do this if we are going to use fabric8
Kubernetes client SDK for the leader election. The official Kubernetes Java
client[1] also could not support it. Unless we implement a new
LeaderElector in Flink based on the
Hi,
Generally +1 for a native k8s HA service.
For leader election & publish leader information, there was a discussion[1]
pointed out that since these two actions is NOT atomic, there will be always
edge case where a previous leader overwrite leader information, even with
versioned write. Version
And I got this error log
Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException:
Hadoop is not in the classpath/dependencies.
Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could
not find a file system implementation for scheme 'hdfs'. The scheme is
