I checked the logs, but no information indicates what happens.
In fact, in the same app, there is another stream, but its kafka
source is low traffic, and I aggregate some field of that source too,
and flink gives correct results continuously.
So I doubt if keyby() could not handle high traffic we
Hi:
Referring to documentation
(https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/operators/index.html)
for ConnectedStreams:
"Connects" two data streams retaining their types. Connect allowing for shared
state between the two streams.DataStream someStream = //...
DataStream
Hi:
Apache Flink documentation
(https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/operators/index.html)
indicates that a reduce function on a KeyedStream as follows:
A "rolling" reduce on a keyed data stream. Combines the current element with
the last reduced value and emit
Thanks Gordon for your explanation.
Mans
On Wednesday, December 20, 2017 2:16 PM, Tzu-Li (Gordon) Tai
wrote:
#yiv6533607487 body{font-family:Helvetica, Arial;font-size:13px;}Hi Mans,
What's the difference between an operator and a function ?
An operator in Flink needs to handle pr
> but soon later, no results produced, and flink seems busy doing
something forever.
Jinhua, don't know if you have checked these things. if not, maybe worth a
look.
have you tried to do a thread dump?
How is the GC pause?
do you see flink restart? check the exception tab in Flink web UI for you
Thanks Eron
I have tried to read an EOF symbol and invoke FlinkKafkaConsumer's cancel
method, it doesn't work. But I invoke the method in a FlatMap operator
which is next to source operator, I guess that is the problem. I will try
your answer, thanks for your suggestion.
--
Sent from: http://a
Flink 1.4 added regex pattern matching for FlinkKafkaConsumer's which is a
neat feature. I would like to use this feature, but I'm wondering how that
impacts the FsStateBackend checkpointing mechanism. Before I would
subscribe to one topic and set a checkpoint path specific to that topic for
exampl
Hi,
When testing Flink 1.4 locally the error below keeps getting thrown. I've
followed the setup by moving the flink-s3-fs-presto.jar from the opt/
folder to the lib/ folder. Is there something additional I need to do?
java.lang.NoClassDefFoundError: Could not initialize class
org.apache.flink.fs
I take time to read some source codes about the keyed stream
windowing, and I make below understanding:
a) the keyed stream would be split and dispatched to downstream tasks
in hash manner, and the hash base is the parallelism of the downstream
operator:
See
org.apache.flink.runtime.state.KeyGro
