Hi Manish,
Appreciate the way you presented Apache Flink. While it's like an 'Intro' to
beginners, I would really encourage you to highlight/present some of the
groundbreaking features that flink offers towards stream processing like -
-> Explicit handling of time with it's notion of 'Event time
Hi Till,
Excellent - I'll check out the current snapshot version! Thank you for
taking the time to look into this.
Regards,
David
On Tue, Nov 8, 2016 at 3:25 PM, Till Rohrmann wrote:
> Hi David,
>
> sorry for my late reply. I just found time to look into the problem. You
> were right with you
Hi Dominik,
It logs to taskmanager log. But if you are using localruntime via ide or
localcluster it is not logging them. If you start jobmanager and
taskmanager separately then you can see logs.
On Nov 11, 2016 23:02, "Dominik Safaric" wrote:
> If taskmanager.debug.memory.startLogThread is set
I am running Apache Flink 1.1.3 - Hadoop version 1.2.1 with the NiFi connector.
When I run a program with a single NiFi Source, I receive the following Stack
trace in the logs:
2016-11-11 19:28:25,661 WARN org.apache.flink.client.CliFrontend
- Unable to locate custom CLI class
Yes, I mean aborting the checkpoint alignment directly from an operator.
This is because I am introducing an operator that performs some asynchronous
stuff that also involves side effects on its internal state.
I wanted to abort a checkpoint directly from that operator if a barrier comes
in while
If taskmanager.debug.memory.startLogThread is set to true, where does the task
manager output the logs to?
Unfortunately I couldn’t find this information in the documentation, hence the
question.
Thanks in advance,
Dominik
Hi,
I have a streaming application based on event time. When I issue a
watermark that will close more than 1 window (and trigger their
processment), I can see that windows are computed sequentially (at least
using local machine) and that the computing order is not defined. Can I
change this behavi
What do you mean exactly with aborting a checkpoint? Continuing the
processing despite failed checkpoints?
You can have a look at these recent changes, they cleanly abort checkpoint
alignment in certain conditions:
https://issues.apache.org/jira/browse/FLINK-4976
https://github.com/apache/flink/p
Hi,
I think the jar files in the lib folder are checked first so shipping the
WindowOperator with the job should not work.
Cheers,
Aljoscha
On Thu, 10 Nov 2016 at 17:48 Konstantin Knauf
wrote:
> Hi Aljoscha,
>
> alright, for the time being I have modified the WindowOperator and built
> flink-st
Hi Guys,
I have contributed a blog post on Apache Flink introduction, architecture
and execution model. Please provide the feedback:
http://data-flair.training/blogs/apache-flink-comprehensive-guide-tutorial-for-beginners/
I hope I have covered all the concepts correctly.
-Malini
Use-case: I am trying to see how to use flink with s3, where we use our own
client libraries or things like AWS firehose to put data into S3, then
process it in batch using flink. This clients are putting data into S3
with out HDFS - Aka we aren't using HDFS on top of S3.
Most of what I can find
Hi everybody, I am using Flink v1.1.2
is it possible to programmatically abort a snapshot from the method
public StreamTaskState snapshotOperatorState(long checkpointId, long
timestamp)
In an operator?
Thank you!
Lorenzo
Hello,
I'm struggling to understand the following behaviour of the
`WindowOperator` and would appreciate some insight from experts:
In particular I'm thinking about the following hypothetical data flow:
input.keyBy(..)
.window(TumblingEventTimeWindows.of(..))
.apply(..)
...
Hi All,
Does anyone know if Flink can read and write Avro schema to Kafka?
Thanks
daviD
Hello,
I'm new to Flink and I need some advicees regarding the best approach to do
the following:
- read some items from a Kafka topic
- on Flink stream side, after some simple filtering steps, group these items
in batches by flink processing time.
- insert the items in a PostgreSql database using
Hi Mich,
at the moment there is not much support handle such data driven exceptions
(badly formatted data, late data, ...).
However, there is a proposal to improve this: FLIP-13 [1]. So it is work in
progress.
It would be very helpful if you could check if the proposal would address
your use case
16 matches
Mail list logo
