The reason I posted this now is that we need to think about the API and
windowing before proceeding with the PRs of Gabor (inverse reduce) and
Gyula (removal of "aggregate" functions on DataStream).
For the windowing, I think that the current model does not work for
out-of-order processing. Theref
May bad, (4) is clear now.
On Mon, Jun 22, 2015 at 8:59 PM, Aljoscha Krettek
wrote:
> Marton referred to the KafkaSink in 4). For sources the job will keep
> running by reading from a different broker.
>
> On Mon, 22 Jun 2015 at 18:45 Stephan Ewen wrote:
>
> > I would like to consolidate those
For the windowing designs, we should also have in mind what requirements we
have on the way we keep/store the elements (in external stores, Flink
managed memory, ...)
On Tue, Jun 23, 2015 at 9:55 AM, Aljoscha Krettek
wrote:
> The reason I posted this now is that we need to think about the API an
Hi,
a recent build failed for me, due to an AssertionError:
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.479 sec <<<
> FAILURE! - in org.apache.flink.test.javaApiOperators.SumMinMaxITCase
> testGroupedAggregate[Execution mode =
> CLUSTER](org.apache.flink.test.javaApiOper
Hey
I think we should not block PRs unnecessarily if your suggested changes
might touch them at some point.
Also I still think we should not put everything in the Datastream because
it will be a huge mess.
Also we need to agree on the out of order processing, whether we want it
the way you propo
+1
2015-06-22 17:44 GMT+02:00 Stephan Ewen :
> +1
>
> On Fri, Jun 19, 2015 at 10:48 AM, Matthias J. Sax <
> mj...@informatik.hu-berlin.de> wrote:
>
> > +1
> >
> > On 06/19/2015 10:35 AM, Ufuk Celebi wrote:
> > >
> > > On 19 Jun 2015, at 10:29, Maximilian Michels wrote:
> > >
> > >> Bringing this
That would help to get around many cases, it still leaves some open, like
forgetting to create a sink after transformations after a collect() call.
Would probably be a good improvement over the status quo, though...
On Mon, Jun 22, 2015 at 9:37 PM, Alexander Alexandrov <
alexander.s.alexand...@gm
Márton Balassi created FLINK-2263:
-
Summary: ExecutionGraph uses Thread.sleep to delay execution
retries
Key: FLINK-2263
URL: https://issues.apache.org/jira/browse/FLINK-2263
Project: Flink
I agree that there should be multiple alternatives the user(!) can
choose from. Partial out-of-order processing works for many/most
aggregates. However, if you consider Event-Pattern-Matching, global
ordering in necessary (even if the performance penalty might be high).
I would also keep "system-t
Hi!
On Travis, there seems to be the weird issue that the file writing in the
tests is not completely reliable. Sometimes the test results are read from
incomplete/stale files.
I lazily try to migrate failed tests to using "collect()". This seems
reliable on Travis.
Cheers,
Stephan
On Tue, Jun
@Stephan I understand your concerns that the user might wonder that nothing
happens when executing. However, in this case a warning will provide a hint
to the user that he didn't define any sinks. In the case where he
immediately calls execute() after an eager execution, the program is
actually exe
What I like a lot about Aljoscha's proposed design is that we need no
different code for "system time" vs. "event time". It only differs in where
the timestamps are assigned.
The OOP approach also gives you the semantics of total ordering without
imposing merges on the streams.
On Tue, Jun 23, 20
Shall I provide a PR to fix this test using .collect()? JIRA?
-Matthias
On 06/23/2015 12:22 PM, Stephan Ewen wrote:
> Hi!
>
> On Travis, there seems to be the weird issue that the file writing in the
> tests is not completely reliable. Sometimes the test results are read from
> incomplete/stale
Dear Flink community,
The voting time is over. The vote has passed!
The following binding votes were cast:
+1 votes:
Aljoscha Krettek
Ufuk Celebi
Robert Metzger
Till Rohrmann
==> +4
no -1s or 0s
Thank you everybody for verifying the release candidate. I'll release the
files to the Maven reposi
Hi everyone,
With the vote now being passed, we should think about the release
announcement. I've created a document which basically contains the
milestone release announcement with a few changes.
https://docs.google.com/document/d/1s5H0q961Ucvu6fR56RleY4oN4YFqmeGkN77XPPsco3k/edit#heading=h.5fiop
Yes, please go ahead.
On Tue, Jun 23, 2015 at 1:38 PM, Matthias J. Sax <
mj...@informatik.hu-berlin.de> wrote:
> Shall I provide a PR to fix this test using .collect()? JIRA?
>
> -Matthias
>
> On 06/23/2015 12:22 PM, Stephan Ewen wrote:
> > Hi!
> >
> > On Travis, there seems to be the weird issue
Thanks, Max. I would like to present the streaming stuff a bit differently,
updating it.
On Tue, Jun 23, 2015 at 1:59 PM, Maximilian Michels wrote:
> Hi everyone,
>
> With the vote now being passed, we should think about the release
> announcement. I've created a document which basically contain
Is the static code analysis thing worth it to write a paragraph about it
or is better located in the "More Improvements and Fixes" section?
On 23.06.2015 14:13, Márton Balassi wrote:
Thanks, Max. I would like to present the streaming stuff a bit differently,
updating it.
On Tue, Jun 23, 2015 a
+1
On Tue, Jun 23, 2015 at 11:31 AM, Fabian Hueske wrote:
> +1
>
> 2015-06-22 17:44 GMT+02:00 Stephan Ewen :
>
> > +1
> >
> > On Fri, Jun 19, 2015 at 10:48 AM, Matthias J. Sax <
> > mj...@informatik.hu-berlin.de> wrote:
> >
> > > +1
> > >
> > > On 06/19/2015 10:35 AM, Ufuk Celebi wrote:
> > >
I would add a separate paragraph about it!
On Tue, Jun 23, 2015 at 2:21 PM, Timo Walther wrote:
> Is the static code analysis thing worth it to write a paragraph about it
> or is better located in the "More Improvements and Fixes" section?
>
>
> On 23.06.2015 14:13, Márton Balassi wrote:
>
>> Th
I also like a separate paragraph about it :-)
On Tue, Jun 23, 2015 at 3:20 PM, Robert Metzger wrote:
> I would add a separate paragraph about it!
>
> On Tue, Jun 23, 2015 at 2:21 PM, Timo Walther wrote:
>
> > Is the static code analysis thing worth it to write a paragraph about it
> > or is bet
I think you forgot Henry's vote.
So the result is:
+1 votes:
Aljoscha Krettek
Ufuk Celebi
Robert Metzger
Till Rohrmann
Henry Saputra
==> +5
no -1s or 0s
On Tue, Jun 23, 2015 at 1:57 PM, Maximilian Michels wrote:
> Dear Flink community,
>
> The voting time is over. The vote has passed!
>
> Th
Ah yes, thanks for correcting, Stephan. Sorry for overlooking you, Henry.
On Tue, Jun 23, 2015 at 3:31 PM, Stephan Ewen wrote:
> I think you forgot Henry's vote.
>
> So the result is:
>
> +1 votes:
> Aljoscha Krettek
> Ufuk Celebi
> Robert Metzger
> Till Rohrmann
> Henry Saputra
>
> ==> +5
> no
Vasia Kalavri created FLINK-2264:
Summary: Migrate integration tests for Gelly
Key: FLINK-2264
URL: https://issues.apache.org/jira/browse/FLINK-2264
Project: Flink
Issue Type: Sub-task
Hello fellow Flinksters,
I currently work on implementing Stale Synchronous Parallel iterations
from the current bulk iterations. I have replacement classes for
IterationHeadPactTask, IterationSynchronizationTask and corresponding
event handlers to do the job. Among the generated events, I have
Cl
I also don't like big changes but sometimes they are necessary. The reason
why I'm so adamant about out-of-order processing is that out-of-order
elements are not some exception that occurs once in a while; they occur
constantly in a distributed system. For example, in this:
https://gist.github.com/
Hello everyone,
I am a new contributor (well soon to be!) to this project and I am just
getting to know the ropes around the framework.
I love what flink is about which is what makes me want to contribute to
it.
I have read the documentation and will still re-read it a couple more
times. I
Out of order is ubiquitous in the real-world. Typically, what happens is
that businesses will declare a maximum allowable delay for delayed
transactions and will commit to results when that delay is reached.
Transactions that arrive later than this cutoff are collected specially as
corrections whi
Hey Tran Nam-Luc,
You don't have to register with a serializer. Can you share the event code?
I will look into it asap.
The runtime is buffer oriented and events arrive as buffers before they are
deserialized. That's why you see the getNextBuffer call in the stack trace.
– Ufuk
On Tuesday, June
Hey nuno! Welcome to the Flink community. :) The points you mentioned sound
very reasonable. There is also a how to contribute guide and a coding
guidelines document on the web page you can check out. Is there a specific
starter issue you are interested in? Then it will be easier to give
pointers t
30 matches
Mail list logo
