[jira] [Commented] (FLINK-3684) CEP operator does not forward watermarks properly
[ https://issues.apache.org/jira/browse/FLINK-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219662#comment-15219662 ] ASF GitHub Bot commented on FLINK-3684: --- GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/1842 [FLINK-3684] [cep] Add proper watermark emission to CEP operators Adds watermark emission to CEP stream operators. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tillrohrmann/flink fixCEPWatermarkPropagation Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/1842.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1842 commit 9feb6eac17a0b90cf6f30f06adb4a7b97e2d9382 Author: Till Rohrmann Date: 2016-03-31T09:35:19Z [FLINK-3684] [cep] Add proper watermark emission to CEP operators > CEP operator does not forward watermarks properly > - > > Key: FLINK-3684 > URL: https://issues.apache.org/jira/browse/FLINK-3684 > Project: Flink > Issue Type: Bug > Components: CEP >Affects Versions: 1.0.0 >Reporter: Till Rohrmann >Assignee: Till Rohrmann > Fix For: 1.1.0, 1.0.1 > > > The CEP stream operator don't emit a proper watermark when using event time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-3684) CEP operator does not forward watermarks properly
Till Rohrmann created FLINK-3684: Summary: CEP operator does not forward watermarks properly Key: FLINK-3684 URL: https://issues.apache.org/jira/browse/FLINK-3684 Project: Flink Issue Type: Bug Components: CEP Affects Versions: 1.0.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.1.0, 1.0.1 The CEP stream operator don't emit a proper watermark when using event time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-3684] [cep] Add proper watermark emissi...
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/1842 [FLINK-3684] [cep] Add proper watermark emission to CEP operators Adds watermark emission to CEP stream operators. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tillrohrmann/flink fixCEPWatermarkPropagation Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/1842.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1842 commit 9feb6eac17a0b90cf6f30f06adb4a7b97e2d9382 Author: Till Rohrmann Date: 2016-03-31T09:35:19Z [FLINK-3684] [cep] Add proper watermark emission to CEP operators --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (FLINK-3669) WindowOperator registers a lot of timers at StreamTask
[ https://issues.apache.org/jira/browse/FLINK-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek updated FLINK-3669: Summary: WindowOperator registers a lot of timers at StreamTask (was: WindowOperator registers a log of timers at StreamTask) > WindowOperator registers a lot of timers at StreamTask > -- > > Key: FLINK-3669 > URL: https://issues.apache.org/jira/browse/FLINK-3669 > Project: Flink > Issue Type: Bug > Components: Streaming >Affects Versions: 1.0.1 >Reporter: Aljoscha Krettek >Priority: Blocker > > Right now, the WindowOperator registers a timer at the StreamTask for every > processing-time timer that a Trigger registers. We should combine several > registered trigger timers to only register one low-level timer (timer > coalescing). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (FLINK-2111) Add "stop" signal to cleanly shutdown streaming jobs
[ https://issues.apache.org/jira/browse/FLINK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax closed FLINK-2111. -- > Add "stop" signal to cleanly shutdown streaming jobs > > > Key: FLINK-2111 > URL: https://issues.apache.org/jira/browse/FLINK-2111 > Project: Flink > Issue Type: Improvement > Components: Distributed Runtime, JobManager, Local Runtime, > Streaming, TaskManager, Webfrontend >Reporter: Matthias J. Sax >Assignee: Matthias J. Sax >Priority: Minor > > Currently, streaming jobs can only be stopped using "cancel" command, what is > a "hard" stop with no clean shutdown. > The new introduced "stop" signal, will only affect streaming source tasks > such that the sources can stop emitting data and shutdown cleanly, resulting > in a clean shutdown of the whole streaming job. > This feature is a pre-requirment for > https://issues.apache.org/jira/browse/FLINK-1929 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3659) Allow ConnectedStreams to Be Keyed on Only One Side
[ https://issues.apache.org/jira/browse/FLINK-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219594#comment-15219594 ] ASF GitHub Bot commented on FLINK-3659: --- Github user aljoscha closed the pull request at: https://github.com/apache/flink/pull/1831 > Allow ConnectedStreams to Be Keyed on Only One Side > --- > > Key: FLINK-3659 > URL: https://issues.apache.org/jira/browse/FLINK-3659 > Project: Flink > Issue Type: Improvement > Components: Streaming >Affects Versions: 1.0.0 >Reporter: Aljoscha Krettek >Assignee: Aljoscha Krettek > > Currently, we only allow partitioned state when both inputs are keyed (with > the same key type). I think a valid use case is to have only one side keyed > and have the other side broadcast to publish some updates that are relevant > for all keys. > When relaxing the requirement to have only one side keyed we must still > ensure that the same key is used if both sides are keyed. > [~gyfora] Did you try this with what you're working on? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3659) Allow ConnectedStreams to Be Keyed on Only One Side
[ https://issues.apache.org/jira/browse/FLINK-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219592#comment-15219592 ] ASF GitHub Bot commented on FLINK-3659: --- Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/1831#issuecomment-203821406 Yes, I'll try and come up with Ideas in that direction then. :+1: > Allow ConnectedStreams to Be Keyed on Only One Side > --- > > Key: FLINK-3659 > URL: https://issues.apache.org/jira/browse/FLINK-3659 > Project: Flink > Issue Type: Improvement > Components: Streaming >Affects Versions: 1.0.0 >Reporter: Aljoscha Krettek >Assignee: Aljoscha Krettek > > Currently, we only allow partitioned state when both inputs are keyed (with > the same key type). I think a valid use case is to have only one side keyed > and have the other side broadcast to publish some updates that are relevant > for all keys. > When relaxing the requirement to have only one side keyed we must still > ensure that the same key is used if both sides are keyed. > [~gyfora] Did you try this with what you're working on? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-3659] Allow ConnectedStreams to Be Keye...
Github user aljoscha closed the pull request at: https://github.com/apache/flink/pull/1831 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-3659] Allow ConnectedStreams to Be Keye...
Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/1831#issuecomment-203821429 Closing this for now... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-3659) Allow ConnectedStreams to Be Keyed on Only One Side
[ https://issues.apache.org/jira/browse/FLINK-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219593#comment-15219593 ] ASF GitHub Bot commented on FLINK-3659: --- Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/1831#issuecomment-203821429 Closing this for now... > Allow ConnectedStreams to Be Keyed on Only One Side > --- > > Key: FLINK-3659 > URL: https://issues.apache.org/jira/browse/FLINK-3659 > Project: Flink > Issue Type: Improvement > Components: Streaming >Affects Versions: 1.0.0 >Reporter: Aljoscha Krettek >Assignee: Aljoscha Krettek > > Currently, we only allow partitioned state when both inputs are keyed (with > the same key type). I think a valid use case is to have only one side keyed > and have the other side broadcast to publish some updates that are relevant > for all keys. > When relaxing the requirement to have only one side keyed we must still > ensure that the same key is used if both sides are keyed. > [~gyfora] Did you try this with what you're working on? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-3659] Allow ConnectedStreams to Be Keye...
Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/1831#issuecomment-203821406 Yes, I'll try and come up with Ideas in that direction then. :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (FLINK-3682) CEP operator does not set the processing timestamp correctly
[
https://issues.apache.org/jira/browse/FLINK-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-3682.
Resolution: Fixed
Fixed via bdf55b90d58e3d7cb1f9b4ee0c2780bcca1c8f8a
> CEP operator does not set the processing timestamp correctly
>
>
> Key: FLINK-3682
> URL: https://issues.apache.org/jira/browse/FLINK-3682
> Project: Flink
> Issue Type: Bug
> Components: CEP
>Affects Versions: 1.0.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Fix For: 1.1.0, 1.0.1
>
>
> In the wake of reworking the timestamp assignment where the processing
> timestamp has to be set now by the {{StreamOperator}}, the CEP operators have
> not been adapted. This causes that the timestamp value assigned to the
> {{StreamRecord}} is used. In case of processing time this is
> {{Long.MIN_VALUE}}. In combination with a CEP time window, this can lead to
> an underflow in the NFA where the window time is subtracted from the current
> timestamp value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (FLINK-3681) CEP library does not support Java 8 lambdas as select function
[
https://issues.apache.org/jira/browse/FLINK-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-3681.
Resolution: Fixed
Fixed via 89f3a0faafbab9af90433488393421ae7171
> CEP library does not support Java 8 lambdas as select function
> --
>
> Key: FLINK-3681
> URL: https://issues.apache.org/jira/browse/FLINK-3681
> Project: Flink
> Issue Type: Bug
> Components: CEP
>Affects Versions: 1.0.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.1.0, 1.0.1
>
>
> Currently, the CEP library does not support Java 8 lambdas to be used as
> {{select}} or {{flatSelect}} function. The problem is that the
> {{TypeExtractor}} has different semantics when calling
> {{TypeExtractor.getUnaryOperatorReturnType}} either with a Java 8 lambda or
> an instance of an UDF function.
> To illustrate the problem assume we have the following UDF function
> {code}
> public interface MyFunction[T, O] {
> O foobar(Map inputElements);
> }
> {code}
> When calling the {{TypeExtractor}} with an anonymous class which implements
> this interface, the first type parameter is considered being the input type
> of the function, namely {{T}}.
> In contrast, when providing a Java 8 lambda for this interface, the
> {{TypeExtractor}} will see an input type of {{Map}}.
> This problem also occurs with a {{FlatMapFunction}} whose first type argument
> is {{T}} but whose first parameter of a Java 8 lambda is {{Iterable}}. In
> order to solve the problem here, the
> {{TypeExtractor.getUnaryOperatorReturnType}} method has the parameters
> {{hasIterable}} and {{hasCollector}}. If these values are {{true}}, then a
> special code path is taken (in case of a Java 8 lambda), where the input type
> is compared to the first type argument of the first input parameter of the
> lambda (here an {{Iterable}}). This hand-knitted solution does not
> generalize well, as it will fail for all parameterized types which have the
> input type at a different position (e.g. {{Map}}.
> In order to solve the problem, I propose to generalize the
> {{getUnaryOperatorReturnType}} a little bit so that one can specify at which
> position the input type is specified by a parameterized type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
