Re: [DISCUSS] Adding a dispose() method in the RichFunction.

gt; wrote: > > RichFunctions are used in the DataStream and DataSet APIs. > How would that change affect the DataSet API? > > Best, Fabian > > > 2016-11-10 11:37 GMT+01:00 Kostas Kloudas <k.klou...@data-artisans.com>: > >> Hello, >> >> I would lik

[DISCUSS] Adding a dispose() method in the RichFunction.

Hello, I would like to propose the addition of a dispose() method, in addition to the already existing close(), in the RichFunction interface. This will align the lifecycle of a RichFunction, with that of an Operator. After this, the code paths followed when finishing successfully and when

[jira] [Created] (FLINK-5020) Make the GenericWriteAheadSink rescalable.

Kostas Kloudas created FLINK-5020: - Summary: Make the GenericWriteAheadSink rescalable. Key: FLINK-5020 URL: https://issues.apache.org/jira/browse/FLINK-5020 Project: Flink Issue Type

[jira] [Created] (FLINK-4960) Enable testing of different re-scaling scenarios in the AbstractStreamOperatorTestHarness to

Kostas Kloudas created FLINK-4960: - Summary: Enable testing of different re-scaling scenarios in the AbstractStreamOperatorTestHarness to Key: FLINK-4960 URL: https://issues.apache.org/jira/browse/FLINK-4960

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hello, As Aljoshca said, the TriggerDSL depends on another PR, before it can be merged. In addition, this will change also the whole structure of how we test window/trigger related functionality, so some refactoring will be required. Kostas > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek

[jira] [Created] (FLINK-4833) Unstable test OperatorStatsAccumulatorTest.testAccumulatorHeavyHitterCountMinSketch

Kostas Kloudas created FLINK-4833: - Summary: Unstable test OperatorStatsAccumulatorTest.testAccumulatorHeavyHitterCountMinSketch Key: FLINK-4833 URL: https://issues.apache.org/jira/browse/FLINK-4833

Re: [DISCUSS] Removing delete*Timer from the WindowOperator.Context

triggers can not delete timers. I am not familiar with the new > Trigger DSL though. > > Cheers, > > Konstantin > > On 12.10.2016 15:38, Kostas Kloudas wrote: >> Hi all, >> >> This thread has been dormant for some time now. >> >> Given that th

Re: [DISCUSS] Removing delete*Timer from the WindowOperator.Context

created themselves? > > I guessing most of these use cases will be obsolete with the new > Trigger DSL because the trigger logic can be expressed more easily. So > +1 for removing the delete methods from the context. > > On Tue, Sep 27, 2016 at 3:43 PM, Kostas Kloudas > <

[jira] [Created] (FLINK-4800) Refactor the ContinuousFileMonitoringFunction code and the related tests.

Kostas Kloudas created FLINK-4800: - Summary: Refactor the ContinuousFileMonitoringFunction code and the related tests. Key: FLINK-4800 URL: https://issues.apache.org/jira/browse/FLINK-4800 Project

Re: [VOTE] Release Apache Flink 1.1.3 (RC1)

Hi all, I tested the Kafka source and continuous file sources and everything seems to be working fine. Kostas > On Oct 6, 2016, at 3:37 PM, Fabian Hueske wrote: > > +1 to release (binding) > > - checked hashes and signatures > - checked diffs against 1.1.2: no

Re: Releasing Flink 1.1.3

erge https://github.com/apache/flink/pull/2548 for the release. >>> It's cosmetic but it avoids a NPE in case the user jar doesn't contain >>> Flink jobs. >>> >>> On Wed, Oct 5, 2016 at 12:59 PM, Kostas Kloudas >>> <k.klou...@data-artisans.com>

Re: Releasing Flink 1.1.3

cus >>>> on getting a RC out. I would really like to do this today in order to >>>> start testing. >>>> >>>> The current state of affairs is as follows: >>>> >>>> - FLINK-4732 (Maven junction plugin security issue) has a fix in

[jira] [Created] (FLINK-4700) Harden the TimeProvider test

Kostas Kloudas created FLINK-4700: - Summary: Harden the TimeProvider test Key: FLINK-4700 URL: https://issues.apache.org/jira/browse/FLINK-4700 Project: Flink Issue Type: Bug

[DISCUSS] Removing delete*Timer from the WindowOperator.Context

Hi all, As the title of this email suggests, I am proposing to remove the methods deleteProcessingTimeTimer(long time) and deleteEventTimeTimer(long time) from the WindowOperator.Context. With this change, registered timers that have nothing to do (e.g. because their state has already been

[jira] [Created] (FLINK-4651) Re-register processing time timers at the WindowOperator upon recovery.

Kostas Kloudas created FLINK-4651: - Summary: Re-register processing time timers at the WindowOperator upon recovery. Key: FLINK-4651 URL: https://issues.apache.org/jira/browse/FLINK-4651 Project

[jira] [Created] (FLINK-4496) Refactor the TimeServiceProvider to take a Trigerable instead of a Runnable.

Kostas Kloudas created FLINK-4496: - Summary: Refactor the TimeServiceProvider to take a Trigerable instead of a Runnable. Key: FLINK-4496 URL: https://issues.apache.org/jira/browse/FLINK-4496 Project

[jira] [Created] (FLINK-4494) Expose the TimeServiceProvider from the Task to each Operator.

Kostas Kloudas created FLINK-4494: - Summary: Expose the TimeServiceProvider from the Task to each Operator. Key: FLINK-4494 URL: https://issues.apache.org/jira/browse/FLINK-4494 Project: Flink

[jira] [Created] (FLINK-4433) Refactor the StreamSource.

Kostas Kloudas created FLINK-4433: - Summary: Refactor the StreamSource. Key: FLINK-4433 URL: https://issues.apache.org/jira/browse/FLINK-4433 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-9: Trigger DSL

As I said, in some cases keeping that extra bit can be avoided. For >>> example, if you have Repeat.forever(Some.trigger()) you know that the >>> finished bit will always be false and so you don't keep any state in the >>> TriggerRunner. If every trigger manually does that bookke

Re: [DISCUSS] FLIP-9: Trigger DSL

combine the two we get: >> >> ((E and ET) or WT) and ((!E and LT) or WT) >> >> now we can eliminate the two parts with E and !E because they can never be >> true and are in an "or": >> >> WT and WT >> >> which yield just "WT".

Re: [DISCUSS] FLIP-9: Trigger DSL

yTrigger(earlyFiringTrigger), > EventTimeTrigger.afterEndOfWindow() > .withLateTrigger(lateFiringTrigger)) > > Is it legal? Will the earlyFiringTrigger and lateFiringTrigger both work ? > > > - Jark Wu > >> 在 2016年8月17日，上午12:24，

[jira] [Created] (FLINK-4407) Implement the trigger DSL

Kostas Kloudas created FLINK-4407: - Summary: Implement the trigger DSL Key: FLINK-4407 URL: https://issues.apache.org/jira/browse/FLINK-4407 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-9: Trigger DSL

ver(CountTrigger.of(5))) > > this gives me early firings whenever I see 5 new elements plus the > watermark firing. > > What do you think? > > On Tue, 16 Aug 2016 at 15:31 Kostas Kloudas <k.klou...@data-artisans.com> > wrote: > >> Thanks Till! >> >>

Re: [DISCUSS] FLIP-9: Trigger DSL

Thanks Till! Kostas > On Aug 16, 2016, at 3:30 PM, Till Rohrmann <trohrm...@apache.org> wrote: > > Cool design doc Klou. It's well described with a lot of details. I like it > a lot :-) +1 for implementing the trigger DSL. > > Cheers, > Till > > On Tue, Aug 16

Re: [DISCUSS] FLIP-9: Trigger DSL

nk they form one unit. > > On Tue, Aug 16, 2016 at 12:34 PM, Kostas Kloudas > <k.klou...@data-artisans.com> wrote: >> Hi all! >> >> I've created a FLIP for the trigger DSL. This is the triggers >> that we want Apache Flink to support out-of-the-box. This p

[DISCUSS] FLIP-9: Trigger DSL

Hi all! I've created a FLIP for the trigger DSL. This is the triggers that we want Apache Flink to support out-of-the-box. This proposal builds on various discussions on the mailing list and aims at serving as a base for further ones.

[jira] [Created] (FLINK-4397) Unstable test SlotCountExceedingParallelismTest.tearDown

Kostas Kloudas created FLINK-4397: - Summary: Unstable test SlotCountExceedingParallelismTest.tearDown Key: FLINK-4397 URL: https://issues.apache.org/jira/browse/FLINK-4397 Project: Flink

[jira] [Created] (FLINK-4398) Unstable test KvStateServerHandlerTest.testSimpleQuery

Kostas Kloudas created FLINK-4398: - Summary: Unstable test KvStateServerHandlerTest.testSimpleQuery Key: FLINK-4398 URL: https://issues.apache.org/jira/browse/FLINK-4398 Project: Flink Issue

Re: Question about Apache Flink Use Case

Hi Suma Cherukuri, From what I understand you have many small files and you want to aggregate them into bigger ones containing the logs of the last 24h. As Max said RollingSinks will allow you to have exactly-once semantics when writing your aggregated results to your FS. As far as reading

Re: [DISCUSS] Allowed Lateness in Flink

you have > any ideas. > > Cheers, > Aljoscha > > On Fri, 22 Jul 2016 at 13:10 Kostas Kloudas <k.klou...@data-artisans.com> > wrote: > >> Forgot to say that the signature for the onFire() that I think fits should >> be: >> >> void onFire(Window

Re: [DISCUSS] Allowed Lateness in Flink

Forgot to say that the signature for the onFire() that I think fits should be: void onFire(Window window, TriggerContext ctx) throws Exception; > On Jul 22, 2016, at 12:47 PM, Kostas Kloudas <k.klou...@data-artisans.com> > wrote: > > Hi, > > I started working on the ne

Re: [DISCUSS] Allowed Lateness in Flink

;>>> Only a rough sketch so that we don't forget about it. >>>>>> >>>>>> On Fri, 8 Jul 2016 at 07:47 Chen Qin <qinnc...@gmail.com> wrote: >>>>>> >>>>>>> +1 for allowedLateness scenario. >>>>>>&g

Re: Flink 1.1.0 Preview RC0

I think Aljoscha is having a look. > On Jul 21, 2016, at 3:39 PM, Kostas Kloudas <k.klou...@data-artisans.com> > wrote: > > There is an open PR now for https://issues.apache.org/jira/browse/FLINK-4207 > <https://issues.apache.org/jira/browse/FLINK-4207> > > K

Re: Flink 1.1.0 Preview RC0

There is an open PR now for https://issues.apache.org/jira/browse/FLINK-4207 Kostas > On Jul 21, 2016, at 3:38 PM, Stephan Ewen wrote: > > Looking into - https://github.com/apache/flink/pull/2226: Move Metrics API > to

[jira] [Created] (FLINK-4237) ClassLoaderITCase.testDisposeSavepointWithCustomKvState fails due to Timeout Futures

Kostas Kloudas created FLINK-4237: - Summary: ClassLoaderITCase.testDisposeSavepointWithCustomKvState fails due to Timeout Futures Key: FLINK-4237 URL: https://issues.apache.org/jira/browse/FLINK-4237

[jira] [Created] (FLINK-4224) Exception after successful execution when submitting job through the web interface.

Kostas Kloudas created FLINK-4224: - Summary: Exception after successful execution when submitting job through the web interface. Key: FLINK-4224 URL: https://issues.apache.org/jira/browse/FLINK-4224

[jira] [Created] (FLINK-4199) Wrong client behavior when submitting job to non-existing cluster

Kostas Kloudas created FLINK-4199: - Summary: Wrong client behavior when submitting job to non-existing cluster Key: FLINK-4199 URL: https://issues.apache.org/jira/browse/FLINK-4199 Project: Flink

Re: sampling function

Hi Do, In DataStream you can always implement your own sampling function, hopefully without too much effort. Adding such functionality it to the API could be a good idea. But given that in sampling there is no “one-size-fits-all” solution (as not every use case needs random sampling and not

[jira] [Created] (FLINK-4126) Unstable test ZooKeeperLeaderElectionTest.testZooKeeperReelection

Kostas Kloudas created FLINK-4126: - Summary: Unstable test ZooKeeperLeaderElectionTest.testZooKeeperReelection Key: FLINK-4126 URL: https://issues.apache.org/jira/browse/FLINK-4126 Project: Flink

[jira] [Created] (FLINK-4124) Unstable test WrapperSetupHelperTest.testCreateTopologyContext

Kostas Kloudas created FLINK-4124: - Summary: Unstable test WrapperSetupHelperTest.testCreateTopologyContext Key: FLINK-4124 URL: https://issues.apache.org/jira/browse/FLINK-4124 Project: Flink

Re: [PROPOSAL] Structure the Flink Open Source Development

Hello, You can also add me to the DataStream API. Kostas > On Jun 16, 2016, at 7:02 PM, Robert Metzger wrote: > > Cool, thank you. > > So now we have at least one shepherd for each component. > Since there were no other comments / complaints about this proposal, I >

Re: [jira] [Updated] (FLINK-2155) Add an additional checkstyle validation for illegal imports

- >> >>Key: FLINK-2155 >>URL: https://issues.apache.org/jira/browse/FLINK-2155 >>Project: Flink >> Issue Type: Improvement >> Components: Build System >> Affects Versions: 1.1.0

[jira] [Created] (FLINK-4065) Unstable Kafka09ITCase.testMultipleSourcesOnePartition test

Kostas Kloudas created FLINK-4065: - Summary: Unstable Kafka09ITCase.testMultipleSourcesOnePartition test Key: FLINK-4065 URL: https://issues.apache.org/jira/browse/FLINK-4065 Project: Flink

[jira] [Created] (FLINK-3986) Rename the readFileStream(String filePath, long intervalMillis, WatchType watchType) from the StreamExecutionEnvironment

Kostas Kloudas created FLINK-3986: - Summary: Rename the readFileStream(String filePath, long intervalMillis, WatchType watchType) from the StreamExecutionEnvironment Key: FLINK-3986 URL: https://issues.apache.org

[jira] [Created] (FLINK-3956) Make FileInputFormats in Streaming independent from the Configuration object

Kostas Kloudas created FLINK-3956: - Summary: Make FileInputFormats in Streaming independent from the Configuration object Key: FLINK-3956 URL: https://issues.apache.org/jira/browse/FLINK-3956 Project

[jira] [Created] (FLINK-3896) Allow a StreamTask to be Externally Cancelled.

Kostas Kloudas created FLINK-3896: - Summary: Allow a StreamTask to be Externally Cancelled. Key: FLINK-3896 URL: https://issues.apache.org/jira/browse/FLINK-3896 Project: Flink Issue Type

Re: [DISCUSS] Release Flink 1.0.3

I am working on: https://issues.apache.org/jira/browse/FLINK-2314?filter=-1 which I believe will also affect: https://issues.apache.org/jira/browse/FLINK-3796 Essentially the FileSourceFunction will become obsolete. > On Apr 22,

[jira] [Created] (FLINK-3717) Add functionality to get the current offset of a source with a FileInputFormat

Kostas Kloudas created FLINK-3717: - Summary: Add functionality to get the current offset of a source with a FileInputFormat Key: FLINK-3717 URL: https://issues.apache.org/jira/browse/FLINK-3717

[jira] [Created] (FLINK-3683) Unstable tests: in the Kafka09ITCase, the testMultipleSourcesOnePartition() and the testOneToOneSources()

Kostas Kloudas created FLINK-3683: - Summary: Unstable tests: in the Kafka09ITCase, the testMultipleSourcesOnePartition() and the testOneToOneSources() Key: FLINK-3683 URL: https://issues.apache.org/jira/browse

Re: [ANNOUNCE] Flink 0.10.2 Released

Yes thanks a lot Ufuk! > On Feb 12, 2016, at 3:09 PM, Till Rohrmann wrote: > > Thanks for being our release manager Ufuk :-) Great work! > > On Fri, Feb 12, 2016 at 2:15 PM, Robert Metzger wrote: > >> Thank you for doing a release Ufuk! >> >> I

<    1   2   3   4   5