HAQueryableStateRocksDBBackendITCase test cause outofmemoryerror and never stop

2018-02-21 Thread mingleizhang
Hi, I run HAQueryableStateRocksDBBackendITCase#testValueStateDefault test class on my local machine. And it hangs always and in the end gives me a outofmemory error. Cheers Minglei.

Re: Verifying watermarks in integration test

2018-02-21 Thread Xingcan Cui
Hi Thomas, some test cases in JoinHarnessTest show how to verify the emitted watermarks. Hope this helps. Best, Xingcan > On 21 Feb 2018,

[jira] [Created] (FLINK-8721) Client blocks indefinitely if job archiving fails

2018-02-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-8721: --- Summary: Client blocks indefinitely if job archiving fails Key: FLINK-8721 URL: https://issues.apache.org/jira/browse/FLINK-8721 Project: Flink Issue T

Re: HAQueryableStateRocksDBBackendITCase test cause outofmemoryerror and never stop

2018-02-21 Thread Kostas Kloudas
Hi Minglei, Can you send us where the execution is when it hangs? Threaddump, what objects are created, or whatever can help track down the problem. Thanks, Kostas > On Feb 21, 2018, at 9:07 AM, mingleizhang wrote: > > Hi, > I run HAQueryableStateRocksDBBackendITCase#testValueStateDefault te

[jira] [Created] (FLINK-8722) Refactor BroadcastState examples.

2018-02-21 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-8722: - Summary: Refactor BroadcastState examples. Key: FLINK-8722 URL: https://issues.apache.org/jira/browse/FLINK-8722 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-8723) Remove existing BroadcastState examples.

2018-02-21 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-8723: - Summary: Remove existing BroadcastState examples. Key: FLINK-8723 URL: https://issues.apache.org/jira/browse/FLINK-8723 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-8724) Add BroadcastState examples with predetermined state transitions for the broadcast state

2018-02-21 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-8724: - Summary: Add BroadcastState examples with predetermined state transitions for the broadcast state Key: FLINK-8724 URL: https://issues.apache.org/jira/browse/FLINK-8724

[jira] [Created] (FLINK-8725) Separate NFA-state from NFA in CEP library

2018-02-21 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-8725: --- Summary: Separate NFA-state from NFA in CEP library Key: FLINK-8725 URL: https://issues.apache.org/jira/browse/FLINK-8725 Project: Flink Issue Type: Bu

[jira] [Created] (FLINK-8726) Code highlighting partially broken

2018-02-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-8726: --- Summary: Code highlighting partially broken Key: FLINK-8726 URL: https://issues.apache.org/jira/browse/FLINK-8726 Project: Flink Issue Type: Improvemen

[jira] [Created] (FLINK-8727) Test instability in SlotPoolRpcTest.testCancelSlotAllocationWithoutResourceManager

2018-02-21 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-8727: --- Summary: Test instability in SlotPoolRpcTest.testCancelSlotAllocationWithoutResourceManager Key: FLINK-8727 URL: https://issues.apache.org/jira/browse/FLINK-8727 Projec

[jira] [Created] (FLINK-8728) Fail new worker requests immediately in Standalone mode

2018-02-21 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-8728: Summary: Fail new worker requests immediately in Standalone mode Key: FLINK-8728 URL: https://issues.apache.org/jira/browse/FLINK-8728 Project: Flink Issue T

[jira] [Created] (FLINK-8729) Migrate JSONGenerator from Slink to Jackson

2018-02-21 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-8729: --- Summary: Migrate JSONGenerator from Slink to Jackson Key: FLINK-8729 URL: https://issues.apache.org/jira/browse/FLINK-8729 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-8730) Make SerializedThrowable deserialized from JSON re-throwable

2018-02-21 Thread Gary Yao (JIRA)
Gary Yao created FLINK-8730: --- Summary: Make SerializedThrowable deserialized from JSON re-throwable Key: FLINK-8730 URL: https://issues.apache.org/jira/browse/FLINK-8730 Project: Flink Issue Type:

[jira] [Created] (FLINK-8731) TwoInputStreamTaskTest flaky on travis

2018-02-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-8731: --- Summary: TwoInputStreamTaskTest flaky on travis Key: FLINK-8731 URL: https://issues.apache.org/jira/browse/FLINK-8731 Project: Flink Issue Type: Improv

[jira] [Created] (FLINK-8732) Cancel scheduling operation when cancelling the ExecutionGraph

2018-02-21 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-8732: Summary: Cancel scheduling operation when cancelling the ExecutionGraph Key: FLINK-8732 URL: https://issues.apache.org/jira/browse/FLINK-8732 Project: Flink

[jira] [Created] (FLINK-8733) fix SpillableSubpartition#spillFinishedBufferConsumers() not counting spilled bytes

2018-02-21 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-8733: -- Summary: fix SpillableSubpartition#spillFinishedBufferConsumers() not counting spilled bytes Key: FLINK-8733 URL: https://issues.apache.org/jira/browse/FLINK-8733 Project

[jira] [Created] (FLINK-8734) SpillableSubpartition does not update byte statistics when spilled by a SpillableSubpartitionView

2018-02-21 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-8734: -- Summary: SpillableSubpartition does not update byte statistics when spilled by a SpillableSubpartitionView Key: FLINK-8734 URL: https://issues.apache.org/jira/browse/FLINK-8734

[jira] [Created] (FLINK-8735) Add savepoint migration ITCase that covers operator state

2018-02-21 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-8735: --- Summary: Add savepoint migration ITCase that covers operator state Key: FLINK-8735 URL: https://issues.apache.org/jira/browse/FLINK-8735 Project: Flink

[jira] [Created] (FLINK-8736) Memory segment offsets for slices of slices are wrong

2018-02-21 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-8736: -- Summary: Memory segment offsets for slices of slices are wrong Key: FLINK-8736 URL: https://issues.apache.org/jira/browse/FLINK-8736 Project: Flink Issue Type: S

FINAL REMINDER: CFP for Apache EU Roadshow Closes 25th February

2018-02-21 Thread Sharan F
Hello Apache Supporters and Enthusiasts This is your FINAL reminder that the Call for Papers (CFP) for the Apache EU Roadshow is closing soon. Our Apache EU Roadshow will focus on Cloud, IoT, Apache Tomcat, Apache Http and will run from 13-14 June 2018 in Berlin. Note that the CFP deadline has

[jira] [Created] (FLINK-8737) Creating a union of UnionGate instances will fail with UnsupportedOperationException when retrieving buffers

2018-02-21 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-8737: -- Summary: Creating a union of UnionGate instances will fail with UnsupportedOperationException when retrieving buffers Key: FLINK-8737 URL: https://issues.apache.org/jira/browse/FLINK-

[jira] [Created] (FLINK-8738) Converge runtime dependency versions for 'scala-lang' and for 'com.typesafe:config'

2018-02-21 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-8738: --- Summary: Converge runtime dependency versions for 'scala-lang' and for 'com.typesafe:config' Key: FLINK-8738 URL: https://issues.apache.org/jira/browse/FLINK-8738 Proje

[DISCUSS] Move Flink Docker Files from Flink to 'docker-flink'

2018-02-21 Thread Stephan Ewen
Hi all! I want to start the discussion to whether to drop the " *flink-contrib/docker-flink*" subproject and instead point users to "*github.com/docker-flink/docker-flink *". There are a few reasons for this thought: - The projects have gone out of

[jira] [Created] (FLINK-8739) Optimize runtime support for distinct filter to reuse same distinct accumulator for filtering

2018-02-21 Thread Rong Rong (JIRA)
Rong Rong created FLINK-8739: Summary: Optimize runtime support for distinct filter to reuse same distinct accumulator for filtering Key: FLINK-8739 URL: https://issues.apache.org/jira/browse/FLINK-8739 P

[jira] [Created] (FLINK-8740) Job-level metrics lost during job re-submission in HA mode

2018-02-21 Thread Joshua DeWald (JIRA)
Joshua DeWald created FLINK-8740: Summary: Job-level metrics lost during job re-submission in HA mode Key: FLINK-8740 URL: https://issues.apache.org/jira/browse/FLINK-8740 Project: Flink Issu

Re: [DISCUSS] Move Flink Docker Files from Flink to 'docker-flink'

2018-02-21 Thread Bowen Li
+1. flink-contrib is an experiment land, docker-flink should graduate from it. This also helps to clean up flink-contrib/ On Wed, Feb 21, 2018 at 1:41 PM, Stephan Ewen wrote: > Hi all! > > I want to start the discussion to whether to drop the " > *flink-contrib/docker-flink*" subproject and i

Re: Timestamp/watermark support in Kinesis consumer

2018-02-21 Thread Jamie Grier
Big +1 on trying to come up with a common framework for partition-based, replayable sources. There is so much common code to be written that makes it possible to write correct connectors and Gordon's bullet points are exactly those -- and it's not just Kinesis and Kafka. It's also true for readin

Re: Timestamp/watermark support in Kinesis consumer

2018-02-21 Thread Jamie Grier
I know this is a very simplistic idea but... In general the issue Eron is describing occurs whenever two (or more) parallel partitions are assigned to the same Flink sub-task and there is large time delta between them. This problem exists though largely because we are not making any decisions abo

Re: [DISCUSS] Flink security improvements

2018-02-21 Thread Shuyi Chen
Hi Eron, thanks a lot for taking a look at the proposal, the comments are very useful. I've updated the document to address your concerns. Could you please help take another look, and suggest what the next step is? Highly appreciated. Shuyi On Thu, Feb 15, 2018 at 4:19 AM, Shuyi Chen wrote: > H