Re: Possible bug in Kafka producer partitioning logic

2017-04-12 Thread Gyula Fóra
No worries, we have already pushed the fix for our jobs on a custom build :) Have a safe trip! Gyula Tzu-Li (Gordon) Tai ezt írta (időpont: 2017. ápr. 11., K, 23:11): > That workaround should work, yes. > The proper fix would also be something similar I guess, only just exposing > extra APIs to

RE: Question about the process order in stream aggregate

2017-04-12 Thread Stefano Bortoli
I'm afraid that to keep order either you have to process it in a serial way (parallelism 1), or provide an element that allows to sort the objects when these are processed in parallel (i.e. rowTime). When you distribute the computation, as Fabian explained, you get a round-robin assignment to th

[jira] [Created] (FLINK-6299) make all IT cases extend from TestLogger

2017-04-12 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-6299: -- Summary: make all IT cases extend from TestLogger Key: FLINK-6299 URL: https://issues.apache.org/jira/browse/FLINK-6299 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-6300) PID1 of docker images does not behave correctly

2017-04-12 Thread kathleen sharp (JIRA)
kathleen sharp created FLINK-6300: - Summary: PID1 of docker images does not behave correctly Key: FLINK-6300 URL: https://issues.apache.org/jira/browse/FLINK-6300 Project: Flink Issue Type: B

Re: Question about the process order in stream aggregate

2017-04-12 Thread Fabian Hueske
Hi Xingcan, the 0L timestamp literal is an artifact of how the Calcite query is translated by Flink. It represents the value of the procTime() function that is logically used to sort the data. Calcite expects this attribute in the schema but Flink's OVER operator actually processes the data based

[jira] [Created] (FLINK-6301) Flink KafkaConnector09 leaks memory on reading compressed messages due to a Kafka consumer bug

2017-04-12 Thread Rahul Yadav (JIRA)
Rahul Yadav created FLINK-6301: -- Summary: Flink KafkaConnector09 leaks memory on reading compressed messages due to a Kafka consumer bug Key: FLINK-6301 URL: https://issues.apache.org/jira/browse/FLINK-6301

[VOTE] Release Apache Flink 1.2.1 (RC2)

2017-04-12 Thread Robert Metzger
Dear Flink community, Please vote on releasing the following candidate as Apache Flink version 1.2 .1. The commit to be voted on: 76eba4e0 (*http://git-wip-us.apache.org/repos/asf/flink/commit/76eba4e0

Jobmanager drops upon submitting a jar

2017-04-12 Thread amir bahmanyari
Hi Colleagues,I have a simple test job when I submit it to the Flink cluster the JM seems to disconnect.Its a one node cluster implemented in a VirtualBox Centos 7 VM.Flink starts fine and everything else look fine. Following is stack trace.I appreciate a feedback.Cheers 17/04/12 15:53:04 IN

Re: Question about the process order in stream aggregate

2017-04-12 Thread Xingcan Cui
Hi, @Radu @Stefano, sorry that I misunderstood it before. We considered the problem from different viewpoints. I agree that (ingestion) timestamp injection could be a good solution for this problem in some scenarios. Thanks. @Fabian, thanks for your explanation. That makes sense. Best, Xingcan

Re: Jobmanager drops upon submitting a jar

2017-04-12 Thread wenlong.lwl
Hi, amir, I think you could check the log of job manager to make sure that job manager [192.168.56.101:6123 ] is running well firstly, you may get what is wrong in the log. On 13 April 2017 at 08:54, amir bahmanyari wrote: > > > > Hi Colleagues,I have a simple

Re: [VOTE] Release Apache Flink 1.2.1 (RC2)

2017-04-12 Thread Ted Yu
I ran test suite where the following failed: Failed tests: StreamExecutionEnvironmentTest.testDefaultParallelismIsDefault:143 expected:<-1> but was:<24> StreamExecutionEnvironmentTest.testMaxParallelismMustBeBiggerEqualParallelism Expected test to throw an instance of java.lang.IllegalArgumentE

Splitting input stream in to multiple windows and process individually

2017-04-12 Thread madhairsilence
down vote favorite I have a Datastream . I have to apply window function as 1 hour, 2 hour, 3 hour...24 hour. And each split has its own window function to be done How do I split the window in an efficient way. The dirtiest way I can think of is for(int i=1; i < 24 ;i++){ inputStream.keyBy("i

Splitting input stream in to multiple windows and process individually

2017-04-12 Thread madhairsilence
I have a Datastream . I have to apply window function as 1 hour, 2 hour, 3 hour...24 hour. And each split has its own window function to be done How do I split the window in an efficient way. The dirtiest way I can think of is for(int i=1; i < 24 ;i++){ inputStream.keyBy("id").timeWindow(Time

Re: [VOTE] Release Apache Flink 1.2.1 (RC2)

2017-04-12 Thread Andrew Psaltis
Ted -- I did not see those errors. My environment is: Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T11:41:47-05:00) Maven home: /usr/local/Cellar/maven/3.3.9/libexec Java version: 1.8.0_121, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_

Re: [VOTE] Release Apache Flink 1.2.1 (RC2)

2017-04-12 Thread Andrew Psaltis
+1 -- checked out all code, built with all tests, ran local cluster, deployed example streaming jobs On Thu, Apr 13, 2017 at 2:26 AM, Andrew Psaltis wrote: > Ted -- I did not see those errors. My environment is: > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > 2015-11-10T11:41:4