[jira] [Created] (FLINK-4562) table example make an divided module in flink-examples

2016-09-01 Thread shijinkui (JIRA)
shijinkui created FLINK-4562: Summary: table example make an divided module in flink-examples Key: FLINK-4562 URL: https://issues.apache.org/jira/browse/FLINK-4562 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-4561) replace all the scala version as a property

2016-09-01 Thread shijinkui (JIRA)
shijinkui created FLINK-4561: Summary: replace all the scala version as a property Key: FLINK-4561 URL: https://issues.apache.org/jira/browse/FLINK-4561 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4560) enforcer java version as 1.7

2016-09-01 Thread shijinkui (JIRA)
shijinkui created FLINK-4560: Summary: enforcer java version as 1.7 Key: FLINK-4560 URL: https://issues.apache.org/jira/browse/FLINK-4560 Project: Flink Issue Type: Improvement Report

Re: [VOTE] Release Apache Flink 1.1.2 (RC1)

2016-09-01 Thread Neelesh Salian
+1 Downloaded the bits. Ran the examples. A suggestion: "The vote is open for the next 3 days and passes if a majority of at least three +1 PMC votes are cast." Could the duration of the vote be more specific with the date and time since we want to avoid timezone discrepancies? On Thu, Sep 1, 2

[jira] [Created] (FLINK-4559) Kinesis Producer not setting credentials provider properly when AWS_CREDENTIALS_PROVIDER is "AUTO"

2016-09-01 Thread Tzu-Li (Gordon) Tai (JIRA)
Tzu-Li (Gordon) Tai created FLINK-4559: -- Summary: Kinesis Producer not setting credentials provider properly when AWS_CREDENTIALS_PROVIDER is "AUTO" Key: FLINK-4559 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (FLINK-4558) Add support for synchronizing streams

2016-09-01 Thread Elias Levy (JIRA)
Elias Levy created FLINK-4558: - Summary: Add support for synchronizing streams Key: FLINK-4558 URL: https://issues.apache.org/jira/browse/FLINK-4558 Project: Flink Issue Type: Improvement

[DISCUSS] FLIP-11: Table API Stream Aggregations

2016-09-01 Thread Timo Walther
Hi all! Fabian and I worked on a FLIP for Stream Aggregations in the Table API. You can find the FLIP-11 here: https://cwiki.apache.org/confluence/display/FLINK/FLIP-11%3A+Table+API+Stream+Aggregations Motivation for the FLIP: The Table API is a declarative API to define queries on static an

[jira] [Created] (FLINK-4557) Table API Stream Aggregations

2016-09-01 Thread Timo Walther (JIRA)
Timo Walther created FLINK-4557: --- Summary: Table API Stream Aggregations Key: FLINK-4557 URL: https://issues.apache.org/jira/browse/FLINK-4557 Project: Flink Issue Type: New Feature C

Re: [VOTE] Release Apache Flink 1.1.2 (RC1)

2016-09-01 Thread Aljoscha Krettek
+1 I went over all the changes that we introduced since 1.1.1 and they look good. On Wed, 31 Aug 2016 at 14:53 Maximilian Michels wrote: > Found a minor bug for detached job submissions but I wouldn't cancel > the release for it: https://issues.apache.org/jira/browse/FLINK-4540 > > On Wed, Aug

[ANNOUNCE] Queryable State is Broken on Master

2016-09-01 Thread Aljoscha Krettek
Our recent changes to make keyed state rescalable/key-group aware are breaking queryable state because it is not yet made key-group aware. I opened this Jira issue to track the fix for that: https://issues.apache.org/jira/browse/FLINK-4556. Sorry for the inconvenience. Cheers, Aljoscha

[jira] [Created] (FLINK-4556) Make Queryable State Key-Group Aware

2016-09-01 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-4556: --- Summary: Make Queryable State Key-Group Aware Key: FLINK-4556 URL: https://issues.apache.org/jira/browse/FLINK-4556 Project: Flink Issue Type: Improvem

Re: Scheduling of Flink jobs

2016-09-01 Thread Stephan Ewen
You can control the resource sharing of tasks pretty fine grained. The packing heuristic makes it simpler to initially configure and balance clusters, because you need not to task-math to compute the resources. On Thu, Sep 1, 2016 at 1:27 PM, Bhupesh Chawda wrote: > Thanks Stephan for your re

Queryable state deactivated on current master branch

2016-09-01 Thread Stefan Richter
Hi, I want to announce that the queryable state feature is deactivated on the current master. In our efforts to implement dynamic scaling for Flink, we introduced key-groups for keyed state (see [FLINK-3755]). However, queryable state does not support key-groups, yet. We will reactivate the fea

Re: Scheduling of Flink jobs

2016-09-01 Thread Bhupesh Chawda
Thanks Stephan for your reply. If I understand correctly, if my parallelism is 1, then all of the operators, not matter how many (say 20), will still run on just one task manager. What happens in case the resources on that task manager are not sufficient for all of these operators? ~ Bhupesh On

[jira] [Created] (FLINK-4555) Explicitly kill TaskManager on YARN when ApplicationMaster is shutting down

2016-09-01 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-4555: - Summary: Explicitly kill TaskManager on YARN when ApplicationMaster is shutting down Key: FLINK-4555 URL: https://issues.apache.org/jira/browse/FLINK-4555 Project:

Re: Scheduling of Flink jobs

2016-09-01 Thread Stephan Ewen
In the default configuration, the job uses as many slots as the parallelism of the operators states. I assume you run with a parallelism of 2, so it occupies two slots. if you run 5 taskmanagers with each one slot, you should set the parallelism to 5 as well. On Mon, Aug 29, 2016 at 4:04 PM, Bhup

[jira] [Created] (FLINK-4554) Add support for array types

2016-09-01 Thread Timo Walther (JIRA)
Timo Walther created FLINK-4554: --- Summary: Add support for array types Key: FLINK-4554 URL: https://issues.apache.org/jira/browse/FLINK-4554 Project: Flink Issue Type: New Feature Com

[jira] [Created] (FLINK-4553) Add support for table sampling

2016-09-01 Thread Timo Walther (JIRA)
Timo Walther created FLINK-4553: --- Summary: Add support for table sampling Key: FLINK-4553 URL: https://issues.apache.org/jira/browse/FLINK-4553 Project: Flink Issue Type: New Feature

Re: Extending FLIP template

2016-09-01 Thread Timo Walther
+1 Make sense especially since FLIPs are always large changes that need to be done in several steps. Am 01/09/16 um 11:24 schrieb Aljoscha Krettek: +1 If you think it worthwhile you can add it to the template(s). On Thu, 1 Sep 2016 at 10:38 Fabian Hueske wrote: Hi, I'm currently preparin

Re: Extending FLIP template

2016-09-01 Thread Aljoscha Krettek
+1 If you think it worthwhile you can add it to the template(s). On Thu, 1 Sep 2016 at 10:38 Fabian Hueske wrote: > Hi, > > I'm currently preparing a FLIP for Table API streaming aggregates and > noticed that there is no section about how the task can be divided into > subtasks. > > I think it w

[jira] [Created] (FLINK-4552) Refactor WindowOperator/Trigger Tests

2016-09-01 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-4552: --- Summary: Refactor WindowOperator/Trigger Tests Key: FLINK-4552 URL: https://issues.apache.org/jira/browse/FLINK-4552 Project: Flink Issue Type: Improve

[jira] [Created] (FLINK-4551) Heartbeat Manager integration with JobMaster

2016-09-01 Thread zhangjing (JIRA)
zhangjing created FLINK-4551: Summary: Heartbeat Manager integration with JobMaster Key: FLINK-4551 URL: https://issues.apache.org/jira/browse/FLINK-4551 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-4550) Clearly define SQL operator table

2016-09-01 Thread Timo Walther (JIRA)
Timo Walther created FLINK-4550: --- Summary: Clearly define SQL operator table Key: FLINK-4550 URL: https://issues.apache.org/jira/browse/FLINK-4550 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4549) Test and document implicitly supported SQL functions

2016-09-01 Thread Timo Walther (JIRA)
Timo Walther created FLINK-4549: --- Summary: Test and document implicitly supported SQL functions Key: FLINK-4549 URL: https://issues.apache.org/jira/browse/FLINK-4549 Project: Flink Issue Type:

Re: NullPointerException in beam stream runner

2016-09-01 Thread Maximilian Michels
Hi Alexey, You don't have to set the streaming mode. The Flink Runner will automatically choose to use streaming mode when it discovers UnboundedSources like Kafka. I'm wondering why that didn't work in your case. I just ran your example and it chose streaming mode and didn't return an error durin

Extending FLIP template

2016-09-01 Thread Fabian Hueske
Hi, I'm currently preparing a FLIP for Table API streaming aggregates and noticed that there is no section about how the task can be divided into subtasks. I think it would make sense to extend the template by a section "Work Plan" or "Implementation Plan" that explains in which steps or subtask

Re: [DISCUSS] Automated code quality check in Flink

2016-09-01 Thread Maximilian Michels
Hi Ivan, I don't have any experience with the sites but I wouldn't mind trying out one of these. As far as I understand they help to visualize code coverage and perform static analysis to find code problems. As for the code coverage, we have to build in coverage checks into our build system oursel

[jira] [Created] (FLINK-4548) Refactor ContinuousFileMonitoringTest to use OneShotLatch

2016-09-01 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-4548: --- Summary: Refactor ContinuousFileMonitoringTest to use OneShotLatch Key: FLINK-4548 URL: https://issues.apache.org/jira/browse/FLINK-4548 Project: Flink