Re: [DISCUSS] Merging the FLIP-6 feature branch into the Master branch

2016-12-02 Thread Stephan Ewen
Hi Greg! Yes, originally I was arguing to merge Flip-6 after forking off the 1.2 branch. That would be the preferable solution, but since it seems that the 1.2 branch may still take a few weeks, I was thinking that we may want to do that earlier. Many flip-6 contributors would like to be active a

Re: [DISCUSS] Proposed updates to Flink project site

2016-12-02 Thread Maximilian Michels
The changes look great, Mike! Here's a quick screenshot of the front page: http://pasteboard.co/images/51Tqpm8Ke.png A quick one liner to paste in your terminal to open a browser window with the changes: git clone https://github.com/wints/flink-web-updates web-updates && cd web-updates && ./b

[jira] [Created] (FLINK-5241) Introduce AbstractStreamOperator get CloseableRegistry

2016-12-02 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-5241: Summary: Introduce AbstractStreamOperator get CloseableRegistry Key: FLINK-5241 URL: https://issues.apache.org/jira/browse/FLINK-5241 Project: Flink Issue Ty

Re: [DISCUSS] Merging the FLIP-6 feature branch into the Master branch

2016-12-02 Thread Greg Hogan
Hi Stephan, How soon are you expecting the "release-1.2" fork? I am sure you have considered merging the FLIP-6 branch after the fork. Do we anticipate the new tests pushing Flink over Travis CI's new 50 minute limit? This might be a good opportunity to rebalance the test ranges as the most recen

[jira] [Created] (FLINK-5240) Properly Close StateBackend in StreamTask when closing/canceling

2016-12-02 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-5240: --- Summary: Properly Close StateBackend in StreamTask when closing/canceling Key: FLINK-5240 URL: https://issues.apache.org/jira/browse/FLINK-5240 Project: Flink

[jira] [Created] (FLINK-5239) Properly unpack thrown exceptions in RPC methods

2016-12-02 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-5239: --- Summary: Properly unpack thrown exceptions in RPC methods Key: FLINK-5239 URL: https://issues.apache.org/jira/browse/FLINK-5239 Project: Flink Issue Type: Sub-

[DISCUSS] Merging the FLIP-6 feature branch into the Master branch

2016-12-02 Thread Stephan Ewen
Hi all! I want to start a discussion about merging the FLIP-6 feature branch into the master branch. The feature branch implements the following: - A new RPC abstraction - A new High-Availability Services Abstraction - New JobManager / TaskManager / ResourceManager with dynamic slot allocat

[jira] [Created] (FLINK-5238) Flip-6 mini cluster should start local TaskManagers if there is only one TaskManager

2016-12-02 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-5238: --- Summary: Flip-6 mini cluster should start local TaskManagers if there is only one TaskManager Key: FLINK-5238 URL: https://issues.apache.org/jira/browse/FLINK-5238 Proj

[jira] [Created] (FLINK-5237) Consolidate and harmonize Window Translation Tests

2016-12-02 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-5237: --- Summary: Consolidate and harmonize Window Translation Tests Key: FLINK-5237 URL: https://issues.apache.org/jira/browse/FLINK-5237 Project: Flink Issue

Re: [ANNOUNCE] Stability/Scalability effort and Pull Requests

2016-12-02 Thread Paris Carbone
This is really huge. Highly appreciated Stephan and the rest :) The unfair input handling you mentioned was maybe responsible of the incredibly slow processing of records through not-so-active feedback edges (e.g. in ML projects). cheers Paris > On 2 Dec 2016, at 14:48, Stephan Ewen wrote: >

[jira] [Created] (FLINK-5236) User jar classpath should come first in classpath

2016-12-02 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-5236: - Summary: User jar classpath should come first in classpath Key: FLINK-5236 URL: https://issues.apache.org/jira/browse/FLINK-5236 Project: Flink Iss

[jira] [Created] (FLINK-5235) HBase classpath automatically added to Flink classpath

2016-12-02 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-5235: - Summary: HBase classpath automatically added to Flink classpath Key: FLINK-5235 URL: https://issues.apache.org/jira/browse/FLINK-5235 Project: Flink

[jira] [Created] (FLINK-5234) Inform user about classpath configuration

2016-12-02 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-5234: - Summary: Inform user about classpath configuration Key: FLINK-5234 URL: https://issues.apache.org/jira/browse/FLINK-5234 Project: Flink Issue Type:

[jira] [Created] (FLINK-5233) Upgrade Jackson version because of class loader leak

2016-12-02 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-5233: - Summary: Upgrade Jackson version because of class loader leak Key: FLINK-5233 URL: https://issues.apache.org/jira/browse/FLINK-5233 Project: Flink Issue Ty

[jira] [Created] (FLINK-5232) Add a Thread default uncaught exception handler on the JobManager

2016-12-02 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-5232: --- Summary: Add a Thread default uncaught exception handler on the JobManager Key: FLINK-5232 URL: https://issues.apache.org/jira/browse/FLINK-5232 Project: Flink

[jira] [Created] (FLINK-5231) JobManager should exit on "OutOfMemoryError"

2016-12-02 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-5231: --- Summary: JobManager should exit on "OutOfMemoryError" Key: FLINK-5231 URL: https://issues.apache.org/jira/browse/FLINK-5231 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-5230) Safety nets against leaving dysfunctional JobManagers

2016-12-02 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-5230: --- Summary: Safety nets against leaving dysfunctional JobManagers Key: FLINK-5230 URL: https://issues.apache.org/jira/browse/FLINK-5230 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-5 Only send data to each taskmanager once for broadcasts

2016-12-02 Thread Felix Neutatz
Hi everybody, I implemented the second approach (see https://cwiki.apache.org/confl uence/display/FLINK/FLIP-5%3A+Only+send+data+to+each+taskman ager+once+for+broadcasts). So each subpartition will be read by m tasks (m = number of task managers) and the other tasks will notify the subpartition th

[jira] [Created] (FLINK-5229) Cleanup StreamTaskStates if a checkpoint operation of a subsequent operator fails

2016-12-02 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-5229: Summary: Cleanup StreamTaskStates if a checkpoint operation of a subsequent operator fails Key: FLINK-5229 URL: https://issues.apache.org/jira/browse/FLINK-5229 Proj

[ANNOUNCE] Stability/Scalability effort and Pull Requests

2016-12-02 Thread Stephan Ewen
Hi fellow Flink enthusiasts! Over the past weeks, several community members have been working hard on a thread of issues to make Flink more stable and scalable for some demanding large scale deployments. It was quite a lot of work digging through the masses of logs, setting up clusters to reprodu

Re: Join to FLINK Team

2016-12-02 Thread Fabian Hueske
Thanks! I assigned the issue to you and gave you Contributor permissions. You can now assign issues to yourself. Thanks, Fabian 2016-12-02 12:24 GMT+01:00 Andrey Efimov : > Hi Fabian, > Username: Andrew Efimov > > Thanks, > Andrew > > > 2016-12-02 13:56 GMT+03:00 Fabian Hueske : > > > Hi Andrew,

Re: Join to FLINK Team

2016-12-02 Thread Andrey Efimov
Hi Fabian, Username: Andrew Efimov Thanks, Andrew 2016-12-02 13:56 GMT+03:00 Fabian Hueske : > Hi Andrew, > > welcome to the Flink community. > I need your JIRA user to give you contributor permissions and assign > FLINK-4905 to you. > > Thanks, > Fabian > > 2016-12-02 11:47 GMT+01:00 Andrey Ef

[jira] [Created] (FLINK-5228) LocalInputChannel re-trigger request and release deadlock

2016-12-02 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-5228: -- Summary: LocalInputChannel re-trigger request and release deadlock Key: FLINK-5228 URL: https://issues.apache.org/jira/browse/FLINK-5228 Project: Flink Issue Typ

Re: Join to FLINK Team

2016-12-02 Thread Fabian Hueske
Hi Andrew, welcome to the Flink community. I need your JIRA user to give you contributor permissions and assign FLINK-4905 to you. Thanks, Fabian 2016-12-02 11:47 GMT+01:00 Andrey Efimov : > Hi FLINK Team, > > I would like to join to Flink team, dig deeper and develop the product. > At the mome

Join to FLINK Team

2016-12-02 Thread Andrey Efimov
Hi FLINK Team, I would like to join to Flink team, dig deeper and develop the product. At the moment I am busy with FLINK-4905 Could you please assign the FLINK-4905 to me. I am digging the Check

Re: separation of JVMs for different applications

2016-12-02 Thread Fabian Hueske
Hi Manu, As far as I know, there are not plans to change the stand-alone deployment. FLIP-6 is focusing on deployments via resource providers (YARN, Mesos, etc.) which allow to start Flink processes per job. Till (in CC) is more familiar with the FLIP-6 effort and might be able to add more detail

[jira] [Created] (FLINK-5227) Add warning to include flink-table in job fat jars

2016-12-02 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-5227: Summary: Add warning to include flink-table in job fat jars Key: FLINK-5227 URL: https://issues.apache.org/jira/browse/FLINK-5227 Project: Flink Issue Type:

[jira] [Created] (FLINK-5226) Eagerly project unused attributes

2016-12-02 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-5226: Summary: Eagerly project unused attributes Key: FLINK-5226 URL: https://issues.apache.org/jira/browse/FLINK-5226 Project: Flink Issue Type: Improvement