Re: Adding TaskManager's

No, the "slaves" file is still used to ssh into the machines and start the task manager processes in the start-cluster.sh script. So you still need password-less ssh into the machines if you want to use that. The task managers discover the job manager via ZooKeeper though (therefore you don't need

Re: Adding TaskManager's

Hi Max and Stephan, Does this mean that I can start Flink HA cluster without keeping any entry in "slaves" file ? I'm asking this because then I should not worry about copying public key for password-less ssh in Flink HA cluster On Wed, Feb 17, 2016 at 12:38 PM, Deepak Jha

Re: Adding TaskManager's

Sorry for the typo Stephan On Wednesday, February 17, 2016, Deepak Jha wrote: > Thanks Max and Steven for the response. > > On Wednesday, February 17, 2016, Stephan Ewen > wrote: > >> Hi Deepak! >> >> The

Re: Adding TaskManager's

Thanks Max and Steven for the response. On Wednesday, February 17, 2016, Stephan Ewen wrote: > Hi Deepak! > > The "slaves" file is only used by the SSH script to start a standalone > cluster. > > As Max said, TaskManagers register dynamically at the JobManager. > > Discovery

Re: Affiity Propagation

Hi Josep, welcome to the the Flink dev list! There exists a JIRA issue for Affinity Propagation that is currently unassigned (FLINK-1707 [1]). You can see in the comments that someone started working on it, but they dropped it. As far as I know, nobody is working on this right now, so let me

Hi all

Hi All, I will introduce myself as I’m new in the list, my name is Josep Rubió and I’m from Barcelona. About Flink I just started with, I haven’t gone further than reading some documentation and compiling it. Even being that new I’d like to do a small contribution to it, I will send another

Affiity Propagation

Hi, I’ve been working on an implementation of the binary approach of Affinity Propagation for Giraph (https://github.com/joseprupi/okapi/tree/master/src/main/java/ml/grafos/okapi/clustering/ap) and willing to help Flink I’d like to use it as a starting point to implement it for Gelly. Just

Re: Hi all

Welcome, Josep! On Wed, Feb 17, 2016 at 11:27 AM, Josep Rubio wrote: > Hi All, > > I will introduce myself as I’m new in the list, my name is Josep Rubió and > I’m from Barcelona. > > About Flink I just started with, I haven’t gone further than reading some > documentation and

Hi all

Hi All, I will introduce myself as I’m new in the list, my name is Josep Rubió and I’m from Barcelona. About Flink I just started with, I haven’t gone further than reading some documentation and compiling it. I’m new with this but I’d like to do a small contribution, I will send another

Re: Adding TaskManager's

Hi Deepak! The "slaves" file is only used by the SSH script to start a standalone cluster. As Max said, TaskManagers register dynamically at the JobManager. Discovery works via: - config in non-HA mode - ZooKeeper in HA mode On Wed, Feb 17, 2016 at 10:11 AM, Maximilian Michels

Guarantees for object reuse modes and documentation

Hi, Flink's DataSet API features a configuration parameter called enableObjectReuse(). If activated, Flink's runtime will create fewer objects which results in better performance and lower garbage collection overhead. Depending on whether the configuration switch is enabled or not, user

[jira] [Created] (FLINK-3438) ExternalProcessRunner fails to detect ClassNotFound exception because of locale settings

Stefano Baghino created FLINK-3438: -- Summary: ExternalProcessRunner fails to detect ClassNotFound exception because of locale settings Key: FLINK-3438 URL: https://issues.apache.org/jira/browse/FLINK-3438

[jira] [Created] (FLINK-3437) Fix UI router state for job plan

Greg Hogan created FLINK-3437: - Summary: Fix UI router state for job plan Key: FLINK-3437 URL: https://issues.apache.org/jira/browse/FLINK-3437 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-3436) Remove ComplexIntegrationITCase

Aljoscha Krettek created FLINK-3436: --- Summary: Remove ComplexIntegrationITCase Key: FLINK-3436 URL: https://issues.apache.org/jira/browse/FLINK-3436 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-3435) Change interplay of Ingestion Time and Event Time

Stephan Ewen created FLINK-3435: --- Summary: Change interplay of Ingestion Time and Event Time Key: FLINK-3435 URL: https://issues.apache.org/jira/browse/FLINK-3435 Project: Flink Issue Type:

[jira] [Created] (FLINK-3434) Return value from flinkYarnClient#deploy() should be checked against null

Ted Yu created FLINK-3434: - Summary: Return value from flinkYarnClient#deploy() should be checked against null Key: FLINK-3434 URL: https://issues.apache.org/jira/browse/FLINK-3434 Project: Flink

[jira] [Created] (FLINK-3433) Return value from ObjectInputStream#read() should be checked in AvroOutputFormat#readObject

Ted Yu created FLINK-3433: - Summary: Return value from ObjectInputStream#read() should be checked in AvroOutputFormat#readObject Key: FLINK-3433 URL: https://issues.apache.org/jira/browse/FLINK-3433 Project:

[jira] [Created] (FLINK-3432) ZookeeperOffsetHandlerTest fails on windows

Chesnay Schepler created FLINK-3432: --- Summary: ZookeeperOffsetHandlerTest fails on windows Key: FLINK-3432 URL: https://issues.apache.org/jira/browse/FLINK-3432 Project: Flink Issue Type:

[jira] [Created] (FLINK-3431) Add retrying logic for RocksDB snapshots

Gyula Fora created FLINK-3431: - Summary: Add retrying logic for RocksDB snapshots Key: FLINK-3431 URL: https://issues.apache.org/jira/browse/FLINK-3431 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-3430) Remove "no POJO" warning in TypeAnalyzer

Aljoscha Krettek created FLINK-3430: --- Summary: Remove "no POJO" warning in TypeAnalyzer Key: FLINK-3430 URL: https://issues.apache.org/jira/browse/FLINK-3430 Project: Flink Issue Type:

[jira] [Created] (FLINK-3429) Add histogram-based timestamp/watermark extractor

Robert Metzger created FLINK-3429: - Summary: Add histogram-based timestamp/watermark extractor Key: FLINK-3429 URL: https://issues.apache.org/jira/browse/FLINK-3429 Project: Flink Issue

[jira] [Created] (FLINK-3428) Add fixed time trailing timestamp/watermark extractor

Robert Metzger created FLINK-3428: - Summary: Add fixed time trailing timestamp/watermark extractor Key: FLINK-3428 URL: https://issues.apache.org/jira/browse/FLINK-3428 Project: Flink Issue

[jira] [Created] (FLINK-3427) Add watermark monitoring to JobManager web frontend

Robert Metzger created FLINK-3427: - Summary: Add watermark monitoring to JobManager web frontend Key: FLINK-3427 URL: https://issues.apache.org/jira/browse/FLINK-3427 Project: Flink Issue

[jira] [Created] (FLINK-3426) JobManagerLeader[Re]ElectionTest.testleaderElection fails on Windows

Chesnay Schepler created FLINK-3426: --- Summary: JobManagerLeader[Re]ElectionTest.testleaderElection fails on Windows Key: FLINK-3426 URL: https://issues.apache.org/jira/browse/FLINK-3426 Project:

[jira] [Created] (FLINK-3425) DataSinkTaskTest.Failing[Sorting]DataSinkTask fails on Windows

Chesnay Schepler created FLINK-3425: --- Summary: DataSinkTaskTest.Failing[Sorting]DataSinkTask fails on Windows Key: FLINK-3425 URL: https://issues.apache.org/jira/browse/FLINK-3425 Project: Flink

[jira] [Created] (FLINK-3424) FileStateBackendtest.testStateOutputStream fails on windows

Chesnay Schepler created FLINK-3424: --- Summary: FileStateBackendtest.testStateOutputStream fails on windows Key: FLINK-3424 URL: https://issues.apache.org/jira/browse/FLINK-3424 Project: Flink

Re: Adding TaskManager's

Hi Deepak, The job manager doesn't have to know about task managers. They will simply register at the job manager using the provided configuration. In HA mode, they will lookup the currently leading job manager first and then connect to it. The job manager can then assign work. Cheers, Max On

[jira] [Created] (FLINK-3423) ExternalProcessRunnerTest fails on Windows

Chesnay Schepler created FLINK-3423: --- Summary: ExternalProcessRunnerTest fails on Windows Key: FLINK-3423 URL: https://issues.apache.org/jira/browse/FLINK-3423 Project: Flink Issue Type: