What can Mnemonic do with Flink ?

Hi, I'm a contributor from Mnemonic community, I'm looking for an entry point to leverage some features. e.g. durable object/computing models for Flink. Mnemonic has supported some use cases of Hadoop MR and Spark e.g. https://mnemonic.apache.org/docs/sparkusecases.html

[jira] [Created] (FLINK-7723) DispatcherTest instable

Till Rohrmann created FLINK-7723: Summary: DispatcherTest instable Key: FLINK-7723 URL: https://issues.apache.org/jira/browse/FLINK-7723 Project: Flink Issue Type: Bug Components:

[jira] [Created] (FLINK-7722) MiniCluster does not appear to honor Log4j settings

Elias Levy created FLINK-7722: - Summary: MiniCluster does not appear to honor Log4j settings Key: FLINK-7722 URL: https://issues.apache.org/jira/browse/FLINK-7722 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-7721) StatusWatermarkValve should output a new min watermark only if it was aggregated from aligned chhanels

Tzu-Li (Gordon) Tai created FLINK-7721: -- Summary: StatusWatermarkValve should output a new min watermark only if it was aggregated from aligned chhanels Key: FLINK-7721 URL:

Re: JIRA contributor rights for FLINK

Welcome to the Flink community Malcolm :-) I've given you contributor permissions. On Wed, Sep 27, 2017 at 5:07 PM, Malcolm Taylor wrote: > Hi, > I am a new contributor to FLINK. Can someone please assign me contributor > rights in JIRA, so that I can assign an issue ? > My

[jira] [Created] (FLINK-7719) Send checkpoint id to task as part of deployment descriptor when resuming

Stefan Richter created FLINK-7719: - Summary: Send checkpoint id to task as part of deployment descriptor when resuming Key: FLINK-7719 URL: https://issues.apache.org/jira/browse/FLINK-7719 Project:

[jira] [Created] (FLINK-7717) Port TaskManagerMetricsHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7717: -- Summary: Port TaskManagerMetricsHandler to new REST endpoint Key: FLINK-7717 URL: https://issues.apache.org/jira/browse/FLINK-7717 Project: Flink

[jira] [Created] (FLINK-7716) Port JobManagerMetricsHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7716: -- Summary: Port JobManagerMetricsHandler to new REST endpoint Key: FLINK-7716 URL: https://issues.apache.org/jira/browse/FLINK-7716 Project: Flink

[jira] [Created] (FLINK-7713) Port JarUploadHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7713: -- Summary: Port JarUploadHandler to new REST endpoint Key: FLINK-7713 URL: https://issues.apache.org/jira/browse/FLINK-7713 Project: Flink Issue

JIRA contributor rights for FLINK

Hi, I am a new contributor to FLINK. Can someone please assign me contributor rights in JIRA, so that I can assign an issue ? My user name is malcolmt thanks, Malcolm

[jira] [Created] (FLINK-7715) Port JarRunHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7715: -- Summary: Port JarRunHandler to new REST endpoint Key: FLINK-7715 URL: https://issues.apache.org/jira/browse/FLINK-7715 Project: Flink Issue

[jira] [Created] (FLINK-7711) Port JarListHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7711: -- Summary: Port JarListHandler to new REST endpoint Key: FLINK-7711 URL: https://issues.apache.org/jira/browse/FLINK-7711 Project: Flink Issue

[jira] [Created] (FLINK-7709) Port CheckpointStatsDetailsHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7709: -- Summary: Port CheckpointStatsDetailsHandler to new REST endpoint Key: FLINK-7709 URL: https://issues.apache.org/jira/browse/FLINK-7709 Project: Flink

[jira] [Created] (FLINK-7710) Port CheckpointStatsHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7710: -- Summary: Port CheckpointStatsHandler to new REST endpoint Key: FLINK-7710 URL: https://issues.apache.org/jira/browse/FLINK-7710 Project: Flink

[jira] [Created] (FLINK-7707) Port CheckpointStatsDetailsSubtasksHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7707: -- Summary: Port CheckpointStatsDetailsSubtasksHandler to new REST endpoint Key: FLINK-7707 URL: https://issues.apache.org/jira/browse/FLINK-7707 Project:

[jira] [Created] (FLINK-7708) Port CheckpointConfigHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7708: -- Summary: Port CheckpointConfigHandler to new REST endpoint Key: FLINK-7708 URL: https://issues.apache.org/jira/browse/FLINK-7708 Project: Flink

[jira] [Created] (FLINK-7706) Port JobAccumulatorsHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7706: -- Summary: Port JobAccumulatorsHandler to new REST endpoint Key: FLINK-7706 URL: https://issues.apache.org/jira/browse/FLINK-7706 Project: Flink

[jira] [Created] (FLINK-7705) Port JobDetailsHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7705: -- Summary: Port JobDetailsHandler to new REST endpoint Key: FLINK-7705 URL: https://issues.apache.org/jira/browse/FLINK-7705 Project: Flink Issue

[jira] [Created] (FLINK-7704) Port JobPlanHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7704: -- Summary: Port JobPlanHandler to new REST endpoint Key: FLINK-7704 URL: https://issues.apache.org/jira/browse/FLINK-7704 Project: Flink Issue

[jira] [Created] (FLINK-7703) Port JobExceptionsHandler to new REST endpoint

Tzu-Li (Gordon) Tai created FLINK-7703: -- Summary: Port JobExceptionsHandler to new REST endpoint Key: FLINK-7703 URL: https://issues.apache.org/jira/browse/FLINK-7703 Project: Flink

[jira] [Created] (FLINK-7702) Javadocs link broken

Gabor Gevay created FLINK-7702: -- Summary: Javadocs link broken Key: FLINK-7702 URL: https://issues.apache.org/jira/browse/FLINK-7702 Project: Flink Issue Type: Bug Components:

[jira] [Created] (FLINK-7701) IllegalArgumentException in Netty bootstrap with small memory state segment size

Nico Kruber created FLINK-7701: -- Summary: IllegalArgumentException in Netty bootstrap with small memory state segment size Key: FLINK-7701 URL: https://issues.apache.org/jira/browse/FLINK-7701 Project:

[jira] [Created] (FLINK-7700) State merging in RocksDB backend leaves old state

Aljoscha Krettek created FLINK-7700: --- Summary: State merging in RocksDB backend leaves old state Key: FLINK-7700 URL: https://issues.apache.org/jira/browse/FLINK-7700 Project: Flink Issue

Re: Zookeeper failure handling

Thanks Till for pointing these issues out, you are right this is exactly the same problem! Gyula Till Rohrmann ezt írta (időpont: 2017. szept. 27., Sze, 11:18): > I agree that this is not very nice and can put a lot of stress on your > cluster. > > There is actually an

[jira] [Created] (FLINK-7699) Define the BufferListener interface to replace EventlListener in BufferProvider

zhijiang created FLINK-7699: --- Summary: Define the BufferListener interface to replace EventlListener in BufferProvider Key: FLINK-7699 URL: https://issues.apache.org/jira/browse/FLINK-7699 Project: Flink

Re: got Warn message - "the expected leader session ID did not equal the received leader session ID " when using LocalFlinkMiniCluster to interpret scala code

Hi Till, I’ve found that a StandaloneMiniCluster doesn’t startup web fronted when it is running.so,how can i cancel a running job on it with restful method. Cheers, Till > 在 2017年9月20日，15:43，Till Rohrmann 写道： > > Hi XiangWei, > > programmatically there

[jira] [Created] (FLINK-7698) Join with null literals leads to NPE

Timo Walther created FLINK-7698: --- Summary: Join with null literals leads to NPE Key: FLINK-7698 URL: https://issues.apache.org/jira/browse/FLINK-7698 Project: Flink Issue Type: Bug

Re: Zookeeper failure handling

I agree that this is not very nice and can put a lot of stress on your cluster. There is actually an open issue for exactly this [1] and also a PR [2]. The problem is that in the general case it will allow for split-brain situations and therefore it has not been merged yet. I'm actually not

Re: Question about job canceling in Flink

Hi Wangsan, what you are observing is the intended behaviour of Flink in the case of cancelling a job. The assumption is that if a job is cancelled, then there should be no clean shut down. This entails that the Tasks are interrupted in order to terminate them as fast as possible. Thus, the

Re: Zookeeper failure handling

Hi Till, Thanks for the explanation, yes this sounds like a hard problem but it just seems wrong that whenever the ZK leader is restarted all the Flink jobs fail on a cluster. This might be within the overall guarantees of the system but can lead to some cascading failures if every job recovers at

Re: [E] Re: Elasticsearch connector

The problem are the different signatures. The old version seem to take an `ActionRequest` whereas the old version requires a `IndexRequest`. Cheers, Till On Tue, Sep 26, 2017 at 10:09 PM, Premkumar, Ramprabhu < ramprabhu.premku...@verizon.com> wrote: > Thank you for the response! > > It turns

Re: Zookeeper failure handling

Hi Gyula, if we don't listen to the LeaderLatch#notLeader call but instead wait until we see (via the NodeCache) a new leader information being written to the leader path in order to revoke leadership, then we potentially end up running the same job twice. Even though this can theoretically

Re: Zookeeper failure handling

On a second iteration, the whole problem seems to stem from the fact that we revoke leadership from the JM when the notLeader method is called before waiting for a new leader to be elected. Ideally we should wait until isLeader is called again to check who was the previous leader but I can see how

Question about job canceling in Flink

Hi all, We are currently using BucketingSink to save data into HDFS in parquet format. But when the flink job was canceled, we always got Exception in BucketingSink’s close method. The detailed exception info is as below: [ERROR] [2017-09-26 20:51:58,893]

[jira] [Created] (FLINK-7697) Add metrics for Elasticsearch Sink

Hai Zhou UTC+8 created FLINK-7697: - Summary: Add metrics for Elasticsearch Sink Key: FLINK-7697 URL: https://issues.apache.org/jira/browse/FLINK-7697 Project: Flink Issue Type: Wish