How to register and send custom metrics from Flink?

Hi guys, I'm retrying to send some app related custom metrics from Flink to Datadog via StatsD. I followed https://ci.apache.org/projects/flink/flink-docs- release-1.2/monitoring/metrics.html to set up flink-conf.yaml and test code like this // flink-conf.yaml metrics.reporters: stsd

[jira] [Created] (FLINK-6026) Cannot name flatMap operations

Luke Hutchison created FLINK-6026: - Summary: Cannot name flatMap operations Key: FLINK-6026 URL: https://issues.apache.org/jira/browse/FLINK-6026 Project: Flink Issue Type: Bug Affects Ve

Re: How to register and send custom metrics from Flink?

Hello, yes, Flink can handle custom metrics. It is quite odd that you can see the system metrics but not your own; I don't see a problem with your code. How long is the job running? Since you create a metric for the flatMap operation the metric will only be exposed as long as the flatMap oper

[jira] [Created] (FLINK-6027) Ignore the exception thrown by the subsuming of old completed checkpoints

Xiaogang Shi created FLINK-6027: --- Summary: Ignore the exception thrown by the subsuming of old completed checkpoints Key: FLINK-6027 URL: https://issues.apache.org/jira/browse/FLINK-6027 Project: Flink

[jira] [Created] (FLINK-6028) Unnamed operations in main method logged as "main(null:-1)"

Luke Hutchison created FLINK-6028: - Summary: Unnamed operations in main method logged as "main(null:-1)" Key: FLINK-6028 URL: https://issues.apache.org/jira/browse/FLINK-6028 Project: Flink

[jira] [Created] (FLINK-6029) Strange linebreaks in web dashboard graph view make it hard to read text

Luke Hutchison created FLINK-6029: - Summary: Strange linebreaks in web dashboard graph view make it hard to read text Key: FLINK-6029 URL: https://issues.apache.org/jira/browse/FLINK-6029 Project: Fli

Re: How to register and send custom metrics from Flink?

Hi Chesnay, You saved my day! Yes, the WordCount example runs too fast, about 20ms. I extended its running time, and also set 'metrics

Request permissions for wiki

Hi there, I would like to create a FLIP for FLINK-5734 on the wiki. Could you please assign relevant permissions to me? Best, Pat -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Request-permissions-for-wiki-tp16449.html Sent from the Apache Fli

[jira] [Created] (FLINK-6030) Job metrics are not updated correctly

Avihai Berkovitz created FLINK-6030: --- Summary: Job metrics are not updated correctly Key: FLINK-6030 URL: https://issues.apache.org/jira/browse/FLINK-6030 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-6031) Add parameter for per job yarn clusters to control whether the user code jar is included into the system classloader.

Robert Metzger created FLINK-6031: - Summary: Add parameter for per job yarn clusters to control whether the user code jar is included into the system classloader. Key: FLINK-6031 URL: https://issues.apache.org/jir

Re: Frontend classpath issue

I think the change reduces the chances to run into classloading issues in case there's a bug in Flink (= it is using the wrong CL) I've filed a JIRA for the problem: https://issues.apache.org/jira/browse/FLINK-6031 On Fri, Feb 24, 2017 at 9:29 PM, Gyula Fóra wrote: > Hi, > I am wondering whethe

Wrong argument used in StreamTask when user-defined backend is used

https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L716 This should be : LOG.info("Using user-defined state backend: {}.", fromJob); Regards, Vladislav

Re: Frontend classpath issue

I think we need to get away from the dynamic class loading as much as possible. It breaks way to soon and causes easily class leaks. I would be in favor if understanding how to fix this on the Flink side, i.e., either: - Having flags for disabling it optionally - Having an option of "user cod

Re: [DISCUSS] FLIP-17 Side Inputs

Ha! this is turning out to be quite the discussion. :-) Also, thanks Kenn, for chiming in with the Beam perspective! I'll try and address some stuff. It seems we have some consensus on using N-ary operator to implement side inputs. I see two ways forward there: - Have a "pure" N-ary operator tha

[jira] [Created] (FLINK-6032) CEP-Clean up the operator state when not needed.

Kostas Kloudas created FLINK-6032: - Summary: CEP-Clean up the operator state when not needed. Key: FLINK-6032 URL: https://issues.apache.org/jira/browse/FLINK-6032 Project: Flink Issue Type:

[jira] [Created] (FLINK-6033) Support UNNEST query in the stream SQL api

Haohui Mai created FLINK-6033: - Summary: Support UNNEST query in the stream SQL api Key: FLINK-6033 URL: https://issues.apache.org/jira/browse/FLINK-6033 Project: Flink Issue Type: Improvement

Re: Request permissions for wiki

I have given you the permissions. Please let me know if something does not work. Thanks, Fabian 2017-03-13 11:29 GMT+01:00 Pattarawat Chormai : > Hi there, > > I would like to create a FLIP for FLINK-5734 on the wiki. > > Could you please assign relevant permissions to me? > > Best, > Pat > > >

[jira] [Created] (FLINK-6034) Add KeyedStateHandle for the snapshots in keyed streams

Xiaogang Shi created FLINK-6034: --- Summary: Add KeyedStateHandle for the snapshots in keyed streams Key: FLINK-6034 URL: https://issues.apache.org/jira/browse/FLINK-6034 Project: Flink Issue Typ

Flink as a Service (FaaS)

Hi there, I am very happy about Flink 1.2 release. It was much more robust and feature rich compare to previous versions. In the following section, I would like to discuss a non typical use case in flink community. With ever increasing popularity of micro services[1] to scale out popular online s

[jira] [Created] (FLINK-6035) The method name(String) is undefined for the type UnsortedGrouping

Luke Hutchison created FLINK-6035: - Summary: The method name(String) is undefined for the type UnsortedGrouping Key: FLINK-6035 URL: https://issues.apache.org/jira/browse/FLINK-6035 Project: Flink