Re: Flink Streaming Counter

2021-03-24 Thread Matthias Pohl 
Hi Vijayendra,
what about the example from the docs you already referred to [1]?

Best,
Matthias

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/ops/metrics.html#counter

On Tue, Mar 23, 2021 at 6:48 PM Vijayendra Yadav 
wrote:

> Hi Pohl,
>
> Thanks for getting back to me so quickly. I am looking for a sample
> example where I can increment counters on each stage #1 thru #3 for
> DATASTREAM.
> Then probably I can print it using slf4j.
>
> Thanks,
> Vijay
>
> On Tue, Mar 23, 2021 at 6:35 AM Matthias Pohl 
> wrote:
>
>> Hi Vijayendra,
>> thanks for reaching out to the Flink community. What do you mean by
>> displaying it in your local IDE? Would it be ok to log the information out
>> onto stdout? You might want to have a look at the docs about setting up a
>> slf4j metrics report [1] if that's the case.
>>
>> Best,
>> Matthias
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.10/monitoring/metrics.html#slf4j-orgapacheflinkmetricsslf4jslf4jreporter
>>
>> On Tue, Mar 23, 2021 at 2:09 AM Vijayendra Yadav 
>> wrote:
>>
>>> Hi Team,
>>>
>>> Could you provide a sample how to pass Flink Datastream Source and sink
>>> results to increment COUNTER and then I want to display the Counter in
>>> Local IDE.
>>> Counter to display for #1 through #3.
>>>
>>> 1) DataStream messageStream = env.addSource(Kinesis Source);
>>> 2) DataStream outputStream =
>>> messageStream.rebalance().map(CustomMapFunction());
>>> 3) outputStream.addSink(Streaming File Sink).
>>>
>>> public class MyMapper extends RichMapFunction {
>>>   private transient Counter counter;
>>>
>>>   @Override
>>>   public void open(Configuration config) {
>>> this.counter = getRuntimeContext()
>>>   .getMetricGroup()
>>>   .counter("myCounter");
>>>   }
>>>
>>>   @Override
>>>   public String map(String value) throws Exception {
>>> this.counter.inc();
>>> return value;
>>>   }}
>>>
>>>
>>> Thanks,
>>> Vijay
>>>
>>

Re: Flink Streaming Counter

2021-03-23 Thread Vijayendra Yadav 
Hi Pohl,

Thanks for getting back to me so quickly. I am looking for a sample example
where I can increment counters on each stage #1 thru #3 for DATASTREAM.
Then probably I can print it using slf4j.

Thanks,
Vijay

On Tue, Mar 23, 2021 at 6:35 AM Matthias Pohl 
wrote:

> Hi Vijayendra,
> thanks for reaching out to the Flink community. What do you mean by
> displaying it in your local IDE? Would it be ok to log the information out
> onto stdout? You might want to have a look at the docs about setting up a
> slf4j metrics report [1] if that's the case.
>
> Best,
> Matthias
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/monitoring/metrics.html#slf4j-orgapacheflinkmetricsslf4jslf4jreporter
>
> On Tue, Mar 23, 2021 at 2:09 AM Vijayendra Yadav 
> wrote:
>
>> Hi Team,
>>
>> Could you provide a sample how to pass Flink Datastream Source and sink
>> results to increment COUNTER and then I want to display the Counter in
>> Local IDE.
>> Counter to display for #1 through #3.
>>
>> 1) DataStream messageStream = env.addSource(Kinesis Source);
>> 2) DataStream outputStream =
>> messageStream.rebalance().map(CustomMapFunction());
>> 3) outputStream.addSink(Streaming File Sink).
>>
>> public class MyMapper extends RichMapFunction {
>>   private transient Counter counter;
>>
>>   @Override
>>   public void open(Configuration config) {
>> this.counter = getRuntimeContext()
>>   .getMetricGroup()
>>   .counter("myCounter");
>>   }
>>
>>   @Override
>>   public String map(String value) throws Exception {
>> this.counter.inc();
>> return value;
>>   }}
>>
>>
>> Thanks,
>> Vijay
>>
>

Re: Flink Streaming Counter

2021-03-23 Thread Matthias Pohl 
Hi Vijayendra,
thanks for reaching out to the Flink community. What do you mean by
displaying it in your local IDE? Would it be ok to log the information out
onto stdout? You might want to have a look at the docs about setting up a
slf4j metrics report [1] if that's the case.

Best,
Matthias

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/monitoring/metrics.html#slf4j-orgapacheflinkmetricsslf4jslf4jreporter

On Tue, Mar 23, 2021 at 2:09 AM Vijayendra Yadav 
wrote:

> Hi Team,
>
> Could you provide a sample how to pass Flink Datastream Source and sink
> results to increment COUNTER and then I want to display the Counter in
> Local IDE.
> Counter to display for #1 through #3.
>
> 1) DataStream messageStream = env.addSource(Kinesis Source);
> 2) DataStream outputStream =
> messageStream.rebalance().map(CustomMapFunction());
> 3) outputStream.addSink(Streaming File Sink).
>
> public class MyMapper extends RichMapFunction {
>   private transient Counter counter;
>
>   @Override
>   public void open(Configuration config) {
> this.counter = getRuntimeContext()
>   .getMetricGroup()
>   .counter("myCounter");
>   }
>
>   @Override
>   public String map(String value) throws Exception {
> this.counter.inc();
> return value;
>   }}
>
>
> Thanks,
> Vijay
>

Flink Streaming Counter

2021-03-22 Thread Vijayendra Yadav 
Hi Team,

Could you provide a sample how to pass Flink Datastream Source and sink
results to increment COUNTER and then I want to display the Counter in
Local IDE.
Counter to display for #1 through #3.

1) DataStream messageStream = env.addSource(Kinesis Source);
2) DataStream outputStream =
messageStream.rebalance().map(CustomMapFunction());
3) outputStream.addSink(Streaming File Sink).

public class MyMapper extends RichMapFunction {
  private transient Counter counter;

  @Override
  public void open(Configuration config) {
this.counter = getRuntimeContext()
  .getMetricGroup()
  .counter("myCounter");
  }

  @Override
  public String map(String value) throws Exception {
this.counter.inc();
return value;
  }}


Thanks,
Vijay