Muchl created FLINK-16839:
-----------------------------

             Summary: Connectors Kinesis metrics can be disabled
                 Key: FLINK-16839
                 URL: https://issues.apache.org/jira/browse/FLINK-16839
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / Kinesis
    Affects Versions: 1.10.0
            Reporter: Muchl


Currently, there are 9 metrics in the kinesis connector, each of which is 
recorded according to the kinesis shard dimension. If there are enough shards, 
taskmanager mtrics will be unavailable.

In our production environment, a case of a job that reads dynamodb stream 
kinesis adapter, this dynamodb has more than 10,000 shards, multiplied by 9 
metrics, there are more than 100,000 metrics for kinesis, and the entire 
metrics output reaches tens of MB , Cause prometheus cannot collect metrics.

We should have a configuration item that can disable kinesis metrics



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to