[
https://issues.apache.org/jira/browse/FLINK-16611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062284#comment-17062284
]
Stephen Whelan commented on FLINK-16611:
----------------------------------------
Currently, the `DatadogHttpReporter` builds a request of all metrics and sends
it wholesome[1]. Datadog has a request size limit. If exceeded, you will see a
response such as:
{code:java}
Response{protocol=http/1.1, code=413, message=Request Entity Too Large,
url=https://app.datadoghq.com/api/v1/series?api_key=some_key}
{code}
We made the following trivial patch to `DatadogHttpReporter.report()`which
solved the issue. Probably more intelligent solutions. We set
`MAX_METRICS_PER_REQUEST` equal to 100 and have had no issues. We're running
Flink 1.9.
{code:java}
int currentCount = 0;
for (DCounter c : counters.values()) {
request.addCounter(c);
currentCount += 1;
if (currentCount >= MAX_METRICS_PER_REQUEST) {
sendRequest(request);
request = new DatadogHttpRequest();
currentCount = 0;
}
}
// send any leftover metrics
sendRequest(request);
{code}
[[1]
https://github.com/apache/flink/blob/master/flink-metrics/flink-metrics-datadog/src/main/java/org/apache/flink/metrics/datadog/DatadogHttpReporter.java#L128|https://github.com/apache/flink/blob/master/flink-metrics/flink-metrics-datadog/src/main/java/org/apache/flink/metrics/datadog/DatadogHttpReporter.java#L128]
> Datadog reporter should chunk large reports
> -------------------------------------------
>
> Key: FLINK-16611
> URL: https://issues.apache.org/jira/browse/FLINK-16611
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Metrics
> Affects Versions: 1.9.0
> Reporter: Chesnay Schepler
> Priority: Major
>
> Datadog has a maximum size for reports that it accepts.
> If the report exceeds this size it is simply rejected, rendering the reporter
> unusable.
> We should investigate what this size limit is, and split the report into
> multiple chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
