xBis7 commented on code in PR #74046: URL: https://github.com/apache/airflow/pull/74046#discussion_r4326624557
########## airflow-core/docs/administration-and-deployment/logging-monitoring/metrics.rst: ########## @@ -184,6 +185,37 @@ How the attributes surface depends on the backend. Those implementing the OpenTe `Prometheus compatibility <https://opentelemetry.io/docs/specs/otel/compatibility/prometheus_and_openmetrics/>`_ spec expose them as the ``job`` and ``instance`` labels. +.. _short-lived-processes: + +Short-lived processes +^^^^^^^^^^^^^^^^^^^^^ + +A task runs in a process that exits when the attempt ends, so a running total exported by that +process never grows if the values are cumulative. Task SDK processes therefore export counters +and histograms as **delta** values and share ``service.instance.id=task-sdk``, so that every +attempt adds to the same series. ``OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE`` and +``OTEL_RESOURCE_ATTRIBUTES`` override either, if set. + +For backends that support delta values natively, no change is needed. Backends that only +accept cumulative values, such as Prometheus, need the Collector to convert the +values back and to keep idle series long enough for the next run of a Dag to add to them: + +.. code-block:: yaml + + processors: + deltatocumulative: + max_stale: 168h # 1 week Review Comment: I'm looking into this. I'm running manual tests in breeze and taking some measurements. This part below is unnecessary and actually leads to unwanted side effects. I'll remove it. ``` exporters: prometheus: endpoint: 0.0.0.0:8889 metric_expiration: 168h # 1 week ``` Every stream is approximately 1kB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
