[
https://issues.apache.org/jira/browse/AIRFLOW-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726763#comment-16726763
]
Ash Berlin-Taylor commented on AIRFLOW-3545:
--------------------------------------------
Right now this is not possible, and due to the heavy use of (sub)processes by
Airflow this isn't a trivial change.
I'd suggest taking a look at https://github.com/prometheus/statsd_exporter
> Can't use Prometheus or other pull based instrumentation systems to monitor
> Tasks launched on Kubernetes
> --------------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-3545
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3545
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Aditya Vishwakarma
> Priority: Major
>
> Prometheus, which is a common way to instrument services on Kubernetes, uses
> a pull based mechanism to fetch metrics. This involves a service exposing a
> `/metrics` endpoint. This endpoint is scraped every 30 secs by prometheus to
> collect metrics.
> This requires a port to be specified in the generated Pod config. Something
> like below.
> {code:java}
> // Sample Pod Spec
> apiVersion: v1
> kind: Job
> metadata:
> name: batch-job
> spec:
> ports:
> - name: metrics
> port: 9091 # port to fetch metrics from
> protocol: TCP
> targetPort: 9091
> {code}
> Currently KubernetesPodOperator doesn't have any options to open ports like
> this.
> Is it possible to have an option to do this?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
