Hi Luca, Thanks for your reply, which is very helpful for me :)
I am trying other metrics sinks with cAdvisor to see the effect. If it works well, I will share it with the community. On Fri, Feb 10, 2023 at 4:26 PM Luca Canali <luca.can...@cern.ch> wrote: > Hi Qian, > > > > Indeed the metrics available with the Prometheus servlet sink (which is > marked still as experimental) are limited, compared to the full > instrumentation, and this is due to the way it is implemented with a > servlet and cannot be easily extended from what I can see. > > You can use another supported metrics sink (see > https://spark.apache.org/docs/latest/monitoring.html#metrics ) if you > want to collect all the metrics are exposed by Spark executors. > > For example, I use the graphite sink and then collect metrics into an > InfluxDB instance (see https://github.com/cerndb/spark-dashboard ) > > An additional comment is that there is room for having more sinks > available for Apache Spark metrics, notably for InfluxDB and for Prometheus > (gateway), if someone is interested in working on that. > > > > Best, > > Luca > > > > > > *From:* Qian Sun <qian.sun2...@gmail.com> > *Sent:* Friday, February 10, 2023 05:05 > *To:* dev <d...@spark.apache.org>; user.spark <user@spark.apache.org> > *Subject:* Executor metrics are missing on prometheus sink > > > > Setting up prometheus sink in this way: > > -c spark.ui.prometheus.enabled=true > > -c spark.executor.processTreeMetrics.enabled=true > > -c spark.metrics.conf=/spark/conf/metric.properties > > *metric.properties:*{} > > *.sink.prometheusServlet.class=org.apache.spark.metrics.sink.PrometheusServlet > > *.sink.prometheusServlet.path=/metrics/prometheus > > Result: > > Both of these endpoints have some metrics > > <driver-ip>:4040/metrics/prometheus > > <driver-ip>:4040/metrics/executors/prometheus > > > > But the executor one misses metrics under the executor namespace > described here: > > > https://spark.apache.org/docs/latest/monitoring.html#component-instance--executor > > > > *How to expose executor metrics on spark exeuctors pod?* > > > > *Any help will be appreciated.* > > -- > > Regards, > > Qian Sun > -- Regards, Qian Sun
