If you're using just Spark you could try turning on the history server <https://spark.apache.org/docs/latest/monitoring.html> and try to glean statistics from there. But there is no one location or log file which stores them all. Databricks, which is a managed Spark solution, provides such features in an Enterprise setting. I am unsure whether AWS EMR or Google Data Proc does the same.
Femi On Mon, Apr 8, 2024 at 5:34 AM casel.chen <casel_c...@126.com> wrote: > Hello, I have a spark application with jdbc source and do some > calculation. > To monitor application healthy, I need db related metrics per database > like number of connections, sql execution time and sql fired time > distribution etc. > Does anybody know how to get them? Thanks! > > -- http://dataphantik.com "Great spirits have always encountered violent opposition from mediocre minds." - Albert Einstein.
