Github user squito commented on the issue: https://github.com/apache/spark/pull/18935 thanks for the added info @jerryshao. (a) ok makes sense now about the use of this -- it is not exposed now, you plan to expose it in future changes. That is fine, however I would like to at least see a *plan* for how you want to expose this before we merge this. I think past efforts have gotten tripped on how we'd expose it in the driver; perhaps the simplest thing to do is expose it to the metric system? There just isn't any point in this unless we're going to do something with it. (b) thanks for including the example of the metrics. But one minor thing I noticed -- are you at all surprised that the direct & heap memory are exactly the same? Are things getting mixed up somewhere? Or maybe those are just the values from the initialization netty always does. I definitely think that is more metrics than most users care about, and would want verbose off by default. I am wondering if those metrics are useful at all ... but I guess they are pretty cheap, and maybe there is some scenario where a user would care about some of them, eg. `numHugeAllocations`.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org