Copilot commented on code in PR #1982:
URL: https://github.com/apache/auron/pull/1982#discussion_r3032037467
##########
spark-extension/src/main/scala/org/apache/spark/sql/execution/auron/plan/NativeWindowBase.scala:
##########
@@ -67,8 +67,7 @@ abstract class NativeWindowBase(
override lazy val metrics: Map[String, SQLMetric] = SortedMap[String,
SQLMetric]() ++ Map(
NativeHelper
- .getDefaultNativeMetrics(sparkContext)
- .filterKeys(Set("stage_id", "output_rows", "elapsed_compute"))
+ .getDefaultNativeMetrics(sparkContext, Set("stage_id", "output_rows",
"elapsed_compute"))
.toSeq: _*)
Review Comment:
`getDefaultNativeMetrics` now returns a `Map` directly, but this call site
still does `.toSeq` and rebuilds another `Map` just to merge into a
`SortedMap`. You can reduce allocations (and keep deterministic ordering) by
merging the returned map directly (e.g., `SortedMap.empty ++
NativeHelper.getDefaultNativeMetrics(...)`).
##########
spark-extension/src/main/scala/org/apache/spark/sql/auron/NativeHelper.scala:
##########
@@ -233,7 +245,8 @@ object NativeHelper extends Logging {
}
def getNativeFileScanMetrics(sc: SparkContext): Map[String, SQLMetric] =
TreeMap(
- getDefaultNativeMetrics(sc)
- .filterKeys(Set("stage_id", "output_rows", "elapsed_compute"))
- .toSeq ++ getDefaultNativeFileMetrics(sc).toSeq: _*)
+ getDefaultNativeMetrics(
+ sc,
+ Set("stage_id", "output_rows", "elapsed_compute")).toSeq ++
getDefaultNativeFileMetrics(
+ sc).toSeq: _*)
Review Comment:
`getNativeFileScanMetrics` still materializes
`getDefaultNativeMetrics(...).toSeq` and reconstructs a `TreeMap` via varargs.
Since `getDefaultNativeMetrics` now returns a map directly, you can avoid the
intermediate sequence/varargs by combining maps with `++` (e.g.,
`getDefaultNativeMetrics(...) ++ getDefaultNativeFileMetrics(sc)`), preserving
sorted output while being a bit lighter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
