Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21642#discussion_r200160518 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -73,6 +74,10 @@ private[spark] class AppStatusListener( // around liveExecutors. @volatile private var activeExecutorCount = 0 + private val inputDataSetId = new AtomicLong(0) + private val outputDataSetId = new AtomicLong(0) + private val maxRecords = conf.getInt("spark.data.maxRecords", 1000) --- End diff -- What's this `spark.data.maxRecords` for? Maybe you should follow the config in core/src/main/scala/org/apache/spark/status/config.scala
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org