[GitHub] spark pull request #23216: [SPARK-26264][CORE]It is better to add @transient...
Github user 10110346 closed the pull request at: https://github.com/apache/spark/pull/23216 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23216: [SPARK-26264][CORE]It is better to add @transient...
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23216#discussion_r238892679 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -56,7 +56,7 @@ private[spark] class ResultTask[T, U]( stageAttemptId: Int, taskBinary: Broadcast[Array[Byte]], partition: Partition, -locs: Seq[TaskLocation], +@transient private var locs: Seq[TaskLocation], --- End diff -- why is it `var` BTW? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23216: [SPARK-26264][CORE]It is better to add @transient...
GitHub user 10110346 opened a pull request: https://github.com/apache/spark/pull/23216 [SPARK-26264][CORE]It is better to add @transient to field 'locs' for class `ResultTask`. ## What changes were proposed in this pull request? The field 'locs' is only used in driver side for class `ResultTask`, so it is not needed to serialize when sending the `ResultTask` to executor. Although it's not very big, it's very frequent, so we can add` transient` for it like `ShuffleMapTask`. ## How was this patch tested? Existed unit tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/10110346/spark locs_transient Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/23216.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #23216 commit b3ede8be1a9073f057cc46fb82eacd7fa3ec36c6 Author: liuxian Date: 2018-12-04T08:55:40Z fix --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
