Github user squito commented on the issue:
https://github.com/apache/spark/pull/15512
serializing will create buffers, but since these are only used for
deserializing, I don't think there should even be any buffers created. I guess
the time saved is all the registration which can be skipped?
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala#L85.
I suppose in this case, this is the result of
`spark.sql.shuffle.partitions=100000`, so you end up calling this method a lot.
The only wrinkle I can see here is if reference-tracking is turned on
(which it is, by default). But I think this is taken care of anyway by the way
`borrowKryo`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
