Hi, It's just a (minor?) example of how to use catalyst.dsl package [1], but am currently reviewing deserialize [2] and got a question.
CatalystSerde.deserialize [3] is exactly the deserialize operator (referred above) and since CatalystSerde.deserialize's used in few places like Dataset.rdd [4] as follows: val deserialized = CatalystSerde.deserialize[T](logicalPlan) I'm wondering why the deserialize dsl operator is not used instead that would make the line as follows: val deserialized = deserialize(logicalPlan) which looks so much nicer to my eyes. Any reason for using CatalystSerde.deserialize[T](logicalPlan) instead of this seemingly simpler deserialize operator? Is this because it's just a matter of find-and-replace and no one had time for this? Please help me understand this area better. Thanks! [1] https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala [2] https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala#L304 [3] https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala#L32 [4] https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L2498 Pozdrawiam, Jacek Laskowski ---- https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
