Hi,

I'm curious why Spark SQL uses two different methods for the seemingly very
same code?

* DataFrameWriter.runCommand -->
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala#L663

* Dataset.withAction -->
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L3317

It looks like the relationship is as follows:

DataFrameWriter.runCommand == Dataset.withAction(_.execute)

Should one be removed for the other? I'd first change runCommand to use
withAction(_.execute) or even remove runCommand altogether.

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski

Reply via email to