Hi Ashwin,
This sounds like it might be a good use for Apache Arrow, if you are open
to the type of format to exchange. As of Spark 2.3, Dataset has a method
"toArrowPayload" that will convert a Dataset of Rows to a byte array in
Arrow format, although the API is currently not public. Your
Hi!
I'm building a spark app which runs a spark-sql query and send results to
client over grpc(my proto file is configured to send the sql output as
"bytes"). The client then displays the output rows. When I run spark.sql, I
get a DataSet. How do I convert this to byte array?
Also is there a
