Bjørn, that is the project of "spark connect" which dissociates client and
server from Spark driver. Not related to data source (which is also known
as connector).
Mitch, as I understand correctly, unfortunately we don't have dedicated
documentation for implementing data source/connectors. It's
This is from the vote for spark connector. Is this you are looking for?
The goal of the SPIP is to introduce a Dataframe based client/server API
for Spark
Please also refer to:
- Previous discussion in dev mailing list: [DISCUSS] SPIP: Spark Connect -
A client and server interface for Apache
Hello,
I’m wondering if anyone can point me in the right direction for a Spark
connector developer guide.
I’m looking for information on writing a new connector for Spark to move data
between Apache Spark and other systems.
Any information would be helpful. I found a similar thing for
