Re: Writing Custom Spark Readers and Writers

2022-04-06 Thread Enrico Minack
Another project implementing DataSource V2 in Scala with Python wrapper: https://github.com/G-Research/spark-dgraph-connector Cheers, Enrico Am 06.04.22 um 12:01 schrieb Cheng Pan: There are some projects based on Spark DataSource V2 that I hope will help you.

Re: Writing Custom Spark Readers and Writers

2022-04-06 Thread Cheng Pan
There are some projects based on Spark DataSource V2 that I hope will help you. https://github.com/datastax/spark-cassandra-connector https://github.com/housepower/spark-clickhouse-connector https://github.com/oracle/spark-oracle https://github.com/pingcap/tispark Thanks, Cheng Pan On Wed, Apr

Re: Writing Custom Spark Readers and Writers

2022-04-06 Thread Dyanesh Varun
Thanks a lot ! On Wed, 6 Apr, 2022, 15:21 daniel queiroz, wrote: > > https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/connector/read/index.html > > https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/connector/write/index.html > > >

Re: Writing Custom Spark Readers and Writers

2022-04-06 Thread daniel queiroz
https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/connector/read/index.html https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/connector/write/index.html https://developer.here.com/documentation/java-scala-dev/dev_guide/spark-connector/index.html Grato, Daniel

Writing Custom Spark Readers and Writers

2022-04-06 Thread Dyanesh Varun
Hey team, Can you please share some documentation/blogs where we can get to know how we can write custom sources and sinks for both streaming and static datasets. Thanks in advance Dyanesh Varun