There’s another pr https://github.com/apache/spark/pull/21861 but which is based the old V2 APIs.
We’d better link the JIRAs, SPARK-24907, SPARK-25547, and SPARK-28380 and finalize a plan. Xianyin From: Shiv Prashant Sood <shivprash...@gmail.com> Date: Sunday, July 14, 2019 at 2:59 AM To: Gabor Somogyi <gabor.g.somo...@gmail.com> Cc: Xianyin Xin <xianyin....@alibaba-inc.com>, Ryan Blue <rb...@netflix.com>, <gengliang.w...@databricks.com>, Spark Dev List <dev@spark.apache.org> Subject: Re: JDBC connector for DataSourceV2 To me this looks like refactoring of DS1 JDBC to enable user provided connection factories. In itself a good change, but IMO not DSV2 related. I created a JIRA and added some goals. Please comments/add as relevant. https://issues.apache.org/jira/browse/SPARK-28380 JIRA for DataSourceV2 API based JDBC connector. Goals : Generic connector based on JDBC that supports all databases (min bar is support for all V1 data bases). Reference implementation and Interface for any specialized JDBC connectors. Regards, Shiv On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <gabor.g.somo...@gmail.com> wrote: Hi Guys, Don't know what's the intention exactly here but there is such a PR: https://github.com/apache/spark/pull/22560 If that's what we need maybe we can resurrect it. BTW, I'm also interested in... BR, G On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <shivprash...@gmail.com> wrote: Thanks all. I can also contribute toward this effort. Regards, Shiv Sent from my iPhone On Jul 12, 2019, at 6:51 PM, Xianyin Xin <xianyin....@alibaba-inc.com> wrote: If there’s nobody working on that, I’d like to contribute. Loop in @Gengliang Wang. Xianyin From: Ryan Blue <rb...@netflix.com.INVALID> Reply-To: <rb...@netflix.com> Date: Saturday, July 13, 2019 at 6:54 AM To: Shiv Prashant Sood <shivprash...@gmail.com> Cc: Spark Dev List <dev@spark.apache.org> Subject: Re: JDBC connector for DataSourceV2 I'm not aware of a JDBC connector effort. It would be great to have someone build one! On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <shivprash...@gmail.com> wrote: Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA. DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path. Thanks & Regards, Shiv -- Ryan Blue Software Engineer Netflix
