[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22560 Sorry for the late reply. Could we hold this PR since we are actively working on the data source API V2? Will revisit it after that. Also cc @gengliangwang @cloud-fan FYI, I just marked its target version to 3.0. We will address the issue in the upcoming 3.0 release. Thanks for your PR! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user fsauer65 commented on the issue: https://github.com/apache/spark/pull/22560 @maropu : This would certainly be another (perhaps better?) way to accomplish the behavior I'm looking for. However, I can't find where JdbcDialect is used on master to abstract connections, JDBCRDD is still using JdbcUtils to get a connection - does this exist on some other branch? : https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22560 Just a question about the design: what's the relationship between `JdbcDialect` and tihs provider? In the master, we now abstract JDBC connections by `JdbcDialect`. BTW, you can't reuse `PostgresDialect` by connecting an AWS Aurora postgres? We need to add a new interface to do so? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22560 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22560 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97370/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22560 **[Test build #97370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97370/testReport)** for PR 22560 at commit [`f5671a3`](https://github.com/apache/spark/commit/f5671a3552322f96bf26ec78392f4383bb6d6cd3). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait ConnectionFactoryProvider extends Serializable ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22560 **[Test build #97370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97370/testReport)** for PR 22560 at commit [`f5671a3`](https://github.com/apache/spark/commit/f5671a3552322f96bf26ec78392f4383bb6d6cd3). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user fsauer65 commented on the issue: https://github.com/apache/spark/pull/22560 tests are failing with ``` Caused by: sbt.ForkMain$ForkError: java.io.NotSerializableException: org.apache.spark.sql.execution.datasources.jdbc.DefaultConnectionFactoryProvider$ Serialization stack: - object not serializable (class: org.apache.spark.sql.execution.datasources.jdbc.DefaultConnectionFactoryProvider$, value: org.apache.spark.sql.execution.datasources.jdbc.DefaultConnectionFactoryProvider$@7d4b4fa8) ``` Not sure what the right way is to fix this but could it be as simple as extending Serializable? I don't understand why I'm not seeing this in our environment. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22560 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22560 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97369/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22560 **[Test build #97369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97369/testReport)** for PR 22560 at commit [`526a1d0`](https://github.com/apache/spark/commit/526a1d0ac7922bc104864f3137aa993f9c4e7c67). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22560: [SPARK-25547][SQL] Pluggable JDBC connection factory
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22560 **[Test build #97369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97369/testReport)** for PR 22560 at commit [`526a1d0`](https://github.com/apache/spark/commit/526a1d0ac7922bc104864f3137aa993f9c4e7c67). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
