[GitHub] spark pull request #20370: Changing JDBC relation to better process quotes

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20370


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20370: Changing JDBC relation to better process quotes

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/20370#discussion_r163403070
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala
 ---
@@ -78,7 +78,8 @@ private[sql] object JDBCRelation extends Logging {
 // Overflow and silliness can happen if you subtract then divide.
 // Here we get a little roundoff, but that's (hopefully) OK.
 val stride: Long = upperBound / numPartitions - lowerBound / 
numPartitions
-val column = partitioning.column
+val dialect = JdbcDialects.get(jdbcOptions.url)
+val column = dialect.quoteIdentifier(partitioning.column)
--- End diff --

We also need to add a test case in `PostgresIntegrationSuite`


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20370: Changing JDBC relation to better process quotes

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/20370#discussion_r163402979
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala
 ---
@@ -78,7 +78,8 @@ private[sql] object JDBCRelation extends Logging {
 // Overflow and silliness can happen if you subtract then divide.
 // Here we get a little roundoff, but that's (hopefully) OK.
 val stride: Long = upperBound / numPartitions - lowerBound / 
numPartitions
-val column = partitioning.column
+val dialect = JdbcDialects.get(jdbcOptions.url)
+val column = dialect.quoteIdentifier(partitioning.column)
--- End diff --

We should do it in `class JDBCOptions`. To avoid breaking the behavior, we 
should eat the quotes if users manually specify them. 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20370: Changing JDBC relation to better process quotes

[conorbmurphy]

GitHub user conorbmurphy opened a pull request:

https://github.com/apache/spark/pull/20370

Changing JDBC relation to better process quotes

## What changes were proposed in this pull request?

The way JDBC writes currently work, they do not properly account for mixed 
case column names.  Instead, the user has to use quotes on each column name.  
This change avoids that.

## How was this patch tested?

Manual tests and working with @dougbateman and @gatorsmile 

Please review http://spark.apache.org/contributing.html before opening a 
pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/conorbmurphy/spark-1 master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20370.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20370


commit d2864d06c039cb0c0b0c9d9271c9757309017e1a
Author: conorbmurphy 
Date:   2018-01-23T22:43:32Z

Changing JDBC relation to better process quotes




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org