Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11063#discussion_r54375174
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala
---
@@ -75,12 +83,63 @@ private[sql] object JDBCRelation {
}
ans.toArray
}
+
+ /**
+ * Returns null value predicate if the column is nullable, otherwise
empty string. Uses
+ * nullable information available in the catalyst schema generated for
the source table.
+ * This method avoids querying the database metadata to find if the
unquoted column names
+ * are interpreted as uppercase or lower case. In rare cases when the
schema has on column
+ * with uppercase name, and another one with lowercase, it will return
null value
+ * partition predicate if either one of the column is nullable.
+ *
+ * @param column name of the column
+ * @param url the jdbc url of the database
+ * @param schema table's Catalyst schema
+ * @return null value predicate or empty string.
+ */
+ private def getParitionNullValuePredicate(column: String,
+ url: String, schema: StructType): String = {
--- End diff --
Sorry for the delay. I'd definitely just go for the simpler approach (i.e.
always put the null in 1st partition), and only make it more complicated in the
future if it is big enough of a problem.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
