[
https://issues.apache.org/jira/browse/SPARK-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-13167:
------------------------------------
Assignee: Apache Spark
> JDBC data source does not include null value partition columns rows in the
> result.
> ----------------------------------------------------------------------------------
>
> Key: SPARK-13167
> URL: https://issues.apache.org/jira/browse/SPARK-13167
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0, 2.0.0
> Reporter: Suresh Thalamati
> Assignee: Apache Spark
>
> Reading from am JDBC data source using a partition column that is nullable
> can return incorrect number of rows, if there are rows with null value for
> partition column.
> {code}
> val emp =
> sqlContext.read.jdbc("jdbc:h2:mem:testdb0;user=testUser;password=testPass",
> "TEST.EMP", "theid", 0, 4, 3, new Properties)
> emp.count()
> {code}
> Above jdbc read call sets up the partitions of the following form. It does
> not include null predicate.
> {code}
> JDBCPartition(THEID < 1,0),JDBCPartition(THEID >= 1 AND THEID <
> 2,1),JDBCPartition(THEID >= 2,2)
> {code}
> Rows with null values in partition column are not included in the results
> because the partition predicate does not specify is null predicates.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
