[ https://issues.apache.org/jira/browse/SPARK-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reynold Xin resolved SPARK-13167. --------------------------------- Resolution: Fixed Assignee: Suresh Thalamati Fix Version/s: 2.0.0 > JDBC data source does not include null value partition columns rows in the > result. > ---------------------------------------------------------------------------------- > > Key: SPARK-13167 > URL: https://issues.apache.org/jira/browse/SPARK-13167 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.6.0, 2.0.0 > Reporter: Suresh Thalamati > Assignee: Suresh Thalamati > Fix For: 2.0.0 > > > Reading from am JDBC data source using a partition column that is nullable > can return incorrect number of rows, if there are rows with null value for > partition column. > {code} > val emp = > sqlContext.read.jdbc("jdbc:h2:mem:testdb0;user=testUser;password=testPass", > "TEST.EMP", "theid", 0, 4, 3, new Properties) > emp.count() > {code} > Above jdbc read call sets up the partitions of the following form. It does > not include null predicate. > {code} > JDBCPartition(THEID < 1,0),JDBCPartition(THEID >= 1 AND THEID < > 2,1),JDBCPartition(THEID >= 2,2) > {code} > Rows with null values in partition column are not included in the results > because the partition predicate does not specify is null predicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org