spark git commit: [SPARK-19832][SQL] DynamicPartitionWriteTask get partitionPath should escape the partition name

2017-03-06 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 1f6c090c1 -> e52499ea9 [SPARK-19832][SQL] DynamicPartitionWriteTask get partitionPath should escape the partition name ## What changes were proposed in this pull request? Currently in DynamicPartitionWriteTask, when we get the

spark git commit: [SPARK-19818][SPARKR] rbind should check for name consistency of input data frames

2017-03-06 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 9909f6d36 -> 1f6c090c1 [SPARK-19818][SPARKR] rbind should check for name consistency of input data frames ## What changes were proposed in this pull request? Added checks for name consistency of input data frames in union. ## How was

spark git commit: [SPARK-19350][SQL] Cardinality estimation of Limit and Sample

2017-03-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/master b0a5cd890 -> 9909f6d36 [SPARK-19350][SQL] Cardinality estimation of Limit and Sample ## What changes were proposed in this pull request? Before this pr, LocalLimit/GlobalLimit/Sample propagates the same row count and column stats from

spark git commit: [SPARK-19719][SS] Kafka writer for both structured streaming and batch queires

2017-03-06 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.1 ca7a7e8a8 -> fd6c6d5c3 [SPARK-19719][SS] Kafka writer for both structured streaming and batch queires ## What changes were proposed in this pull request? Add a new Kafka Sink and Kafka Relation for writing streaming and batch

spark git commit: [SPARK-19719][SS] Kafka writer for both structured streaming and batch queires

2017-03-06 Thread tdas
Repository: spark Updated Branches: refs/heads/master f6471dc0d -> b0a5cd890 [SPARK-19719][SS] Kafka writer for both structured streaming and batch queires ## What changes were proposed in this pull request? Add a new Kafka Sink and Kafka Relation for writing streaming and batch queries,

spark git commit: [SPARK-19709][SQL] Read empty file with CSV data source

2017-03-06 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 926543664 -> f6471dc0d [SPARK-19709][SQL] Read empty file with CSV data source ## What changes were proposed in this pull request? Bugfix for reading empty file with CSV data source. Instead of throwing `NoSuchElementException`, an empty

spark git commit: [SPARK-16845][SQL][BRANCH-1.6] GeneratedClass$SpecificOrdering` grows beyond 64 KB

2017-03-06 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-1.6 a50ef3d9a -> 23f9faa40 [SPARK-16845][SQL][BRANCH-1.6] GeneratedClass$SpecificOrdering` grows beyond 64 KB ## What changes were proposed in this pull request? This is a backport pr of #15480 into `branch-1.6`. ## How was this patch

spark git commit: [SPARK-16845][SQL][BRANCH-2.0] GeneratedClass$SpecificOrdering` grows beyond 64 KB

2017-03-06 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 c7e7b042d -> 0cc992c89 [SPARK-16845][SQL][BRANCH-2.0] GeneratedClass$SpecificOrdering` grows beyond 64 KB ## What changes were proposed in this pull request? This is a backport pr of #15480 into `branch-2.0`. ## How was this patch

spark git commit: [SPARK-19382][ML] Test sparse vectors in LinearSVCSuite

2017-03-06 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 9991c2dad -> 926543664 [SPARK-19382][ML] Test sparse vectors in LinearSVCSuite ## What changes were proposed in this pull request? Add unit tests for testing SparseVector. We can't add mixed DenseVector and SparseVector test case, as

spark git commit: [SPARK-19211][SQL] Explicitly prevent Insert into View or Create View As Insert

2017-03-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 12bf83240 -> 9991c2dad [SPARK-19211][SQL] Explicitly prevent Insert into View or Create View As Insert ## What changes were proposed in this pull request? Currently we don't explicitly forbid the following behaviors: 1. The statement

spark git commit: [SPARK-19304][STREAMING][KINESIS] fix kinesis slow checkpoint recovery

2017-03-06 Thread brkyvz
Repository: spark Updated Branches: refs/heads/master 339b53a13 -> 46a64d1e0 [SPARK-19304][STREAMING][KINESIS] fix kinesis slow checkpoint recovery ## What changes were proposed in this pull request? added a limit to getRecords api call call in KinesisBackedBlockRdd. This helps reduce the

[1/2] spark git commit: [SPARK-19257][SQL] location for table/partition/database should be java.net.URI

2017-03-06 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 46a64d1e0 -> 096df6d93 http://git-wip-us.apache.org/repos/asf/spark/blob/096df6d9/sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala -- diff

[2/2] spark git commit: [SPARK-19257][SQL] location for table/partition/database should be java.net.URI

2017-03-06 Thread wenchen
[SPARK-19257][SQL] location for table/partition/database should be java.net.URI ## What changes were proposed in this pull request? Currently we treat the location of table/partition/database as URI string. It will be safer if we can make the type of location as java.net.URI. In this PR, there

spark git commit: [SPARK-19737][SQL] New analysis rule for reporting unregistered functions without relying on relation resolution

2017-03-06 Thread lian
Repository: spark Updated Branches: refs/heads/master 2a0bc867a -> 339b53a13 [SPARK-19737][SQL] New analysis rule for reporting unregistered functions without relying on relation resolution ## What changes were proposed in this pull request? This PR adds a new `Once` analysis rule batch

spark git commit: [SPARK-17495][SQL] Support Decimal type in Hive-hash

2017-03-06 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 207067ead -> 2a0bc867a [SPARK-17495][SQL] Support Decimal type in Hive-hash ## What changes were proposed in this pull request? Hive hash to support Decimal datatype. [Hive internally normalises