spark git commit: [SPARK-8928] [SQL] Makes CatalystSchemaConverter sticking to 1.4.x- when handling Parquet LISTs in compatible mode

Closes #7314 from liancheng/spark-8928 and squashes the following commits: 75267fe [Cheng Lian] Makes CatalystSchemaConverter sticking to 1.4.x- when handling LISTs in compatible mode Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark

spark git commit: [SPARK-8928] [SQL] Makes CatalystSchemaConverter sticking to 1.4.x- when handling Parquet LISTs in compatible mode

Closes #7304 from liancheng/spark-8928 and squashes the following commits: 75267fe [Cheng Lian] Makes CatalystSchemaConverter sticking to 1.4.x- when handling LISTs in compatible mode Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark

spark git commit: Revert [SPARK-8928] [SQL] Makes CatalystSchemaConverter sticking to 1.4.x- when handling Parquet LISTs in compatible mode

: c056484c0741e2a03d4a916538e1b9e3e65e71c3 Parents: 3dab0da Author: Cheng Lian l...@databricks.com Authored: Wed Jul 8 22:14:38 2015 -0700 Committer: Cheng Lian l...@databricks.com Committed: Wed Jul 8 22:14:38 2015 -0700 -- .../spark/sql/parquet

[2/4] spark git commit: [SPARK-6123] [SPARK-6775] [SPARK-6776] [SQL] Refactors Parquet read path for interoperability and backwards-compatibility

http://git-wip-us.apache.org/repos/asf/spark/blob/4ffc27ca/sql/core/src/test/gen-java/org/apache/spark/sql/parquet/test/thrift/ParquetThriftCompat.java -- diff --git

[3/4] spark git commit: [SPARK-6123] [SPARK-6775] [SPARK-6776] [SQL] Refactors Parquet read path for interoperability and backwards-compatibility

http://git-wip-us.apache.org/repos/asf/spark/blob/4ffc27ca/sql/core/src/test/gen-java/org/apache/spark/sql/parquet/test/avro/CompatibilityTest.java -- diff --git

[1/4] spark git commit: [SPARK-6123] [SPARK-6775] [SPARK-6776] [SQL] Refactors Parquet read path for interoperability and backwards-compatibility

Repository: spark Updated Branches: refs/heads/master 5687f7655 - 4ffc27caa http://git-wip-us.apache.org/repos/asf/spark/blob/4ffc27ca/sql/core/src/test/gen-java/org/apache/spark/sql/parquet/test/thrift/Suit.java -- diff --git

spark git commit: [SPARK-8841] [SQL] Fix partition pruning percentage log message

://git-wip-us.apache.org/repos/asf/spark/diff/39e4e7e4 Branch: refs/heads/master Commit: 39e4e7e4d89077a637c4cad3a986e0e3447d1ae7 Parents: 86768b7 Author: Steve Lindemann steve.lindem...@engineersgatelp.com Authored: Mon Jul 6 10:17:05 2015 -0700 Committer: Cheng Lian l...@databricks.com Committed: Mon

svn commit: r1689292 - /spark/js/downloads.js

Author: lian Date: Sun Jul 5 21:36:13 2015 New Revision: 1689292 URL: http://svn.apache.org/r1689292 Log: Bumps old version threshold to 1.2.0 Modified: spark/js/downloads.js Modified: spark/js/downloads.js URL: http://svn.apache.org/viewvc/spark/js/downloads.js?rev=1689292r1=1689291r2

svn commit: r1689293 - /spark/site/js/downloads.js

Author: lian Date: Sun Jul 5 21:40:14 2015 New Revision: 1689293 URL: http://svn.apache.org/r1689293 Log: Checks in updated site/ directory Modified: spark/site/js/downloads.js Modified: spark/site/js/downloads.js URL: http://svn.apache.org/viewvc/spark/site/js/downloads.js?rev=1689293r1

spark git commit: [SPARK-8501] [SQL] Avoids reading schema from empty ORC files

. But it causes trouble when reading raw ORC files via Spark SQL since we have to discover the schema from the files. Notice that the ORC data source always avoids writing empty ORC files, but it's still problematic when reading Hive tables which contain empty part-files. Author: Cheng Lian l

spark git commit: [SPARK-8501] [SQL] Avoids reading schema from empty ORC files (backport to 1.4)

Repository: spark Updated Branches: refs/heads/branch-1.4 3f1e4efbd - de0802499 [SPARK-8501] [SQL] Avoids reading schema from empty ORC files (backport to 1.4) This PR backports #7199 to branch-1.4 Author: Cheng Lian l...@databricks.com Closes #7200 from liancheng/spark-8501-for-1.4

spark git commit: [SPARK-8690] [SQL] Add a setting to disable SparkSQL parquet schema merge by using datasource API

/repos/asf/spark/diff/246265f2 Branch: refs/heads/master Commit: 246265f2bb056d5e9011d3331b809471a24ff8d7 Parents: 1bbdf9e Author: Wisely Chen wiselyc...@appier.com Authored: Thu Jul 2 09:58:12 2015 -0700 Committer: Cheng Lian l...@databricks.com Committed: Thu Jul 2 09:58:12 2015 -0700

spark git commit: [SPARK-8615] [DOCUMENTATION] Fixed Sample deprecated code

Committer: Cheng Lian l...@databricks.com Committed: Tue Jun 30 10:50:45 2015 -0700 -- docs/sql-programming-guide.md | 10 +- 1 file changed, 5 insertions(+), 5 deletions

spark git commit: [SPARK-8692] [SQL] re-order the case statements that handling catalyst data types

/heads/master Commit: ed413bcc78d8d97a1a0cd0871d7a20f7170476d0 Parents: ea88b1a Author: Wenchen Fan cloud0...@outlook.com Authored: Mon Jun 29 11:41:26 2015 -0700 Committer: Cheng Lian l...@databricks.com Committed: Mon Jun 29 11:41:26 2015 -0700

spark git commit: [SQL] [MINOR] Skip unresolved expression for InConversion

(cherry picked from commit edf09ea1bd4bf7692e0085ad9c70cb1bfc8d06d8) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/457d07ea Tree: http://git-wip-us.apache.org/repos/asf/spark/tree

spark git commit: [SPARK-8604] [SQL] HadoopFsRelation subclasses should set their output format class

committer can be setup correctly when doing appending (where we ignore user defined output committers). Author: Cheng Lian l...@databricks.com Closes #6998 from liancheng/spark-8604 and squashes the following commits: 9be51d1 [Cheng Lian] Adds more comments 6db1368 [Cheng Lian] HadoopFsRelation

spark git commit: [SPARK-8604] [SQL] HadoopFsRelation subclasses should set their output format class

committer can be setup correctly when doing appending (where we ignore user defined output committers). Author: Cheng Lian l...@databricks.com Closes #6998 from liancheng/spark-8604 and squashes the following commits: 9be51d1 [Cheng Lian] Adds more comments 6db1368 [Cheng Lian] HadoopFsRelation

spark git commit: [SPARK-8138] [SQL] Improves error message when conflicting partition columns are found

file:/tmp/foo/b=1 file:/tmp/foo/b=1/c=1 file:/tmp/foo/b=0/c=0 ``` Author: Cheng Lian l...@databricks.com Closes #6610 from liancheng/part-errmsg and squashes the following commits: 7d05f2c [Cheng Lian] Fixes Scala style issue a149250 [Cheng Lian] Adds test case for the error

spark git commit: [SPARK-6777] [SQL] Implements backwards compatibility rules in CatalystSchemaConverter

`. These would be done in follow-up PRs. TODO - [x] More schema conversion test cases for legacy schema patterns. [1]: https://github.com/apache/parquet-format/blob/ea095226597fdbecd60c2419d96b54b2fdb4ae6c/LogicalTypes.md [2]: https://github.com/apache/parquet-mr/ Author: Cheng Lian l

spark git commit: [DOC] [SQL] Addes Hive metastore Parquet table conversion section

[this comment] [1] in #5188) 2. Metadata refreshing requirement introduced in #5339 [1]: https://github.com/apache/spark/pull/5188#issuecomment-86531248 Author: Cheng Lian l...@databricks.com Closes #5348 from liancheng/sql-doc-parquet-conversion and squashes the following commits: 42ae0d0 [Cheng Lian

spark git commit: [SPARK-8139] [SQL] Updates docs and comments of data sources and Parquet output committer options

appear in Parquet 1.7.0. Author: Cheng Lian l...@databricks.com Closes #6683 from liancheng/output-committer-docs and squashes the following commits: b4648b8 [Cheng Lian] Removes spark.sql.sources.outputCommitterClass as it's not a public option ee63923 [Cheng Lian] Updates docs and comments

spark git commit: [SPARK-8508] [SQL] Ignores a test case to cleanup unnecessary testing output until #6882 is merged

build output unnecessarily large and it's hard to debug other build errors. A proper fix is on the way in #6882. This PR ignores this test case temporarily until #6882 is merged. [1]: https://github.com/apache/spark/pull/6404/files#diff-1ea02a6fab84e938582f7f87cc4d9ea1R641 Author: Cheng Lian l

spark git commit: [SPARK-8379] [SQL] avoid speculative tasks write to the same file

Author: jeanlyn jeanly...@gmail.com Authored: Sun Jun 21 00:13:40 2015 -0700 Committer: Cheng Lian l...@databricks.com Committed: Sun Jun 21 00:13:40 2015 -0700 -- .../spark/sql/hive/execution/InsertIntoHiveTable.scala | 1

spark git commit: [SPARK-8379] [SQL] avoid speculative tasks write to the same file

a1e3649c8775d71ca78796b6544284e942ac1331) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f0e40402 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f0e40402 Diff: http://git-wip-us.apache.org/repos

spark git commit: [SPARK-8458] [SQL] Don't strip scheme part of output path when writing ORC files

to the file system configured in Hadoop configuration. Should use `Path.toString` instead. Author: Cheng Lian l...@databricks.com Closes #6892 from liancheng/spark-8458 and squashes the following commits: 87f8199 [Cheng Lian] Don't strip scheme of output path when writing ORC files (cherry picked

spark git commit: [SPARK-8458] [SQL] Don't strip scheme part of output path when writing ORC files

to the file system configured in Hadoop configuration. Should use `Path.toString` instead. Author: Cheng Lian l...@databricks.com Closes #6892 from liancheng/spark-8458 and squashes the following commits: 87f8199 [Cheng Lian] Don't strip scheme of output path when writing ORC files Project: http

spark git commit: [SQL] [MINOR] correct semanticEquals logic

Branch: refs/heads/master Commit: c19c78577a211eefe1112ebd4670a4ce7c3cc3be Parents: e428b3a Author: Wenchen Fan cloud0...@outlook.com Authored: Fri Jun 12 16:38:28 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Fri Jun 12 16:38:28 2015 +0800

spark git commit: [SPARK-6566] [SQL] Related changes for newer parquet version

Authored: Fri Jun 12 13:44:09 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Fri Jun 12 13:44:09 2015 +0800 -- .../spark/sql/parquet/ParquetFilters.scala | 42 - .../sql/parquet/ParquetTableOperations.scala

spark git commit: [SPARK-7886] Added unit test for HAVING aggregate pushdown.

/repos/asf/spark/diff/e90035e6 Branch: refs/heads/master Commit: e90035e676e492de840f44b61b330db526313019 Parents: 57c60c5 Author: Reynold Xin r...@databricks.com Authored: Wed Jun 10 18:58:01 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Wed Jun 10 18:58:01 2015 +0800

spark git commit: [SPARK-7886] Add built-in expressions to FunctionRegistry.

: 1b43ad185b04dd5065facb565cbe7e249521 Parents: 0902a11 Author: Reynold Xin r...@databricks.com Authored: Tue Jun 9 16:24:38 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Tue Jun 9 16:24:38 2015 +0800 -- python/pyspark/sql

spark git commit: [SPARK-7939] [SQL] Add conf to enable/disable partition column type inference

: refs/heads/master Commit: 03ef6be9ce61a13dcd9d8c71298fb4be39119411 Parents: eacd4a9 Author: Liang-Chi Hsieh vii...@gmail.com Authored: Mon Jun 8 17:50:38 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Mon Jun 8 17:50:38 2015 +0800

spark git commit: [SPARK-8118] [SQL] Mutes noisy Parquet log output reappeared after upgrading Parquet to 1.7.0

Repository: spark Updated Branches: refs/heads/master 0ac47083f - 8c321d66d [SPARK-8118] [SQL] Mutes noisy Parquet log output reappeared after upgrading Parquet to 1.7.0 Author: Cheng Lian l...@databricks.com Closes #6670 from liancheng/spark-8118 and squashes the following commits

spark git commit: [SPARK-8141] [SQL] Precompute datatypes for partition columns and reuse it

Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/26d07f1e Branch: refs/heads/master Commit: 26d07f1ece4174788b0bcdc338a14d0bbc0e3602 Parents: 081db94 Author: Liang-Chi Hsieh vii...@gmail.com Authored: Sun Jun 7 15:33:48 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sun Jun 7

spark git commit: [SPARK-8141] [SQL] Precompute datatypes for partition columns and reuse it

/reuse_partition_column_types and squashes the following commits: dab0688 [Liang-Chi Hsieh] Reuse partitionColumnTypes. (cherry picked from commit 26d07f1ece4174788b0bcdc338a14d0bbc0e3602) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http

spark git commit: [SPARK-8079] [SQL] Makes InsertIntoHadoopFsRelation job/task abortion more robust

exception, an unexpected NPE will be thrown during job abortion. (This issue doesn't bring much damage since the job is failing anyway.) This PR makes the job/task abortion logic in `InsertIntoHadoopFsRelation` more robust to avoid such confusing exceptions. Author: Cheng Lian l...@databricks.com

spark git commit: [SQL] Simplifies binary node pattern matching

, StringType))) ``` can be simplified to ```scala case p BinaryComparison(l StringType(), r DateType()) = p.makeCopy(Array(l, Cast(r, StringType))) ``` Author: Cheng Lian l...@databricks.com Closes #6537 from liancheng/binary-node-patmat and squashes the following commits: a3bf5fe [Cheng Lian] Fixes

spark git commit: [SPARK-8037] [SQL] Ignores files whose name starts with dot in HadoopFsRelation

Repository: spark Updated Branches: refs/heads/master bd97840d5 - 1bb5d716c [SPARK-8037] [SQL] Ignores files whose name starts with dot in HadoopFsRelation Author: Cheng Lian l...@databricks.com Closes #6581 from liancheng/spark-8037 and squashes the following commits: d08e97b [Cheng Lian

spark git commit: [SQL] [MINOR] Fixes a minor comment mistake in IsolatedClientLoader

Repository: spark Updated Branches: refs/heads/master 00a713790 - f7fe9e474 [SQL] [MINOR] Fixes a minor comment mistake in IsolatedClientLoader Author: Cheng Lian l...@databricks.com Closes #6521 from liancheng/classloader-comment-fix and squashes the following commits: fc09606 [Cheng Lian

spark git commit: [SPARK-7684] [SQL] Invoking HiveContext.newTemporaryConfiguration() shouldn't create new metastore directory

by `HiveContext.newTemporaryConfiguration()`, which always creates a new temporary metastore directory and returns a metastore configuration pointing that directory. This makes `TestHive.reset()` always replaces old temporary metastore with an empty new one. Author: Cheng Lian l...@databricks.com Closes #6359 from liancheng/spark

spark git commit: [SPARK-7684] [SQL] Invoking HiveContext.newTemporaryConfiguration() shouldn't create new metastore directory

by `HiveContext.newTemporaryConfiguration()`, which always creates a new temporary metastore directory and returns a metastore configuration pointing that directory. This makes `TestHive.reset()` always replaces old temporary metastore with an empty new one. Author: Cheng Lian l...@databricks.com Closes #6359 from liancheng

spark git commit: [SPARK-7842] [SQL] Makes task committing/aborting in InsertIntoHadoopFsRelation more robust

. Author: Cheng Lian l...@databricks.com Closes #6378 from liancheng/spark-7838 and squashes the following commits: f18253a [Cheng Lian] Makes task committing/aborting in InsertIntoHadoopFsRelation more robust (cherry picked from commit 8af1bf10b70b9b67f18f618174e84365d69caa48) Signed-off-by: Cheng

spark git commit: [MINOR] [SQL] Ignores Thrift server UISeleniumSuite

solution. Author: Cheng Lian l...@databricks.com Closes #6345 from liancheng/ignore-selenium-test and squashes the following commits: 09996fe [Cheng Lian] Ignores Thrift server UISeleniumSuite (cherry picked from commit 4e5220c3171b6a2f4970409bd16be2db930df65d) Signed-off-by: Cheng Lian l

spark git commit: [MINOR] [SQL] Ignores Thrift server UISeleniumSuite

. Author: Cheng Lian l...@databricks.com Closes #6345 from liancheng/ignore-selenium-test and squashes the following commits: 09996fe [Cheng Lian] Ignores Thrift server UISeleniumSuite Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark

spark git commit: [SPARK-7737] [SQL] Use leaf dirs having data files to discover partitions.

/347b5010 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/347b5010 Branch: refs/heads/master Commit: 347b50106bd1bcd40049f1ca29cefbb0baf53413 Parents: 147b6be Author: Yin Huai yh...@databricks.com Authored: Fri May 22 07:10:26 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Fri May

spark git commit: [SPARK-7737] [SQL] Use leaf dirs having data files to discover partitions.

and squashes the following commits: 7e0dfc7 [Yin Huai] Use leaf dirs having data files to discover partitions. (cherry picked from commit 347b50106bd1bcd40049f1ca29cefbb0baf53413) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit

spark git commit: [SPARK-7320] [SQL] Add Cube / Rollup for dataframe

Committer: Cheng Lian l...@databricks.com Committed: Wed May 20 19:09:47 2015 +0800 -- .../scala/org/apache/spark/sql/DataFrame.scala | 104 ++- .../org/apache/spark/sql/GroupedData.scala | 92

spark git commit: [SQL] [MINOR] Skip unresolved expression for InConversion

: edf09ea1bd4bf7692e0085ad9c70cb1bfc8d06d8 Parents: 1a7b9ce Author: scwf wangf...@huawei.com Authored: Sun May 17 15:17:11 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sun May 17 15:17:11 2015 +0800 -- .../org/apache/spark/sql

spark git commit: [SPARK-7447] [SQL] Don't re-merge Parquet schema when the relation is deserialized

Committer: Cheng Lian l...@databricks.com Committed: Sun May 17 15:42:21 2015 +0800 -- .../apache/spark/sql/parquet/newParquet.scala | 32 +++- 1 file changed, 18 insertions(+), 14 deletions

spark git commit: [SPARK-7447] [SQL] Don't re-merge Parquet schema when the relation is deserialized

-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into without_remerge_schema b0fc09b [Liang-Chi Hsieh] Don't generate and merge parquetSchema multiple times. (cherry picked from commit 339905578790fa37fcad9684b859b443313a5aa2) Signed-off-by: Cheng Lian l...@databricks.com Project: http

spark git commit: [HOTFIX] [SQL] Fixes DataFrameWriter.mode(String)

Repository: spark Updated Branches: refs/heads/master 47e7ffe36 - ce6391296 [HOTFIX] [SQL] Fixes DataFrameWriter.mode(String) We forgot an assignment there. /cc rxin Author: Cheng Lian l...@databricks.com Closes #6212 from liancheng/fix-df-writer and squashes the following commits

spark git commit: [HOTFIX] [SQL] Fixes DataFrameWriter.mode(String)

Repository: spark Updated Branches: refs/heads/branch-1.4 ad5b0b1ce - 856619d48 [HOTFIX] [SQL] Fixes DataFrameWriter.mode(String) We forgot an assignment there. /cc rxin Author: Cheng Lian l...@databricks.com Closes #6212 from liancheng/fix-df-writer and squashes the following commits

[1/2] spark git commit: [SPARK-7591] [SQL] Partitioning support API tweaks

Repository: spark Updated Branches: refs/heads/master 94761485b - fdf5bba35 http://git-wip-us.apache.org/repos/asf/spark/blob/fdf5bba3/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala -- diff

spark git commit: [MINOR] [SQL] Removes debugging println

Repository: spark Updated Branches: refs/heads/branch-1.4 9ca28d9de - d78f0e1b4 [MINOR] [SQL] Removes debugging println Author: Cheng Lian l...@databricks.com Closes #6123 from liancheng/remove-println and squashes the following commits: 03356b6 [Cheng Lian] Removes debugging println

spark git commit: [SPARK-6734] [SQL] Add UDTF.close support in Generate

] Support UDTF.close (cherry picked from commit 0da254fb2903c01e059fa7d0dc81df5740312b35) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/42cf4a2a Tree: http://git-wip-us.apache.org

spark git commit: [SPARK-6734] [SQL] Add UDTF.close support in Generate

Commit: 0da254fb2903c01e059fa7d0dc81df5740312b35 Parents: aa6ba3f Author: Cheng Hao hao.ch...@intel.com Authored: Thu May 14 00:14:59 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Thu May 14 00:14:59 2015 +0800

[1/2] spark git commit: [SPARK-3928] [SPARK-5182] [SQL] Partitioning support for the data sources API

Repository: spark Updated Branches: refs/heads/branch-1.4 a9d84a9bf - d2328137f http://git-wip-us.apache.org/repos/asf/spark/blob/d2328137/sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala -- diff --git

spark git commit: [SPARK-5100] [SQL] add webui for thriftserver

: tianyi tianyi.asiai...@gmail.com Authored: Mon May 4 16:59:34 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Mon May 4 16:59:34 2015 +0800 -- .../scala/org/apache/spark/sql/SQLConf.scala| 2 + sql/hive

spark git commit: [SPARK-6505] [SQL] Remove the reflection call in HiveFunctionWrapper

2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Mon Apr 27 14:08:05 2015 +0800 -- .../org/apache/spark/sql/hive/Shim13.scala | 44 ++-- 1 file changed, 22 insertions(+), 22 deletions

spark git commit: [SPARK-6694][SQL]SparkSQL CLI must be able to specify an option --database on the command line.

/tree/3ae37b93 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/3ae37b93 Branch: refs/heads/master Commit: 3ae37b93a7c299bd8b22a36248035bca5de3422f Parents: de4fa6b Author: Jin Adachi adachij2...@yahoo.co.jp Authored: Thu Apr 16 23:41:04 2015 +0800 Committer: Cheng Lian l...@databricks.com

spark git commit: [SPARK-6352] [SQL] Add DirectParquetOutputCommitter

/diff/b29663ee Branch: refs/heads/master Commit: b29663eeea440b1d1a288d41b5ddf67e77c5bd54 Parents: 202ebf0 Author: Pei-Lun Lee pl...@appier.com Authored: Mon Apr 13 21:52:00 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Mon Apr 13 21:52:00 2015 +0800

spark git commit: [SPARK-6835] [SQL] Fix bug of Hive UDTF in Lateral View (ClassNotFound)

...@intel.com Authored: Sat Apr 11 22:11:03 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sat Apr 11 22:11:03 2015 +0800 -- .../src/main/scala/org/apache/spark/sql/hive/HiveQl.scala | 9 - .../org/apache

spark git commit: [SPARK-6748] [SQL] Makes QueryPlan.schema a lazy val

:///tmp/src.parquet).collect() ``` !-- Reviewable:start -- [img src=https://reviewable.io/review_button.png; height=40 alt=Review on Reviewable/](https://reviewable.io/reviews/apache/spark/5398) !-- Reviewable:end -- Author: Cheng Lian l...@databricks.com Closes #5398 from liancheng/spark-6748

spark git commit: [SQL] Use path.makeQualified in newParquet.

/heads/master Commit: da25c86d64ff9ce80f88186ba083f6c21dd9a568 Parents: 9b40c17 Author: Yin Huai yh...@databricks.com Authored: Sat Apr 4 23:26:10 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sat Apr 4 23:26:10 2015 +0800

spark git commit: [SPARK-6607][SQL] Check invalid characters for Parquet schema and show error messages

vii...@gmail.com Authored: Sun Apr 5 00:20:43 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sun Apr 5 00:20:43 2015 +0800 -- .../org/apache/spark/sql/parquet/ParquetTypes.scala | 14 ++ .../org/apache

spark git commit: [SPARK-6575][SQL] Converted Parquet Metastore tables no longer cache metadata

by Committer: Cheng Lian l...@databricks.com Closes #5339 from yhuai/parquetRelationCache and squashes the following commits: b0e1a42 [Yin Huai] Address comments. 83d9846 [Yin Huai] Remove unnecessary change. c0dc7a4 [Yin Huai] Cache converted parquet relations. Project: http://git-wip

spark git commit: [SPARK-6575][SQL] Converted Parquet Metastore tables no longer cache metadata

by Committer: Cheng Lian l...@databricks.com Closes #5339 from yhuai/parquetRelationCache and squashes the following commits: b0e1a42 [Yin Huai] Address comments. 83d9846 [Yin Huai] Remove unnecessary change. c0dc7a4 [Yin Huai] Cache converted parquet relations. (cherry picked from commit

spark git commit: [SPARK-5203][SQL] fix union with different decimal type

Apr 4 02:02:30 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sat Apr 4 02:02:30 2015 +0800 -- .../catalyst/analysis/HiveTypeCoercion.scala| 190 --- .../org/apache/spark/sql/types

spark git commit: [SPARK-6608] [SQL] Makes DataFrame.rdd a lazy val

) !-- Reviewable:end -- Author: Cheng Lian l...@databricks.com Closes #5265 from liancheng/spark-6608 and squashes the following commits: 7500968 [Cheng Lian] Updates javadoc 7f37d21 [Cheng Lian] Makes DataFrame.rdd a lazy val Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http

spark git commit: Revert [SPARK-6618][SQL] HiveMetastoreCatalog.lookupRelation should use fine-grained lock

Author: Cheng Lian l...@databricks.com Authored: Thu Apr 2 12:59:38 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Thu Apr 2 12:59:38 2015 +0800 -- .../apache/spark/sql/hive/HiveMetastoreCatalog.scala| 12

spark git commit: Revert [SPARK-6618][SQL] HiveMetastoreCatalog.lookupRelation should use fine-grained lock

: Cheng Lian l...@databricks.com Authored: Thu Apr 2 12:56:34 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Thu Apr 2 12:56:34 2015 +0800 -- .../apache/spark/sql/hive/HiveMetastoreCatalog.scala| 12

spark git commit: [SPARK-6618][SQL] HiveMetastoreCatalog.lookupRelation should use fine-grained lock

/314afd0e Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/314afd0e Branch: refs/heads/master Commit: 314afd0e2f08dd8dd3143712c2c79fa40d1e Parents: b80a030 Author: Yin Huai yh...@databricks.com Authored: Tue Mar 31 16:28:40 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Tue

spark git commit: [SPARK-6618][SQL] HiveMetastoreCatalog.lookupRelation should use fine-grained lock

/lookupRelationLock and squashes the following commits: 591b4be [Yin Huai] A test? b3a9625 [Yin Huai] Just protect client. (cherry picked from commit 314afd0e2f08dd8dd3143712c2c79fa40d1e) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit

spark git commit: [SPARK-6542][SQL] add CreateStruct

: a05835b89fe2086e460f0b80f7c22e284c0c32d0 Parents: 314afd0 Author: Xiangrui Meng m...@databricks.com Authored: Tue Mar 31 17:05:23 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Tue Mar 31 17:05:23 2015 +0800 -- .../spark/sql

spark git commit: [SPARK-6369] [SQL] Uses commit coordinator to help committing Hive and Parquet tables

/](https://reviewable.io/reviews/apache/spark/5139) !-- Reviewable:end -- Author: Cheng Lian l...@databricks.com Closes #5139 from liancheng/spark-6369 and squashes the following commits: 72eb628 [Cheng Lian] Fixes typo in javadoc 9a4b82b [Cheng Lian] Adds javadoc and addresses @aarondav's comments

spark git commit: [SPARK-6369] [SQL] Uses commit coordinator to help committing Hive and Parquet tables

/](https://reviewable.io/reviews/apache/spark/5139) !-- Reviewable:end -- Author: Cheng Lian l...@databricks.com Closes #5139 from liancheng/spark-6369 and squashes the following commits: 72eb628 [Cheng Lian] Fixes typo in javadoc 9a4b82b [Cheng Lian] Adds javadoc and addresses @aarondav's

spark git commit: [SPARK-6538][SQL] Add missing nullable Metastore fields when merging a Parquet schema

/repos/asf/spark/diff/5909f097 Branch: refs/heads/master Commit: 5909f0973de15f685836c2828e6d4c38f57d2c19 Parents: 3af7334 Author: Adam Budde bu...@amazon.com Authored: Sat Mar 28 09:14:09 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sat Mar 28 09:14:09 2015 +0800

spark git commit: [SPARK-6538][SQL] Add missing nullable Metastore fields when merging a Parquet schema

] Add missing nullable Metastore fields when merging a Parquet schema (cherry picked from commit 5909f0973de15f685836c2828e6d4c38f57d2c19) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark

spark git commit: [SPARK-6465][SQL] Fix serialization of GenericRowWithSchema using kryo

[Michael Armbrust] Fix serialization of GenericRowWithSchema using kryo f914f16 [Michael Armbrust] Add no arg constructor to GenericRowWithSchema (cherry picked from commit f88f51bbd461e0a42ad7021147268509b9c3c56e) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org

spark git commit: [SQL][SPARK-6471]: Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

Branch: refs/heads/master Commit: 1c05027a143d1b0bf3df192984e6cac752b1e926 Parents: 0c88ce5 Author: Yash Datta yash.da...@guavus.com Authored: Thu Mar 26 21:13:38 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Thu Mar 26 21:13:38 2015 +0800

spark git commit: [SQL][SPARK-6471]: Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

dropping of columns using replace columns (cherry picked from commit 1c05027a143d1b0bf3df192984e6cac752b1e926) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/836c9216 Tree: http://git

spark git commit: [SPARK-6483][SQL]Improve ScalaUdf called performance.

Authored: Wed Mar 25 19:11:04 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Wed Mar 25 19:11:04 2015 +0800 -- .../sql/catalyst/expressions/ScalaUdf.scala | 1016 -- 1 file changed, 661 insertions

spark git commit: [SPARK-6397][SQL] Check the missingInput simply

/repos/asf/spark/diff/9f3273bd Branch: refs/heads/master Commit: 9f3273bd9c919f6c48a95383b3d5be357c89998c Parents: bf044de Author: Yadong Qi qiyadong2...@gmail.com Authored: Mon Mar 23 18:16:49 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Mon Mar 23 18:16:49 2015 +0800

spark git commit: [SPARK-6397][SQL] Check the missingInput simply

the following commits: 1e5bdc5 [Yadong Qi] Check the missingInput simply (cherry picked from commit 9f3273bd9c919f6c48a95383b3d5be357c89998c) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark

spark git commit: [SPARK-6408] [SQL] Fix JDBCRDD filtering string literals

: 9b1e1f20d4498bda72dd53a832110883a7ca41b5 Parents: b6090f9 Author: ypcat ypc...@gmail.com Authored: Sun Mar 22 15:49:13 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sun Mar 22 15:49:13 2015 +0800 -- .../org/apache/spark/sql/jdbc

spark git commit: [SPARK-6408] [SQL] Fix JDBCRDD filtering string literals

picked from commit 9b1e1f20d4498bda72dd53a832110883a7ca41b5) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e60fbf6c Tree: http://git-wip-us.apache.org/repos/asf/spark/tree

spark git commit: [SPARK-4985] [SQL] parquet support for date type

4659468f369d69e7f777130e5e3b4c5d47a624f1) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/60b9b96b Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/60b9b96b Diff: http://git-wip

spark git commit: [SPARK-6397][SQL] Check the missingInput simply

: e566fe5982bac5d24e6be76e5d7d6270544a85e6 Parents: 4659468 Author: q00251598 qiyad...@huawei.com Authored: Mon Mar 23 12:06:13 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Mon Mar 23 12:06:13 2015 +0800 -- .../org

spark git commit: Revert [SPARK-6397][SQL] Check the missingInput simply

/repos/asf/spark/commit/bf044def Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/bf044def Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/bf044def Branch: refs/heads/master Commit: bf044def4c3a37a0fd4d5e70c2d57685cfd9fd71 Parents: e566fe5 Author: Cheng Lian l...@databricks.com Authored

spark git commit: [SPARK-5821] [SQL] ParquetRelation2 CTAS should check if delete is successful

/tree/bc37c974 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/bc37c974 Branch: refs/heads/master Commit: bc37c9743e065a0c756363c7b70e88f22a6e6edd Parents: 25e271d Author: Yanbo Liang yblia...@gmail.com Authored: Sat Mar 21 10:53:04 2015 +0800 Committer: Cheng Lian l...@databricks.com

spark git commit: [SPARK-6315] [SQL] Also tries the case class string parser while reading Parquet schema

://reviewable.io/reviews/apache/spark/5034) !-- Reviewable:end -- Author: Cheng Lian l...@databricks.com Closes #5034 from liancheng/spark-6315 and squashes the following commits: a182f58 [Cheng Lian] Adds a regression test b9c6dbe [Cheng Lian] Also tries the case class string parser while reading Parquet

spark git commit: [SPARK-6315] [SQL] Also tries the case class string parser while reading Parquet schema

://reviewable.io/reviews/apache/spark/5034) !-- Reviewable:end -- Author: Cheng Lian l...@databricks.com Closes #5034 from liancheng/spark-6315 and squashes the following commits: a182f58 [Cheng Lian] Adds a regression test b9c6dbe [Cheng Lian] Also tries the case class string parser while reading

spark git commit: [SPARK-5821] [SQL] ParquetRelation2 CTAS should check if delete is successful

-parquet and squashes the following commits: 7092c8d [Yanbo Liang] ParquetRelation2 CTAS should check if delete is successful (cherry picked from commit bc37c9743e065a0c756363c7b70e88f22a6e6edd) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-5821] [SQL] JSON CTAS command should throw error message when delete path failure

: e5d2c37c68ac00a57c2542e62d1c5b4ca267c89e Parents: 937c1e5 Author: Yanbo Liang yblia...@gmail.com Authored: Sat Mar 21 11:23:28 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sat Mar 21 11:23:28 2015 +0800 -- .../apache

spark git commit: [SPARK-6366][SQL] In Python API, the default save mode for save and saveAsTable should be error instead of append.

#5053 from yhuai/SPARK-6366 and squashes the following commits: fc81897 [Yin Huai] Use error as the default save mode for save/saveAsTable. (cherry picked from commit dc9c9196d63aa465e86ac52f0e86e10c12472100) Signed-off-by: Cheng Lian l...@databricks.com Project: http://git-wip-us.apache.org

spark git commit: [SPARK-6366][SQL] In Python API, the default save mode for save and saveAsTable should be error instead of append.

/repos/asf/spark/tree/dc9c9196 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/dc9c9196 Branch: refs/heads/master Commit: dc9c9196d63aa465e86ac52f0e86e10c12472100 Parents: 4633a87 Author: Yin Huai yh...@databricks.com Authored: Wed Mar 18 09:41:06 2015 +0800 Committer: Cheng Lian l

spark git commit: [SPARK-6330] [SQL] Add a test case for SPARK-6330

://github.com/apache/spark into spark-6351 896e80a [Pei-Lun Lee] [SPARK-6351] [SQL] Add test case 2ae0916 [Pei-Lun Lee] [SPARK-6351] [SQL] ParquetRelation2 supporting multiple file systems (cherry picked from commit 4633a87b86a6ef01fa724d31763dcb97cb5bc746) Signed-off-by: Cheng Lian l...@databricks.com

spark git commit: [SPARK-2087] [SQL] Multiple thriftserver sessions with single HiveContext instance

: 12a345adcbaee359199ddfed4f41bf0e19d66d48 Parents: 00e730b Author: Cheng Hao hao.ch...@intel.com Authored: Tue Mar 17 01:09:27 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Tue Mar 17 01:09:27 2015 +0800 -- .../scala/org/apache

spark git commit: [SPARK-6285][SQL]Remove ParquetTestData in SparkBuild.scala and in README.md

Commit: 62ede5383f64b69570a66d46939638f4bf38d1b1 Parents: c49d156 Author: OopsOutOfMemory victorshen...@126.com Authored: Sun Mar 15 20:44:45 2015 +0800 Committer: Cheng Lian l...@databricks.com Committed: Sun Mar 15 20:44:45 2015 +0800

spark git commit: [SPARK-6195] [SQL] Adds in-memory column type for fixed-precision decimals

=https://reviewable.io/review_button.png; height=40 alt=Review on Reviewable/](https://reviewable.io/reviews/apache/spark/4938) !-- Reviewable:end -- Author: Cheng Lian l...@databricks.com Closes #4938 from liancheng/decimal-column-type and squashes the following commits: fef5338 [Cheng Lian

spark git commit: [SPARK-6285] [SQL] Removes unused ParquetTestData and duplicated TestGroupWriteSupport

of the old Parquet test suites. !-- Reviewable:start -- [img src=https://reviewable.io/review_button.png; height=40 alt=Review on Reviewable/](https://reviewable.io/reviews/apache/spark/5010) !-- Reviewable:end -- Author: Cheng Lian l...@databricks.com Closes #5010 from liancheng/spark-6285 and squashes

<    1   2   3   4   5   >