http://git-wip-us.apache.org/repos/asf/spark/blob/52cb1ad3/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/CatalystWriteSupport.scala
--
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/executio
Repository: spark
Updated Branches:
refs/heads/branch-2.0 35c0a60a6 -> 52cb1ad38
http://git-wip-us.apache.org/repos/asf/spark/blob/52cb1ad3/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala
---
Repository: spark
Updated Branches:
refs/heads/branch-1.6 0a8ada506 -> a4485c3b5
Update branch-1.6 for 1.6.2 release.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a4485c3b
Tree: http://git-wip-us.apache.org/repos/asf/sp
Repository: spark
Updated Tags: refs/tags/v1.6.2 [deleted] f16649304
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/branch-2.0 2280ad8a3 -> 8f7138859
[SPARK-15966][DOC] Add closing tag to fix rendering issue for Spark monitoring
## What changes were proposed in this pull request?
Adds the missing closing tag for spark.ui.view.acls.groups
## How was this patch
Repository: spark
Updated Branches:
refs/heads/master e5d703bca -> ef43b4ed8
[SPARK-15811][SQL] fix the Python UDF in Scala 2.10
## What changes were proposed in this pull request?
Iterator can't be serialized in Scala 2.10, we should force it into a array to
make sure that .
## How was thi
Repository: spark
Updated Branches:
refs/heads/branch-2.0 b82abde06 -> 3c3865d0b
[SPARK-15811][SQL] fix the Python UDF in Scala 2.10
## What changes were proposed in this pull request?
Iterator can't be serialized in Scala 2.10, we should force it into a array to
make sure that .
## How was
ass name start with a verb. The main important "fix" is
probably SamplePushDown -> PushProjectThroughSample. SamplePushDown is actually
the wrong name, since the rule is not about pushing Sample down.
## How was this patch tested?
Updated test cases.
Author: Reynold Xin
Closes
ame start with a verb. The main important "fix" is
probably SamplePushDown -> PushProjectThroughSample. SamplePushDown is actually
the wrong name, since the rule is not about pushing Sample down.
## How was this patch tested?
Updated test cases.
Author: Reynold Xin
Closes #13732
Repository: spark
Updated Branches:
refs/heads/master 898cb6525 -> 35a2f3c01
[SPARK-16023][SQL] Move InMemoryRelation to its own file
## What changes were proposed in this pull request?
Improve readability of `InMemoryTableScanExec.scala`, which has too much stuff
in it.
## How was this pat
Repository: spark
Updated Branches:
refs/heads/branch-2.0 feeef497d -> 329dfe706
[SPARK-16023][SQL] Move InMemoryRelation to its own file
## What changes were proposed in this pull request?
Improve readability of `InMemoryTableScanExec.scala`, which has too much stuff
in it.
## How was this
Repository: spark
Updated Branches:
refs/heads/master 35a2f3c01 -> e574c9973
[SPARK-15973][PYSPARK] Fix GroupedData Documentation
*This contribution is my original work and that I license the work to the
project under the project's open source license.*
## What changes were proposed in this
Repository: spark
Updated Branches:
refs/heads/branch-2.0 329dfe706 -> f159eb521
[SPARK-15973][PYSPARK] Fix GroupedData Documentation
*This contribution is my original work and that I license the work to the
project under the project's open source license.*
## What changes were proposed in t
Repository: spark
Updated Branches:
refs/heads/branch-2.0 80c6d4e3a -> d11f533de
[SPARK-16031] Add debug-only socket source in Structured Streaming
## What changes were proposed in this pull request?
This patch adds a text-based socket source similar to the one in Spark
Streaming for debuggi
Repository: spark
Updated Branches:
refs/heads/master 5930d7a2e -> 4f17fddcd
[SPARK-16031] Add debug-only socket source in Structured Streaming
## What changes were proposed in this pull request?
This patch adds a text-based socket source similar to the one in Spark
Streaming for debugging a
Repository: spark
Updated Branches:
refs/heads/branch-2.0 54001cb12 -> b40663541
[SPARK-16061][SQL][MINOR] The property
"spark.streaming.stateStore.maintenanceInterval" should be renamed to
"spark.sql.streaming.stateStore.maintenanceInterval"
## What changes were proposed in this pull reques
Repository: spark
Updated Branches:
refs/heads/master b99129cc4 -> 6daa8cf1a
[SPARK-16061][SQL][MINOR] The property
"spark.streaming.stateStore.maintenanceInterval" should be renamed to
"spark.sql.streaming.stateStore.maintenanceInterval"
## What changes were proposed in this pull request?
T
een
found on this partition. Malformed records from now on will not be logged.
```
Closes #12173
## How was this patch tested?
Manually tested.
Author: Reynold Xin
Closes #13795 from rxin/SPARK-13792.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.
een
found on this partition. Malformed records from now on will not be logged.
```
Closes #12173
## How was this patch tested?
Manually tested.
Author: Reynold Xin
Closes #13795 from rxin/SPARK-13792.
(cherry picked from commit c775bf09e0c3540f76de3f15d3fd35112a4912c1)
Signed-off-by: Reynold
Repository: spark
Updated Branches:
refs/heads/master c775bf09e -> d9a3a2a0b
[SPARK-16056][SPARK-16057][SPARK-16058][SQL] Fix Multiple Bugs in Column
Partitioning in JDBC Source
What changes were proposed in this pull request?
This PR is to fix the following bugs:
**Issue 1: Wrong Resul
Repository: spark
Updated Branches:
refs/heads/branch-2.0 603424c16 -> b998c33c0
[SPARK-16056][SPARK-16057][SPARK-16058][SQL] Fix Multiple Bugs in Column
Partitioning in JDBC Source
What changes were proposed in this pull request?
This PR is to fix the following bugs:
**Issue 1: Wrong R
Repository: spark
Updated Branches:
refs/heads/branch-2.0 b998c33c0 -> 9d513b8d2
[SPARK-16074][MLLIB] expose VectorUDT/MatrixUDT in a public API
## What changes were proposed in this pull request?
Both VectorUDT and MatrixUDT are private APIs, because UserDefinedType itself
is private in Spa
Repository: spark
Updated Branches:
refs/heads/master d9a3a2a0b -> 18a8a9b1f
[SPARK-16074][MLLIB] expose VectorUDT/MatrixUDT in a public API
## What changes were proposed in this pull request?
Both VectorUDT and MatrixUDT are private APIs, because UserDefinedType itself
is private in Spark.
Repository: spark
Updated Branches:
refs/heads/master 18a8a9b1f -> 4f7f1c436
[SPARK-16044][SQL] input_file_name() returns empty strings in data sources
based on NewHadoopRDD
## What changes were proposed in this pull request?
This PR makes `input_file_name()` function return the file paths n
Repository: spark
Updated Branches:
refs/heads/branch-2.0 9d513b8d2 -> 12f00b6ed
[SPARK-16044][SQL] input_file_name() returns empty strings in data sources
based on NewHadoopRDD
## What changes were proposed in this pull request?
This PR makes `input_file_name()` function return the file pat
Repository: spark
Updated Branches:
refs/heads/branch-2.0 12f00b6ed -> 4fc4eb943
[SPARK-16025][CORE] Document OFF_HEAP storage level in 2.0
This has changed from 1.6, and now stores memory off-heap using spark's
off-heap support instead of in tachyon.
Author: Eric Liang
Closes #13744 from
Repository: spark
Updated Branches:
refs/heads/master 4f7f1c436 -> 07367533d
[SPARK-16025][CORE] Document OFF_HEAP storage level in 2.0
This has changed from 1.6, and now stores memory off-heap using spark's
off-heap support instead of in tachyon.
Author: Eric Liang
Closes #13744 from eric
of this, I also make the Python option
setting for both CSV and JSON more robust against positional errors.
## How was this patch tested?
N/A
Author: Reynold Xin
Closes #13800 from rxin/SPARK-13792-2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-
As part of this, I also make the Python option
setting for both CSV and JSON more robust against positional errors.
## How was this patch tested?
N/A
Author: Reynold Xin
Closes #13800 from rxin/SPARK-13792-2.
(cherry picked from commit 93338807aafdb2db9fb036ceadee1467cd367cdd)
Signed-off-by: Reyn
Repository: spark
Updated Branches:
refs/heads/branch-2.0 aeda9a153 -> 8d5a62d5f
[SPARK-16117][MLLIB] hide LibSVMFileFormat and move its doc to LibSVMDataSource
## What changes were proposed in this pull request?
LibSVMFileFormat implements data source for LIBSVM format. However, users do
no
Repository: spark
Updated Branches:
refs/heads/master dbfdae4e4 -> f4e8c31ad
[SPARK-16117][MLLIB] hide LibSVMFileFormat and move its doc to LibSVMDataSource
## What changes were proposed in this pull request?
LibSVMFileFormat implements data source for LIBSVM format. However, users do
not re
Repository: spark
Updated Branches:
refs/heads/branch-2.0 f2413736d -> fe41f68fd
Update branch-2.0 for 2.0.0 release.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fe41f68f
Tree: http://git-wip-us.apache.org/repos/asf/sp
ong. This is a small refactoring to move it into its own functions.
Eventually we would be able to move the partitioning functions into a physical
operator, rather than doing it in physical planning.
## How was this patch tested?
This is a simple code move.
Author: Reynold Xin
Closes #13862 from r
Repository: spark
Updated Branches:
refs/heads/branch-2.0 5b4a9a4c3 -> 4ad731ed6
[SQL][MINOR] Fix minor formatting issues in SHOW CREATE TABLE output
## What changes were proposed in this pull request?
This PR fixes two minor formatting issues appearing in `SHOW CREATE TABLE`
output.
Before
Repository: spark
Updated Branches:
refs/heads/master 925884a61 -> f34b5c62b
[SQL][MINOR] Fix minor formatting issues in SHOW CREATE TABLE output
## What changes were proposed in this pull request?
This PR fixes two minor formatting issues appearing in `SHOW CREATE TABLE`
output.
Before:
`
Repository: spark
Updated Branches:
refs/heads/master f34b5c62b -> 4374a46bf
[SPARK-16162] Remove dead code OrcTableScan.
## What changes were proposed in this pull request?
SPARK-14535 removed all calls to class OrcTableScan. This removes the dead code.
## How was this patch tested?
Existi
Repository: spark
Updated Branches:
refs/heads/master 5f8de2160 -> 4435de1bd
[SPARK-16179][PYSPARK] fix bugs for Python udf in generate
## What changes were proposed in this pull request?
This PR fix the bug when Python UDF is used in explode (generator),
GenerateExec requires that all the a
Repository: spark
Updated Branches:
refs/heads/branch-2.0 4bb8cca44 -> df137e3e0
[SPARK-16179][PYSPARK] fix bugs for Python udf in generate
## What changes were proposed in this pull request?
This PR fix the bug when Python UDF is used in explode (generator),
GenerateExec requires that all t
Repository: spark
Updated Tags: refs/tags/v1.6.2 [created] 54b1121f3
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.6.2-rc2 [deleted] 54b1121f3
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.6.2-rc1 [deleted] 4168d9c94
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Author: rxin
Date: Sat Jun 25 07:07:25 2016
New Revision: 14145
Log:
Add spark-1.6.2
Added:
dev/spark/spark-1.6.2/
dev/spark/spark-1.6.2/spark-1.6.2-bin-cdh4.tgz (with props)
dev/spark/spark-1.6.2/spark-1.6.2-bin-cdh4.tgz.asc
dev/spark/spark-1.6.2/spark-1.6.2-bin-cdh4.tgz.md5
Author: rxin
Date: Sat Jun 25 07:16:01 2016
New Revision: 14146
Log:
Spark 1.6.2 release
Added:
release/spark/spark-1.6.2/
- copied from r14145, dev/spark/spark-1.6.2/
Removed:
dev/spark/spark-1.6.2/
-
To
Author: rxin
Date: Sat Jun 25 08:02:58 2016
New Revision: 1750186
URL: http://svn.apache.org/viewvc?rev=1750186&view=rev
Log:
add 1.6.2 doc
[This commit notification would consist of 931 parts,
which exceeds the limit of 50 ones, so it was shortened to the sum
Added: spark/site/news/spark-1-6-2-released.html
URL:
http://svn.apache.org/viewvc/spark/site/news/spark-1-6-2-released.html?rev=1750423&view=auto
==
--- spark/site/news/spark-1-6-2-released.html (added)
+++ spark/site/new
Author: rxin
Date: Mon Jun 27 23:22:21 2016
New Revision: 1750423
URL: http://svn.apache.org/viewvc?rev=1750423&view=rev
Log:
Add 1.6.2 release
Added:
spark/news/_posts/2016-06-25-spark-1-6-2-released.md
spark/releases/_posts/2016-06-25-spark-release-1-6-2.md
spark/site/news/spa
Modified: spark/site/releases/spark-release-1-1-0.html
URL:
http://svn.apache.org/viewvc/spark/site/releases/spark-release-1-1-0.html?rev=1750423&r1=1750422&r2=1750423&view=diff
==
--- spark/site/releases/spark-release-1-1
Author: rxin
Date: Mon Jun 27 23:32:42 2016
New Revision: 1750424
URL: http://svn.apache.org/viewvc?rev=1750424&view=rev
Log:
doc update for 1.6.2
Modified:
spark/documentation.md
spark/site/docs/latest
spark/site/documentation.html
Modified: spark/documentation.md
URL:
Author: rxin
Date: Mon Jun 27 23:34:10 2016
New Revision: 1750425
URL: http://svn.apache.org/viewvc?rev=1750425&view=rev
Log:
update doc header
Modified:
spark/_layouts/global.html
Modified: spark/_layouts/global.html
URL:
http://svn.apache.org/viewvc/spark/_layouts/global.html?rev=175
Author: rxin
Date: Mon Jun 27 23:35:03 2016
New Revision: 1750426
URL: http://svn.apache.org/viewvc?rev=1750426&view=rev
Log:
update doc header
Modified:
spark/site/community.html
spark/site/documentation.html
spark/site/downloads.html
spark/site/examples.html
spark/
Modified: spark/site/screencasts/3-transformations-and-caching.html
URL:
http://svn.apache.org/viewvc/spark/site/screencasts/3-transformations-and-caching.html?rev=1750426&r1=1750425&r2=1750426&view=diff
==
--- spark/site/
Repository: spark
Updated Branches:
refs/heads/master c15b552dd -> 02a029df4
[SPARK-16220][SQL] Add scope to show functions
## What changes were proposed in this pull request?
Spark currently shows all functions when issue a `SHOW FUNCTIONS` command. This
PR refines the `SHOW FUNCTIONS` comma
Repository: spark
Updated Branches:
refs/heads/branch-2.0 27f3462d0 -> c7704099d
[SPARK-16220][SQL] Add scope to show functions
## What changes were proposed in this pull request?
Spark currently shows all functions when issue a `SHOW FUNCTIONS` command. This
PR refines the `SHOW FUNCTIONS` c
Repository: spark
Updated Branches:
refs/heads/master 1b7fc5817 -> 50fdd866b
[SPARK-16111][SQL][DOC] Hide SparkOrcNewRecordReader in API docs
## What changes were proposed in this pull request?
Currently, Spark Scala/Java API documents shows
**org.apache.hadoop.hive.ql.io.orc** package at th
Repository: spark
Updated Branches:
refs/heads/branch-2.0 4c4f7775c -> 7177e1843
[SPARK-16111][SQL][DOC] Hide SparkOrcNewRecordReader in API docs
## What changes were proposed in this pull request?
Currently, Spark Scala/Java API documents shows
**org.apache.hadoop.hive.ql.io.orc** package a
Repository: spark
Updated Branches:
refs/heads/branch-2.0 7177e1843 -> af70ad028
[SPARK-15863][SQL][DOC][FOLLOW-UP] Update SQL programming guide.
## What changes were proposed in this pull request?
This PR makes several updates to SQL programming guide.
Author: Yin Huai
Closes #13938 from y
Repository: spark
Updated Branches:
refs/heads/master a0da854fb -> dd6b7dbe7
[SPARK-15863][SQL][DOC][FOLLOW-UP] Update SQL programming guide.
## What changes were proposed in this pull request?
This PR makes several updates to SQL programming guide.
Author: Yin Huai
Closes #13938 from yhuai
Repository: spark
Updated Branches:
refs/heads/master dd6b7dbe7 -> 4cbf611c1
[SPARK-16202][SQL][DOC] Correct The Description of CreatableRelationProvider's
createRelation
What changes were proposed in this pull request?
The API description of `createRelation` in `CreatableRelationProvide
Repository: spark
Updated Branches:
refs/heads/master ae14f3623 -> 1aad8c6e5
[SPARK-16259][PYSPARK] cleanup options in DataFrame read/write API
## What changes were proposed in this pull request?
There are some duplicated code for options in DataFrame reader/writer API, this
PR clean them up
Repository: spark
Updated Branches:
refs/heads/master 35438fb0a -> 25520e976
[SPARK-16236][SQL] Add Path Option back to Load API in DataFrameReader
What changes were proposed in this pull request?
koertkuipers identified the PR https://github.com/apache/spark/pull/13727/
changed the beha
Repository: spark
Updated Branches:
refs/heads/branch-2.0 43bd612f3 -> 5626a0af5
[SPARK-16236][SQL] Add Path Option back to Load API in DataFrameReader
What changes were proposed in this pull request?
koertkuipers identified the PR https://github.com/apache/spark/pull/13727/
changed the
and adds only a small number of functions to the whitelist,
i.e. the ones we intend to support in the long run in Spark.
## How was this patch tested?
Updated tests to reflect the change.
Author: Reynold Xin
Closes #13939 from rxin/hive-whitelist.
Project: http://git-wip-us.apache.org/repos/
ist and adds only a small number of functions to the whitelist,
i.e. the ones we intend to support in the long run in Spark.
## How was this patch tested?
Updated tests to reflect the change.
Author: Reynold Xin
Closes #13939 from rxin/hive-whitelist.
(cherry picked from com
Repository: spark
Updated Branches:
refs/heads/master 0df5ce1bc -> 153c2f9ac
[SPARK-16271][SQL] Implement Hive's UDFXPathUtil
## What changes were proposed in this pull request?
This patch ports Hive's UDFXPathUtil over to Spark, which can be used to
implement xpath functionality in Spark in
Repository: spark
Updated Branches:
refs/heads/master 39f2eb1da -> cb1b9d34f
[SPARK-14480][SQL] Remove meaningless StringIteratorReader for CSV data source.
## What changes were proposed in this pull request?
This PR removes meaningless `StringIteratorReader` for CSV data source.
In `CSVPars
Repository: spark
Updated Branches:
refs/heads/branch-1.6 0cb06c993 -> 1ac830aca
[SPARK-16044][SQL] Backport input_file_name() for data source based on
NewHadoopRDD to branch 1.6
## What changes were proposed in this pull request?
This PR backports https://github.com/apache/spark/pull/13759.
Repository: spark
Updated Branches:
refs/heads/master 8b5a8b25b -> 9b1b3ae77
[SPARK-16006][SQL] Attemping to write empty DataFrame with no fields throws
non-intuitive exception
## What changes were proposed in this pull request?
This PR allows `emptyDataFrame.write` since the user didn't spe
Repository: spark
Updated Branches:
refs/heads/branch-2.0 a7f66ef62 -> ef0253ff6
[SPARK-16006][SQL] Attemping to write empty DataFrame with no fields throws
non-intuitive exception
## What changes were proposed in this pull request?
This PR allows `emptyDataFrame.write` since the user didn't
Repository: spark
Updated Branches:
refs/heads/branch-2.0 ef0253ff6 -> c4cebd572
[SPARK-16238] Metrics for generated method and class bytecode size
## What changes were proposed in this pull request?
This extends SPARK-15860 to include metrics for the actual bytecode size of
janino-generated
Repository: spark
Updated Branches:
refs/heads/master 9b1b3ae77 -> 23c58653f
[SPARK-16238] Metrics for generated method and class bytecode size
## What changes were proposed in this pull request?
This extends SPARK-15860 to include metrics for the actual bytecode size of
janino-generated met
Repository: spark
Updated Branches:
refs/heads/master 23c58653f -> 2eaabfa41
[SPARK-16228][SQL] HiveSessionCatalog should return `double`-param functions
for decimal param lookups
## What changes were proposed in this pull request?
This PR supports a fallback lookup by casting `DecimalType`
Repository: spark
Updated Branches:
refs/heads/branch-2.0 c4cebd572 -> 011befd20
[SPARK-16228][SQL] HiveSessionCatalog should return `double`-param functions
for decimal param lookups
## What changes were proposed in this pull request?
This PR supports a fallback lookup by casting `DecimalTy
Repository: spark
Updated Branches:
refs/heads/master d063898be -> 831a04f5d
[SPARK-16267][TEST] Replace deprecated `CREATE TEMPORARY TABLE ... USING` from
testsuites.
## What changes were proposed in this pull request?
After SPARK-15674, `DDLStrategy` prints out the following deprecation me
Repository: spark
Updated Branches:
refs/heads/branch-2.0 e1bdf1e02 -> b52bd8070
[SPARK-16267][TEST] Replace deprecated `CREATE TEMPORARY TABLE ... USING` from
testsuites.
## What changes were proposed in this pull request?
After SPARK-15674, `DDLStrategy` prints out the following deprecatio
Repository: spark
Updated Branches:
refs/heads/master fdf9f94f8 -> 46395db80
[SPARK-16289][SQL] Implement posexplode table generating function
## What changes were proposed in this pull request?
This PR implements `posexplode` table generating function. Currently, master
branch raises the fo
Repository: spark
Updated Branches:
refs/heads/master 5d00a7bc1 -> fb41670c9
[SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException
## What changes were proposed in this pull request?
This patch appends a message to suggest users running refresh table or
reloading data fram
Repository: spark
Updated Branches:
refs/heads/branch-2.0 03008e049 -> 4dc7d377f
[SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException
## What changes were proposed in this pull request?
This patch appends a message to suggest users running refresh table or
reloading data
ior because it can mask legitimate errors and the resulting plan will
silently have 0 rows. This patch changes it to not silently drop the errors.
## How was this patch tested?
Manually verified.
Author: Reynold Xin
Closes #13987 from rxin/SPARK-16313.
Project: http://git-wip-us.apache.org/repos/
ior because it can mask legitimate errors and the resulting plan will
silently have 0 rows. This patch changes it to not silently drop the errors.
## How was this patch tested?
Manually verified.
Author: Reynold Xin
Closes #13987 from rxin/SPARK-16313.
(cherry picked from com
was this patch tested?
This is a test only change.
Author: Reynold Xin
Closes #14005 from rxin/SPARK-15954.
(cherry picked from commit 38f4d6f44eaa03bdc703662e4a7be9c09ba86e16)
Signed-off-by: Reynold Xin
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-
his patch tested?
This is a test only change.
Author: Reynold Xin
Closes #14005 from rxin/SPARK-15954.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/38f4d6f4
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/38f4d
Repository: spark
Updated Branches:
refs/heads/master aa6564f37 -> 14cf61e90
[SPARK-16331][SQL] Reduce code generation time
## What changes were proposed in this pull request?
During the code generation, a `LocalRelation` often has a huge `Vector` object
as `data`. In the simple example below
ses #14002 from rxin/SPARK-16335.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d601894c
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d601894c
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d601894c
Branch: r
ses #14002 from rxin/SPARK-16335.
(cherry picked from commit d601894c0494d415e7f330e02168c43a2dacfb02)
Signed-off-by: Reynold Xin
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/78387ce0
Tree: http://git-wip-us.apache.org/re
ion
(or alternatively just create a new QueryExecution).
## How was this patch tested?
Re-enabled a previously ignored test, and added a new test suite for Hive
testing behavior of temporary views against MetastoreRelation.
Author: Reynold Xin
Author: petermaxlee
Closes #14009 from rxin/SPARK-16
(or alternatively just create a new QueryExecution).
## How was this patch tested?
Re-enabled a previously ignored test, and added a new test suite for Hive
testing behavior of temporary views against MetastoreRelation.
Author: Reynold Xin
Author: petermaxlee
Closes #14009 from rxin/SPARK-16
Repository: spark
Updated Branches:
refs/heads/master 1fca9da95 -> 5b7a1770a
[HOTFIX] Fix build break.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5b7a1770
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/5b7a17
Repository: spark
Updated Branches:
refs/heads/branch-2.0 cabee2324 -> 9c1596b6c
[SPARK-15730][SQL] Respect the --hiveconf in the spark-sql command line
## What changes were proposed in this pull request?
This PR makes spark-sql (backed by SparkSQLCLIDriver) respects confs set by
hiveconf, wh
Repository: spark
Updated Branches:
refs/heads/master 5b7a1770a -> 920cb5fe4
[SPARK-15730][SQL] Respect the --hiveconf in the spark-sql command line
## What changes were proposed in this pull request?
This PR makes spark-sql (backed by SparkSQLCLIDriver) respects confs set by
hiveconf, which
Repository: spark
Updated Branches:
refs/heads/master 1f0d02130 -> 4db63fd2b
[SPARK-16383][SQL] Remove `SessionState.executeSql`
## What changes were proposed in this pull request?
This PR removes `SessionState.executeSql` in favor of `SparkSession.sql`. We
can remove this safely since the v
Repository: spark
Updated Branches:
refs/heads/master ec18cd0af -> ec79183ac
[SPARK-16340][SQL] Support column arguments for `regexp_replace` Dataset
operation
## What changes were proposed in this pull request?
Currently, `regexp_replace` function supports `Column` arguments in a query.
Th
Repository: spark
Updated Branches:
refs/heads/master 478b71d02 -> 23eff5e51
[SPARK-15979][SQL] Renames CatalystWriteSupport to ParquetWriteSupport
## What changes were proposed in this pull request?
PR #13696 renamed various Parquet support classes but left
`CatalystWriteSupport` behind. Th
Repository: spark
Updated Branches:
refs/heads/master 23eff5e51 -> b1310425b
[DOC][SQL] update out-of-date code snippets using SQLContext in all documents.
## What changes were proposed in this pull request?
I search the whole documents directory using SQLContext, and update the
following pl
Repository: spark
Updated Branches:
refs/heads/branch-2.0 e956bd775 -> 091cd5f26
[DOC][SQL] update out-of-date code snippets using SQLContext in all documents.
## What changes were proposed in this pull request?
I search the whole documents directory using SQLContext, and update the
followin
Repository: spark
Updated Branches:
refs/heads/branch-2.0 091cd5f26 -> 03f336d89
[MINOR][PYSPARK][DOC] Fix wrongly formatted examples in PySpark documentation
## What changes were proposed in this pull request?
This PR fixes wrongly formatted examples in PySpark documentation as below:
- **`
Repository: spark
Updated Branches:
refs/heads/master b1310425b -> 4e14199ff
[MINOR][PYSPARK][DOC] Fix wrongly formatted examples in PySpark documentation
## What changes were proposed in this pull request?
This PR fixes wrongly formatted examples in PySpark documentation as below:
- **`Spar
Repository: spark
Updated Branches:
refs/heads/master 4e14199ff -> 480357cc6
[SPARK-16304] LinkageError should not crash Spark executor
## What changes were proposed in this pull request?
This patch updates the failure handling logic so Spark executor does not crash
when seeing LinkageError.
Repository: spark
Updated Branches:
refs/heads/master 480357cc6 -> 4f8ceed59
[SPARK-16371][SQL] Do not push down filters incorrectly when inner name and
outer name are the same in Parquet
## What changes were proposed in this pull request?
Currently, if there is a schema as below:
```
root
Repository: spark
Updated Branches:
refs/heads/branch-2.0 03f336d89 -> 2465f0728
[SPARK-16371][SQL] Do not push down filters incorrectly when inner name and
outer name are the same in Parquet
## What changes were proposed in this pull request?
Currently, if there is a schema as below:
```
r
Repository: spark
Updated Branches:
refs/heads/branch-2.0 d7926da5e -> 88be66b93
[SPARK-16379][CORE][MESOS] Spark on mesos is broken due to race condition in
Logging
## What changes were proposed in this pull request?
The commit
https://github.com/apache/spark/commit/044971eca0ff3c2ce62afa6
501 - 600 of 4541 matches
Mail list logo
