Repository: spark
Updated Branches:
refs/heads/branch-1.3 09feecc7c - 11dbf7137
[HOTFIX] Fix the maven build after adding sqlContext to spark-shell
Follow up to #4387 to fix the build break.
Author: Michael Armbrust mich...@databricks.com
Closes #4443 from marmbrus/fixMaven and squashes
Repository: spark
Updated Branches:
refs/heads/branch-1.3 1b148adfc - 2ef9853e7
[SPARK-5586][Spark Shell][SQL] Make `sqlContext` available in spark shell
Result is like this
```
15/02/05 13:41:22 INFO SparkILoop: Created spark context..
Spark context available as sc.
15/02/05 13:41:22 INFO
Repository: spark
Updated Branches:
refs/heads/branch-1.3 af6ddf8b6 - c950058e9
[SQL] Use TestSQLContext in Java tests
Sometimes tests were failing due to the creation of multiple `SparkContext`s in
a single JVM.
Author: Michael Armbrust mich...@databricks.com
Closes #4441 from marmbrus
Repository: spark
Updated Branches:
refs/heads/master 61073f832 - 76c4bf59f
[SQL] Use TestSQLContext in Java tests
Sometimes tests were failing due to the creation of multiple `SparkContext`s in
a single JVM.
Author: Michael Armbrust mich...@databricks.com
Closes #4441 from marmbrus
and
field renaming to the select clause in an `INSERT INTO/OVERWRITE` statement.
Also, with the change of this PR, we always invalidate our in memory data cache
after inserting into a BaseRelation.
cc marmbrus liancheng
Author: Yin Huai yh...@databricks.com
Closes #4373 from yhuai/insertFollowUp
Repository: spark
Updated Branches:
refs/heads/branch-1.3 59798cb44 - 50c48ebbe
http://git-wip-us.apache.org/repos/asf/spark/blob/50c48ebb/sql/core/src/test/scala/org/apache/spark/sql/parquet/ParquetFilterSuite.scala
--
diff
Repository: spark
Updated Branches:
refs/heads/master c19152cd2 - a9ed51178
http://git-wip-us.apache.org/repos/asf/spark/blob/a9ed5117/sql/core/src/test/scala/org/apache/spark/sql/parquet/ParquetFilterSuite.scala
--
diff --git
[SPARK-5182] [SPARK-5528] [SPARK-5509] [SPARK-3575] [SQL] Parquet data source
improvements
This PR adds three major improvements to Parquet data source:
1. Partition discovery
While reading Parquet files resides in Hive style partition directories,
`ParquetRelation2` automatically
[SPARK-5182] [SPARK-5528] [SPARK-5509] [SPARK-3575] [SQL] Parquet data source
improvements
This PR adds three major improvements to Parquet data source:
1. Partition discovery
While reading Parquet files resides in Hive style partition directories,
`ParquetRelation2` automatically
Repository: spark
Updated Branches:
refs/heads/branch-1.3 c79dd1e48 - 792079119
[SPARK-5587][SQL] Support change database owner
Support change database owner, here i do not add the golden files since the
golden answer is related to the tmp dir path (see
Repository: spark
Updated Branches:
refs/heads/master a9f0db1fb - b90dd3979
[SPARK-5587][SQL] Support change database owner
Support change database owner, here i do not add the golden files since the
golden answer is related to the tmp dir path (see
Repository: spark
Updated Branches:
refs/heads/master 0a89b1568 - a9f0db1fb
[SPARK-5591][SQL] Fix NoSuchObjectException for CTAS
Now CTAS runs successfully but will throw a NoSuchObjectException.
```
create table sc as select *
from (select '2011-01-11', '2011-01-11+14:18:26' from src
Repository: spark
Updated Branches:
refs/heads/branch-1.3 f9bb3cb02 - c79dd1e48
[SPARK-5591][SQL] Fix NoSuchObjectException for CTAS
Now CTAS runs successfully but will throw a NoSuchObjectException.
```
create table sc as select *
from (select '2011-01-11', '2011-01-11+14:18:26' from src
Repository: spark
Updated Branches:
refs/heads/master 424cb699e - 417d1118c
[SPARK-5367][SQL] Support star expression in udfs
A follow up for #4163: support `select array(key, *) from src`
Since array(key, *) will not go into this case
```
case Alias(f UnresolvedFunction(_, args), name)
Repository: spark
Updated Branches:
refs/heads/branch-1.3 513bb2c84 - cb4c3e55c
[SQL] Use HiveContext's sessionState in
HiveMetastoreCatalog.hiveDefaultTableFilePath
`client.getDatabaseCurrent` uses SessionState's local variable which can be an
issue.
Author: Yin Huai yh...@databricks.com
Repository: spark
Updated Branches:
refs/heads/master 417d1118c - b73d5fff7
[SQL][Hiveconsole] Bring hive console code up to date and update README.md
Add `import org.apache.spark.sql.Dsl._` to make DSL query works.
Since queryExecution is not avaliable in DataFrame, so remove it.
Author:
Repository: spark
Updated Branches:
refs/heads/branch-1.3 2cdcfe382 - 513bb2c84
[SQL] Correct the default size of TimestampType and expose NumericType
Author: Yin Huai yh...@databricks.com
Closes #4314 from yhuai/minor and squashes the following commits:
d3870a7 [Yin Huai] Update test.
Repository: spark
Updated Branches:
refs/heads/master 548c9c2b2 - e0490e271
[SPARK-5118][SQL] Fix: create table test stored as parquet as select ..
Author: guowei2 guow...@asiainfo.com
Closes #3921 from guowei2/SPARK-5118 and squashes the following commits:
b1ba3be [guowei2] add table file
Repository: spark
Updated Branches:
refs/heads/branch-1.3 abc184e59 - bf43781bd
[SPARK-5606][SQL] Support plus sign in HiveContext
Now spark version is only support ```SELECT -key FROM DECIMAL_UDF;``` in
HiveContext.
This patch is used to support ```SELECT +key FROM DECIMAL_UDF;``` in
Repository: spark
Updated Branches:
refs/heads/master db3469046 - 9d3a75ef8
[SPARK-5606][SQL] Support plus sign in HiveContext
Now spark version is only support ```SELECT -key FROM DECIMAL_UDF;``` in
HiveContext.
This patch is used to support ```SELECT +key FROM DECIMAL_UDF;``` in
Repository: spark
Updated Branches:
refs/heads/branch-1.3 cb4c3e55c - 06da8682b
[SPARK-5118][SQL] Fix: create table test stored as parquet as select ..
Author: guowei2 guow...@asiainfo.com
Closes #3921 from guowei2/SPARK-5118 and squashes the following commits:
b1ba3be [guowei2] add table
Repository: spark
Updated Branches:
refs/heads/master 4204a1271 - 0c20ce69f
[SPARK-4987] [SQL] parquet timestamp type support
Author: Daoyuan Wang daoyuan.w...@intel.com
Closes #3820 from adrian-wang/parquettimestamp and squashes the following
commits:
b1e2a0d [Daoyuan Wang] fix for nanos
Repository: spark
Updated Branches:
refs/heads/branch-1.3 4204a1271 - 67d52207b
[SPARK-4987] [SQL] parquet timestamp type support
Author: Daoyuan Wang daoyuan.w...@intel.com
Closes #3820 from adrian-wang/parquettimestamp and squashes the following
commits:
b1e2a0d [Daoyuan Wang] fix for
Repository: spark
Updated Branches:
refs/heads/master 0c20ce69f - ca7a6cdff
[SPARK-5550] [SQL] Support the case insensitive for UDF
SQL in HiveContext, should be case insensitive, however, the following query
will fail.
```scala
udf.register(random0, () = { Math.random()})
Repository: spark
Updated Branches:
refs/heads/branch-1.3 5dbeb2104 - 6e244cf4e
[SPARK-4508] [SQL] build native date type to conform behavior to Hive
The previous #3732 is reverted due to some test failure.
Have fixed that.
Author: Daoyuan Wang daoyuan.w...@intel.com
Closes #4325 from
Repository: spark
Updated Branches:
refs/heads/master 62a93a169 - 683e93824
http://git-wip-us.apache.org/repos/asf/spark/blob/683e9382/sql/hive/v0.13.1/src/main/scala/org/apache/spark/sql/hive/Shim13.scala
--
diff --git
Repository: spark
Updated Branches:
refs/heads/branch-1.2 b978c9fee - 54864403c
[SPARK-5195][sql]Update HiveMetastoreCatalog.scala(override the
MetastoreRelation's sameresult method only compare databasename and table name)
override the MetastoreRelation's sameresult method only compare
Repository: spark
Updated Branches:
refs/heads/master b1aa8fe98 - dca6faa29
[SPARK-5195][sql]Update HiveMetastoreCatalog.scala(override the
MetastoreRelation's sameresult method only compare databasename and table name)
override the MetastoreRelation's sameresult method only compare
Repository: spark
Updated Branches:
refs/heads/master 1bcd46574 - 8f471a66d
[SPARK-5472][SQL] A JDBC data source for Spark SQL.
This pull request contains a Spark SQL data source that can pull data from, and
can put data into, a JDBC database.
I have tested both read and write support with
Repository: spark
Updated Branches:
refs/heads/master 883bc88d5 - ef89b82d8
[Minor][SQL] Little refactor DataFrame related codes
Simplify some codes related to DataFrame.
* Calling `toAttributes` instead of a `map`.
* Original `createDataFrame` creates the `StructType` and its attributes
Repository: spark
Updated Branches:
refs/heads/master 8cf4a1f02 - ec1003219
[SPARK-5465] [SQL] Fixes filter push-down for Parquet data source
Not all Catalyst filter expressions can be converted to Parquet filter
predicates. We should try to convert each individual predicate and then collect
Repository: spark
Updated Branches:
refs/heads/master 7712ed5b1 - 1b56f1d6b
[SPARK-5196][SQL] Support `comment` in Create Table Field DDL
Support `comment` in create a table field.
__CREATE TEMPORARY TABLE people(name string `comment` the name of a person)__
Author: OopsOutOfMemory
Repository: spark
Updated Branches:
refs/heads/master 1b56f1d6b - 8cf4a1f02
[SPARK-5262] [SPARK-5244] [SQL] add coalesce in SQLParser and widen types for
parameters of coalesce
I'll add test case in #4040
Author: Daoyuan Wang daoyuan.w...@intel.com
Closes #4057 from adrian-wang/coal and
fail before the fix in
ScalaReflection. The fix makes the test suite succeed.
CC: marmbrus
Author: Joseph K. Bradley jos...@databricks.com
Closes #4295 from jkbradley/SPARK-5504 and squashes the following commits:
6b7276d [Joseph K. Bradley] Fixed issue in ScalaReflection.convertToCatalyst
Repository: spark
Updated Branches:
refs/heads/master bce0ba1fb - 940f37561
[SPARK-5309][SQL] Add support for dictionaries in PrimitiveConverter for
Strin...
...gs.
Parquet Converters allow developers to take advantage of dictionary encoding of
column data to reduce Column Binary decoding.
Repository: spark
Updated Branches:
refs/heads/master c1b3eebf9 - c00d517d6
[SPARK-4296][SQL] Trims aliases when resolving and checking aggregate
expressions
I believe that SPARK-4296 has been fixed by
3684fd21e1ffdc0adaad8ff6b31394b637e866ce. I am adding tests based #3910 (change
the udf
Repository: spark
Updated Branches:
refs/heads/master fbaf9e089 - c1b3eebf9
[SPARK-5373][SQL] Literal in agg grouping expressions leads to incorrect result
`select key, count( * ) from src group by key, 1` will get the wrong answer.
e.g. for this table
```
val testData2 =
Repository: spark
Updated Branches:
refs/heads/master 940f37561 - de221ea03
[SPARK-4786][SQL]: Parquet filter pushdown for castable types
Enable parquet filter pushdown of castable types like short, byte that can be
cast to integer
Author: Yash Datta yash.da...@guavus.com
Closes #4156 from
Repository: spark
Updated Branches:
refs/heads/master 3ee3ab592 - 812d3679f
[SPARK-5244] [SQL] add coalesce() in sql parser
Author: Daoyuan Wang daoyuan.w...@intel.com
Closes #4040 from adrian-wang/coalesce and squashes the following commits:
0ac8e8f [Daoyuan Wang] add coalesce() in sql
Repository: spark
Updated Branches:
refs/heads/master 812d3679f - 8361078ef
[SPARK-5009] [SQL] Long keyword support in SQL Parsers
* The `SqlLexical.allCaseVersions` will cause `StackOverflowException` if the
key word is too long, the patch will fix that by normalizing all of the
keywords
Repository: spark
Updated Branches:
refs/heads/master 4432568aa - cd5da4285
[SPARK-5284][SQL] Insert into Hive throws NPE when a inner complex type field
has a null value
JIRA: https://issues.apache.org/jira/browse/SPARK-5284
Author: Yin Huai yh...@databricks.com
Closes #4077 from
Repository: spark
Updated Branches:
refs/heads/master cd5da4285 - 2604bc35d
[SPARK-5286][SQL] Fail to drop an invalid table when using the data source API
JIRA: https://issues.apache.org/jira/browse/SPARK-5286
Author: Yin Huai yh...@databricks.com
Closes #4076 from yhuai/SPARK-5286 and
Repository: spark
Updated Branches:
refs/heads/master fd3a8a1d1 - ee1c1f3a0
[SPARK-4937][SQL] Adding optimization to simplify the And, Or condition in
spark sql
Adding optimization to simplify the And/Or condition in spark sql.
There are two kinds of Optimization
1 Numeric condition
Repository: spark
Updated Branches:
refs/heads/master 13d240678 - cfa397c12
[SPARK-5193][SQL] Tighten up SQLContext API
1. Removed 2 implicits (logicalPlanToSparkQuery and baseRelationToSchemaRDD)
2. Moved extraStrategies into ExperimentalMethods.
3. Made private methods protected[sql] so
Repository: spark
Updated Branches:
refs/heads/master 8ead999fd - 6463e0b9e
[SPARK-4912][SQL] Persistent tables for the Spark SQL data sources api
With changes in this PR, users can persist metadata of tables created based on
the data source API in metastore through DDLs.
Author: Yin Huai
Repository: spark
Updated Branches:
refs/heads/master 6463e0b9e - 14e3f114e
[SPARK-5168] Make SQLConf a field rather than mixin in SQLContext
This change should be binary and source backward compatible since we didn't
change any user facing APIs.
Author: Reynold Xin r...@databricks.com
from marmbrus/SPARK-5049 and squashes the following commits:
dd03e4e [Michael Armbrust] Fill in the partition values of parquet scans
instead of using JoinedRow
(cherry picked from commit 5d9fa550820543ee1b0ce82997917745973a5d65)
Signed-off-by: Michael Armbrust mich...@databricks.com
Project
marmbrus/SPARK-5049 and squashes the following commits:
dd03e4e [Michael Armbrust] Fill in the partition values of parquet scans
instead of using JoinedRow
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5d9fa550
Tree: http
Repository: spark
Updated Branches:
refs/heads/master 5d9fa5508 - 1e42e96ec
[SPARK-5138][SQL] Ensure schema can be inferred from a namedtuple
When attempting to infer the schema of an RDD that contains namedtuples,
pyspark fails to identify the records as namedtuples, resulting in it raising
Repository: spark
Updated Branches:
refs/heads/master 545dfcb92 - 1e56eba5d
[SPARK-4925][SQL] Publish Spark SQL hive-thriftserver maven artifact
Author: Alex Liu alex_li...@yahoo.com
Closes #3766 from alexliu68/SPARK-SQL-4925 and squashes the following commits:
3137b51 [Alex Liu]
Repository: spark
Updated Branches:
refs/heads/branch-1.2 2f4e73d8f - 09eef3b5c
[SPARK-4925][SQL] Publish Spark SQL hive-thriftserver maven artifact
Author: Alex Liu alex_li...@yahoo.com
Closes #3766 from alexliu68/SPARK-SQL-4925 and squashes the following commits:
3137b51 [Alex Liu]
Repository: spark
Updated Branches:
refs/heads/master 1e56eba5d - 4b39fd1e6
[SPARK-4943][SQL] Allow table name having dot for db/catalog
The pull only fixes the parsing error and changes API to use tableIdentifier.
Joining different catalog datasource related change is not done in this pull.
Repository: spark
Updated Branches:
refs/heads/master 4b39fd1e6 - 693a323a7
[SPARK-4574][SQL] Adding support for defining schema in foreign DDL commands.
Adding support for defining schema in foreign DDL commands. Now foreign DDL
support commands like:
```
CREATE TEMPORARY TABLE avroTable
Repository: spark
Updated Branches:
refs/heads/branch-1.2 09eef3b5c - c6ea6d4b7
[SPARK-4943][SQL] Allow table name having dot for db/catalog
The pull only fixes the parsing error and changes API to use tableIdentifier.
Joining different catalog datasource related change is not done in this
Repository: spark
Updated Branches:
refs/heads/branch-1.2 c6ea6d4b7 - cce003d75
[SPARK-5187][SQL] Fix caching of tables with HiveUDFs in the WHERE clause
Author: Michael Armbrust mich...@databricks.com
Closes #3987 from marmbrus/hiveUdfCaching and squashes the following commits:
8bca2fa
Repository: spark
Updated Branches:
refs/heads/master 77106df69 - 3684fd21e
[SPARK-5187][SQL] Fix caching of tables with HiveUDFs in the WHERE clause
Author: Michael Armbrust mich...@databricks.com
Closes #3987 from marmbrus/hiveUdfCaching and squashes the following commits:
8bca2fa
Repository: spark
Updated Branches:
refs/heads/master 8a29dc716 - 92d9a704c
[SPARK-4871][SQL] Show sql statement in spark ui when run sql with spark-sql
Author: wangfei wangf...@huawei.com
Closes #3718 from scwf/sparksqlui and squashes the following commits:
e0d6b5d [wangfei] format fix
HiveQuerySuite.scala
1893956 [YanTangZhai] Merge pull request #14 from marmbrus/pr/3555
59e4de9 [Michael Armbrust] make hive test
718afeb [YanTangZhai] Merge pull request #12 from apache/master
950b21e [YanTangZhai] Update HiveQuerySuite.scala
74175b4 [YanTangZhai] Update HiveQuerySuite.scala
92242c7
Repository: spark
Updated Branches:
refs/heads/master 92d9a704c - d22a31f5e
[SPARK-5029][SQL] Enable from follow multiple brackets
Enable from follow multiple brackets:
```
select key from ((select * from testData limit 1) union all (select * from
testData limit 1)) x limit 1
```
Author:
Repository: spark
Updated Branches:
refs/heads/master bbcba3a94 - 04d55d8e8
[SPARK-5040][SQL] Support expressing unresolved attributes using $attribute
name notation in SQL DSL.
Author: Reynold Xin r...@databricks.com
Closes #3862 from rxin/stringcontext-attr and squashes the following
.
Author: Michael Armbrust mich...@databricks.com
Closes #3834 from marmbrus/hiveConcurrency and squashes the following commits:
bf25300 [Michael Armbrust] prevent multiple concurrent hive native commands
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org
.
Author: Michael Armbrust mich...@databricks.com
Closes #3834 from marmbrus/hiveConcurrency and squashes the following commits:
bf25300 [Michael Armbrust] prevent multiple concurrent hive native commands
(cherry picked from commit 480bd1d2edd1de06af607b0cf3ff3c0b16089add)
Signed-off-by: Michael
Repository: spark
Updated Branches:
refs/heads/master 480bd1d2e - 94d60b702
[SQL] enable view test
This is a follow up of #3396 , just add a test to white list.
Author: Daoyuan Wang daoyuan.w...@intel.com
Closes #3826 from adrian-wang/viewtest and squashes the following commits:
f105f68
Repository: spark
Updated Branches:
refs/heads/master 94d60b702 - 65357f11c
[SPARK-4975][SQL] Fix HiveInspectorSuite test failure
HiveInspectorSuite test failureï¼
[info] - wrap / unwrap null, constant null and writables *** FAILED *** (21
milliseconds)
[info] 1 did not equal 0
Repository: spark
Updated Branches:
refs/heads/master 65357f11c - 5595eaa74
[SPARK-4959] [SQL] Attributes are case sensitive when using a select query from
a projection
Author: Cheng Hao hao.ch...@intel.com
Closes #3796 from chenghao-intel/spark_4959 and squashes the following commits:
Repository: spark
Updated Branches:
refs/heads/master 5595eaa74 - 63b84b7d6
[SPARK-4904] [SQL] Remove the unnecessary code change in Generic UDF
Since #3429 has been merged, the bug of wrapping to Writable for HiveGenericUDF
is resolved, we can safely remove the foldable checking in
Repository: spark
Updated Branches:
refs/heads/master 63b84b7d6 - daac22130
[SPARK-5002][SQL] Using ascending by default when not specify order in order by
spark sql does not support ```SELECT a, b FROM testData2 ORDER BY a desc, b```.
Author: wangfei wangf...@huawei.com
Closes #3838 from
Repository: spark
Updated Branches:
refs/heads/master daac22130 - 53f0a00b6
[Spark-4512] [SQL] Unresolved Attribute Exception in Sort By
It will cause exception while do query like:
SELECT key+key FROM src sort by value;
Author: Cheng Hao hao.ch...@intel.com
Closes #3386 from
Repository: spark
Updated Branches:
refs/heads/master 53f0a00b6 - 19a8802e7
[SPARK-4493][SQL] Tests for IsNull / IsNotNull in the ParquetFilterSuite
This is a follow-up of #3367 and #3644.
At the time #3644 was written, #3367 hadn't been merged yet, thus `IsNull` and
`IsNotNull` filters are
Repository: spark
Updated Branches:
refs/heads/master 19a8802e7 - f7a41a0e7
[SPARK-4916][SQL][DOCS]Update SQL programming guide about cache section
`SchemeRDD.cache()` now uses in-memory columnar storage.
Author: luogankun luogan...@gmail.com
Closes #3759 from luogankun/SPARK-4916 and
Repository: spark
Updated Branches:
refs/heads/master f7a41a0e7 - 2deac748b
[SPARK-4930][SQL][DOCS]Update SQL programming guide, CACHE TABLE is eager
`CACHE TABLE tbl` is now __eager__ by default not __lazy__
Author: luogankun luogan...@gmail.com
Closes #3773 from luogankun/SPARK-4930 and
Repository: spark
Updated Branches:
refs/heads/master 2deac748b - a75dd83b7
[SPARK-4928][SQL] Fix: Operator ',,=,=' with decimal between different
precision report error
case operator with decimal between different precision, we need change them to
unlimited
Author: guowei2
Repository: spark
Updated Branches:
refs/heads/master a75dd83b7 - 61a99f6a1
[SPARK-4937][SQL] Normalizes conjunctions and disjunctions to eliminate common
predicates
This PR is a simplified version of several filter optimization rules introduced
in #3778 authored by scwf. Newly introduced
Repository: spark
Updated Branches:
refs/heads/master 61a99f6a1 - 7425bec32
[SPARK-4386] Improve performance when writing Parquet files
Convert type of RowWriteSupport.attributes to Array.
Analysis of performance for writing very wide tables shows that time is spent
predominantly in apply
Repository: spark
Updated Branches:
refs/heads/branch-1.2 cde8a310a - 7a245412f
[SPARK-4386] Improve performance when writing Parquet files
Convert type of RowWriteSupport.attributes to Array.
Analysis of performance for writing very wide tables shows that time is spent
predominantly in
Repository: spark
Updated Branches:
refs/heads/master 7425bec32 - 8f29b7caf
[SPARK-4935][SQL] When hive.cli.print.header configured, spark-sql aborted if
passed in a invalid sql
If we passed in a wrong sql like ```abdcdfsfs```, the spark-sql script aborted.
Author: wangfei
Repository: spark
Updated Branches:
refs/heads/master 8f29b7caf - 07fa1910d
[SPARK-4570][SQL]add BroadcastLeftSemiJoinHash
JIRA issue: [SPARK-4570](https://issues.apache.org/jira/browse/SPARK-4570)
We are planning to create a `BroadcastLeftSemiJoinHash` to implement the
broadcast join for
Repository: spark
Updated Branches:
refs/heads/master 9804a759b - f728e0fe7
[SPARK-2663] [SQL] Support the Grouping Set
Add support for `GROUPING SETS`, `ROLLUP`, `CUBE` and the the virtual column
`GROUPING__ID`.
More details on how to use the `GROUPING SETS can be found at:
Repository: spark
Updated Branches:
refs/heads/master f728e0fe7 - b68bc6d26
[SPARK-3928][SQL] Support wildcard matches on Parquet files.
...arquetFile accept hadoop glob pattern in path.
Author: Thu Kyaw trk...@gmail.com
Closes #3407 from tkyaw/master and squashes the following commits:
Repository: spark
Updated Branches:
refs/heads/master b68bc6d26 - 22ddb6e03
[SPARK-4756][SQL] FIX: sessionToActivePool grow infinitely, even as sessions
expire
**sessionToActivePool** in **SparkSQLOperationManager** grow infinitely, even
as sessions expire.
we should remove the pool value
Repository: spark
Updated Branches:
refs/heads/master 22ddb6e03 - e7de7e5f4
[SPARK-4693] [SQL] PruningPredicates may be wrong if predicates contains an
empty AttributeSet() references
The sql select * from spark_test::for_test where abs(20141202) is not null
has predicates=List(IS NOT NULL
Repository: spark
Updated Branches:
refs/heads/master e7de7e5f4 - 7687415c2
[SPARK-2554][SQL] Supporting SumDistinct partial aggregation
Adding support to the partial aggregation of SumDistinct
Author: ravipesala ravindra.pes...@huawei.com
Closes #3348 from ravipesala/SPARK-2554 and
Repository: spark
Updated Branches:
refs/heads/master 7687415c2 - ae9f12860
[SPARK-4573] [SQL] Add SettableStructObjectInspector support in wrap function
Hive UDAF may create an customized object constructed by
SettableStructObjectInspector, this is critical when integrate Hive UDAF with
Repository: spark
Updated Branches:
refs/heads/master cf50631a6 - 5fdcbdc0c
[SPARK-4625] [SQL] Add sort by for DSL SimpleSqlParser
Add `sort by` support for both DSL SqlParser.
This PR is relevant with #3386, either one merged, will cause the other rebased.
Author: Cheng Hao
Repository: spark
Updated Branches:
refs/heads/master 5fdcbdc0c - 4782def09
[SPARK-4694]Fix HiveThriftServer2 cann't stop In Yarn HA mode.
HiveThriftServer2 can not exit automactic when changing the standy resource
manager in Yarn HA mode.
The scheduler backend was aware of the AM had been
Repository: spark
Updated Branches:
refs/heads/master 4782def09 - 7ad579ee9
[SPARK-3698][SQL] Fix case insensitive resolution of GetField.
Based on #2543.
Author: Michael Armbrust mich...@databricks.com
Closes #3724 from marmbrus/resolveGetField and squashes the following commits:
0a47aae
Repository: spark
Updated Branches:
refs/heads/master 7ad579ee9 - 627713537
[SPARK-4493][SQL] Don't pushdown Eq, NotEq, Lt, LtEq, Gt and GtEq predicates
with nulls for Parquet
Predicates like `a = NULL` and `a NULL` can't be pushed down since Parquet
`Lt`, `LtEq`, `Gt`, `GtEq` doesn't
Repository: spark
Updated Branches:
refs/heads/master 8d0d2a65e - f33d55046
[SPARK-3891][SQL] Add array support to percentile, percentile_approx and
constant inspectors support
Supported passing array to percentile and percentile_approx UDAFs
To support percentile_approx, constant
Repository: spark
Updated Branches:
refs/heads/master d12c0711f - 17688d142
[SQL] SPARK-4700: Add HTTP protocol spark thrift server
Add HTTP protocol support and test cases to spark thrift server, so users can
deploy thrift server in both TCP and http mode.
Author: Judy Nash
Repository: spark
Updated Branches:
refs/heads/master 17688d142 - 1a9e35e57
[DOCS][SQL] Add a Note on jsonFile having separate JSON objects per line
* This commit hopes to avoid the confusion I faced when trying
to submit a regular, valid multi-line JSON file, also see
Repository: spark
Updated Branches:
refs/heads/master 1a9e35e57 - dc8280dcc
[SPARK-4847][SQL]Fix extraStrategies cannot take effect in SQLContext issue
Author: jerryshao saisai.s...@intel.com
Closes #3698 from jerryshao/SPARK-4847 and squashes the following commits:
4741130 [jerryshao] Make
Repository: spark
Updated Branches:
refs/heads/branch-1.2 4f9916f1e - 1b6fc237c
[SPARK-4847][SQL]Fix extraStrategies cannot take effect in SQLContext issue
Author: jerryshao saisai.s...@intel.com
Closes #3698 from jerryshao/SPARK-4847 and squashes the following commits:
4741130 [jerryshao]
Repository: spark
Updated Branches:
refs/heads/master dc8280dcc - 6530243a5
[SPARK-4812][SQL] Fix the initialization issue of 'codegenEnabled'
The problem is `codegenEnabled` is `val`, but it uses a `val` `sqlContext`,
which can be override by subclasses. Here is a simple example to show
Repository: spark
Updated Branches:
refs/heads/master b0dfdbdd1 - ea1315e3e
[SPARK-4527][SQl]Add BroadcastNestedLoopJoin operator selection testsuite
In `JoinSuite` add BroadcastNestedLoopJoin operator selection testsuite
Author: wangxiaojing u9j...@gmail.com
Closes #3395 from
Repository: spark
Updated Branches:
refs/heads/master ea1315e3e - 30f6b85c8
[SPARK-4483][SQL]Optimization about reduce memory costs during the HashOuterJoin
In `HashOuterJoin.scala`, spark read data from both side of join operation
before zip them together. It is a waste for memory. We are
Repository: spark
Updated Branches:
refs/heads/master b85044ecf - 3b395e105
[SPARK-4798][SQL] A new set of Parquet testing API and test suites
This PR provides a set Parquet testing API (see trait `ParquetTest`) that
enables developers to write more concise test cases. A new set of Parquet
Repository: spark
Updated Branches:
refs/heads/master 3b395e105 - 0aa834ade
[SPARK-4744] [SQL] Short circuit evaluation for AND OR in CodeGen
Author: Cheng Hao hao.ch...@intel.com
Closes #3606 from chenghao-intel/codegen_short_circuit and squashes the
following commits:
f466303 [Cheng
Repository: spark
Updated Branches:
refs/heads/master ddc7ba31c - 770d8153a
[SPARK-4375] [SQL] Add 0 argument support for udf
Author: Cheng Hao hao.ch...@intel.com
Closes #3595 from chenghao-intel/udf0 and squashes the following commits:
a858973 [Cheng Hao] Add 0 arguments support for udf
Repository: spark
Updated Branches:
refs/heads/master 770d8153a - ec5c4279e
[SPARK-4866] support StructType as key in MapType
This PR brings support of using StructType(and other hashable types) as key in
MapType.
Author: Davies Liu dav...@databricks.com
Closes #3714 from
Repository: spark
Updated Branches:
refs/heads/master ec5c4279e - 60698801e
[SPARK-4618][SQL] Make foreign DDL commands options case-insensitive
Using lowercase for ```options``` key to make it case-insensitive, then we
should use lower case to get value from parameters.
So flowing cmd work
1001 - 1100 of 1624 matches
Mail list logo
