Repository: spark
Updated Branches:
refs/heads/master 21fac5434 -> e8833dd12
[SPARK-11679][SQL] Invoking method " apply(fields:
java.util.List[StructField])" in "StructType" gets ClassCastException
In the previous method, fields.toArray will cast java.util.List[StructField]
into
Repository: spark
Updated Branches:
refs/heads/master 0158ff773 -> d92514966
[SPARK-10186][SQL] support postgre array type in JDBCRDD
Add ARRAY support to `PostgresDialect`.
Nested ARRAY is not allowed for now because it's hard to get the array
dimension info. See
Repository: spark
Updated Branches:
refs/heads/branch-1.6 38fe092ff -> fbe65c592
[SPARK-11754][SQL] consolidate `ExpressionEncoder.tuple` and `Encoders.tuple`
These 2 are very similar, we can consolidate them into one.
Also add tests for it and fix a bug.
Author: Wenchen Fan
Repository: spark
Updated Branches:
refs/heads/branch-1.6 fbe65c592 -> 90d71bff0
[SPARK-11390][SQL] Query plan with/without filterPushdown indistinguishable
â¦ishable
Propagate pushed filters to PhyicalRDD in DataSourceStrategy.apply
Author: Zee Chen
Closes #9679 from
Repository: spark
Updated Branches:
refs/heads/master b1a966262 -> 985b38dd2
[SPARK-11390][SQL] Query plan with/without filterPushdown indistinguishable
â¦ishable
Propagate pushed filters to PhyicalRDD in DataSourceStrategy.apply
Author: Zee Chen
Closes #9679 from
Repository: spark
Updated Branches:
refs/heads/master 24477d270 -> b1a966262
[SPARK-11754][SQL] consolidate `ExpressionEncoder.tuple` and `Encoders.tuple`
These 2 are very similar, we can consolidate them into one.
Also add tests for it and fix a bug.
Author: Wenchen Fan
Repository: spark
Updated Branches:
refs/heads/master 75ee12f09 -> fd14936be
[SPARK-11625][SQL] add java test for typed aggregate
Author: Wenchen Fan
Closes #9591 from cloud-fan/agg-test.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/branch-1.6 3bd72eafc -> 6c8e0c0ff
[SPARK-11553][SQL] Primitive Row accessors should not convert null to default
value
Invocation of getters for type extending AnyVal returns default value (if field
value is null) instead of throwing NPE. Please
Repository: spark
Updated Branches:
refs/heads/branch-1.6 6c8e0c0ff -> e042780cd
[SPARK-8658][SQL] AttributeReference's equals method compares all the members
This fix is to change the equals method to check all of the specified fields
for equality of AttributeReference.
Author: gatorsmile
Repository: spark
Updated Branches:
refs/heads/master 31296628a -> 75ee12f09
[SPARK-8658][SQL] AttributeReference's equals method compares all the members
This fix is to change the equals method to check all of the specified fields
for equality of AttributeReference.
Author: gatorsmile
Repository: spark
Updated Branches:
refs/heads/branch-1.6 e042780cd -> 4f8c7e18f
[SPARK-11625][SQL] add java test for typed aggregate
Author: Wenchen Fan
Closes #9591 from cloud-fan/agg-test.
(cherry picked from commit fd14936be7beff543dbbcf270f2f9749f7a803c4)
Repository: spark
Updated Branches:
refs/heads/master bcea0bfda -> 31296628a
[SPARK-11553][SQL] Primitive Row accessors should not convert null to default
value
Invocation of getters for type extending AnyVal returns default value (if field
value is null) instead of throwing NPE. Please
Repository: spark
Updated Branches:
refs/heads/branch-1.6 6459a6747 -> 3035e9d23
[SPARK-11654][SQL][FOLLOW-UP] fix some mistakes and clean up
* rename `AppendColumn` to `AppendColumns` to be consistent with the physical
plan name.
* clean up stale comments.
* always pass in resolved encoder
Repository: spark
Updated Branches:
refs/heads/branch-1.6 3035e9d23 -> 8757221a3
[SPARK-11727][SQL] Split ExpressionEncoder into FlatEncoder and ProductEncoder
also add more tests for encoders, and fix bugs that I found:
* when convert array to catalyst array, we can only skip element
Repository: spark
Updated Branches:
refs/heads/master a24477996 -> 23b8188f7
[SPARK-11654][SQL][FOLLOW-UP] fix some mistakes and clean up
* rename `AppendColumn` to `AppendColumns` to be consistent with the physical
plan name.
* clean up stale comments.
* always pass in resolved encoder to
Repository: spark
Updated Branches:
refs/heads/master df0e31815 -> 4fe99c72c
[SPARK-11191][SQL] Looks up temporary function using execution Hive client
When looking up Hive temporary functions, we should always use the
`SessionState` within the execution Hive client, since temporary
Repository: spark
Updated Branches:
refs/heads/branch-1.6 ecf027edd -> 68fa5c713
[SPARK-10113][SQL] Explicit error message for unsigned Parquet logical types
Parquet supports some unsigned datatypes. However, Since Spark does not support
unsigned datatypes, it needs to emit an exception with
rcing the above
constraints in the type system (i.e. `fromRow` only exists on a
`ResolvedEncoder`), but we should probably wait before spending too much time
on this.
Author: Michael Armbrust <mich...@databricks.com>
Author: Wenchen Fan <wenc...@databricks.com>
Closes #9673 from
sider enforcing the above
constraints in the type system (i.e. `fromRow` only exists on a
`ResolvedEncoder`), but we should probably wait before spending too much time
on this.
Author: Michael Armbrust <mich...@databricks.com>
Author: Wenchen Fan <wenc...@databrick
Repository: spark
Updated Branches:
refs/heads/branch-1.6 4aacbe9e6 -> ecf027edd
[SPARK-11191][SQL] Looks up temporary function using execution Hive client
When looking up Hive temporary functions, we should always use the
`SessionState` within the execution Hive client, since temporary
Repository: spark
Updated Branches:
refs/heads/master c964fc101 -> 9c57bc0ef
[SPARK-11656][SQL] support typed aggregate in project list
insert `aEncoder` like we do in `agg`
Author: Wenchen Fan
Closes #9630 from cloud-fan/select.
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.6 f9aeb961e -> 9bf988555
[SPARK-11564][SQL][FOLLOW-UP] clean up java tuple encoder
We need to support custom classes like java beans and combine them into tuple,
and it's very hard to do it with the TypeTag-based approach.
We should
Repository: spark
Updated Branches:
refs/heads/master 9c57bc0ef -> ec2b80721
[SPARK-11564][SQL][FOLLOW-UP] clean up java tuple encoder
We need to support custom classes like java beans and combine them into tuple,
and it's very hard to do it with the TypeTag-based approach.
We should keep
Repository: spark
Updated Branches:
refs/heads/master ec2b80721 -> e71ba5658
[SQL][MINOR] remove newLongEncoder in functions
it may shadows the one from implicits in some case.
Author: Wenchen Fan
Closes #9629 from cloud-fan/minor.
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.6 9bf988555 -> 47cc1fe06
[SQL][MINOR] remove newLongEncoder in functions
it may shadows the one from implicits in some case.
Author: Wenchen Fan
Closes #9629 from cloud-fan/minor.
(cherry picked from commit
Repository: spark
Updated Branches:
refs/heads/master e71ba5658 -> 529a1d338
[SPARK-6152] Use shaded ASM5 to support closure cleaning of Java 8 compiled
classes
This patch modifies Spark's closure cleaner (and a few other places) to use ASM
5, which is necessary in order to support cleaning
Repository: spark
Updated Branches:
refs/heads/branch-1.6 47cc1fe06 -> 1fbfc1b48
[SPARK-6152] Use shaded ASM5 to support closure cleaning of Java 8 compiled
classes
This patch modifies Spark's closure cleaner (and a few other places) to use ASM
5, which is necessary in order to support
Repository: spark
Updated Branches:
refs/heads/master 47735cdc2 -> dfcfcbcc0
[SPARK-11578][SQL][FOLLOW-UP] complete the user facing api for typed aggregation
Currently the user facing api for typed aggregation has some limitations:
* the customized typed aggregation must be the first of
Repository: spark
Updated Branches:
refs/heads/branch-1.6 d2405cb5e -> 6e2e84f3e
[SPARK-11578][SQL][FOLLOW-UP] complete the user facing api for typed aggregation
Currently the user facing api for typed aggregation has some limitations:
* the customized typed aggregation must be the first of
Repository: spark
Updated Branches:
refs/heads/master 53600854c -> 87aedc48c
[SPARK-10371][SQL] Implement subexpr elimination for UnsafeProjections
This patch adds the building blocks for codegening subexpr elimination and
implements
it end to end for UnsafeProjection. The building blocks
Repository: spark
Updated Branches:
refs/heads/branch-1.6 5ccc1eb08 -> f38509a76
[SPARK-10371][SQL] Implement subexpr elimination for UnsafeProjections
This patch adds the building blocks for codegening subexpr elimination and
implements
it end to end for UnsafeProjection. The building
der as part of
the `assembly` task.
`dev/mima` also depended on `lib_managed` in a hacky way in order to set
classpaths when generating MiMa excludes; I've updated this to obtain the
classpaths directly from SBT instead.
/cc dragos marmbrus pwendell srowen
Author: Josh Rosen <joshro...@databric
http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Utils.scala
--
diff --git
http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala
--
diff --git
http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Utils.scala
--
diff --git
http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala
--
diff --git
Repository: spark
Updated Branches:
refs/heads/master 6e5fc3788 -> e0701c756
http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala
--
diff --git
[SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to
evaluate AggregateExpression1s
https://issues.apache.org/jira/browse/SPARK-9830
This PR contains the following main changes.
* Removing `AggregateExpression1`.
* Removing `Aggregate` operator, which is used to evaluate
Repository: spark
Updated Branches:
refs/heads/branch-1.6 825e971d0 -> 7c4ade0d7
http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala
--
diff --git
[SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to
evaluate AggregateExpression1s
https://issues.apache.org/jira/browse/SPARK-9830
This PR contains the following main changes.
* Removing `AggregateExpression1`.
* Removing `Aggregate` operator, which is used to evaluate
Repository: spark
Updated Branches:
refs/heads/master dba1a62cf -> 724cf7a38
[SPARK-11616][SQL] Improve toString for Dataset
Author: Michael Armbrust <mich...@databricks.com>
Closes #9586 from marmbrus/dataset-toString.
Project: http://git-wip-us.apache.org/repos/asf/spark/re
Repository: spark
Updated Branches:
refs/heads/branch-1.6 f0180106a -> 8fb7b8304
[SPARK-11616][SQL] Improve toString for Dataset
Author: Michael Armbrust <mich...@databricks.com>
Closes #9586 from marmbrus/dataset-toString.
(cherry picked from commit 724cf7a38c551bf2a79b87a8158bbe1
Repository: spark
Updated Branches:
refs/heads/branch-1.6 1585f559d -> b9adfdf9c
[SPARK-11564][SQL][FOLLOW-UP] improve java api for GroupedDataset
created `MapGroupFunction`, `FlatMapGroupFunction`, `CoGroupFunction`
Author: Wenchen Fan
Closes #9564 from
Repository: spark
Updated Branches:
refs/heads/branch-1.6 b9adfdf9c -> c42433d02
[SPARK-9557][SQL] Refactor ParquetFilterSuite and remove old ParquetFilters code
Actually this was resolved by https://github.com/apache/spark/pull/8275.
But I found the JIRA issue for this is not marked as
ing
primitive types.
I chose snake_case here instead of camelCase because it seems to be used in the
majority of the multi-word fns.
Do we also want to add these to `functions.py`?
This approach was recommended here:
https://github.com/apache/spark/pull/8592#issuecomment-154247089
marmbrus r
ort aggregating
primitive types.
I chose snake_case here instead of camelCase because it seems to be used in the
majority of the multi-word fns.
Do we also want to add these to `functions.py`?
This approach was recommended here:
https://github.com/apache/spark/pull/8592#issuecomment-154247089
marmbrus r
t;, 30, 30, 2L), ("b", 3, 3, 2L), ("c", 1, 1, 1L)
```
The current implementation focuses on integrating this into the typed API, but
currently only supports running aggregations that return a single long value as
explained in `TypedAggregateExpression`. This will be improved in a
t;, 30, 30, 2L), ("b", 3, 3, 2L), ("c", 1, 1, 1L)
```
The current implementation focuses on integrating this into the typed API, but
currently only supports running aggregations that return a single long value as
explained in `TypedAggregateExpression`. This will be improved in a
Repository: spark
Updated Branches:
refs/heads/master 26739059b -> b2d195e13
[SPARK-11554][SQL] add map/flatMap to GroupedDataset
Author: Wenchen Fan
Closes #9521 from cloud-fan/map.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/branch-1.6 27161f59e -> 6ade67e5f
[SPARK-11554][SQL] add map/flatMap to GroupedDataset
Author: Wenchen Fan
Closes #9521 from cloud-fan/map.
(cherry picked from commit b2d195e137fad88d567974659fa7023ff4da96cd)
Repository: spark
Updated Branches:
refs/heads/master 6d0ead322 -> 1c80d66e5
[SPARK-11546] Thrift server makes too many logs about result schema
SparkExecuteStatementOperation logs result schema for each getNextRowSet()
calls which is by default every 1000 rows, overwhelming whole log file.
Repository: spark
Updated Branches:
refs/heads/branch-1.6 162a7704c -> 9bf77d5c3
[SPARK-11546] Thrift server makes too many logs about result schema
SparkExecuteStatementOperation logs result schema for each getNextRowSet()
calls which is by default every 1000 rows, overwhelming whole log
Repository: spark
Updated Branches:
refs/heads/branch-1.6 40a5db561 -> 162a7704c
[SPARK-9241][SQL] Supporting multiple DISTINCT columns (2) - Rewriting Rule
The second PR for SPARK-9241, this adds support for multiple distinct columns
to the new aggregation code path.
This PR solves the
Repository: spark
Updated Branches:
refs/heads/master 1ab72b086 -> 6d0ead322
[SPARK-9241][SQL] Supporting multiple DISTINCT columns (2) - Rewriting Rule
The second PR for SPARK-9241, this adds support for multiple distinct columns
to the new aggregation code path.
This PR solves the
Repository: spark
Updated Branches:
refs/heads/master 49f1a8203 -> f328fedaf
[SPARK-11450] [SQL] Add Unsafe Row processing to Expand
This PR enables the Expand operator to process and produce Unsafe Rows.
Author: Herman van Hovell
Closes #9414 from
Repository: spark
Updated Branches:
refs/heads/branch-1.6 7755b50b4 -> efa1e4a25
[SPARK-11450] [SQL] Add Unsafe Row processing to Expand
This PR enables the Expand operator to process and produce Unsafe Rows.
Author: Herman van Hovell
Closes #9414 from
Repository: spark
Updated Branches:
refs/heads/branch-1.6 [created] f80f7b69a
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/branch-1.4 6c5e9a3a0 -> 4f98014b9
[SPARK-11188] [SQL] Elide stacktraces in bin/spark-sql for AnalysisExceptions
Only print the error message to the console for Analysis Exceptions in sql-shell
Author: Dilip Biswal
Closes
), ("c", 1))
```
Author: Michael Armbrust <mich...@databricks.com>
Closes #9359 from marmbrus/columnGroupBy and squashes the following commits:
bbcb03b [Michael Armbrust] Update DatasetSuite.scala
8fd2908 [Michael Armbrust] Update DatasetSuite.scala
0b0e2f8 [Michael Armbrust] [SP
Repository: spark
Updated Branches:
refs/heads/master 67e23b39a -> 425ff03f5
[SPARK-11436] [SQL] rebind right encoder when join 2 datasets
When we join 2 datasets, we will combine 2 encoders into a tupled one, and use
it as the encoder for the jioned dataset. Assume both of the 2 encoders
Repository: spark
Updated Branches:
refs/heads/master 1d04dc95c -> f6fcb4874
[SPARK-11477] [SQL] support create Dataset from RDD
Author: Wenchen Fan
Closes #9434 from cloud-fan/rdd2ds and squashes the following commits:
0892d72 [Wenchen Fan] support create Dataset
Repository: spark
Updated Branches:
refs/heads/branch-1.5 b85bf8f49 -> 5604ce9c1
[SPARK-11188] [SQL] Elide stacktraces in bin/spark-sql for AnalysisExceptions
Only print the error message to the console for Analysis Exceptions in sql-shell
Author: Dilip Biswal
Closes
Repository: spark
Updated Branches:
refs/heads/master 59db9e9c3 -> 14d08b990
[SPARK-11393] [SQL] CoGroupedIterator should respect the fact that
GroupedIterator.hasNext is not idempotent
When we cogroup 2 `GroupedIterator`s in `CoGroupedIterator`, if the right side
is smaller, we will
Repository: spark
Updated Branches:
refs/heads/master f7a51deeb -> 8185f038c
[SPARK-11188][SQL] Elide stacktraces in bin/spark-sql for AnalysisExceptions
Only print the error message to the console for Analysis Exceptions in
sql-shell.
Author: Dilip Biswal
Closes #9194
Repository: spark
Updated Branches:
refs/heads/master 87f28fc24 -> f79ebf2a9
[SPARK-11370] [SQL] fix a bug in GroupedIterator and create unit test for it
Before this PR, user has to consume the iterator of one group before process
next group, or we will get into infinite loops.
Author:
Repository: spark
Updated Branches:
refs/heads/master 3dfa4ea52 -> 87f28fc24
[SPARK-11379][SQL] ExpressionEncoder can't handle top level primitive type
correctly
For inner primitive type(e.g. inside `Product`), we use `schemaFor` to get the
catalyst type for it,
Repository: spark
Updated Branches:
refs/heads/master 5f1cee6f1 -> 075ce4914
[SPARK-11313][SQL] implement cogroup on DataSets (support 2 datasets)
A simpler version of https://github.com/apache/spark/pull/9279, only support 2
datasets.
Author: Wenchen Fan
Closes
Repository: spark
Updated Branches:
refs/heads/branch-1.5 86ee81e5c -> 3bd596de4
[SPARK-11303][SQL] filter should not be pushed down into sample
When sampling and then filtering DataFrame, the SQL Optimizer will push down
filter into sample and produce wrong result. This is due to the
Repository: spark
Updated Branches:
refs/heads/master 17f499920 -> 958a0ec8f
[SPARK-11277][SQL] sort_array throws exception scala.MatchError
I'm new to spark. I was trying out the sort_array function then hit this
exception. I looked into the spark source code. I found the root cause is that
Repository: spark
Updated Branches:
refs/heads/master 958a0ec8f -> 360ed832f
[SPARK-11303][SQL] filter should not be pushed down into sample
When sampling and then filtering DataFrame, the SQL Optimizer will push down
filter into sample and produce wrong result. This is due to the sampler is
Repository: spark
Updated Branches:
refs/heads/master f6d06adf0 -> 42d225f44
[SPARK-11216][SQL][FOLLOW-UP] add encoder/decoder for external row
address comments in https://github.com/apache/spark/pull/9184
Author: Wenchen Fan
Closes #9212 from cloud-fan/encoder.
Repository: spark
Updated Branches:
refs/heads/master 16906ef23 -> 8b877cc4e
[SPARK-11088][SQL] Merges partition values using UnsafeProjection
`DataSourceStrategy.mergeWithPartitionValues` is essentially a projection
implemented in a quite inefficient way. This PR optimizes this method with
Repository: spark
Updated Branches:
refs/heads/master 6a2359ff1 -> eb0b4d6e2
[SPARK-11135] [SQL] Exchange incorrectly skips sorts when existing ordering is
non-empty subset of required ordering
In Spark SQL, the Exchange planner tries to avoid unnecessary sorts in cases
where the data has
Repository: spark
Updated Branches:
refs/heads/master 0d1b73b78 -> ef72673b2
[SPARK-11080] [SQL] Incorporate per-JVM id into ExprId to prevent unsafe
cross-JVM comparisions
In the current implementation of named expressions' `ExprIds`, we rely on a
per-JVM AtomicLong to ensure that
Repository: spark
Updated Branches:
refs/heads/branch-1.5 15d2736af -> 94e6d8f72
[SPARK-10389] [SQL] [1.5] support order by non-attribute grouping expression on
Aggregate
backport https://github.com/apache/spark/pull/8548 to 1.5
Author: Wenchen Fan
Closes #9102 from
;
Closes #9100 from marmbrus/productContructor.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/328d1b3e
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/328d1b3e
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff
Repository: spark
Updated Branches:
refs/heads/master 328d1b3e4 -> e170c2216
[SPARK-11032] [SQL] correctly handle having
We should not stop resolving having when the having condtion is resolved, or
something like `count(1)` will crash.
Author: Wenchen Fan
Closes #9105
Repository: spark
Updated Branches:
refs/heads/master 5c9fdf74e -> dcbd58a92
[SPARK-8654] [SQL] Fix Analysis exception when using NULL IN (...)
In the analysis phase , while processing the rules for IN predicate, we
compare the in-list types to the lhs expression type and generate
cast
t#1, intField, IntegerType)`
Author: Michael Armbrust <mich...@databricks.com>
Closes #9022 from marmbrus/expressionToString.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5c9fdf74
Tree: http://git-wip-us.apache.org/repos/
9034 from marmbrus/revert8654.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a8226a9f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a8226a9f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a8226a9f
Branch: r
Repository: spark
Updated Branches:
refs/heads/master 9672602c7 -> f5d154bc7
http://git-wip-us.apache.org/repos/asf/spark/blob/f5d154bc/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala
a variable for input row instead of hardcoding "i" everywhere
- rename `primitive` -> `value` (since its often actually an object)
Author: Michael Armbrust <mich...@databricks.com>
Closes #9006 from marmbrus/codegen-cleanup.
Project: http://git-wip-us.apache.org/repos/asf/spar
Repository: spark
Updated Branches:
refs/heads/branch-1.5 6c6cadb8f -> 64cc62cb5
[SPARK-10403] Allow UnsafeRowSerializer to work with tungsten-sort
ShuffleManager
This patch attempts to fix an issue where Spark SQL's UnsafeRowSerializer was
incompatible with the `tungsten-sort`
Repository: spark
Updated Branches:
refs/heads/master 27bfa9ab3 -> a18208047
[SPARK-10403] Allow UnsafeRowSerializer to work with tungsten-sort
ShuffleManager
This patch attempts to fix an issue where Spark SQL's UnsafeRowSerializer was
incompatible with the `tungsten-sort` ShuffleManager.
uce this is to
`test:compile` before running `unidoc`. To prevent this from happening again,
I've added a clean before doc generation.
Author: Michael Armbrust <mich...@databricks.com>
Closes #8787 from marmbrus/testsInDocs.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Com
can reproduce this is to
`test:compile` before running `unidoc`. To prevent this from happening again,
I've added a clean before doc generation.
Author: Michael Armbrust <mich...@databricks.com>
Closes #8787 from marmbrus/testsInDocs.
(cherry picked from commit e0dc2bc232206d2f4da4278502c1f88
Repository: spark
Updated Branches:
refs/heads/master e0dc2bc23 -> aad644fbe
[SPARK-10639] [SQL] Need to convert UDAF's result from scala to sql type
https://issues.apache.org/jira/browse/SPARK-10639
Author: Yin Huai
Closes #8788 from yhuai/udafConversion.
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.5 fd58ed48d -> 464d6e7d1
[SPARK-10639] [SQL] Need to convert UDAF's result from scala to sql type
https://issues.apache.org/jira/browse/SPARK-10639
Author: Yin Huai
Closes #8788 from yhuai/udafConversion.
Repository: spark
Updated Branches:
refs/heads/master 841972e22 -> 31a229aa7
[SPARK-10475] [SQL] improve column prunning for Project on Sort
Sometimes we can't push down the whole `Project` though `Sort`, but we still
have a chance to push down part of it.
Author: Wenchen Fan
Repository: spark
Updated Branches:
refs/heads/master b42059d2e -> 841972e22
[SPARK-10437] [SQL] Support aggregation expressions in Order By
JIRA: https://issues.apache.org/jira/browse/SPARK-10437
If an expression in `SortOrder` is a resolved one, such as `count(1)`, the
corresponding rule
Repository: spark
Updated Branches:
refs/heads/master 7e32387ae -> 64f04154e
[SPARK-6981] [SQL] Factor out SparkPlanner and QueryExecution from SQLContext
Alternative to PR #6122; in this case the refactored out classes are replaced
by inner classes with the same name for backwards binary
Repository: spark
Updated Branches:
refs/heads/master d5d647380 -> 1eede3b25
[SPARK-7142] [SQL] Minor enhancement to BooleanSimplification Optimizer rule.
Incorporate review comments
Adding changes suggested by cloud-fan in #5700
cc marmbrus
Author: Yash Datta <yash.da...@guav
Repository: spark
Updated Branches:
refs/heads/master 4f1daa1ef -> f892d927d
[SPARK-7142] [SQL] Minor enhancement to BooleanSimplification Optimizer rule
Use these in the optimizer as well:
A and (not(A) or B) => A and B
not(A and B) => not(A) or not(B)
Repository: spark
Updated Branches:
refs/heads/master 5b2192e84 -> 5fd57955e
[SPARK-10316] [SQL] respect nondeterministic expressions in PhysicalOperation
We did a lot of special handling for non-deterministic expressions in
`Optimizer`. However, `PhysicalOperation` just collects all
Repository: spark
Updated Branches:
refs/heads/master f7b55dbfc -> 7a9dcbc91
[SPARK-10441] [SQL] Save data correctly to json.
https://issues.apache.org/jira/browse/SPARK-10441
Author: Yin Huai
Closes #8597 from yhuai/timestampJson.
Project:
Repository: spark
Updated Branches:
refs/heads/master 52b24a602 -> d637a666d
[SPARK-10327] [SQL] Cache Table is not working while subquery has alias in its
project list
```scala
import org.apache.spark.sql.hive.execution.HiveTableScan
sql("select key, value, key + 1 from
Repository: spark
Updated Branches:
refs/heads/master d637a666d -> 2143d592c
[HOTFIX] Fix build break caused by #8494
Author: Michael Armbrust <mich...@databricks.com>
Closes #8659 from marmbrus/testBuildBreak.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: h
Repository: spark
Updated Branches:
refs/heads/master c3b881a7d -> 56c4c172e
[SPARK-10034] [SQL] add regression test for Sort on Aggregate
Before #8371, there was a bug for `Sort` on `Aggregate` that we can't use
aggregate expressions named `_aggOrdering` and can't use more than one ordering
Repository: spark
Updated Branches:
refs/heads/master 56c4c172e -> fc4830779
[SPARK-10389] [SQL] support order by non-attribute grouping expression on
Aggregate
For example, we can write `SELECT MAX(value) FROM src GROUP BY key + 1 ORDER BY
key + 1` in PostgreSQL, and we should support this
Repository: spark
Updated Branches:
refs/heads/master 5369be806 - 24ffa85c0
[SPARK-10289] [SQL] A direct write API for testing Parquet
This PR introduces a direct write API for testing Parquet. It's a DSL flavored
version of the [`writeDirect` method] [1] comes with parquet-avro testing
401 - 500 of 1624 matches
Mail list logo