Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17433#discussion_r108112572
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala ---
@@ -179,88 +132,295 @@ private[sql] class SessionState
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17432#discussion_r108074254
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -597,12 +597,14 @@ object CollapseRepartition
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17432
The project will get eliminated by the optimizer. From the JIRA ticket:
```scala
import org.apache.spark.sql.expressions.Window
val win_spec_max =
Window.partitionBy("x"
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17432
Thanks for the review. Merging this to master/2.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17164
@maropu I do think this is useful. However we really need to refactor the
planner, if we want to get the most value from this.
---
If your project is set up for it, you can reply to this email
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17432
cc @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17433
cc @kunalkhamar
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17433
cc @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17433
[SPARK-20100][SQL] Refactor SessionState initialization
## What changes were proposed in this pull request?
The current SessionState initialization code path is quite complex. A part
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17432
cc @dongjoon-hyun
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17432
[SPARK-20086][SQL] CollapseWindow should not collapse dependent adjacent
windows
## What changes were proposed in this pull request?
The `CollapseWindow` is currently to aggressive when
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17378
LGTM - merging to master thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17397
@ueshin here you go: https://github.com/apache/spark/pull/17420
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17420
[SPARK-20070][SQL] Fix 2.10 build
## What changes were proposed in this pull request?
Commit
https://github.com/apache/spark/commit/91fa80fe8a2480d64c430bd10f97b3d44c007bcc
broke the build
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17397
cc @liufengdb
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17397
[SPARK-20070][SQL] Redact DataSourceScanExec treeString
## What changes were proposed in this pull request?
The explain output of `DataSourceScanExec` can contain sensitive
information
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17376
Merging to branch-2.0. Thanks! Can you close?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17376
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17350
@wzhfy why wouldn't we change the nullability? This is obviously a bug in
the current master:
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17292
LGTM - merging to master. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17292
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17244#discussion_r106363391
--- Diff: core/src/main/scala/org/apache/spark/TaskContextImpl.scala ---
@@ -52,62 +63,79 @@ private[spark] class TaskContextImpl(
@volatile
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16909
LGTM - merging to master. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17301
Merging this one. @rxin could you close?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16909
@tejasapatil there is a small merge conflict. Can you update this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17244
Thanks for the reviews! Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17289
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17292
@bogdanrdc can you add the ticket to the title?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17294
@nsyca that makes sense. Lets not backport for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17294
@nsyca should we backport this to 2.1/2.0?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17294
Merging to master. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17294
LGTM - pending jenkins.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17278
Merging to master. Thanks for the review!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17264
LGTM merging to master. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17289
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17288
Ok merging this to master. Thanks for the approval.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17289
cc @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17289
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17245
Merging to master. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17285#discussion_r105878506
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SimpleCatalystConf.scala
---
@@ -0,0 +1,48 @@
+/*
+ * Licensed
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17285#discussion_r105877761
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -251,7 +247,7 @@ object SQLConf
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17285#discussion_r105874978
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SimpleCatalystConf.scala
---
@@ -0,0 +1,48 @@
+/*
+ * Licensed
GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17288
[SPARK-18874][SQL] Fix 2.10 build after moving the subquery rules to
optimization
## What changes were proposed in this pull request?
Commit
https://github.com/apache/spark/commit
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17278#discussion_r105862033
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -140,7 +140,8 @@ abstract class Optimizer
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16954
LGTM. Merging to master. Thanks for the hard work, and your patience!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17244
Ok, had a small discussion offline. It seems weird that we have different
calling policies for failure and completion listeners. I am going to change the
invocation of completion listeners
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17244#discussion_r105788774
--- Diff: core/src/main/scala/org/apache/spark/TaskContext.scala ---
@@ -126,14 +134,14 @@ abstract class TaskContext extends Serializable
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105780981
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala
---
@@ -40,19 +43,189 @@ abstract class PlanExpression[T
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105780845
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala
---
@@ -40,19 +43,189 @@ abstract class PlanExpression[T
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17278#discussion_r105773239
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -140,7 +140,8 @@ abstract class Optimizer
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17278#discussion_r105770937
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -140,7 +140,8 @@ abstract class Optimizer
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105766906
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -365,17 +368,73 @@ object TypeCoercion
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17278#discussion_r105764212
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -140,7 +140,8 @@ abstract class Optimizer
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16954
@dilipbiswal can you update. I left a few minor comments, but it is good to
go anyway (pending the update).
---
If your project is set up for it, you can reply to this email and have your
reply
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105724836
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala
---
@@ -40,19 +43,189 @@ abstract class PlanExpression[T
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105723410
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1204,80 +1250,44 @@ class Analyzer
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105719841
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala
---
@@ -40,19 +43,189 @@ abstract class PlanExpression[T
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105718880
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala
---
@@ -40,19 +43,189 @@ abstract class PlanExpression[T
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105717565
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala
---
@@ -40,19 +43,189 @@ abstract class PlanExpression[T
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105717078
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -123,19 +123,36 @@ case class Not(child
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105716983
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -123,19 +123,36 @@ case class Not(child
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105715265
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -365,17 +368,73 @@ object TypeCoercion
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105714984
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -365,17 +385,66 @@ object TypeCoercion
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105716052
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -365,17 +368,73 @@ object TypeCoercion
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105714546
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -365,17 +368,73 @@ object TypeCoercion
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105712270
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -109,6 +109,26 @@ object TypeCoercion
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/16954#discussion_r105711886
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -109,6 +109,26 @@ object TypeCoercion
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17241#discussion_r105709959
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -555,6 +555,17 @@ object SQLConf {
.booleanConf
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17241#discussion_r105709767
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -555,6 +555,17 @@ object SQLConf {
.booleanConf
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17241#discussion_r105709364
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -595,6 +594,11 @@ class Analyzer(
case
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17241#discussion_r105709239
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -595,6 +594,11 @@ class Analyzer(
case
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17278
cc @wzhfy
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17278
[SPARK-19933][SQL] Do not change output of a subquery
## What changes were proposed in this pull request?
The `RemoveRedundantAlias` rule can change the output attributes (the
expression
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17278
cc @cloud-fan @ueshin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17244#discussion_r105559913
--- Diff: core/src/main/scala/org/apache/spark/TaskContextImpl.scala ---
@@ -57,57 +68,75 @@ private[spark] class TaskContextImpl(
// Whether
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17244#discussion_r105559906
--- Diff: core/src/main/scala/org/apache/spark/TaskContextImpl.scala ---
@@ -57,57 +68,75 @@ private[spark] class TaskContextImpl(
// Whether
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17244#discussion_r105559898
--- Diff: core/src/main/scala/org/apache/spark/TaskContextImpl.scala ---
@@ -57,57 +68,75 @@ private[spark] class TaskContextImpl(
// Whether
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17244#discussion_r105559900
--- Diff: core/src/main/scala/org/apache/spark/TaskContextImpl.scala ---
@@ -57,57 +68,75 @@ private[spark] class TaskContextImpl(
// Whether
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17264
@maropu this looks like a nice improvement. It is an idea to check all uses
of `wrap`, and see if we can replace them by `wrapperFor`?
---
If your project is set up for it, you can reply
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17244#discussion_r105491214
--- Diff: core/src/main/scala/org/apache/spark/TaskContextImpl.scala ---
@@ -57,57 +68,75 @@ private[spark] class TaskContextImpl(
// Whether
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17241
Yeah you have a point there.
cc @rxin wdyt?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17122
LGTM merging to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17245
[SPARK-19850][SQL] Allow the use of aliases in SQL function calls
## What changes were proposed in this pull request?
We currently cannot use aliases in SQL function calls
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17244
cc @rxin @sameeragarwal @zsxwing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17244
[SPARK-19889][SQL] Make TaskContext callbacks thread safe
## What changes were proposed in this pull request?
It is sometimes useful to use multiple threads in a task to parallelize
tasks
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17236#discussion_r105383505
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -1703,4 +1703,17 @@ class DataFrameSuite extends QueryTest
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17241
@jiangxb1987 I am not entirely sure about the usefulness of this (I thought
it might be good to enforce some restriction when we wrote the design doc). I
don't think it matters much what we do
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17240#discussion_r105371649
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala
---
@@ -122,46 +123,59 @@ case class
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17236
LGTM pending jenkins
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17240#discussion_r105368556
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala
---
@@ -122,46 +123,59 @@ case class
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17240#discussion_r105368297
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala
---
@@ -87,8 +88,8 @@ case class
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17240#discussion_r105368212
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala
---
@@ -75,8 +75,9 @@ case class
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17240#discussion_r105368096
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala
---
@@ -36,27 +36,27 @@ case class
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17236
@cloud-fan can we also check `Distinct`? We rewrite this into an
`Aggregate` in the optimizer, which has the same problem with `MapType`.
---
If your project is set up for it, you can reply
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17236#discussion_r105352149
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
---
@@ -319,7 +322,7 @@ trait CheckAnalysis extends
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17236#discussion_r105351724
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
---
@@ -319,7 +322,7 @@ trait CheckAnalysis extends
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17214#discussion_r105047064
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ---
@@ -46,9 +46,13 @@ class QueryExecution(val sparkSession
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17138
Spark can process bushy joins. This implementation of `DPSize` should be
able to create bushy joins (or I have missed something).
---
If your project is set up for it, you can reply
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/17214#discussion_r105037075
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ---
@@ -46,9 +46,13 @@ class QueryExecution(val sparkSession
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/15363#discussion_r105030111
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -20,19 +20,340 @@ package
901 - 1000 of 4165 matches
Mail list logo
