Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22647#discussion_r223182831
--- Diff:
external/spark-ganglia-lgpl/src/main/scala/org/apache/spark/metrics/sink/GangliaSink.scala
---
@@ -64,11 +64,12 @@ class GangliaSink(val
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22647#discussion_r223182808
--- Diff:
external/spark-ganglia-lgpl/src/main/scala/org/apache/spark/metrics/sink/GangliaSink.scala
---
@@ -64,11 +64,12 @@ class GangliaSink(val
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223182600
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -452,46 +452,68 @@ case class RangeExec(range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223182488
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala
---
@@ -345,6 +345,27 @@ trait CodegenSupport extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223182235
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SortExec.scala ---
@@ -132,6 +132,13 @@ case class SortExec(
// a stop check
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223049448
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -452,46 +452,68 @@ case class RangeExec(range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22603#discussion_r223048108
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala
---
@@ -257,3 +277,19 @@ class
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22603
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22603#discussion_r223048148
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala
---
@@ -257,3 +277,19 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r223039662
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CsvExpressionsSuite.scala
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r223039004
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala
---
@@ -0,0 +1,116 @@
+/*
+ * Licensed
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22047
is it possible to rewrite these 3 new functions with existing expression?
e.g.
`every(col)` -> `count(if (col) null else 1) == 0`
`any(col)` -> `count(if (col) 1 else nul
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223027872
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -452,46 +452,68 @@ case class RangeExec(range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223027028
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
---
@@ -518,56 +521,81 @@ class SQLMetricsSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223025094
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ---
@@ -66,27 +75,22 @@ trait BaseLimitExec extends UnaryExecNode
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223024194
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ---
@@ -46,6 +46,15 @@ case class CollectLimitExec(limit: Int, child
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223024598
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ---
@@ -46,6 +46,15 @@ case class CollectLimitExec(limit: Int, child
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223024087
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -452,46 +452,68 @@ case class RangeExec(range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223023197
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
---
@@ -705,13 +712,16 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r223022650
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala
---
@@ -345,6 +345,27 @@ trait CodegenSupport extends
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22572
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22237#discussion_r222924928
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ParseMode.scala
---
@@ -51,6 +56,8 @@ object ParseMode extends Logging
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22375#discussion_r222920691
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala
---
@@ -113,7 +113,7 @@ class
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22637
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r222894891
--- Diff: python/pyspark/sql/functions.py ---
@@ -2664,6 +2664,28 @@ def sequence(start, stop, step=None):
_to_java_column(start
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r222895375
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r222895573
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22637
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22399
Jenkins was not triggered in this PR, I'm reverting it. Please re-open it
and make sure jenkins pass, thanks
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r222733663
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
---
@@ -705,13 +709,16 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r222733405
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
---
@@ -159,6 +159,10 @@ case class HashAggregateExec
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r222727599
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
---
@@ -518,56 +521,81 @@ class SQLMetricsSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r222728236
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ---
@@ -66,27 +75,22 @@ trait BaseLimitExec extends UnaryExecNode
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22630#discussion_r222726409
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -452,46 +452,68 @@ case class RangeExec(range
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22630
cc @viirya @mgaido91 @kiszk
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/22630
[SPARK-25497][SQL] Limit operation within whole stage codegen should not
consume all the inputs
## What changes were proposed in this pull request?
This PR is inspired by https
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22237#discussion_r222653808
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ParseMode.scala
---
@@ -51,6 +56,8 @@ object ParseMode extends Logging
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22237#discussion_r222651484
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
---
@@ -550,59 +550,93 @@ case class JsonToStructs
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22237#discussion_r222651368
--- Diff: docs/sql-programming-guide.md ---
@@ -1890,6 +1890,10 @@ working with timestamps in `pandas_udf`s to get the
best performance, see
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22572
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22621
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22621#discussion_r222642107
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
---
@@ -517,4 +517,57 @@ class SQLMetricsSuite extends
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22572
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22621
I simplified this PR to focus on `SparkPlan.getByteArrayRdd` only. Will
submit PR to fix range later.
---
-
To unsubscribe, e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22512#discussion_r222533085
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala
---
@@ -238,7 +262,9 @@ object DecimalLiteral
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22512#discussion_r222532952
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InterpretedMutableProjection.scala
---
@@ -53,6 +55,47 @@ class
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22572
Can we update the PR to use `description.uuid` first?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20433
do we have an umbrella JIRA to track all the breaking changes that make
Spark ANSI SQL compatible?
---
-
To unsubscribe, e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22621#discussion_r222520483
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -453,45 +453,89 @@ case class RangeExec(range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22621#discussion_r222510592
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -453,45 +453,89 @@ case class RangeExec(range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22621#discussion_r222510006
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
---
@@ -517,4 +517,93 @@ class SQLMetricsSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22621#discussion_r222342196
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -453,45 +453,89 @@ case class RangeExec(range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22621#discussion_r222341169
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -453,45 +453,89 @@ case class RangeExec(range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22614#discussion_r222340141
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ---
@@ -746,34 +746,20 @@ private[client] class Shim_v0_13 extends
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22621
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22621#discussion_r222319861
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
---
@@ -453,45 +453,89 @@ case class RangeExec(range
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22621
cc @kiszk @viirya @mgaido91
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/22621
[SPARK-25602][SQL] range metrics can be wrong if the result rows are not
fully consumed
## What changes were proposed in this pull request?
This is a long-standing bug. When `Range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10989#discussion_r222144135
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala
---
@@ -117,6 +120,87 @@ case class BroadcastHashJoin
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22602
LGTM, pending jenkins
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10989#discussion_r221961271
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala
---
@@ -117,6 +120,87 @@ case class BroadcastHashJoin
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22602#discussion_r221955172
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala
---
@@ -0,0 +1,43
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22602#discussion_r221913801
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala
---
@@ -0,0 +1,43
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22602#discussion_r221913364
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeWriterSuite.scala
---
@@ -0,0 +1,43
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22603
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22607
cc @HyukjinKwon @vanzin @jerryshao @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/22607
[SPARK-24530][followup] run Sphinx with python 3 in docker
## What changes were proposed in this pull request?
SPARK-24530 discovered a problem of generation python doc, and provided
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22602
I think we can create a `UnsafeWriterSuite` to do some low-level checking.
We can leave the end-to-end test if it's too hard to write
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22602#discussion_r221809826
--- Diff:
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java
---
@@ -185,13 +185,13 @@ public void write
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22602
good catch! LGTM, waiting for the UT.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22574#discussion_r221277340
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -488,26 +494,27 @@ private[parquet
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22577
Can you create a jira ticket and put more description? This is a bug fix
and strictly speaking it's nothing about scala 2.12. It's just the scala 2.12.7
compiler helps you to find this bug
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22573
I think the problem is, the current public `Filter` API uses string as the
attribute type, which is hard to represent nested fields.
Ideally we should extend the API, create a new
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22574
LGTM, pending jenkins
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22577
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22572#discussion_r221157274
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala
---
@@ -183,15 +183,16 @@ object FileFormatWriter
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22577
ah so scala 2.12.7 is smarter on finding mistakes. LGTM then. Do you know
how long this bug exists? It will decide how far we should backport
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22574#discussion_r221152126
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -488,26 +494,25 @@ private[parquet
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22574#discussion_r221151966
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -488,26 +494,25 @@ private[parquet
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22577#discussion_r221136204
--- Diff:
core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala
---
@@ -175,7 +175,7 @@ private[v1] class
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22558
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22563
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22010
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22562
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22544
After thinking more, I'm going it to merge it to 2.4 directly, because:
1. `ImplicitTypeCasts` handles array type but not map type, which looks a
bug to me
2. I checked all
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22556
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22569
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22563#discussion_r220830491
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -155,37 +155,35 @@ case class InSubquery(values
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22010
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22563
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22326
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22543
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22544
I'm merging it to master, please send another PR to 2.4, without touching
the `findTightestCommonType`. Thanks
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22010
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22237#discussion_r220793666
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
---
@@ -554,18 +554,30 @@ case class JsonToStructs
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22545
SGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22545
The scala side change is already in 2.3.0. If we are ok with the behavior
inconsistency between python and scala, it's fine to merge it to master only
(and revert #22552 from 2.4 as well
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22563
cc @mgaido91
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22563#discussion_r220783617
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -155,37 +155,35 @@ case class InSubquery(values
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/22563
[SPARK-24341][SQL][followup] remove duplicated error checking
## What changes were proposed in this pull request?
There are 2 places we check for problematic `InSubquery`: the rule
1101 - 1200 of 17635 matches
Mail list logo
