Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22775#discussion_r227237973
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
---
@@ -770,8 +776,17 @@ case class SchemaOfJson
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22797
Hi @dilipbiswal sorry for the back and forth, can you try one more
approach? Basically we want to evaluate how the simplest logical rewrite looks
like.
We can create unevaluatable EVERY
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22800
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22800
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22790
cc @mengxr @WeichenXu123 how serious is it? shall we treat it as a blocker?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22799
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22788#discussion_r227203199
--- Diff:
sql/core/src/test/resources/sql-tests/results/columnresolution-negative.sql.out
---
@@ -81,7 +81,7 @@ SELECT t1.i1 FROM t1, mydb1.t1
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22788#discussion_r227202767
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala
---
@@ -99,7 +99,7 @@ case class
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22512
LGTM, we also need a unit test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22512#discussion_r227202171
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala ---
@@ -140,6 +141,14 @@ class SQLQueryTestSuite extends QueryTest
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22512#discussion_r227200656
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InterpretedMutableProjection.scala
---
@@ -49,10 +51,54 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22512#discussion_r227200458
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InterpretedMutableProjection.scala
---
@@ -49,10 +51,54 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22745#discussion_r227037566
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala
---
@@ -278,24 +278,20 @@ object JavaTypeInference
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19788
BTW, let's add a config for this feature. We may enable adaptive execution
by default in the future, and we should still allow users to run spark with
legacy shuffle service. We should also throw
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22786
also cc @WeichenXu123
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20820
This is good to have, but we should follow `BroadcastHashJoinExec` and make
the implementation more structured. e.g. `codegenInner`, `codegenOuter`, etc
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22788#discussion_r226988117
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2702,7 +2702,7 @@ class SQLQuerySuite extends QueryTest
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22788#discussion_r226987977
--- Diff:
sql/core/src/test/resources/sql-tests/results/columnresolution-negative.sql.out
---
@@ -81,7 +81,7 @@ SELECT t1.i1 FROM t1, mydb1.t1
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17520
is it time to revisit it?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22047#discussion_r226985475
--- Diff: python/pyspark/sql/functions.py ---
@@ -403,6 +403,28 @@ def countDistinct(col, *cols):
return Column(jc)
+def every
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21860
LGTM except the naming
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21860#discussion_r226981919
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
---
@@ -744,6 +744,7 @@ case class HashAggregateExec
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22745#discussion_r226981527
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala
---
@@ -278,24 +278,20 @@ object JavaTypeInference
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22785#discussion_r226978705
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala
---
@@ -187,7 +187,7 @@ object RowEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22785#discussion_r226978368
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala
---
@@ -171,7 +171,7 @@ object RowEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22785#discussion_r226978811
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala
---
@@ -273,6 +273,16 @@ class RowEncoderSuite
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22785#discussion_r226977895
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala
---
@@ -171,7 +171,7 @@ object RowEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22785#discussion_r226977568
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala
---
@@ -273,6 +273,16 @@ class RowEncoderSuite
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21402
shall we close it since #22173 is merged?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19788
Hi @yucai , good points on the performance concerns. Let's go with the
previous approach:
https://github.com/apache/spark/pull/19788#issuecomment-366887404
sorry for the back and forth
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22575
do we have a full story about stream sql? is the `STREAM` keyword the only
difference between stream sql and normal sql?
also cc @tdas @zsxwing
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22575
do we have a full story about stream sql? is the `STREAM` keyword the only
difference between stream sql and normal sql? how could users define watermark
with SQL?
also cc @tdas @zsxwing
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22575
do we have a full story about stream sql? is the `STREAM` keyword the only
difference between stream sql and normal sql? how could users define watermark
with SQL?
also cc @tdas @zsxwing
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22575
do we have a full story about stream sql? is the `STREAM` keyword the only
difference between stream sql and normal sql? how could users define watermark
with SQL?
also cc @tdas @zsxwing
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22575
do we have a full story about stream sql? is the `STREAM` keyword the only
difference between stream sql and normal sql? how could users define watermark
with SQL?
also cc @tdas @zsxwing
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22575
Do we have a full story about streaming SQL? is the `STREAM` keyword the
only difference between stream sql and normal sql?
also cc @tdas @zsxwing
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22756
reverted from master. Let's move the discussion to
https://github.com/apache/spark/pull/22764
---
-
To unsubscribe, e-mail
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22763
ah i see. @mgaido91 can you resubmit it and update the description? The
method is not deprecated now.
---
-
To unsubscribe, e
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22501
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22547
Let's move the high-level discussion to
https://docs.google.com/document/d/1uUmKCpWLdh9vHxP7AWJ9EgbwB_U6T3EJYNjhISGmiQg/edit?usp=sharing
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22750#discussion_r226819049
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
---
@@ -168,10 +168,11 @@ case class FileSourceScanExec
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22756
shall we revert it from master as well? At least we need to update the
message `This method is deprecated and will be removed in 3.0.0
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22781#discussion_r226816630
--- Diff: docs/building-spark.md ---
@@ -12,7 +12,7 @@ redirect_from: "building-with-maven.html"
## Apache Maven
The Maven-b
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22501
seems jenkins is broken, cc @shaneknapp
```
Command "/tmp/tmp.JfFHaoRFPU/3.5/bin/python -c "import setuptools,
tokenize;__file__='/home/jenkins/workspace/SparkPullRequestBuil
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22750
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22547#discussion_r226812577
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/Format.java ---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22750#discussion_r226812447
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala
---
@@ -164,12 +162,11 @@ private[sql] trait ColumnarBatchScan
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22501
thank you guys for refreshing the benchmarks and results! It's very helpful.
If possible, can we post the perf regressions we found in the umbrella
JIRA? Then people can see if the perf
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22763
This has been reverted from master/2.4
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22764
Since this PR is a little more complicated than we expect, we decided to
not have it in 2.4.0. I'm not sure if we can treat it as a special case and put
it in 2.4.1, cc @mengxr
Anyway
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22466#discussion_r226655438
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -207,6 +207,14 @@ class SessionCatalog
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22764#discussion_r226652377
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/clustering/BisectingKMeansModel.scala
---
@@ -225,13 +227,14 @@ object BisectingKMeansModel
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22766
thanks, merging to master/2.4/2.3!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22743
> Datasource table will not cache in tableRelationCache.
I don't think so. Spark caches data source table in `FindDataSourceTa
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22666#discussion_r226641023
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3886,6 +3886,31 @@ object functions {
withExpr(new CsvToStructs
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22666#discussion_r226640860
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CsvExpressionsSuite.scala
---
@@ -155,4 +155,15 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22666#discussion_r226640362
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala
---
@@ -60,7 +63,7 @@ case class CsvToStructs
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22732
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22750
`DataSourceScanExec` does not have `needsUnsafeRowConversion`
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22743
why it's only a problem for hive tables?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22732#discussion_r226520350
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -932,6 +935,23 @@ trait ScalaReflection
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226519284
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -43,10 +44,11 @@ import
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22743
can you explain more about how this happens?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22732#discussion_r226517584
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala
---
@@ -39,29 +42,29 @@ import
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22501#discussion_r226516354
--- Diff: sql/core/benchmarks/WideSchemaBenchmark-results.txt ---
@@ -1,117 +1,145 @@
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_92-b14 on Mac OS X
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22763
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22764#discussion_r226512051
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/clustering/BisectingKMeans.scala ---
@@ -310,4 +317,6 @@ class BisectingKMeansSummary private
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22750
which description is inaccurate?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22766#discussion_r226511589
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
---
@@ -340,39 +340,39 @@ private[hive] case class HiveUDAFFunction
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22766#discussion_r226511506
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
---
@@ -340,39 +340,39 @@ private[hive] case class HiveUDAFFunction
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22766#discussion_r226511479
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
---
@@ -340,39 +340,39 @@ private[hive] case class HiveUDAFFunction
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22764#discussion_r226384584
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/clustering/BisectingKMeans.scala ---
@@ -310,4 +317,6 @@ class BisectingKMeansSummary private
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22766#discussion_r226375347
--- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
---
@@ -339,40 +339,38 @@ private[hive] case class HiveUDAFFunction
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22766
OK to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22764#discussion_r226372701
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/clustering/BisectingKMeans.scala ---
@@ -310,4 +317,6 @@ class BisectingKMeansSummary private
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22547
A major part of this PR is to update existing streaming sources, which is
just moving code around. There are 3 things we need to pay attention to during
review:
1. the naming
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22547#discussion_r226363445
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -381,7 +390,7 @@ class StreamSuite extends StreamTest
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22547#discussion_r226363020
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -154,21 +159,25 @@ class StreamSuite extends StreamTest
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22547#discussion_r226361309
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamProviderSuite.scala
---
@@ -319,29 +307,18 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22547#discussion_r226359031
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/RateStreamMicroBatchInputStream.scala
---
@@ -60,6 +59,14 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22547#discussion_r226355931
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala
---
@@ -90,6 +140,8 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22547#discussion_r226338580
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/SupportsBatchRead.java
---
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22764
does the example need to be updated with this new API?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22749
I like this idea! waiting for tests pass
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226301402
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -212,21 +183,88 @@ object ExpressionEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226301139
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -212,21 +183,88 @@ object ExpressionEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226299441
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -212,21 +183,88 @@ object ExpressionEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226298803
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -212,21 +183,88 @@ object ExpressionEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226296369
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -103,75 +88,61 @@ object ExpressionEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226295859
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -103,75 +88,61 @@ object ExpressionEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226294255
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -103,75 +88,61 @@ object ExpressionEncoder
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22749#discussion_r226294017
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -43,10 +44,11 @@ import
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22721
I think it's reasonable to follow `InsertIntoHiveTable`, but it's better to
provide more details about what changes in `InsertIntoHadoopFsRelationCommand`:
1. what's refreshed? Previously we
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22721#discussion_r226280121
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -189,6 +189,7 @@ case class
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22756
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22547
Hi @rdblue welcome back! I just rebased it so it's ready for review :)
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22721#discussion_r226208576
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -189,6 +189,7 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22758#discussion_r226198591
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -193,6 +193,16 @@ private[hive] class HiveMetastoreCatalog
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22732#discussion_r226156536
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala ---
@@ -393,4 +393,30 @@ class UDFSuite extends QueryTest with SharedSQLContext
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22732#discussion_r226156400
--- Diff: docs/sql-programming-guide.md ---
@@ -1978,6 +1978,7 @@ working with timestamps in `pandas_udf`s to get the
best performance, see
801 - 900 of 17635 matches
Mail list logo
