Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20611
Any behavior change?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21893
@krisgeus This breaks one of our major assumptions. To provide a proper
support, it requires a lot of code changes. Thus, we do not plan to support it
in the short term.
For your use
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21807
@mauropalsgraaf Do you have a JIRA ID? I can assign the ticket to you.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21807
LGTM
Thanks! Merged to master/2.3
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21883
@wangyum How about our Hive tables? Add a test case for Hive tables? They
are created by CREATE TABLE instead of CREATE TABLE USING. You can use csv/json
format in the test cases
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
If the revert is very risky, we normally change the default from true to
false when the bug is identified in the RC stage
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21883#discussion_r206414524
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala
---
@@ -284,6 +284,11 @@ object PartitioningUtils
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21807
@mauropalsgraaf Could you fix the PR title?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
Normally, we change the default to false or revert the whole PR if the bugs
are found during the RC (release candidate) stage
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
We are still targeting this to 2.4, but we need to fix all the identified
bugs before merging it.
---
-
To unsubscribe, e
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21926
LGTM Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21926#discussion_r206406546
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -529,6 +529,10 @@ class Analyzer
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17648
@dilipbiswal Could you take this over?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17185
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21911
@jzhuge Do you know any other systems that have similar things?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21896
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21922
@rxin This is not to make the actual changes. : ) Just to show the
community how to make it configurable by SQLConf
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21922
I am closing this PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile closed the pull request at:
https://github.com/apache/spark/pull/21922
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21922
cc @dilipbiswal @dongjoon-hyun
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/21922
[WIP] Add an ANSI SQL parser mode
## What changes were proposed in this pull request?
This WIP is just to show how to add a SQLConf in the parser module.
## How was this patch
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21896
streaming tests are very flacky.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21896
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21893
I do not think we will provide this support. This will complicate the whole
design and implementation, and the benefit is limited.
I left two comments about your code. Maybe you can
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21893#discussion_r206235932
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ---
@@ -182,7 +182,9 @@ case class RelationConversions
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21893#discussion_r206234291
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
---
@@ -437,6 +437,8 @@ private[hive] class HiveClientImpl
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21893
The current design of file source reading and writing does not support
this.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21913
cc @zsxwing
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21881
Test case?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21893
A single partitioned table with different file formats, like Parquet, ORC
and csv?
---
-
To unsubscribe, e-mail: reviews
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21898
cc @mengxr @cloud-fan @rxin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21488
https://github.com/apache/kafka/releases KAFKA 2.0.0 release is publicly
available. We can finish it before the code freeze
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21834
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21761
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21834
My only concern is the time zone issues. However, it will not affect the
correctness since the logical partitioning will cover the whole range anyway.
The worse case is the partitioning might
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21834
LGTM pending tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21834
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21892
sounds good to me.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21892
@HyukjinKwon I would suggest to skip this upgrade and then we can get 3.5
times perf improvement for `count
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21892
@HyukjinKwon We need to rerun the perf tests after
https://github.com/apache/spark/pull/21909 is merged.
We are also unable to accept the perf regression larger than `5%`. Based
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21896
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21761
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21892
@MaxGekk @HyukjinKwon We are unable to merge this PR since the performance
regression is very obvious.
---
-
To unsubscribe
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19788
ping @yucai @carsonwang
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21896
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21886
@dilipbiswal The merged PR does not pick up your last commit.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21886
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21886
@dilipbiswal Please address the style issues in your other PRs.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile closed the pull request at:
https://github.com/apache/spark/pull/20331
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21761
LGTM pending Jenkins
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21761
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21886
LGTM pending Jenkins
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20184
cc @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21911
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21886#discussion_r205980691
--- Diff:
sql/core/src/test/resources/sql-tests/results/intersect-all.sql.out ---
@@ -0,0 +1,236 @@
+-- Automatically generated by SQLQueryTestSuite
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21886
The code looks good to me. Let us improve the test cases.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21886#discussion_r205933857
--- Diff:
sql/core/src/test/resources/sql-tests/results/intersect-all.sql.out ---
@@ -0,0 +1,212 @@
+-- Automatically generated by SQLQueryTestSuite
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21886#discussion_r205933861
--- Diff:
sql/core/src/test/resources/sql-tests/results/intersect-all.sql.out ---
@@ -0,0 +1,212 @@
+-- Automatically generated by SQLQueryTestSuite
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21886#discussion_r205933866
--- Diff:
sql/core/src/test/resources/sql-tests/results/intersect-all.sql.out ---
@@ -0,0 +1,212 @@
+-- Automatically generated by SQLQueryTestSuite
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21886#discussion_r205933837
--- Diff:
sql/core/src/test/resources/sql-tests/results/intersect-all.sql.out ---
@@ -0,0 +1,212 @@
+-- Automatically generated by SQLQueryTestSuite
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21886#discussion_r205933692
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -1934,6 +1934,23 @@ class Dataset[T] private[sql](
Intersect
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21886#discussion_r205933541
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -1934,6 +1934,23 @@ class Dataset[T] private[sql](
Intersect
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21906
cc @gengliangwang
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/21906
[SPARK-24924][SQL][FOLLOW-UP] Add mapping for built-in Avro data source
## What changes were proposed in this pull request?
Add one more test case for `com.databricks.spark.avro
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21318
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21897
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21896
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21857
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21886
cc @dilipbiswal Could you resolve the conflicts? I will start the review
after the rebase.
---
-
To unsubscribe, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
I mean https://github.com/apache/spark/pull/21320#issuecomment-408271470
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21879
Thanks! Merged to master/2.3/2.2
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21879
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21889
Just FYI, we are unable to merge it if it has a correctness bug.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21875
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21879
Let us backport it to 2.2 and 2.3.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21857#discussion_r205663395
--- Diff: sql/core/src/test/resources/sql-tests/inputs/except-all.sql ---
@@ -0,0 +1,146 @@
+CREATE TEMPORARY VIEW tab1 AS SELECT * FROM VALUES
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21822
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21873
Thanks! Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21881
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21867
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21758
Thanks! Merged to master.
@jiangxb1987 Please submit the other PRs that are blocked by this ASAP
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21857#discussion_r205554196
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -1947,6 +1947,21 @@ class Dataset[T] private[sql](
Except
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21857#discussion_r205553527
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1400,13 +1401,71 @@ object
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21857#discussion_r205552895
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1400,13 +1401,71 @@ object
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21876
Thanks! Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21608
Also make it configurable. At least, users can turn this feature off.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21608#discussion_r205545427
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala
---
@@ -47,15 +48,23 @@ object CommandUtils extends Logging
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21879
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21608#discussion_r205542099
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala
---
@@ -47,15 +48,23 @@ object CommandUtils extends Logging
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21608#discussion_r205539125
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala
---
@@ -47,15 +48,23 @@ object CommandUtils extends Logging
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21403
`case class In(value: Expression, list: Seq[Expression])` is an internal
class. We do not expect user applications rely on the internal data structure.
If they did, they should understand
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21320
@mallman Really appreciate for your selfless contributions to Apache Spark!
In Spark 2.1, you already made a critical contribution for scalable partition
handling. In the upcoming release
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21857#discussion_r205325587
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
---
@@ -52,7 +52,7 @@ trait CheckAnalysis extends
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21821
Thanks! Merged to master/2.3
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21818
LGTM
Thanks! Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21758
LGTM
@squito Has @jiangxb1987 address your comments?
We want to merge this PR ASAP and a few to-be-submitted PRs depend on this
PR. This feature is targeting to Spark 2.4
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21403#discussion_r205214861
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -2320,6 +2320,27 @@ class DataFrameSuite extends QueryTest
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21403#discussion_r205214601
--- Diff:
sql/core/src/test/resources/sql-tests/results/subquery/negative-cases/subq-input-typecheck.sql.out
---
@@ -113,15 +105,7 @@ WHERE
struct
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21403#discussion_r205213736
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -247,6 +249,20 @@ class Analyzer
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21803
LGTM
Thanks! Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
801 - 900 of 14035 matches
Mail list logo
