Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22467#discussion_r218844704
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSourceSuite.scala
---
@@ -0,0 +1,220 @@
+/*
+ * Licensed to the Apache
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22467#discussion_r218835682
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/ParquetPartitioningTest.scala
---
@@ -0,0 +1,253 @@
+/*
+ * Licensed to the
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22467#discussion_r218835749
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/ParquetPartitioningTest.scala
---
@@ -0,0 +1,253 @@
+/*
+ * Licensed to the
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r219675105
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -995,7 +995,8 @@ class Dataset[T] private[sql](
// After the
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22326
@cloud-fan Great thanks for your offline guidance, as our discussion, I
reimplement this by adding a new rule `HandlePythonUDFInJoinCondition` in
Analyzer, revert all changes in `Optimizer
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22326
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22524
```
Is jenkins down now?
```
Does this means you got a `Reason: Error during SSL Handshake with remote
server` after open the jenkins link
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22524
Got it, same with me :(
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22524
@viirya As @shaneknapp reply in mail-list, you can try
https://hadrian.ist.berkeley.edu/jenkins/. Thanks @shaneknapp
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22326
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22326
```
move this rule to optimizer, as the last batch (but before the
UpdateAttributeReferences batch). Since we apply this rule after filter
pushdown, we can simply pull out any python udf
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220102842
--- Diff: python/pyspark/sql/tests.py ---
@@ -547,6 +547,92 @@ def test_udf_in_filter_on_top_of_join(self):
df = left.crossJoin(right
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220109689
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1234,6 +1237,59 @@ object
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220111919
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1234,6 +1237,59 @@ object
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220128279
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1234,6 +1237,59 @@ object
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220236374
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,60 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220267389
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1304,10 +1307,27 @@ object
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220418201
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1304,10 +1307,27 @@ object
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220432468
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1304,10 +1307,27 @@ object
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220432728
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,56 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220433190
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,56 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220433896
--- Diff: python/pyspark/sql/tests.py ---
@@ -552,6 +552,92 @@ def test_udf_in_filter_on_top_of_join(self):
df = left.crossJoin(right
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220433940
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,56 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220433995
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -165,6 +165,8 @@ abstract class Optimizer
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220433916
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,56 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220433980
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,56 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220437721
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1308,6 +1312,16 @@ object
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220478633
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,56 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220479127
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,56 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220522504
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220523238
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220524840
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220524866
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220526661
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220562279
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220567216
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220567301
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220567381
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220567623
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220568332
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,51 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22165
gental ping @jiangxb1987 @kiszk
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220575840
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,53 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220576062
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,53 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220576115
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,53 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220576188
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,53 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220576239
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -152,3 +153,53 @@ object EliminateOuterJoin extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220628624
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExecSuite.scala
---
@@ -100,6 +104,28 @@ class
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220777383
--- Diff: python/pyspark/sql/tests.py ---
@@ -552,6 +552,96 @@ def test_udf_in_filter_on_top_of_join(self):
df = left.crossJoin(right
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22326#discussion_r220777535
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExecSuite.scala
---
@@ -100,6 +105,29 @@ class
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22165#discussion_r220983212
--- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala ---
@@ -141,7 +145,7 @@ private[spark] class BarrierCoordinator
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22165#discussion_r220984340
--- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala ---
@@ -187,6 +191,12 @@ private[spark] class BarrierCoordinator
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22165#discussion_r220984492
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/BarrierCoordinatorSuite.scala ---
@@ -0,0 +1,166 @@
+/*
+ * Licensed to the Apache
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22165#discussion_r220986263
--- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala ---
@@ -187,6 +191,12 @@ private[spark] class BarrierCoordinator
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22326
Thanks everyone for your review and advise.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/22583
[SPARK-10816][SS] SessionWindow support for Structure Streaming
## What changes were proposed in this pull request?
As the discussion in
[SPARK-10816](https://issues.apache.org/jira
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22590#discussion_r223414086
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -194,6 +195,22 @@ class CSVSuite extends
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/22583
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19773
@jaceklaskowski Thanks for your review and comments, I rebased the branch
and addressed all comments, this patch is now ready for next reviewing
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19773#discussion_r151990044
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -318,16 +318,28 @@ case class AlterTableChangeColumnCommand
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19773#discussion_r152753785
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -318,16 +318,26 @@ case class AlterTableChangeColumnCommand
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19773#discussion_r152957444
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -318,16 +318,26 @@ case class AlterTableChangeColumnCommand
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19773
gental ping @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/17702
gental ping @zsxwing
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/18975#discussion_r155185768
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala
---
@@ -0,0 +1,82
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/18975#discussion_r155518105
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala
---
@@ -0,0 +1,82
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/19941
[SPARK-22753][SQL] Get rid of dataSource.writeAndRead
## What changes were proposed in this pull request?
As the discussion in https://github.com/apache/spark/pull/16481 and
https
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19941
Maybe something wrong with the Jenkins env? I found the build from 84707 to
84733 failed in same reason:
```
[error] running
/home/jenkins/workspace/SparkPullRequestBuilder@2/R/run
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/16135#discussion_r156254083
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionedTablePerfStatsSuite.scala
---
@@ -352,4 +353,34 @@ class
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/17702#discussion_r156256317
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -668,4 +672,31 @@ object DataSource extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19941#discussion_r156258956
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -490,20 +489,19 @@ case class DataSource
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/17702#discussion_r156265041
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -668,4 +672,31 @@ object DataSource extends
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19941
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19941
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking closed the pull request at:
https://github.com/apache/spark/pull/18760
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/18760
The porting work of SPARK-18838 is in progress based on Spark 2.1 in our
product, I'll close this now and open another PR if needed. Thanks @cloud-fan
@v
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/17702#discussion_r156658524
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -668,4 +672,31 @@ object DataSource extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19941#discussion_r156659744
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala
---
@@ -67,8 +67,9 @@ case class
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19941#discussion_r156661304
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -490,20 +489,19 @@ case class DataSource
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19941#discussion_r156958793
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala
---
@@ -67,8 +67,9 @@ case class
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19941#discussion_r156960449
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -490,20 +489,19 @@ case class DataSource
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19941#discussion_r157118985
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala
---
@@ -67,8 +67,9 @@ case class
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19773
gental ping @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/17702#discussion_r150484715
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -432,6 +433,32 @@ case class DataSource
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/17702#discussion_r150484788
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -246,6 +246,18 @@ class SparkHadoopUtil extends Logging
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/17702
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/17702
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/17702
@zsxwing Thanks for your comments, ready for review.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/19745
[SPARK-2926][Core][Follow Up] Sort shuffle reader for Spark 2.x
## What changes were proposed in this pull request?
As comment in
[SPARK-2926][https://issues.apache.org/jira/browse
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/19773
Supporting for changing column dataType
## What changes were proposed in this pull request?
Support user to change column dataType in hive table and datasource table,
here also want
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/19287
[SPARK-22074][Core] Task killed by other attempt task should not be
resubmitted
## What changes were proposed in this pull request?
As the detail scenario described in
[SPARK-22074
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19287
`signal 9`
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19287#discussion_r140143293
--- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala ---
@@ -66,6 +66,12 @@ class TaskInfo(
*/
var finishTime: Long
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19287
ping @cloud-fan @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/18760
The hold mode is still valid, I resolved the conflict and add the logic
into `AsyncEventQueue`, it can confirm by the test case added in this
[patch](https://github.com/apache/spark/pull/18760
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19287#discussion_r141513379
--- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala ---
@@ -74,6 +81,10 @@ class TaskInfo(
gettingResultTime = time
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/19287
@jerryshao Thanks for you review.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19287#discussion_r141784747
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -744,6 +744,100 @@ class TaskSetManagerSuite extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19287#discussion_r141784812
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -744,6 +744,100 @@ class TaskSetManagerSuite extends
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/19287#discussion_r141784872
--- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala ---
@@ -66,6 +66,13 @@ class TaskInfo(
*/
var finishTime: Long
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/18760
@vanzin Hi Vanzin, thanks a lot for your comments.
Firstly answer your question about `Why isn't hold mode just calling
queue.put (blocking) instead of queue.offer (non-blocking)?`
401 - 500 of 777 matches
Mail list logo
