[GitHub] [spark] AmplabJenkins commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672543101 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672542345 **[Test build #127350 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127350/testReport)** for PR 29328 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA removed a comment on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672338989 **[Test build #127350 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127350/testReport)** for PR 29328 at commit

[GitHub] [spark] SparkQA commented on pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

SparkQA commented on pull request #29406: URL: https://github.com/apache/spark/pull/29406#issuecomment-672527689 **[Test build #127355 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127355/testReport)** for PR 29406 at commit

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468975271 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +450,88 @@ class

[GitHub] [spark] cchighman commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

cchighman commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672520195 @maropu You will be very pleased to hear that I have discovered the maven task for scalafmt! This is an

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468974739 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +450,88 @@ class

[GitHub] [spark] AmplabJenkins commented on pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

AmplabJenkins commented on pull request #29406: URL: https://github.com/apache/spark/pull/29406#issuecomment-672516543 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

AmplabJenkins removed a comment on pull request #29406: URL: https://github.com/apache/spark/pull/29406#issuecomment-672516543 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672514403 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672514403 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

SparkQA removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672330830 **[Test build #127349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127349/testReport)** for PR 29396 at commit

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468973773 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -242,8 +242,10 @@ class DecommissionWorkerSuite

[GitHub] [spark] SparkQA commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

SparkQA commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672511502 **[Test build #127349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127349/testReport)** for PR 29396 at commit

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r468972753 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/pathFilters.scala ## @@ -0,0 +1,177 @@ +/* + * Licensed to the Apache

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r468970624 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/pathfilters/PathFilterFactory.scala ## @@ -0,0 +1,35 @@ +/* + *

[GitHub] [spark] wangyum commented on a change in pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

wangyum commented on a change in pull request #29406: URL: https://github.com/apache/spark/pull/29406#discussion_r468970602 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala ## @@ -176,8 +176,12 @@ object

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r468969864 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,557 @@ +/* + * Licensed to the

[GitHub] [spark] wangyum commented on a change in pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

wangyum commented on a change in pull request #29406: URL: https://github.com/apache/spark/pull/29406#discussion_r468970011 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/PruneFileSourcePartitionsSuite.scala ## @@ -109,6 +109,62 @@ class

[GitHub] [spark] Ngone51 commented on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

Ngone51 commented on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672482303 > @Ngone51 Too heavy to show the page Files changes!!! How about reducing the number of the generated files for reviews? Probably, a few of the generated file examples are

[GitHub] [spark] gatorsmile commented on a change in pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

gatorsmile commented on a change in pull request #29372: URL: https://github.com/apache/spark/pull/29372#discussion_r468969252 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -54,6 +55,8 @@ class QueryExecution( val

[GitHub] [spark] Ngone51 commented on a change in pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

Ngone51 commented on a change in pull request #29270: URL: https://github.com/apache/spark/pull/29270#discussion_r468969318 ## File path: sql/core/src/test/scala/org/apache/spark/sql/PlanStabilitySuite.scala ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] mridulm commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

mridulm commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672478005 Jenkins test this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins removed a comment on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672476822 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins commented on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672476822 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

SparkQA commented on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672475149 **[Test build #127354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127354/testReport)** for PR 29270 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

Ngone51 commented on a change in pull request #29270: URL: https://github.com/apache/spark/pull/29270#discussion_r468968692 ## File path: sql/core/src/test/scala/org/apache/spark/sql/PlanStabilitySuite.scala ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

agrawaldevesh commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468964710 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -242,8 +242,10 @@ class DecommissionWorkerSuite

[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

yanxiaole commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-672459512 Hi @zhouyejoe , Yes, you are right. I have checked the pool definition, and `checkForLogs()` and `cleanLogs()` won't run at same time, the only possible is the replay

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

agrawaldevesh commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468955452 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +450,88 @@ class

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

agrawaldevesh commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468955452 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +450,88 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

AmplabJenkins removed a comment on pull request #29412: URL: https://github.com/apache/spark/pull/29412#issuecomment-672440366 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

AmplabJenkins commented on pull request #29412: URL: https://github.com/apache/spark/pull/29412#issuecomment-672440366 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

SparkQA commented on pull request #29412: URL: https://github.com/apache/spark/pull/29412#issuecomment-672439150 **[Test build #127353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127353/testReport)** for PR 29412 at commit

[GitHub] [spark] viirya opened a new pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

viirya opened a new pull request #29412: URL: https://github.com/apache/spark/pull/29412 ### What changes were proposed in this pull request? This is a follow-up of SPARK-25557. In this PR, the case-sensitivity stuff was removed from OrcFiltersBase. ### Why

[GitHub] [spark] Karl-WangSK commented on a change in pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

Karl-WangSK commented on a change in pull request #29360: URL: https://github.com/apache/spark/pull/29360#discussion_r468955369 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1823,3 +1824,32 @@ object OptimizeLimitZero

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r468952791 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,557 @@ +/* + * Licensed to the

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r468952791 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,557 @@ +/* + * Licensed to the

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r468952326 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,557 @@ +/* + * Licensed to the

[GitHub] [spark] zhouyejoe commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

zhouyejoe commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-672428047 @yanxiaole I don't think there will be race condition between the checkForLogs() and cleanLogs() thread. These two threads are launched from the same pool, and the thread

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r468943144 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/pathfilters/PathFilterStrategy.scala ## @@ -0,0 +1,33 @@ +/* + *

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

AmplabJenkins removed a comment on pull request #29409: URL: https://github.com/apache/spark/pull/29409#issuecomment-672418660 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

AmplabJenkins commented on pull request #29409: URL: https://github.com/apache/spark/pull/29409#issuecomment-672418660 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

SparkQA removed a comment on pull request #29409: URL: https://github.com/apache/spark/pull/29409#issuecomment-672267613 **[Test build #127347 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127347/testReport)** for PR 29409 at commit

[GitHub] [spark] SparkQA commented on pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

SparkQA commented on pull request #29409: URL: https://github.com/apache/spark/pull/29409#issuecomment-672417423 **[Test build #127347 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127347/testReport)** for PR 29409 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29403: [SPARK-32585][SQL] Support scala enumeration in ScalaReflection

AmplabJenkins commented on pull request #29403: URL: https://github.com/apache/spark/pull/29403#issuecomment-672415040 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29403: [SPARK-32585][SQL] Support scala enumeration in ScalaReflection

AmplabJenkins removed a comment on pull request #29403: URL: https://github.com/apache/spark/pull/29403#issuecomment-672415040 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29403: [SPARK-32585][SQL] Support scala enumeration in ScalaReflection

SparkQA commented on pull request #29403: URL: https://github.com/apache/spark/pull/29403#issuecomment-672414635 **[Test build #127352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127352/testReport)** for PR 29403 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins removed a comment on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672410147 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672410147 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA removed a comment on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672264110 **[Test build #127346 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127346/testReport)** for PR 29328 at commit

[GitHub] [spark] SparkQA commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672409570 **[Test build #127346 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127346/testReport)** for PR 29328 at commit

[GitHub] [spark] venkata91 commented on a change in pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

venkata91 commented on a change in pull request #29411: URL: https://github.com/apache/spark/pull/29411#discussion_r468942157 ## File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ## @@ -1400,11 +1399,10 @@ private[spark] object SparkSubmitUtils {

[GitHub] [spark] AmplabJenkins commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672399512 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins removed a comment on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672399512 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA removed a comment on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672260512 **[Test build #127344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127344/testReport)** for PR 29328 at commit

[GitHub] [spark] SparkQA commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672397379 **[Test build #127344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127344/testReport)** for PR 29328 at commit

[GitHub] [spark] github-actions[bot] closed pull request #26257: [SPARK-29606][SQL] Improve EliminateOuterJoin performance

github-actions[bot] closed pull request #26257: URL: https://github.com/apache/spark/pull/26257 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] github-actions[bot] commented on pull request #28200: [SPARK-31432][DEPLOY] Make SPARK_JARS_DIR configurable

github-actions[bot] commented on pull request #28200: URL: https://github.com/apache/spark/pull/28200#issuecomment-672397026 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29322: [SPARK-32511][SQL] Add dropFields method to Column class

AmplabJenkins removed a comment on pull request #29322: URL: https://github.com/apache/spark/pull/29322#issuecomment-672394105 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29322: [SPARK-32511][SQL] Add dropFields method to Column class

AmplabJenkins commented on pull request #29322: URL: https://github.com/apache/spark/pull/29322#issuecomment-672394105 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] rednaxelafx commented on a change in pull request #29407: [SPARK-32588][CORE][TEST] Fix SizeEstimator initialization in tests

rednaxelafx commented on a change in pull request #29407: URL: https://github.com/apache/spark/pull/29407#discussion_r468936303 ## File path: core/src/test/scala/org/apache/spark/util/SizeEstimatorSuite.scala ## @@ -214,6 +216,10 @@ class SizeEstimatorSuite }

[GitHub] [spark] ral51 commented on pull request #19410: [SPARK-22184][CORE][GRAPHX] GraphX fails in case of insufficient memory and checkpoints enabled

ral51 commented on pull request #19410: URL: https://github.com/apache/spark/pull/19410#issuecomment-672393523 I ran into same issue myself. Is there a workaround? @szhem @EthanRock This is an automated message from the

[GitHub] [spark] ral51 edited a comment on pull request #19410: [SPARK-22184][CORE][GRAPHX] GraphX fails in case of insufficient memory and checkpoints enabled

ral51 edited a comment on pull request #19410: URL: https://github.com/apache/spark/pull/19410#issuecomment-672393523 I ran into same issue today. Is there a workaround? @szhem @EthanRock This is an automated message from

[GitHub] [spark] SparkQA commented on pull request #29322: [SPARK-32511][SQL] Add dropFields method to Column class

SparkQA commented on pull request #29322: URL: https://github.com/apache/spark/pull/29322#issuecomment-672393733 **[Test build #127351 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127351/testReport)** for PR 29322 at commit

[GitHub] [spark] maropu commented on pull request #29401: [SPARK-32400][SQL] Improve test coverage of HiveScriptTransformationExec

maropu commented on pull request #29401: URL: https://github.com/apache/spark/pull/29401#issuecomment-672392509 Looks okay and I have no more comment. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] rohitmishr1484 commented on pull request #29410: [SPARK-32180][PYSPARK][DOCS] Getting started-Installation guide for pyspark doc

rohitmishr1484 commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672391444 Hi @HyukjinKwon, I was not sure how to add you as a Reviewer for this Pull request, thus adding this comment. I would like to mention a few points: 1.

[GitHub] [spark] maropu commented on a change in pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

maropu commented on a change in pull request #29270: URL: https://github.com/apache/spark/pull/29270#discussion_r468932486 ## File path: sql/core/src/test/scala/org/apache/spark/sql/PlanStabilitySuite.scala ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] maropu commented on a change in pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

maropu commented on a change in pull request #29270: URL: https://github.com/apache/spark/pull/29270#discussion_r468931369 ## File path: sql/core/src/test/scala/org/apache/spark/sql/PlanStabilitySuite.scala ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] maropu commented on a change in pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

maropu commented on a change in pull request #29270: URL: https://github.com/apache/spark/pull/29270#discussion_r468925917 ## File path: sql/core/src/test/scala/org/apache/spark/sql/PlanStabilitySuite.scala ## @@ -0,0 +1,312 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

AmplabJenkins removed a comment on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672354973 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] mridulm commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

mridulm commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672367418 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] mridulm commented on a change in pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

mridulm commented on a change in pull request #29411: URL: https://github.com/apache/spark/pull/29411#discussion_r468923013 ## File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ## @@ -1400,11 +1399,10 @@ private[spark] object SparkSubmitUtils {

[GitHub] [spark] mridulm commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

mridulm commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672367215 +CC @vanzin, @brkyvz This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] maropu commented on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

maropu commented on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672365902 @Ngone51 Too heavy to show the page `Files changes`!!! How about reducing the number of the generated files for reviews? Probably, a few of the generated file examples are

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

AmplabJenkins removed a comment on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672354589 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

AmplabJenkins commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672354973 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

AmplabJenkins commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672354589 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] venkata91 opened a new pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

venkata91 opened a new pull request #29411: URL: https://github.com/apache/spark/pull/29411 ### What changes were proposed in this pull request? Clear Ivy resolution files as part of finally block if not failures while artifacts resolution can leave the resolution files around.

[GitHub] [spark] maropu commented on pull request #29407: [SPARK-32588][TEST] Fix SizeEstimator initialization in tests.

maropu commented on pull request #29407: URL: https://github.com/apache/spark/pull/29407#issuecomment-672345352 cc: @kiszk @rednaxelafx This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins removed a comment on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672339519 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672339519 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672338989 **[Test build #127350 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127350/testReport)** for PR 29328 at commit

[GitHub] [spark] imback82 commented on a change in pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

imback82 commented on a change in pull request #29328: URL: https://github.com/apache/spark/pull/29328#discussion_r468910024 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -245,12 +245,19 @@ class DataFrameReader

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672337529 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672337529 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672264107 **[Test build #127345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127345/testReport)** for PR 29367 at commit

[GitHub] [spark] SparkQA commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672336829 **[Test build #127345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127345/testReport)** for PR 29367 at commit

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468905696 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +450,88 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672331249 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672331249 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

SparkQA commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672330830 **[Test build #127349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127349/testReport)** for PR 29396 at commit

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468902557 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -242,8 +242,10 @@ class DecommissionWorkerSuite

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672304579 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

SparkQA commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672304557 **[Test build #127348 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127348/testReport)** for PR 29396 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672304569 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

SparkQA removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672302622 **[Test build #127348 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127348/testReport)** for PR 29396 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672304569 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672303780 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

AmplabJenkins removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672303780 This is an automated message from the Apache Git Service. To respond to the message, please log on

<    1   2   3   4   5   6   7   >