[GitHub] [spark] SparkQA commented on pull request #32411: [SPARK-28551][SQL]CTAS with LOCATION , should not allow to a non-empty directory.

SparkQA commented on pull request #32411: URL: https://github.com/apache/spark/pull/32411#issuecomment-832855291 **[Test build #138179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138179/testReport)** for PR 32411 at commit

[GitHub] [spark] SparkQA commented on pull request #32444: [SPARK-35111][SQL][FOLLOWUP] Support Cast string to year-month interval

SparkQA commented on pull request #32444: URL: https://github.com/apache/spark/pull/32444#issuecomment-832855178 **[Test build #138177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138177/testReport)** for PR 32444 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32443: [SPARK-35319][K8S][BUILD] Upgrade K8s client to 5.3.1

AmplabJenkins removed a comment on pull request #32443: URL: https://github.com/apache/spark/pull/32443#issuecomment-832852790 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42697/

[GitHub] [spark] AmplabJenkins commented on pull request #32443: [SPARK-35319][K8S][BUILD] Upgrade K8s client to 5.3.1

AmplabJenkins commented on pull request #32443: URL: https://github.com/apache/spark/pull/32443#issuecomment-832852790 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42697/ --

[GitHub] [spark] vinodkc commented on a change in pull request #32411: [SPARK-28551][SQL]In CTAS with LOCATION , should not allow to a non-empty directory.

vinodkc commented on a change in pull request #32411: URL: https://github.com/apache/spark/pull/32411#discussion_r626731935 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/DataWritingCommand.scala ## @@ -96,4 +98,24 @@ object DataWritingCommand {

[GitHub] [spark] xkrogen commented on a change in pull request #32431: [SPARK-35173][SQL][PYTHON] Add multiple columns adding support

xkrogen commented on a change in pull request #32431: URL: https://github.com/apache/spark/pull/32431#discussion_r626731636 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -2398,8 +2398,14 @@ class Dataset[T] private[sql]( /** * Returns a

[GitHub] [spark] vinodkc commented on a change in pull request #32411: [SPARK-28551][SQL]In CTAS with LOCATION , should not allow to a non-empty directory.

vinodkc commented on a change in pull request #32411: URL: https://github.com/apache/spark/pull/32411#discussion_r626730085 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -3707,6 +3716,8 @@ class SQLConf extends Serializable with

[GitHub] [spark] vinodkc commented on a change in pull request #32411: [SPARK-28551][SQL]In CTAS with LOCATION , should not allow to a non-empty directory.

vinodkc commented on a change in pull request #32411: URL: https://github.com/apache/spark/pull/32411#discussion_r626729769 ## File path: docs/sql-migration-guide.md ## @@ -78,7 +78,9 @@ license: | - In Spark 3.2, the timestamps subtraction expression such as `timestamp

[GitHub] [spark] vinodkc commented on a change in pull request #32411: [SPARK-28551][SQL]In CTAS with LOCATION , should not allow to a non-empty directory.

vinodkc commented on a change in pull request #32411: URL: https://github.com/apache/spark/pull/32411#discussion_r626729449 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -1604,6 +1604,15 @@ object SQLConf { .booleanConf

[GitHub] [spark] holdenk commented on pull request #32436: [SPARK-34764][CORE][K8S][UI] Propagate reason for exec loss to Web UI

holdenk commented on pull request #32436: URL: https://github.com/apache/spark/pull/32436#issuecomment-832841187 So if you want it's pretty easy to trigger OOMs with the GroupByKey example (it's how I did the manual test). I'm not sure about event logs though, do we have those committed

[GitHub] [spark] holdenk commented on a change in pull request #32436: [SPARK-34764][CORE][K8S][UI] Propagate reason for exec loss to Web UI

holdenk commented on a change in pull request #32436: URL: https://github.com/apache/spark/pull/32436#discussion_r626727970 ## File path: core/src/main/resources/org/apache/spark/ui/static/executorspage.js ## @@ -25,6 +25,14 @@ function getThreadDumpEnabled() { return

[GitHub] [spark] SparkQA commented on pull request #32443: [SPARK-35319][K8S][BUILD] Upgrade K8s client to 5.3.1

SparkQA commented on pull request #32443: URL: https://github.com/apache/spark/pull/32443#issuecomment-832839986 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] holdenk commented on a change in pull request #32436: [SPARK-34764][CORE][K8S][UI] Propagate reason for exec loss to Web UI

holdenk commented on a change in pull request #32436: URL: https://github.com/apache/spark/pull/32436#discussion_r626727168 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala ## @@ -217,14

[GitHub] [spark] fommil commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

fommil commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-832821382 Oh I see, this is native via ludovic netlib. Can you provide benchmarks vs netlib-java to confirm that there is no obvious perf regression? -- This is an automated message

[GitHub] [spark] AngersZhuuuu opened a new pull request #32444: [SPARK-35111][SQL][FOLLOWUP] Support Cast string to year-month interval

AngersZh opened a new pull request #32444: URL: https://github.com/apache/spark/pull/32444 ### What changes were proposed in this pull request? Refactor code with day-second interval ### Why are the changes needed? refactor code ### Does this PR introduce

[GitHub] [spark] AngersZhuuuu commented on pull request #32444: [SPARK-35111][SQL][FOLLOWUP] Support Cast string to year-month interval

AngersZh commented on pull request #32444: URL: https://github.com/apache/spark/pull/32444#issuecomment-832820948 FYI @MaxGekk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] fommil edited a comment on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

fommil edited a comment on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-832810727 Perhaps I am misunderstanding the benchmarks, but it looks to me like netlib-java native (unclear which impl you're using, I assume MKL?) DGEMM still dramatically wins

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code

AmplabJenkins removed a comment on pull request #31776: URL: https://github.com/apache/spark/pull/31776#issuecomment-832819401 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138173/

[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code

AmplabJenkins commented on pull request #31776: URL: https://github.com/apache/spark/pull/31776#issuecomment-832819401 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138173/ -- This

[GitHub] [spark] fommil edited a comment on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

fommil edited a comment on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-832810727 Perhaps I am misunderstanding the benchmarks, but it looks to me like netlib-java native (unclear which impl you're using, I assume MKL?) DGEMM still dramatically wins

[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code

SparkQA removed a comment on pull request #31776: URL: https://github.com/apache/spark/pull/31776#issuecomment-832628468 **[Test build #138173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138173/testReport)** for PR 31776 at commit

[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code

SparkQA commented on pull request #31776: URL: https://github.com/apache/spark/pull/31776#issuecomment-832817814 **[Test build #138173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138173/testReport)** for PR 31776 at commit

[GitHub] [spark] mridulm commented on pull request #31876: [SPARK-34942][API][CORE] Abstract Location in MapStatus to enable support for custom storage

mridulm commented on pull request #31876: URL: https://github.com/apache/spark/pull/31876#issuecomment-832811272 I have to circle back to this PR and review. @Ngone51 can we update to latest please ? Or are we waiting for some other changes to go in before doing it ? -- This is an

[GitHub] [spark] fommil commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

fommil commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-832810727 Perhaps I am misunderstanding the benchmarks, but it looks to me like netlib-java native (unclear which impl you're using, I assume MKL?) DGEMM still dramatically wins over the

[GitHub] [spark] SparkQA commented on pull request #32443: [SPARK-35319][K8S][BUILD] Upgrade K8s client to 5.3.1

SparkQA commented on pull request #32443: URL: https://github.com/apache/spark/pull/32443#issuecomment-832809158 **[Test build #138176 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138176/testReport)** for PR 32443 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

AmplabJenkins removed a comment on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832808380 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42696/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

AmplabJenkins removed a comment on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-832808379 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138174/

[GitHub] [spark] AmplabJenkins commented on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

AmplabJenkins commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832808380 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42696/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

AmplabJenkins commented on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-832808379 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138174/ -- This

[GitHub] [spark] mridulm commented on a change in pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

mridulm commented on a change in pull request #32287: URL: https://github.com/apache/spark/pull/32287#discussion_r626693376 ## File path: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala ## @@ -317,6 +377,14 @@ final class

[GitHub] [spark] dongjoon-hyun opened a new pull request #32443: [SPARK-35319][K8S][BUILD] Upgrade K8s client to 5.3.1

dongjoon-hyun opened a new pull request #32443: URL: https://github.com/apache/spark/pull/32443 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ###

[GitHub] [spark] mridulm commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

mridulm commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r626661012 ## File path: core/src/main/scala/org/apache/spark/Dependency.scala ## @@ -102,6 +102,12 @@ class ShuffleDependency[K: ClassTag, V: ClassTag, C:

[GitHub] [spark] SparkQA commented on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832800828 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42696/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832793857 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42696/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

SparkQA removed a comment on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-832664537 **[Test build #138174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138174/testReport)** for PR 32442 at commit

[GitHub] [spark] SparkQA commented on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

SparkQA commented on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-832787002 **[Test build #138174 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138174/testReport)** for PR 32442 at commit

[GitHub] [spark] luhenry commented on pull request #30810: [SPARK-33882][ML] Add a vectorized BLAS implementation

luhenry commented on pull request #30810: URL: https://github.com/apache/spark/pull/30810#issuecomment-832774216 > However, it is strange that you're not providing benchmarks against machine-optimised MKL (or otherwise) backends as described in http://fommil.com/scalax14/#/ You

[GitHub] [spark] SparkQA commented on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832758505 **[Test build #138175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138175/testReport)** for PR 32439 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

AmplabJenkins removed a comment on pull request #32441: URL: https://github.com/apache/spark/pull/32441#issuecomment-832755201 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138172/

[GitHub] [spark] AmplabJenkins commented on pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

AmplabJenkins commented on pull request #32441: URL: https://github.com/apache/spark/pull/32441#issuecomment-832755201 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138172/ -- This

[GitHub] [spark] 8bitreid commented on a change in pull request #32436: [SPARK-34764][CORE][K8S][UI] Propagate reason for exec loss to Web UI

8bitreid commented on a change in pull request #32436: URL: https://github.com/apache/spark/pull/32436#discussion_r626640992 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala ## @@ -217,14

[GitHub] [spark] SparkQA removed a comment on pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

SparkQA removed a comment on pull request #32441: URL: https://github.com/apache/spark/pull/32441#issuecomment-832563969 **[Test build #138172 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138172/testReport)** for PR 32441 at commit

[GitHub] [spark] SparkQA commented on pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

SparkQA commented on pull request #32441: URL: https://github.com/apache/spark/pull/32441#issuecomment-832727556 **[Test build #138172 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138172/testReport)** for PR 32441 at commit

[GitHub] [spark] srowen commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

srowen commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-832721754 Hm, it does seem very sensitive to partitioning then. That's not good; maybe kind of understandable if the data set is so small that each partition has just a few elements. The

[GitHub] [spark] srowen commented on a change in pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

srowen commented on a change in pull request #32415: URL: https://github.com/apache/spark/pull/32415#discussion_r626602492 ## File path: LICENSE-binary ## @@ -456,7 +456,6 @@ org.antlr:ST4 org.antlr:stringtemplate org.antlr:antlr4-runtime antlr:antlr

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

AmplabJenkins removed a comment on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-832706882 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42695/

[GitHub] [spark] AmplabJenkins commented on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

AmplabJenkins commented on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-832706882 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42695/ --

[GitHub] [spark] SparkQA commented on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

SparkQA commented on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-832702301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] tgravescs commented on pull request #32136: [SPARK-35022][CORE] Task Scheduling Plugin in Spark

tgravescs commented on pull request #32136: URL: https://github.com/apache/spark/pull/32136#issuecomment-832699455 Personally for me, I'm hesitant about putting a plugin at this point without making sure it can address other use cases. even if its a developer plugin, if I go remove or

[GitHub] [spark] SparkQA commented on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

SparkQA commented on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-832664537 **[Test build #138174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138174/testReport)** for PR 32442 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code

AmplabJenkins removed a comment on pull request #31776: URL: https://github.com/apache/spark/pull/31776#issuecomment-832663431 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42694/

[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code

AmplabJenkins commented on pull request #31776: URL: https://github.com/apache/spark/pull/31776#issuecomment-832663431 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42694/ --

[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code

SparkQA commented on pull request #31776: URL: https://github.com/apache/spark/pull/31776#issuecomment-832658703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] beliefer opened a new pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

beliefer opened a new pull request #32442: URL: https://github.com/apache/spark/pull/32442 ### What changes were proposed in this pull request? Some command used to display metadata, such as: `SHOW TABLES`, `SHOW TABLE EXTENDED`,`SHOW TBLPROPERTIES` and so no. If the output rows much

[GitHub] [spark] jerqi commented on pull request #32426: [SPARK-35297][DOC] modify the comment about the executor

jerqi commented on pull request #32426: URL: https://github.com/apache/spark/pull/32426#issuecomment-832633577 cc @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code

SparkQA commented on pull request #31776: URL: https://github.com/apache/spark/pull/31776#issuecomment-832628468 **[Test build #138173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138173/testReport)** for PR 31776 at commit

[GitHub] [spark] LuciferYang commented on pull request #32374: [WIP][SPARK-35253][BUILD][SQL] Upgrade Janino from 3.0.16 to 3.1.3

LuciferYang commented on pull request #32374: URL: https://github.com/apache/spark/pull/32374#issuecomment-832614287 @maropu After janino-compiler/janino#145 and janino-compiler/janino#146 merged, I will try to use the snapshot version to verify whether there are other blocking issue

[GitHub] [spark] LuciferYang commented on pull request #32374: [WIP][SPARK-35253][BUILD][SQL] Upgrade Janino from 3.0.16 to 3.1.3

LuciferYang commented on pull request #32374: URL: https://github.com/apache/spark/pull/32374#issuecomment-832611309 @maropu Thank you for your feedback, maybe we need to wait until the next version of Janino to upgrade it -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] LuciferYang commented on pull request #32423: [SPARK-35250][SQL][DOCS] Fix duplicated STOP_AT_DELIMITER to SKIP_VALUE at CSV's unescapedQuoteHandling option documentation

LuciferYang commented on pull request #32423: URL: https://github.com/apache/spark/pull/32423#issuecomment-832609695 Thank you for your fix @HyukjinKwon , I'm on vacation, so I'm not online these days :) -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] 8bitreid commented on a change in pull request #32436: [SPARK-34764][CORE][K8S][UI] Propagate reason for exec loss to Web UI

8bitreid commented on a change in pull request #32436: URL: https://github.com/apache/spark/pull/32436#discussion_r626474836 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala ## @@ -217,14

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

AmplabJenkins removed a comment on pull request #32441: URL: https://github.com/apache/spark/pull/32441#issuecomment-832595996 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42693/

[GitHub] [spark] AmplabJenkins commented on pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

AmplabJenkins commented on pull request #32441: URL: https://github.com/apache/spark/pull/32441#issuecomment-832595996 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42693/ --

[GitHub] [spark] SparkQA commented on pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

SparkQA commented on pull request #32441: URL: https://github.com/apache/spark/pull/32441#issuecomment-832586893 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] wangyum commented on pull request #32440: [SPARK-35315][TESTS] Keep benchmark result consistent between spark-submit and SBT

wangyum commented on pull request #32440: URL: https://github.com/apache/spark/pull/32440#issuecomment-832585013 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] wangyum closed pull request #32440: [SPARK-35315][TESTS] Keep benchmark result consistent between spark-submit and SBT

wangyum closed pull request #32440: URL: https://github.com/apache/spark/pull/32440 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] SparkQA commented on pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

SparkQA commented on pull request #32441: URL: https://github.com/apache/spark/pull/32441#issuecomment-832563969 **[Test build #138172 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138172/testReport)** for PR 32441 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

AmplabJenkins removed a comment on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832562832 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138171/

[GitHub] [spark] AmplabJenkins commented on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

AmplabJenkins commented on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832562832 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138171/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32430: [SPARK-35133][SQL] Explain codegen works with AQE

AmplabJenkins removed a comment on pull request #32430: URL: https://github.com/apache/spark/pull/32430#issuecomment-832562149 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138167/

[GitHub] [spark] SparkQA removed a comment on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

SparkQA removed a comment on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832499825 **[Test build #138171 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138171/testReport)** for PR 32410 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

AmplabJenkins removed a comment on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832562150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42692/

[GitHub] [spark] AmplabJenkins commented on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

AmplabJenkins commented on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832562150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42692/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32430: [SPARK-35133][SQL] Explain codegen works with AQE

AmplabJenkins commented on pull request #32430: URL: https://github.com/apache/spark/pull/32430#issuecomment-832562149 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138167/ -- This

[GitHub] [spark] SparkQA commented on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

SparkQA commented on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832561921 **[Test build #138171 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138171/testReport)** for PR 32410 at commit

[GitHub] [spark] SparkQA commented on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

SparkQA commented on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832557356 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42692/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

SparkQA commented on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832554678 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42692/ -- This is an automated message from the Apache

[GitHub] [spark] linhongliu-db opened a new pull request #32441: [SPARK-35318][SQL] Hide internal view properties for describe table cmd

linhongliu-db opened a new pull request #32441: URL: https://github.com/apache/spark/pull/32441 ### What changes were proposed in this pull request? Hide internal view properties for describe table command, because those properties are generated by spark and should be transparent to

[GitHub] [spark] luhenry commented on a change in pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

luhenry commented on a change in pull request #32415: URL: https://github.com/apache/spark/pull/32415#discussion_r626408304 ## File path: LICENSE-binary ## @@ -456,7 +456,6 @@ org.antlr:ST4 org.antlr:stringtemplate org.antlr:antlr4-runtime antlr:antlr

[GitHub] [spark] SparkQA removed a comment on pull request #32430: [SPARK-35133][SQL] Explain codegen works with AQE

SparkQA removed a comment on pull request #32430: URL: https://github.com/apache/spark/pull/32430#issuecomment-832409814 **[Test build #138167 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138167/testReport)** for PR 32430 at commit

[GitHub] [spark] SparkQA commented on pull request #32430: [SPARK-35133][SQL] Explain codegen works with AQE

SparkQA commented on pull request #32430: URL: https://github.com/apache/spark/pull/32430#issuecomment-832541072 **[Test build #138167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138167/testReport)** for PR 32430 at commit

[GitHub] [spark] grarkydev commented on pull request #32289: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes

grarkydev commented on pull request #32289: URL: https://github.com/apache/spark/pull/32289#issuecomment-832538779 cc @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] viirya commented on pull request #32136: [SPARK-35022][CORE] Task Scheduling Plugin in Spark

viirya commented on pull request #32136: URL: https://github.com/apache/spark/pull/32136#issuecomment-832536972 @tgravescs @mridulm @HeartSaVioR @Ngone51 @cloud-fan Any more suggestive comments? As this is not public API, is there concern to have this as an experiment to unblock further

[GitHub] [spark] luhenry commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

luhenry commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-832527738 @srowen I am not observing much convergence even when ramping up `maxIter` to 1000. __`local[2]` + `maxIter=1000`:__

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32422: [DO-NOT-MERGE][WIP][SS] Custom stateful task scheduling

AmplabJenkins removed a comment on pull request #32422: URL: https://github.com/apache/spark/pull/32422#issuecomment-832526779 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138152/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

AmplabJenkins removed a comment on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832526778 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

AmplabJenkins commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832526780 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins commented on pull request #32422: [DO-NOT-MERGE][WIP][SS] Custom stateful task scheduling

AmplabJenkins commented on pull request #32422: URL: https://github.com/apache/spark/pull/32422#issuecomment-832526779 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138152/ -- This

[GitHub] [spark] HeartSaVioR commented on pull request #31944: [SPARK-34854][SQL][SS] Expose source metrics via progress report and add Kafka use-case to report delay.

HeartSaVioR commented on pull request #31944: URL: https://github.com/apache/spark/pull/31944#issuecomment-832511403 Thanks all for thoughtful reviewing and thanks @yijiacui-db for the contribution! Merged to master. -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] SaurabhChawla100 commented on pull request #32437: [SPARK-35279][CORE][SQL] Code change to fix the missing "_SUCCESS" file

SaurabhChawla100 commented on pull request #32437: URL: https://github.com/apache/spark/pull/32437#issuecomment-832510015 cc @cloud-fan @HyukjinKwon @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] HeartSaVioR closed pull request #31944: [SPARK-34854][SQL][SS] Expose source metrics via progress report and add Kafka use-case to report delay.

HeartSaVioR closed pull request #31944: URL: https://github.com/apache/spark/pull/31944 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] HeartSaVioR commented on pull request #31944: [SPARK-34854][SQL][SS] Expose source metrics via progress report and add Kafka use-case to report delay.

HeartSaVioR commented on pull request #31944: URL: https://github.com/apache/spark/pull/31944#issuecomment-832507837 Thanks! Merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA removed a comment on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

SparkQA removed a comment on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832458574 **[Test build #138168 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138168/testReport)** for PR 32439 at commit

[GitHub] [spark] SparkQA commented on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832505409 **[Test build #138168 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138168/testReport)** for PR 32439 at commit

[GitHub] [spark] viirya commented on pull request #32404: [SPARK-35278][SQL] Invoke should find the method with correct number of parameters

viirya commented on pull request #32404: URL: https://github.com/apache/spark/pull/32404#issuecomment-832505283 @cloud-fan Seems to me we still need to support it. One example is `UserDefinedType`. For example, `VectorUDT` extends `UserDefinedType[Vector]`. `VectorUDT` is

[GitHub] [spark] SparkQA removed a comment on pull request #32422: [DO-NOT-MERGE][WIP][SS] Custom stateful task scheduling

SparkQA removed a comment on pull request #32422: URL: https://github.com/apache/spark/pull/32422#issuecomment-832327910 **[Test build #138152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138152/testReport)** for PR 32422 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

SparkQA removed a comment on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832463984 **[Test build #138170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138170/testReport)** for PR 32439 at commit

[GitHub] [spark] SparkQA commented on pull request #32422: [DO-NOT-MERGE][WIP][SS] Custom stateful task scheduling

SparkQA commented on pull request #32422: URL: https://github.com/apache/spark/pull/32422#issuecomment-832501545 **[Test build #138152 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138152/testReport)** for PR 32422 at commit

[GitHub] [spark] SparkQA commented on pull request #32439: [WIP][SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-832500346 **[Test build #138170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138170/testReport)** for PR 32439 at commit

[GitHub] [spark] SparkQA commented on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

SparkQA commented on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832499825 **[Test build #138171 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138171/testReport)** for PR 32410 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32410: [SPARK-35286][SQL] Replace SessionState.start with SessionState.setCurrentSessionState

AmplabJenkins removed a comment on pull request #32410: URL: https://github.com/apache/spark/pull/32410#issuecomment-832492165 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138169/

<    1   2   3   4   >