[GitHub] [spark] MaxGekk closed pull request #33551: [SPARK-36323][SQL] Support ANSI interval literals for TimeWindow

2021-07-28 Thread GitBox
MaxGekk closed pull request #33551: URL: https://github.com/apache/spark/pull/33551 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #33563: [SPARK-36334][K8S] Add a new conf to allow K8s API server-side caching for pod listing

2021-07-28 Thread GitBox
SparkQA commented on pull request #33563: URL: https://github.com/apache/spark/pull/33563#issuecomment-21901 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46318/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

2021-07-28 Thread GitBox
SparkQA removed a comment on pull request #33535: URL: https://github.com/apache/spark/pull/33535#issuecomment-888773347 **[Test build #141803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141803/testReport)** for PR 33535 at commit

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-28 Thread GitBox
SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-20990 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46319/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

2021-07-28 Thread GitBox
SparkQA commented on pull request #33535: URL: https://github.com/apache/spark/pull/33535#issuecomment-20806 **[Test build #141803 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141803/testReport)** for PR 33535 at commit

[GitHub] [spark] SparkQA commented on pull request #33569: [SPARK-35806][PYTHON][FOLLOW-UP] Mapping the mode argument to pandas in DataFrame.to_csv

2021-07-28 Thread GitBox
SparkQA commented on pull request #33569: URL: https://github.com/apache/spark/pull/33569#issuecomment-17603 **[Test build #141809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141809/testReport)** for PR 33569 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-17046 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141804/

[GitHub] [spark] AmplabJenkins commented on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-17046 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141804/ -- This

[GitHub] [spark] SparkQA commented on pull request #33567: [SPARK-36254][INFRA][PYTHON] Install mlflow in Github Actions CI

2021-07-28 Thread GitBox
SparkQA commented on pull request #33567: URL: https://github.com/apache/spark/pull/33567#issuecomment-16903 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46320/ -- This is an automated message from the Apache

[GitHub] [spark] itholic opened a new pull request #33569: [SPARK-35806][PYTHON][FOLLOW-UP] Mapping the mode argument to pandas in DataFrame.to_csv

2021-07-28 Thread GitBox
itholic opened a new pull request #33569: URL: https://github.com/apache/spark/pull/33569 ### What changes were proposed in this pull request? This PR is follow-up for https://github.com/apache/spark/pull/33414 to match the `mode` argument for all APIs that has `mode` argument, not

[GitHub] [spark] SparkQA removed a comment on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
SparkQA removed a comment on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-888773387 **[Test build #141804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141804/testReport)** for PR 33451 at commit

[GitHub] [spark] SparkQA commented on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
SparkQA commented on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-16444 **[Test build #141804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141804/testReport)** for PR 33451 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33535: URL: https://github.com/apache/spark/pull/33535#issuecomment-15656 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46316/

[GitHub] [spark] SparkQA commented on pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

2021-07-28 Thread GitBox
SparkQA commented on pull request #33535: URL: https://github.com/apache/spark/pull/33535#issuecomment-15641 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46316/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33535: URL: https://github.com/apache/spark/pull/33535#issuecomment-15656 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46316/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33568: [SPARK-36335][DOCS] Remove Local-cluster mode reference (and add a missing period)

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33568: URL: https://github.com/apache/spark/pull/33568#issuecomment-15320 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] yutoacts opened a new pull request #33568: [SPARK-36335][DOCS] Remove Local-cluster mode reference (and add a missing period)

2021-07-28 Thread GitBox
yutoacts opened a new pull request #33568: URL: https://github.com/apache/spark/pull/33568 ### What changes were proposed in this pull request? Remove local-cluster mode reference from configuration.md and add a missing period to submitting-application.md. ### Why are

[GitHub] [spark] SparkQA commented on pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

2021-07-28 Thread GitBox
SparkQA commented on pull request #33404: URL: https://github.com/apache/spark/pull/33404#issuecomment-14928 **[Test build #141808 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141808/testReport)** for PR 33404 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33567: [SPARK-36254][INFRA][PYTHON] Install mlflow in Github Actions CI

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33567: URL: https://github.com/apache/spark/pull/33567#issuecomment-14384 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141807/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-14385 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46315/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33566: [SPARK-36271][SQL] Unify V1 insert check field name before prepare writter

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33566: URL: https://github.com/apache/spark/pull/33566#issuecomment-14387 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46313/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33563: [SPARK-36334][K8S] Add a new conf to allow K8s API server-side caching for pod listing

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33563: URL: https://github.com/apache/spark/pull/33563#issuecomment-14383 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141805/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-14386 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46314/

[GitHub] [spark] AmplabJenkins commented on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-14386 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46314/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-14385 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46315/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33566: [SPARK-36271][SQL] Unify V1 insert check field name before prepare writter

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33566: URL: https://github.com/apache/spark/pull/33566#issuecomment-14387 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46313/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33567: [SPARK-36254][INFRA][PYTHON] Install mlflow in Github Actions CI

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33567: URL: https://github.com/apache/spark/pull/33567#issuecomment-14384 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141807/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33563: [SPARK-36334][K8S] Add a new conf to allow K8s API server-side caching for pod listing

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33563: URL: https://github.com/apache/spark/pull/33563#issuecomment-14383 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141805/ -- This

[GitHub] [spark] venkata91 commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

2021-07-28 Thread GitBox
venkata91 commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-13035 @Ngone51 @mridulm Addressed all the review comments, please take a look again. Thanks! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA removed a comment on pull request #33567: [SPARK-36254][INFRA][PYTHON] Install mlflow in Github Actions CI

2021-07-28 Thread GitBox
SparkQA removed a comment on pull request #33567: URL: https://github.com/apache/spark/pull/33567#issuecomment-888796409 **[Test build #141807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141807/testReport)** for PR 33567 at commit

[GitHub] [spark] SparkQA commented on pull request #33566: [SPARK-36271][SQL] Unify V1 insert check field name before prepare writter

2021-07-28 Thread GitBox
SparkQA commented on pull request #33566: URL: https://github.com/apache/spark/pull/33566#issuecomment-09469 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46313/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33567: [SPARK-36254][INFRA][PYTHON] Install mlflow in Github Actions CI

2021-07-28 Thread GitBox
SparkQA commented on pull request #33567: URL: https://github.com/apache/spark/pull/33567#issuecomment-08470 **[Test build #141807 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141807/testReport)** for PR 33567 at commit

[GitHub] [spark] SparkQA commented on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

2021-07-28 Thread GitBox
SparkQA commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-06299 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46314/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
SparkQA commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-00754 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46315/ -- This is an automated message from the

[GitHub] [spark] itholic commented on a change in pull request #33562: [SPARK-36333][PYTHON] Reuse isnull where the null check is needed

2021-07-28 Thread GitBox
itholic commented on a change in pull request #33562: URL: https://github.com/apache/spark/pull/33562#discussion_r678749415 ## File path: python/pyspark/pandas/generic.py ## @@ -3180,6 +3180,7 @@ def __bool__(self) -> NoReturn: def _count_expr(spark_column: Column,

[GitHub] [spark] LuciferYang edited a comment on pull request #33558: [SPARK-36326][SQL] Use Map.computeIfAbsent to simplify the process of bufferPoolsBySize init new item in HeapMemoryAllocator

2021-07-28 Thread GitBox
LuciferYang edited a comment on pull request #33558: URL: https://github.com/apache/spark/pull/33558#issuecomment-888771778 > There is one other instance you could change in AbstractCommandBuilder.getEffectiveConfig The original method as follows: ```java Map

[GitHub] [spark] SparkQA removed a comment on pull request #33563: [SPARK-36334][K8S] Add a new conf to allow K8s API server-side caching for pod listing

2021-07-28 Thread GitBox
SparkQA removed a comment on pull request #33563: URL: https://github.com/apache/spark/pull/33563#issuecomment-888792663 **[Test build #141805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141805/testReport)** for PR 33563 at commit

[GitHub] [spark] SparkQA commented on pull request #33563: [SPARK-36334][K8S] Add a new conf to allow K8s API server-side caching for pod listing

2021-07-28 Thread GitBox
SparkQA commented on pull request #33563: URL: https://github.com/apache/spark/pull/33563#issuecomment-888798434 **[Test build #141805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141805/testReport)** for PR 33563 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-888796757 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46317/

[GitHub] [spark] AmplabJenkins commented on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-888796757 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46317/ --

[GitHub] [spark] SparkQA commented on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
SparkQA commented on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-888796737 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46317/ --

[GitHub] [spark] SparkQA commented on pull request #33567: [SPARK-36254][INFRA][PYTHON] Install mlflow in Github Actions CI

2021-07-28 Thread GitBox
SparkQA commented on pull request #33567: URL: https://github.com/apache/spark/pull/33567#issuecomment-888796409 **[Test build #141807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141807/testReport)** for PR 33567 at commit

[GitHub] [spark] itholic opened a new pull request #33567: [SPARK-36254][INFRA][PYTHON] Install mlflow in Github Actions CI

2021-07-28 Thread GitBox
itholic opened a new pull request #33567: URL: https://github.com/apache/spark/pull/33567 ### What changes were proposed in this pull request? This PR proposes adding a Python package, `mlflow` and `sklearn` to enable the MLflow test in pandas API on Spark. ### Why are

[GitHub] [spark] cloud-fan commented on a change in pull request #33413: [SPARK-36175][SQL] Support TimestampNTZ in Avro data source

2021-07-28 Thread GitBox
cloud-fan commented on a change in pull request #33413: URL: https://github.com/apache/spark/pull/33413#discussion_r678816260 ## File path: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala ## @@ -147,6 +147,21 @@ private[sql] class

[GitHub] [spark] SparkQA commented on pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

2021-07-28 Thread GitBox
SparkQA commented on pull request #33535: URL: https://github.com/apache/spark/pull/33535#issuecomment-888793444 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46316/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33556: [WIP][SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

2021-07-28 Thread GitBox
SparkQA commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-888793226 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46314/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-28 Thread GitBox
SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-888793249 **[Test build #141806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141806/testReport)** for PR 31517 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-888792850 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46311/

[GitHub] [spark] SparkQA commented on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

2021-07-28 Thread GitBox
SparkQA commented on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-888792829 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46311/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-888792850 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46311/ --

[GitHub] [spark] SparkQA commented on pull request #33563: [SPARK-36334][K8S] Add a new conf to allow K8s API server-side caching for pod listing

2021-07-28 Thread GitBox
SparkQA commented on pull request #33563: URL: https://github.com/apache/spark/pull/33563#issuecomment-888792663 **[Test build #141805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141805/testReport)** for PR 33563 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33494: [SPARK-36272][SQL][TEST] Change shuffled hash join metrics test to check relative value of build size

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33494: URL: https://github.com/apache/spark/pull/33494#issuecomment-888791833 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141795/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33541: URL: https://github.com/apache/spark/pull/33541#issuecomment-888791837 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46312/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888791835 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141802/

[GitHub] [spark] AmplabJenkins commented on pull request #33544: [SPARK-34927][INFRA] Support TPCDSQueryBenchmark in Benchmarks

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33544: URL: https://github.com/apache/spark/pull/33544#issuecomment-888791842 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #33494: [SPARK-36272][SQL][TEST] Change shuffled hash join metrics test to check relative value of build size

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33494: URL: https://github.com/apache/spark/pull/33494#issuecomment-888791833 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141795/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33541: URL: https://github.com/apache/spark/pull/33541#issuecomment-888791837 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46312/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888791835 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141802/ -- This

[GitHub] [spark] SparkQA commented on pull request #33566: [SPARK-36271][SQL] Unify V1 insert check field name before prepare writter

2021-07-28 Thread GitBox
SparkQA commented on pull request #33566: URL: https://github.com/apache/spark/pull/33566#issuecomment-888789675 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46313/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33494: [SPARK-36272][SQL][TEST] Change shuffled hash join metrics test to check relative value of build size

2021-07-28 Thread GitBox
SparkQA removed a comment on pull request #33494: URL: https://github.com/apache/spark/pull/33494#issuecomment-888690742 **[Test build #141795 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141795/testReport)** for PR 33494 at commit

[GitHub] [spark] SparkQA commented on pull request #33494: [SPARK-36272][SQL][TEST] Change shuffled hash join metrics test to check relative value of build size

2021-07-28 Thread GitBox
SparkQA commented on pull request #33494: URL: https://github.com/apache/spark/pull/33494#issuecomment-888789018 **[Test build #141795 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141795/testReport)** for PR 33494 at commit

[GitHub] [spark] SparkQA commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
SparkQA commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888787998 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46315/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-28 Thread GitBox
SparkQA commented on pull request #33541: URL: https://github.com/apache/spark/pull/33541#issuecomment-888782025 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46312/ -- This is an automated message from the

[GitHub] [spark] LuciferYang commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-28 Thread GitBox
LuciferYang commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-888779189 @holdenk 6c74fc6 merge with master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA removed a comment on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
SparkQA removed a comment on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888773302 **[Test build #141802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141802/testReport)** for PR 33550 at commit

[GitHub] [spark] SparkQA commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
SparkQA commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888778872 **[Test build #141802 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141802/testReport)** for PR 33550 at commit

[GitHub] [spark] LuciferYang edited a comment on pull request #33558: [SPARK-36326][SQL] Use Map.computeIfAbsent to simplify the process of bufferPoolsBySize init new item in HeapMemoryAllocator

2021-07-28 Thread GitBox
LuciferYang edited a comment on pull request #33558: URL: https://github.com/apache/spark/pull/33558#issuecomment-888771778 > There is one other instance you could change in AbstractCommandBuilder.getEffectiveConfig The original method as follows: ```java Map

[GitHub] [spark] LuciferYang edited a comment on pull request #33558: [SPARK-36326][SQL] Use Map.computeIfAbsent to simplify the process of bufferPoolsBySize init new item in HeapMemoryAllocator

2021-07-28 Thread GitBox
LuciferYang edited a comment on pull request #33558: URL: https://github.com/apache/spark/pull/33558#issuecomment-888771778 > There is one other instance you could change in AbstractCommandBuilder.getEffectiveConfig The original method as follows: ```java Map

[GitHub] [spark] LuciferYang edited a comment on pull request #33558: [SPARK-36326][SQL] Use Map.computeIfAbsent to simplify the process of bufferPoolsBySize init new item in HeapMemoryAllocator

2021-07-28 Thread GitBox
LuciferYang edited a comment on pull request #33558: URL: https://github.com/apache/spark/pull/33558#issuecomment-888771778 > There is one other instance you could change in AbstractCommandBuilder.getEffectiveConfig The original method as follows: ```java Map

[GitHub] [spark] LuciferYang edited a comment on pull request #33558: [SPARK-36326][SQL] Use Map.computeIfAbsent to simplify the process of bufferPoolsBySize init new item in HeapMemoryAllocator

2021-07-28 Thread GitBox
LuciferYang edited a comment on pull request #33558: URL: https://github.com/apache/spark/pull/33558#issuecomment-888771778 > There is one other instance you could change in AbstractCommandBuilder.getEffectiveConfig The original method as follows: ```java Map

[GitHub] [spark] LuciferYang edited a comment on pull request #33558: [SPARK-36326][SQL] Use Map.computeIfAbsent to simplify the process of bufferPoolsBySize init new item in HeapMemoryAllocator

2021-07-28 Thread GitBox
LuciferYang edited a comment on pull request #33558: URL: https://github.com/apache/spark/pull/33558#issuecomment-888771778 > There is one other instance you could change in AbstractCommandBuilder.getEffectiveConfig The original method as follows: ```java Map

[GitHub] [spark] SparkQA commented on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

2021-07-28 Thread GitBox
SparkQA commented on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-888774369 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46311/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33535: [SPARK-36108][SQL] Refactor first set of 20 query parsing errors to use error classes

2021-07-28 Thread GitBox
SparkQA commented on pull request #33535: URL: https://github.com/apache/spark/pull/33535#issuecomment-888773347 **[Test build #141803 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141803/testReport)** for PR 33535 at commit

[GitHub] [spark] SparkQA commented on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
SparkQA commented on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-888773387 **[Test build #141804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141804/testReport)** for PR 33451 at commit

[GitHub] [spark] SparkQA commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
SparkQA commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888773302 **[Test build #141802 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141802/testReport)** for PR 33550 at commit

[GitHub] [spark] SparkQA commented on pull request #33566: [SPARK-36271][SQL] Unify V1 insert check field name before prepare writter

2021-07-28 Thread GitBox
SparkQA commented on pull request #33566: URL: https://github.com/apache/spark/pull/33566#issuecomment-888773286 **[Test build #141800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141800/testReport)** for PR 33566 at commit

[GitHub] [spark] SparkQA commented on pull request #33556: [WIP][SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

2021-07-28 Thread GitBox
SparkQA commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-888773263 **[Test build #141801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141801/testReport)** for PR 33556 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
Ngone51 commented on a change in pull request #33451: URL: https://github.com/apache/spark/pull/33451#discussion_r678795780 ## File path: common/network-common/src/main/java/org/apache/spark/network/corruption/Cause.java ## @@ -0,0 +1,27 @@ +/* + * Licensed to the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33565: [SPARK-33298][CORE][FOLLOWUP] Move `FileNameSpec` into `FileCommitProtocol` object

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33565: URL: https://github.com/apache/spark/pull/33565#issuecomment-888772193 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141796/

[GitHub] [spark] AmplabJenkins commented on pull request #33545: [SPARK-36319][SQL][PySpark] Have Observation return Map instead of Row

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33545: URL: https://github.com/apache/spark/pull/33545#issuecomment-888772199 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #33565: [SPARK-33298][CORE][FOLLOWUP] Move `FileNameSpec` into `FileCommitProtocol` object

2021-07-28 Thread GitBox
AmplabJenkins commented on pull request #33565: URL: https://github.com/apache/spark/pull/33565#issuecomment-888772193 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141796/ -- This

[GitHub] [spark] LuciferYang commented on pull request #33558: [SPARK-36326][SQL] Use Map.computeIfAbsent to simplify the process of bufferPoolsBySize init new item in HeapMemoryAllocator

2021-07-28 Thread GitBox
LuciferYang commented on pull request #33558: URL: https://github.com/apache/spark/pull/33558#issuecomment-888771778 > There is one other instance you could change in AbstractCommandBuilder.getEffectiveConfig The original method as follows: ```java Map

[GitHub] [spark] SparkQA commented on pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-28 Thread GitBox
SparkQA commented on pull request #33541: URL: https://github.com/apache/spark/pull/33541#issuecomment-888770887 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46312/ -- This is an automated message from the Apache

[GitHub] [spark] maryannxue commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-28 Thread GitBox
maryannxue commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r678793485 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala ## @@ -32,8 +32,14 @@ import

[GitHub] [spark] maryannxue commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-28 Thread GitBox
maryannxue commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r678792942 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala ## @@ -251,11 +257,12 @@ object

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #33566: [SPARK-36271][SQL] Unify V1 insert check field name before prepare writter

2021-07-28 Thread GitBox
AngersZh commented on a change in pull request #33566: URL: https://github.com/apache/spark/pull/33566#discussion_r678792113 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetWriteSupport.scala ## @@ -482,7 +482,6 @@ object

[GitHub] [spark] AngersZhuuuu opened a new pull request #33566: [SPARK-36271][SQL] Unify V1 insert check field name before prepare writter

2021-07-28 Thread GitBox
AngersZh opened a new pull request #33566: URL: https://github.com/apache/spark/pull/33566 ### What changes were proposed in this pull request? Unify DataSource V1 insert schema check field name before prepare writer. ### Why are the changes needed? Unify code

[GitHub] [spark] maryannxue commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

2021-07-28 Thread GitBox
maryannxue commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r678789558 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala ## @@ -251,11 +257,12 @@ object

[GitHub] [spark] viirya commented on a change in pull request #33494: [SPARK-36272][SQL][TEST] Change shuffled hash join metrics test to check relative value of build size

2021-07-28 Thread GitBox
viirya commented on a change in pull request #33494: URL: https://github.com/apache/spark/pull/33494#discussion_r678789271 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala ## @@ -396,27 +396,25 @@ class SQLMetricsSuite extends

[GitHub] [spark] ulysses-you commented on a change in pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
ulysses-you commented on a change in pull request #33550: URL: https://github.com/apache/spark/pull/33550#discussion_r678787932 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala ## @@ -70,8

[GitHub] [spark] c21 commented on pull request #33494: [SPARK-36272][SQL][TEST] Change shuffled hash join metrics test to check relative value of build size

2021-07-28 Thread GitBox
c21 commented on pull request #33494: URL: https://github.com/apache/spark/pull/33494#issuecomment-888762611 Thank you @HyukjinKwon for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] ulysses-you commented on a change in pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
ulysses-you commented on a change in pull request #33550: URL: https://github.com/apache/spark/pull/33550#discussion_r678787228 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -256,7 +256,7 @@ private[spark] object

[GitHub] [spark] ulysses-you commented on a change in pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
ulysses-you commented on a change in pull request #33550: URL: https://github.com/apache/spark/pull/33550#discussion_r678787135 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala ## @@ -70,8

[GitHub] [spark] ulysses-you commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

2021-07-28 Thread GitBox
ulysses-you commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888761873 @holdenk @dongjoon-hyun ``` test("Run SparkPi with a very long application name.", k8sTestTag) { sparkAppConf.set("spark.app.name", "long" * 40)

[GitHub] [spark] SparkQA removed a comment on pull request #33565: [SPARK-33298][CORE][FOLLOWUP] Move `FileNameSpec` into `FileCommitProtocol` object

2021-07-28 Thread GitBox
SparkQA removed a comment on pull request #33565: URL: https://github.com/apache/spark/pull/33565#issuecomment-888708317 **[Test build #141796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141796/testReport)** for PR 33565 at commit

[GitHub] [spark] SparkQA commented on pull request #33565: [SPARK-33298][CORE][FOLLOWUP] Move `FileNameSpec` into `FileCommitProtocol` object

2021-07-28 Thread GitBox
SparkQA commented on pull request #33565: URL: https://github.com/apache/spark/pull/33565#issuecomment-888761263 **[Test build #141796 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141796/testReport)** for PR 33565 at commit

[GitHub] [spark] LuciferYang edited a comment on pull request #33558: [SPARK-36326][SQL] Use Map.computeIfAbsent to simplify the process of bufferPoolsBySize init new item in HeapMemoryAllocator

2021-07-28 Thread GitBox
LuciferYang edited a comment on pull request #33558: URL: https://github.com/apache/spark/pull/33558#issuecomment-888751784 > There is one other instance you could change in AbstractCommandBuilder.getEffectiveConfig OK, let me check this -- This is an automated message from the

[GitHub] [spark] Ngone51 commented on a change in pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

2021-07-28 Thread GitBox
Ngone51 commented on a change in pull request #33451: URL: https://github.com/apache/spark/pull/33451#discussion_r678781857 ## File path: common/network-common/src/main/java/org/apache/spark/network/corruption/Cause.java ## @@ -0,0 +1,27 @@ +/* + * Licensed to the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

2021-07-28 Thread GitBox
SparkQA removed a comment on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-888750598 **[Test build #141798 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141798/testReport)** for PR 33432 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

2021-07-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-888754296 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141798/

  1   2   3   4   5   6   7   8   >