[GitHub] [spark] SparkQA commented on pull request #33988: [SPARK-36683][SQL] Add new built-in SQL functions: SEC and CSC

SparkQA commented on pull request #33988: URL: https://github.com/apache/spark/pull/33988#issuecomment-918829375 **[Test build #143242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143242/testReport)** for PR 33988 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins removed a comment on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918828436 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47742/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

AmplabJenkins removed a comment on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918828371 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143231/

[GitHub] [spark] AmplabJenkins commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918828436 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47742/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

AmplabJenkins commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918828371 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143231/ -- This

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918828405 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47742/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

SparkQA removed a comment on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918754464 **[Test build #143231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143231/testReport)** for PR 33987 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33930: [SPARK-36665][SQL] Add more Not operator simplifications

AmplabJenkins removed a comment on pull request #33930: URL: https://github.com/apache/spark/pull/33930#issuecomment-918827010 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47744/

[GitHub] [spark] SparkQA commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

SparkQA commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918827180 **[Test build #143231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143231/testReport)** for PR 33987 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33988: [SPARK-36683][SQL] Add new built-in SQL functions: SEC and CSC

AmplabJenkins removed a comment on pull request #33988: URL: https://github.com/apache/spark/pull/33988#issuecomment-918797307 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] AmplabJenkins commented on pull request #33930: [SPARK-36665][SQL] Add more Not operator simplifications

AmplabJenkins commented on pull request #33930: URL: https://github.com/apache/spark/pull/33930#issuecomment-918827010 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47744/ --

[GitHub] [spark] SparkQA commented on pull request #33930: [SPARK-36665][SQL] Add more Not operator simplifications

SparkQA commented on pull request #33930: URL: https://github.com/apache/spark/pull/33930#issuecomment-918826977 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47744/ --

[GitHub] [spark] sarutak commented on pull request #33988: [SPARK-36683][SQL] Add new built-in SQL functions: SEC and CSC

sarutak commented on pull request #33988: URL: https://github.com/apache/spark/pull/33988#issuecomment-918826460 Jenkins, add to whitelist. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] sarutak commented on a change in pull request #33988: [SPARK-36683][SQL] Add new built-in SQL functions: SEC and CSC

sarutak commented on a change in pull request #33988: URL: https://github.com/apache/spark/pull/33988#discussion_r707925829 ## File path: sql/core/src/test/resources/sql-tests/results/operators.sql.out ## @@ -259,11 +259,83 @@ struct -- !query +select sec(1) +-- !query

[GitHub] [spark] sunchao commented on a change in pull request #33930: [SPARK-36665][SQL] Add more Not operator simplifications

sunchao commented on a change in pull request #33930: URL: https://github.com/apache/spark/pull/33930#discussion_r707916583 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -797,6 +834,19 @@ object NullPropagation

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins removed a comment on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918824358 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47741/

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918824327 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47741/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918824358 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47741/ --

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918824140 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47742/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

AmplabJenkins removed a comment on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918822840 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47740/

[GitHub] [spark] AmplabJenkins commented on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

AmplabJenkins commented on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918822840 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47740/ --

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918820137 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47741/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

SparkQA commented on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918818647 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47740/ -- This is an automated message from the

[GitHub] [spark] rmcyang commented on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

rmcyang commented on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918816927 Thanks for the reviews @mridulm @HyukjinKwon @gengliangwang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] SparkQA commented on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

SparkQA commented on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918813479 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47740/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33958: [SPARK-36718][SQL] Only collapse projects if we don't duplicate expensive expressions

SparkQA commented on pull request #33958: URL: https://github.com/apache/spark/pull/33958#issuecomment-918807664 **[Test build #143241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143241/testReport)** for PR 33958 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins removed a comment on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918805690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47739/

[GitHub] [spark] AmplabJenkins commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918805690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47739/ --

[GitHub] [spark] SparkQA commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

SparkQA commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918805662 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47739/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33930: [SPARK-36665][SQL] Add more Not operator simplifications

SparkQA commented on pull request #33930: URL: https://github.com/apache/spark/pull/33930#issuecomment-918804936 **[Test build #143240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143240/testReport)** for PR 33930 at commit

[GitHub] [spark] kazuyukitanimura commented on a change in pull request #33930: [SPARK-36665][SQL] Add more Not operator simplifications

kazuyukitanimura commented on a change in pull request #33930: URL: https://github.com/apache/spark/pull/33930#discussion_r707912025 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BooleanSimplificationSuite.scala ## @@ -330,4 +330,118 @@

[GitHub] [spark] SparkQA commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

SparkQA commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918801950 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47739/ -- This is an automated message from the Apache

[GitHub] [spark] taroplus commented on a change in pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

taroplus commented on a change in pull request #33981: URL: https://github.com/apache/spark/pull/33981#discussion_r707887231 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SchemaPruning.scala ## @@ -65,16 +67,19 @@ object SchemaPruning

[GitHub] [spark] pralabhkumar commented on pull request #33980: [WIP][SPARK-32285]Add PySpark support for nested timestamps with arrow

pralabhkumar commented on pull request #33980: URL: https://github.com/apache/spark/pull/33980#issuecomment-918801328 > @pralabhkumar mind keeping the GitHub PR template as is? (https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE) @HyukjinKwon Have updated

[GitHub] [spark] taroplus commented on a change in pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

taroplus commented on a change in pull request #33981: URL: https://github.com/apache/spark/pull/33981#discussion_r707907989 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -3768,6 +3768,24 @@ class SQLConf extends Serializable with

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins removed a comment on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918798632 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47737/

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918798604 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47737/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

AmplabJenkins commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918798632 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47737/ --

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918797641 **[Test build #143239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143239/testReport)** for PR 33955 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33988: [SPARK-36683][SQL] Add new built-in SQL functions: SEC and CSC

AmplabJenkins commented on pull request #33988: URL: https://github.com/apache/spark/pull/33988#issuecomment-918797307 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33954: [SPARK-36709][PYTHON] Support new syntax for specifying index type and name in pandas API on Spark

AmplabJenkins removed a comment on pull request #33954: URL: https://github.com/apache/spark/pull/33954#issuecomment-918796356 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

AmplabJenkins removed a comment on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918796355 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143237/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

AmplabJenkins removed a comment on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918796353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins removed a comment on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918796339 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47738/

[GitHub] [spark] AmplabJenkins commented on pull request #33954: [SPARK-36709][PYTHON] Support new syntax for specifying index type and name in pandas API on Spark

AmplabJenkins commented on pull request #33954: URL: https://github.com/apache/spark/pull/33954#issuecomment-918796357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

AmplabJenkins commented on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918796355 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143237/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

AmplabJenkins commented on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918796353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918796339 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47738/ --

[GitHub] [spark] SparkQA commented on pull request #33954: [SPARK-36709][PYTHON] Support new syntax for specifying index type and name in pandas API on Spark

SparkQA commented on pull request #33954: URL: https://github.com/apache/spark/pull/33954#issuecomment-918795993 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47736/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918795424 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47737/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

SparkQA commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918794778 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47738/ --

[GitHub] [spark] SparkQA removed a comment on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

SparkQA removed a comment on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918724273 **[Test build #143229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143229/testReport)** for PR 33984 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33954: [SPARK-36709][PYTHON] Support new syntax for specifying index type and name in pandas API on Spark

SparkQA removed a comment on pull request #33954: URL: https://github.com/apache/spark/pull/33954#issuecomment-918774029 **[Test build #143233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143233/testReport)** for PR 33954 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

SparkQA removed a comment on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918783351 **[Test build #143237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143237/testReport)** for PR 33968 at commit

[GitHub] [spark] SparkQA commented on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

SparkQA commented on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918794329 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47735/ -- This is an automated message from the

[GitHub] [spark] c21 closed pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

c21 closed pull request #33987: URL: https://github.com/apache/spark/pull/33987 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] mridulm commented on pull request #33977: [SPARK-36737][BUILD][CORE][SQL][SS] Upgrade Apache commons-io to 2.11.0 and revert change of SPARK-36456

mridulm commented on pull request #33977: URL: https://github.com/apache/spark/pull/33977#issuecomment-918793111 I like the consumer idea - with a default noop consumer which will ignore the message for the case @HeartSaVioR mentioned (when we truely dont care if exception is thrown).

[GitHub] [spark] c21 commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

c21 commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918793045 @mridulm - ah yes, just saw it. Thanks for review and quick fix! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] SparkQA commented on pull request #33954: [SPARK-36709][PYTHON] Support new syntax for specifying index type and name in pandas API on Spark

SparkQA commented on pull request #33954: URL: https://github.com/apache/spark/pull/33954#issuecomment-918792682 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47736/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

SparkQA commented on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918792253 **[Test build #143237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143237/testReport)** for PR 33968 at commit

[GitHub] [spark] SparkQA commented on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

SparkQA commented on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918790903 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47735/ -- This is an automated message from the Apache

[GitHub] [spark] mridulm commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

mridulm commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918790456 @c21 This has been fixed in #33984 Thanks for sending the PR ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] mridulm commented on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

mridulm commented on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918790091 Merged to master/branch-3.2 +CC @gengliangwang Thanks for fixing this @rmcyang ! Thanks for the reviews @HyukjinKwon, @gengliangwang :-) -- This is an automated

[GitHub] [spark] asfgit closed pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

asfgit closed pull request #33984: URL: https://github.com/apache/spark/pull/33984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

SparkQA commented on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918788625 **[Test build #143229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143229/testReport)** for PR 33984 at commit

[GitHub] [spark] SparkQA commented on pull request #33954: [SPARK-36709][PYTHON] Support new syntax for specifying index type and name in pandas API on Spark

SparkQA commented on pull request #33954: URL: https://github.com/apache/spark/pull/33954#issuecomment-918788416 **[Test build #143233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143233/testReport)** for PR 33954 at commit

[GitHub] [spark] dbtsai commented on a change in pull request #33969: [SPARK-36726] Upgrade Parquet to 1.12.1

dbtsai commented on a change in pull request #33969: URL: https://github.com/apache/spark/pull/33969#discussion_r707896485 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala ## @@ -855,6 +855,12 @@ class

[GitHub] [spark] yutoacts opened a new pull request #33988: [SPARK-36683][SQL] Add new built-in SQL functions: SEC and CSC

yutoacts opened a new pull request #33988: URL: https://github.com/apache/spark/pull/33988 ### What changes were proposed in this pull request? Add new built-in SQL functions: secant and cosecant, and add them as Scala and Python functions. ### Why are the changes

[GitHub] [spark] dgd-contributor commented on a change in pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

dgd-contributor commented on a change in pull request #33968: URL: https://github.com/apache/spark/pull/33968#discussion_r707894902 ## File path: python/pyspark/pandas/series.py ## @@ -4548,22 +4548,31 @@ def update(self, other: "Series") -> None: if not

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33972: [SPARK-36734][SQL][BUILD][3.1] Upgrade ORC to 1.5.13

dongjoon-hyun commented on a change in pull request #33972: URL: https://github.com/apache/spark/pull/33972#discussion_r707894517 ## File path: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala ## @@ -693,11 +693,11 @@ class FileBasedDataSourceSuite

[GitHub] [spark] dgd-contributor commented on a change in pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

dgd-contributor commented on a change in pull request #33968: URL: https://github.com/apache/spark/pull/33968#discussion_r707894293 ## File path: python/pyspark/pandas/series.py ## @@ -4548,22 +4548,31 @@ def update(self, other: "Series") -> None: if not

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918783434 **[Test build #143238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143238/testReport)** for PR 33955 at commit

[GitHub] [spark] SparkQA commented on pull request #33968: [SPARK-36722][PYTHON] Fix Series.update with another in same frame

SparkQA commented on pull request #33968: URL: https://github.com/apache/spark/pull/33968#issuecomment-918783351 **[Test build #143237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143237/testReport)** for PR 33968 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

dongjoon-hyun commented on pull request #33981: URL: https://github.com/apache/spark/pull/33981#issuecomment-918782803 cc @sunchao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

SparkQA commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918780253 **[Test build #143236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143236/testReport)** for PR 33974 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

AmplabJenkins removed a comment on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918777299 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47734/

[GitHub] [spark] SparkQA commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

SparkQA commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918777275 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47734/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

AmplabJenkins commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918777299 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47734/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

AmplabJenkins removed a comment on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918374959 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] SparkQA commented on pull request #33974: [SPARK-36716][SQL] Join estimation support ExistenceJoin join type

SparkQA commented on pull request #33974: URL: https://github.com/apache/spark/pull/33974#issuecomment-918775942 **[Test build #143234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143234/testReport)** for PR 33974 at commit

[GitHub] [spark] SparkQA commented on pull request #33955: [SPARK-36702][SQL] ArrayUnion handle duplicated Double.NaN and Float.Nan

SparkQA commented on pull request #33955: URL: https://github.com/apache/spark/pull/33955#issuecomment-918775974 **[Test build #143235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143235/testReport)** for PR 33955 at commit

[GitHub] [spark] taroplus commented on a change in pull request #33981: [SPARK-36733][SQL] Fix a perf issue in SchemaPruning when a struct has many fields

taroplus commented on a change in pull request #33981: URL: https://github.com/apache/spark/pull/33981#discussion_r707887231 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SchemaPruning.scala ## @@ -65,16 +67,19 @@ object SchemaPruning

[GitHub] [spark] SparkQA commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

SparkQA commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918774374 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47734/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33954: [SPARK-36709][PYTHON] Support new syntax for specifying index type and name in pandas API on Spark

SparkQA commented on pull request #33954: URL: https://github.com/apache/spark/pull/33954#issuecomment-918774029 **[Test build #143233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143233/testReport)** for PR 33954 at commit

[GitHub] [spark] SparkQA commented on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

SparkQA commented on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918773921 **[Test build #143232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143232/testReport)** for PR 33984 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

AmplabJenkins removed a comment on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-918772242 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143228/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33969: [SPARK-36726] Upgrade Parquet to 1.12.1

AmplabJenkins removed a comment on pull request #33969: URL: https://github.com/apache/spark/pull/33969#issuecomment-918772427 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143227/

[GitHub] [spark] AmplabJenkins commented on pull request #33969: [SPARK-36726] Upgrade Parquet to 1.12.1

AmplabJenkins commented on pull request #33969: URL: https://github.com/apache/spark/pull/33969#issuecomment-918772427 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143227/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

AmplabJenkins commented on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-918772242 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143228/ -- This

[GitHub] [spark] SparkQA commented on pull request #33969: [SPARK-36726] Upgrade Parquet to 1.12.1

SparkQA commented on pull request #33969: URL: https://github.com/apache/spark/pull/33969#issuecomment-918771741 **[Test build #143227 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143227/testReport)** for PR 33969 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33969: [SPARK-36726] Upgrade Parquet to 1.12.1

SparkQA removed a comment on pull request #33969: URL: https://github.com/apache/spark/pull/33969#issuecomment-918700640 **[Test build #143227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143227/testReport)** for PR 33969 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

SparkQA removed a comment on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-918701216 **[Test build #143228 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143228/testReport)** for PR 33432 at commit

[GitHub] [spark] SparkQA commented on pull request #33432: [SPARK-32709][SQL] Support writing Hive bucketed table (Parquet/ORC format with Hive hash)

SparkQA commented on pull request #33432: URL: https://github.com/apache/spark/pull/33432#issuecomment-918765386 **[Test build #143228 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143228/testReport)** for PR 33432 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #33873: [SPARK-36624][YARN] In yarn client mode, when ApplicationMaster failed with KILLED/FAILED, driver should exit with code not

AngersZh commented on a change in pull request #33873: URL: https://github.com/apache/spark/pull/33873#discussion_r707878507 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala ## @@ -122,6 +122,13 @@

[GitHub] [spark] rmcyang commented on a change in pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

rmcyang commented on a change in pull request #33984: URL: https://github.com/apache/spark/pull/33984#discussion_r707877884 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -2604,18 +2604,19 @@ private[spark] object Utils extends Logging { * -

[GitHub] [spark] SparkQA commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

SparkQA commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918754464 **[Test build #143231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143231/testReport)** for PR 33987 at commit

[GitHub] [spark] c21 commented on a change in pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

c21 commented on a change in pull request #33987: URL: https://github.com/apache/spark/pull/33987#discussion_r707870619 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -2604,23 +2604,27 @@ private[spark] object Utils extends Logging { * -

[GitHub] [spark] c21 commented on pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

c21 commented on pull request #33987: URL: https://github.com/apache/spark/pull/33987#issuecomment-918754054 cc @gengliangwang and @mridulm, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] c21 opened a new pull request #33987: [SPARK-36705][SHUFFLE][FOLLOWUP] Avoid unnecessary logging for checking push-based shuffle

c21 opened a new pull request #33987: URL: https://github.com/apache/spark/pull/33987 ### What changes were proposed in this pull request? When running unit test for other change after rebasing master, I found the log becomes very noisy with printing a lot of lines of

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33984: [SPARK-36705][FOLLOW-UP] Fix unnecessary logWarning when PUSH_BASED_SHUFFLE_ENABLED is set to false

AmplabJenkins removed a comment on pull request #33984: URL: https://github.com/apache/spark/pull/33984#issuecomment-918753385 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47732/

  1   2   3   4   5   6   7   >