[GitHub] [spark] AmplabJenkins removed a comment on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins removed a comment on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769416155 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134623/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769416155 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134623/ -

[GitHub] [spark] SparkQA commented on pull request #31273: [WIP][Spark-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

SparkQA commented on pull request #31273: URL: https://github.com/apache/spark/pull/31273#issuecomment-769415691 **[Test build #134629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134629/testReport)** for PR 31273 at commit [`bfabe9f`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

SparkQA removed a comment on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769261249 **[Test build #134623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134623/testReport)** for PR 30175 at commit [`fba5f7c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

SparkQA commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769415057 **[Test build #134623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134623/testReport)** for PR 30175 at commit [`fba5f7c`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

AmplabJenkins removed a comment on pull request #16478: URL: https://github.com/apache/spark/pull/16478#issuecomment-765067600 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

AmplabJenkins removed a comment on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769414475 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

AmplabJenkins commented on pull request #16478: URL: https://github.com/apache/spark/pull/16478#issuecomment-769414476 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39213/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

AmplabJenkins commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769414475 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

SparkQA commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769413097 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39215/ ---

[GitHub] [spark] SparkQA commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

SparkQA commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769406990 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39215/ -

[GitHub] [spark] dongjoon-hyun commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

dongjoon-hyun commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-769401995 Merged to master/3.1/3.0/2.4. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] dongjoon-hyun closed pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

dongjoon-hyun closed pull request #31373: URL: https://github.com/apache/spark/pull/31373 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] viirya commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

viirya commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769393929 Got it. Thanks @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [spark] dongjoon-hyun commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

dongjoon-hyun commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-769393624 Thank you so much, @tgravescs ! This is an automated message from the Apache Git Service. To respond to th

[GitHub] [spark] viirya commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

viirya commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r566405558 ## File path: sql/core/src/test/resources/sql-tests/results/group-by-filter.sql.out ## @@ -796,14 +796,16 @@ IN/EXISTS predicate sub-queries can only be us

[GitHub] [spark] dongjoon-hyun commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

dongjoon-hyun commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769391214 Merged to master/3.1 for Apache Spark 3.1.1. cc @HyukjinKwon This is an automated message from the

[GitHub] [spark] dongjoon-hyun closed pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

dongjoon-hyun closed pull request #31386: URL: https://github.com/apache/spark/pull/31386 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

dongjoon-hyun commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769389752 `KUBERNETES_DRIVER_POD_NAME_PREFIX` is technically `spark.kubernetes.driver.resourceNamePrefix` and used for testing only. ``` // For testing only. val KUBERNE

[GitHub] [spark] dongjoon-hyun commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

dongjoon-hyun commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769389171 Thank you, @viirya . Yes, we use the following instead because we have one driver pod. ```scala val KUBERNETES_DRIVER_POD_NAME = ConfigBuilder("spark.k

[GitHub] [spark] dongjoon-hyun commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

dongjoon-hyun commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-769386103 Updated! This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [spark] dongjoon-hyun commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

dongjoon-hyun commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-769384446 Thank you for review, @tgravescs . Yes, it dose. I'll update it. This is an automated message from the Apa

[GitHub] [spark] tgravescs commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

tgravescs commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-769383139 @dongjoon-hyun can you update description with what user sees from this? I think its a bunch of extra log messages and shutdown takes longer? ---

[GitHub] [spark] SparkQA removed a comment on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

SparkQA removed a comment on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769371549 **[Test build #134627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134627/testReport)** for PR 31386 at commit [`c7e1e3c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

SparkQA commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769381511 **[Test build #134627 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134627/testReport)** for PR 31386 at commit [`c7e1e3c`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

SparkQA commented on pull request #16478: URL: https://github.com/apache/spark/pull/16478#issuecomment-769381502 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39213/ ---

[GitHub] [spark] razajafri commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

razajafri commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-769380453 > Another simpler idea is to fix the schema inference: > > https://github.com/apache/spark/blob/3a361cd837eeea5b5c82b0f90f5d1987a8a30328/sql/core/src/main/scala/org/apac

[GitHub] [spark] SparkQA commented on pull request #31387: [WIP][SPARK-34282][SQL][TESTS] Unify v1 and v2 TRUNCATE TABLE tests

SparkQA commented on pull request #31387: URL: https://github.com/apache/spark/pull/31387#issuecomment-769380220 **[Test build #134626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134626/testReport)** for PR 31387 at commit [`ae4389c`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29818: [SPARK-32953][PYTHON] Add Arrow self_destruct support to toPandas

SparkQA commented on pull request #29818: URL: https://github.com/apache/spark/pull/29818#issuecomment-769378767 **[Test build #134628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134628/testReport)** for PR 29818 at commit [`64d0301`](https://github.com

[GitHub] [spark] MaxGekk commented on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

MaxGekk commented on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-769376995 @sunchao @dongjoon-hyun WDYT about the updated comments. This is an automated message from the Apache Git Servic

[GitHub] [spark] dongjoon-hyun commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

dongjoon-hyun commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769375064 Could you review this, @viirya ? This is an automated message from the Apache Git Service. To respond to t

[GitHub] [spark] SparkQA commented on pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

SparkQA commented on pull request #31386: URL: https://github.com/apache/spark/pull/31386#issuecomment-769371549 **[Test build #134627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134627/testReport)** for PR 31386 at commit [`c7e1e3c`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31369: [SPARK-34270][SS] Combine StateStoreMetrics should not override StateStoreCustomMetric

AmplabJenkins removed a comment on pull request #31369: URL: https://github.com/apache/spark/pull/31369#issuecomment-769370157 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39212/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins removed a comment on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769370160 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39211/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

AmplabJenkins removed a comment on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769370153 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134621/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769370160 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39211/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31369: [SPARK-34270][SS] Combine StateStoreMetrics should not override StateStoreCustomMetric

AmplabJenkins commented on pull request #31369: URL: https://github.com/apache/spark/pull/31369#issuecomment-769370157 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39212/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

AmplabJenkins commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769370153 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134621/ -

[GitHub] [spark] imback82 commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

imback82 commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r566323136 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -450,45 +450,40 @@ case class In

[GitHub] [spark] MaxGekk opened a new pull request #31387: [WIP][SPARK-34282][SQL][TESTS] Unify v1 and v2 TRUNCATE TABLE tests

MaxGekk opened a new pull request #31387: URL: https://github.com/apache/spark/pull/31387 ### What changes were proposed in this pull request? 1. Port DS v1 tests from `DDLSuite` to `v1.TruncateTableSuiteBase` and to `v1.TruncateTableSuite`. ### Why are the changes needed? To i

[GitHub] [spark] SparkQA commented on pull request #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

SparkQA commented on pull request #16478: URL: https://github.com/apache/spark/pull/16478#issuecomment-769363529 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39213/ -

[GitHub] [spark] dongjoon-hyun opened a new pull request #31386: [SPARK-34281][K8S] Promote spark.kubernetes.executor.podNamePrefix to the public conf

dongjoon-hyun opened a new pull request #31386: URL: https://github.com/apache/spark/pull/31386 ### What changes were proposed in this pull request? This PR aims to remove `internal()` from `spark.kubernetes.executor.podNamePrefix` in order to make it the configuration public.

[GitHub] [spark] SparkQA removed a comment on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

SparkQA removed a comment on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769176972 **[Test build #134621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134621/testReport)** for PR 31368 at commit [`2587e53`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

SparkQA commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769356754 **[Test build #134621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134621/testReport)** for PR 31368 at commit [`2587e53`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

SparkQA commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769345958 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39211/ ---

[GitHub] [spark] holdenk commented on pull request #31299: [SPARK-32866][K8S] Fix docker cross-build

holdenk commented on pull request #31299: URL: https://github.com/apache/spark/pull/31299#issuecomment-769343025 Merged to the current dev branch and 3.1. This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] asfgit closed pull request #31299: [SPARK-32866][K8S] Fix docker cross-build

asfgit closed pull request #31299: URL: https://github.com/apache/spark/pull/31299 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] SparkQA commented on pull request #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

SparkQA commented on pull request #16478: URL: https://github.com/apache/spark/pull/16478#issuecomment-769333782 **[Test build #134625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134625/testReport)** for PR 16478 at commit [`72414c6`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31369: [SPARK-34270][SS] Combine StateStoreMetrics should not override StateStoreCustomMetric

SparkQA commented on pull request #31369: URL: https://github.com/apache/spark/pull/31369#issuecomment-769333460 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39212/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

AmplabJenkins removed a comment on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-769323374 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39210/

[GitHub] [spark] AmplabJenkins commented on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

AmplabJenkins commented on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-769323374 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39210/ -

[GitHub] [spark] SparkQA commented on pull request #31369: [SPARK-34270][SS] Combine StateStoreMetrics should not override StateStoreCustomMetric

SparkQA commented on pull request #31369: URL: https://github.com/apache/spark/pull/31369#issuecomment-769315008 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39212/ -

[GitHub] [spark] SparkQA commented on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

SparkQA commented on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-769297804 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39210/ ---

[GitHub] [spark] SparkQA commented on pull request #31369: [SPARK-34270][SS] Combine StateStoreMetrics should not override StateStoreCustomMetric

SparkQA commented on pull request #31369: URL: https://github.com/apache/spark/pull/31369#issuecomment-769297293 **[Test build #134624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134624/testReport)** for PR 31369 at commit [`9ac7253`](https://github.com

[GitHub] [spark] srowen commented on pull request #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

srowen commented on pull request #16478: URL: https://github.com/apache/spark/pull/16478#issuecomment-769295348 Jenkins retest this please This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31384: [SPARK-31816][SQL] Added high level description about JDBC connection providers for users/developers

AmplabJenkins removed a comment on pull request #31384: URL: https://github.com/apache/spark/pull/31384#issuecomment-769289260 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134617/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31384: [SPARK-31816][SQL] Added high level description about JDBC connection providers for users/developers

AmplabJenkins commented on pull request #31384: URL: https://github.com/apache/spark/pull/31384#issuecomment-769289260 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134617/ -

[GitHub] [spark] SparkQA commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

SparkQA commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769288691 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39211/ -

[GitHub] [spark] SparkQA commented on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

SparkQA commented on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-769287903 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39210/ -

[GitHub] [spark] viirya commented on a change in pull request #31369: [SPARK-34270][SS] Combine StateStoreMetrics should not override StateStoreCustomMetric

viirya commented on a change in pull request #31369: URL: https://github.com/apache/spark/pull/31369#discussion_r566308008 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala ## @@ -165,10 +165,18 @@ case class StateStoreMetric

[GitHub] [spark] viirya commented on a change in pull request #31369: [SPARK-34270][SS] Combine StateStoreMetrics should not override StateStoreCustomMetric

viirya commented on a change in pull request #31369: URL: https://github.com/apache/spark/pull/31369#discussion_r566304481 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala ## @@ -165,10 +165,18 @@ case class StateStoreMetric

[GitHub] [spark] SparkQA removed a comment on pull request #31384: [SPARK-31816][SQL] Added high level description about JDBC connection providers for users/developers

SparkQA removed a comment on pull request #31384: URL: https://github.com/apache/spark/pull/31384#issuecomment-769063086 **[Test build #134617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134617/testReport)** for PR 31384 at commit [`7115245`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31384: [SPARK-31816][SQL] Added high level description about JDBC connection providers for users/developers

SparkQA commented on pull request #31384: URL: https://github.com/apache/spark/pull/31384#issuecomment-769265863 **[Test build #134617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134617/testReport)** for PR 31384 at commit [`7115245`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

dongjoon-hyun commented on pull request #31373: URL: https://github.com/apache/spark/pull/31373#issuecomment-769263121 Also, cc @mridulm and @tgravescs This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

dongjoon-hyun commented on a change in pull request #31373: URL: https://github.com/apache/spark/pull/31373#discussion_r566277434 ## File path: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ## @@ -128,14 +128,16 @@ private[spark] class HeartbeatReceiver(sc: Spark

[GitHub] [spark] SparkQA commented on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

SparkQA commented on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-769261592 **[Test build #134622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134622/testReport)** for PR 31364 at commit [`92d9b12`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

SparkQA commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769261249 **[Test build #134623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134623/testReport)** for PR 30175 at commit [`fba5f7c`](https://github.com

[GitHub] [spark] c21 commented on pull request #31353: [SPARK-34253][SQL] Object hash aggregate should not fallback if no more input rows

c21 commented on pull request #31353: URL: https://github.com/apache/spark/pull/31353#issuecomment-769258569 Thank you @maropu and @cloud-fan for review! This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] c21 commented on pull request #31353: [SPARK-34253][SQL] Object hash aggregate should not fallback if no more input rows

c21 commented on pull request #31353: URL: https://github.com/apache/spark/pull/31353#issuecomment-769258360 > btw, '0' above is just a typo? @maropu - It was meant to be a link, I was typing `[0]` to give a reference link. Sorry for confusion. -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

AmplabJenkins removed a comment on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769252154 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39209/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

AmplabJenkins removed a comment on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-769252147 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

AmplabJenkins commented on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-769252147 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

AmplabJenkins commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769252154 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39209/ -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31373: [SPARK-34273][CORE] Do not reregister BlockManager when SparkContext is stopped

dongjoon-hyun commented on a change in pull request #31373: URL: https://github.com/apache/spark/pull/31373#discussion_r566277434 ## File path: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ## @@ -128,14 +128,16 @@ private[spark] class HeartbeatReceiver(sc: Spark

[GitHub] [spark] MaxGekk commented on a change in pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

MaxGekk commented on a change in pull request #31364: URL: https://github.com/apache/spark/pull/31364#discussion_r566272528 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -998,7 +998,26 @@ class SessionCatalog( }

[GitHub] [spark] dongjoon-hyun closed pull request #31383: [SPARK-34278][CORE] Make BlockManagerMaster driver heartbeat timeout configurable

dongjoon-hyun closed pull request #31383: URL: https://github.com/apache/spark/pull/31383 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #31383: [SPARK-34278][CORE] Make BlockManagerMaster driver heartbeat timeout configurable

dongjoon-hyun commented on pull request #31383: URL: https://github.com/apache/spark/pull/31383#issuecomment-769244643 Thank you so much, @HyukjinKwon ! Merged to master for Apache Spark 3.2.0. This is an automated message

[GitHub] [spark] warrenzhu25 commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

warrenzhu25 commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-769236526 > @warrenzhu25 Thanks for the fix! It would be great to write down the RCA about why the job hanging in the description. > (Thanks for pinging me @viirya) Added RCA

[GitHub] [spark] warrenzhu25 commented on a change in pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

warrenzhu25 commented on a change in pull request #30175: URL: https://github.com/apache/spark/pull/30175#discussion_r566263872 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala ## @@ -133,6 +133,13 @@ private[cont

[GitHub] [spark] SparkQA commented on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

SparkQA commented on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-769233302 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39207/ ---

[GitHub] [spark] SparkQA removed a comment on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

SparkQA removed a comment on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-769126081 **[Test build #134619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134619/testReport)** for PR 31367 at commit [`c6a1bdc`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

SparkQA commented on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-769229039 **[Test build #134619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134619/testReport)** for PR 31367 at commit [`c6a1bdc`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

SparkQA commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769217884 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39209/ ---

[GitHub] [spark] cloud-fan edited a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

cloud-fan edited a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-769215659 Another simpler idea is to fix the schema inference: https://github.com/apache/spark/blob/3a361cd837eeea5b5c82b0f90f5d1987a8a30328/sql/core/src/main/scala/org/apa

[GitHub] [spark] cloud-fan edited a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

cloud-fan edited a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-769215659 Another simpler idea is to fix the schema inference: https://github.com/apache/spark/blob/v3.1.1-rc1/sql/core/src/main/scala/org/apache/spark/sql/execution/dataso

[GitHub] [spark] cloud-fan commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

cloud-fan commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-769215659 Another simpler idea is to fix the schema inference: https://github.com/apache/spark/blob/v3.1.1-rc1/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/p

[GitHub] [spark] cloud-fan commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

cloud-fan commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-769210624 Ah I got it now, thanks for the explanation @revans2 ! This is an automated message from the Apache Git Servic

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31385: [WIP][SPARK-34279][BUILD][TEST] Upgrade Junit from 4.x to 5.x

AmplabJenkins removed a comment on pull request #31385: URL: https://github.com/apache/spark/pull/31385#issuecomment-769209777 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134618/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

AmplabJenkins removed a comment on pull request #31207: URL: https://github.com/apache/spark/pull/31207#issuecomment-769209778 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39208/

[GitHub] [spark] AmplabJenkins commented on pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

AmplabJenkins commented on pull request #31207: URL: https://github.com/apache/spark/pull/31207#issuecomment-769209778 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39208/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31385: [WIP][SPARK-34279][BUILD][TEST] Upgrade Junit from 4.x to 5.x

AmplabJenkins commented on pull request #31385: URL: https://github.com/apache/spark/pull/31385#issuecomment-769209777 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134618/ -

[GitHub] [spark] SparkQA commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

SparkQA commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769205659 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39209/ -

[GitHub] [spark] xkrogen commented on pull request #31371: [SPARK-34271][SQL] Use majorMinorPatchVersion for Hive version parsing

xkrogen commented on pull request #31371: URL: https://github.com/apache/spark/pull/31371#issuecomment-769198139 Great enhancement, thanks @sunchao ! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #31207: [SPARK-34136][PYTHON][SQL] Add support for complex literals in PySpark

SparkQA commented on pull request #31207: URL: https://github.com/apache/spark/pull/31207#issuecomment-769188647 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39208/ ---

[GitHub] [spark] SparkQA removed a comment on pull request #31385: [WIP][SPARK-34279][BUILD][TEST] Upgrade Junit from 4.x to 5.x

SparkQA removed a comment on pull request #31385: URL: https://github.com/apache/spark/pull/31385#issuecomment-769060388 **[Test build #134618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134618/testReport)** for PR 31385 at commit [`16c0f74`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31385: [WIP][SPARK-34279][BUILD][TEST] Upgrade Junit from 4.x to 5.x

SparkQA commented on pull request #31385: URL: https://github.com/apache/spark/pull/31385#issuecomment-769183594 **[Test build #134618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134618/testReport)** for PR 31385 at commit [`16c0f74`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

SparkQA commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-769176972 **[Test build #134621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134621/testReport)** for PR 31368 at commit [`2587e53`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

AmplabJenkins removed a comment on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-768958757 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #31367: [SPARK-34265][PYTHON][SQL] Instrument Python UDF using SQL Metrics

SparkQA commented on pull request #31367: URL: https://github.com/apache/spark/pull/31367#issuecomment-769171297 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39207/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31385: [WIP][SPARK-34279][BUILD][TEST] Upgrade Junit from 4.x to 5.x

AmplabJenkins removed a comment on pull request #31385: URL: https://github.com/apache/spark/pull/31385#issuecomment-769168006 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39206/

<    1   2   3   4   5   6   >