[GitHub] [spark] SparkQA commented on pull request #32795: [SPARK-35588][PYTHON][DOCS] Update quickstart.ipynb to use pyspark.pandas

2021-06-07 Thread GitBox
SparkQA commented on pull request #32795: URL: https://github.com/apache/spark/pull/32795#issuecomment-855650093 **[Test build #139402 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139402/testReport)** for PR 32795 at commit [`25f6c0b`](https://github.co

[GitHub] [spark] sarutak commented on a change in pull request #32788: [SPARK-35602][SS] Update state schema to be able to accept long length JSON

2021-06-07 Thread GitBox
sarutak commented on a change in pull request #32788: URL: https://github.com/apache/spark/pull/32788#discussion_r646319681 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/SchemaV2Writer.scala ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apa

[GitHub] [spark] sarutak commented on a change in pull request #32788: [SPARK-35602][SS] Update state schema to be able to accept long length JSON

2021-06-07 Thread GitBox
sarutak commented on a change in pull request #32788: URL: https://github.com/apache/spark/pull/32788#discussion_r646319848 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/SchemaV2Writer.scala ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apa

[GitHub] [spark] SparkQA removed a comment on pull request #32795: [SPARK-35588][PYTHON][DOCS] Update quickstart.ipynb to use pyspark.pandas

2021-06-07 Thread GitBox
SparkQA removed a comment on pull request #32795: URL: https://github.com/apache/spark/pull/32795#issuecomment-855626804 **[Test build #139402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139402/testReport)** for PR 32795 at commit [`25f6c0b`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32800: [SPARK-35661][SQL] Allow deserialized off-heap memory entry

2021-06-07 Thread GitBox
SparkQA commented on pull request #32800: URL: https://github.com/apache/spark/pull/32800#issuecomment-855651129 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43918/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32796: [SPARK-35659][SS] Avoid write null to StateStore

2021-06-07 Thread GitBox
SparkQA commented on pull request #32796: URL: https://github.com/apache/spark/pull/32796#issuecomment-855651821 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43920/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855652226 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43919/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r646323000 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -73,23 +75,55 @@ class QueryExecution( sparkSessio

[GitHub] [spark] SparkQA commented on pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
SparkQA commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-855655333 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43921/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r646324270 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -73,23 +75,55 @@ class QueryExecution( sparkSessio

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r646324416 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -73,23 +75,55 @@ class QueryExecution( sparkSessio

[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
SparkQA commented on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855657149 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43922/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r646324804 ## File path: sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala ## @@ -310,8 +310,9 @@ abstract class BaseSessionStat

[GitHub] [spark] SparkQA commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855658255 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43923/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r646327137 ## File path: sql/core/src/test/scala/org/apache/spark/sql/util/DataFrameCallbackSuite.scala ## @@ -227,9 +227,7 @@ class DataFrameCallbackSuite extend

[GitHub] [spark] xuechendi commented on a change in pull request #32717: [SPARK-35396] Manual close for CachedBatch in InMemoryRelation

2021-06-07 Thread GitBox
xuechendi commented on a change in pull request #32717: URL: https://github.com/apache/spark/pull/32717#discussion_r646327426 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/CachedBatchSerializerSuite.scala ## @@ -143,3 +143,109 @@ class CachedBat

[GitHub] [spark] xuechendi commented on a change in pull request #32717: [SPARK-35396] Manual close for CachedBatch in InMemoryRelation

2021-06-07 Thread GitBox
xuechendi commented on a change in pull request #32717: URL: https://github.com/apache/spark/pull/32717#discussion_r646327672 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/CachedBatchSerializerSuite.scala ## @@ -143,3 +143,109 @@ class CachedBat

[GitHub] [spark] AmplabJenkins commented on pull request #32798: [SPARK-35660][BUILD][K8S] Upgrade kubernetes-client to 5.4.1

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32798: URL: https://github.com/apache/spark/pull/32798#issuecomment-855661770 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139393/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32795: [SPARK-35588][PYTHON][DOCS] Update quickstart.ipynb to use pyspark.pandas

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32795: URL: https://github.com/apache/spark/pull/32795#issuecomment-855661771 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139402/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32795: [SPARK-35588][PYTHON][DOCS] Update quickstart.ipynb to use pyspark.pandas

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32795: URL: https://github.com/apache/spark/pull/32795#issuecomment-855661771 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139402/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32798: [SPARK-35660][BUILD][K8S] Upgrade kubernetes-client to 5.4.1

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32798: URL: https://github.com/apache/spark/pull/32798#issuecomment-855661770 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139393/ -

[GitHub] [spark] SparkQA commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855662889 **[Test build #139403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139403/testReport)** for PR 32797 at commit [`984ab81`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #32717: [SPARK-35396] Manual close for CachedBatch in InMemoryRelation

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32717: URL: https://github.com/apache/spark/pull/32717#discussion_r646329447 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/CachedBatchSerializerSuite.scala ## @@ -143,3 +143,109 @@ class CachedBat

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855630871 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139401/ -

[GitHub] [spark] SparkQA commented on pull request #32795: [SPARK-35588][PYTHON][DOCS] Update quickstart.ipynb to use pyspark.pandas

2021-06-07 Thread GitBox
SparkQA commented on pull request #32795: URL: https://github.com/apache/spark/pull/32795#issuecomment-855663878 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43924/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32795: [SPARK-35588][PYTHON][DOCS] Update quickstart.ipynb to use pyspark.pandas

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32795: URL: https://github.com/apache/spark/pull/32795#issuecomment-855663913 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43924/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32795: [SPARK-35588][PYTHON][DOCS] Update quickstart.ipynb to use pyspark.pandas

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32795: URL: https://github.com/apache/spark/pull/32795#issuecomment-855663913 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43924/

[GitHub] [spark] beliefer commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
beliefer commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r646333537 ## File path: sql/core/src/test/scala/org/apache/spark/sql/util/DataFrameCallbackSuite.scala ## @@ -227,9 +227,7 @@ class DataFrameCallbackSuite extends

[GitHub] [spark] AmplabJenkins commented on pull request #32800: [SPARK-35661][SQL] Allow deserialized off-heap memory entry

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32800: URL: https://github.com/apache/spark/pull/32800#issuecomment-855675429 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43918/ -- T

[GitHub] [spark] SparkQA commented on pull request #32800: [SPARK-35661][SQL] Allow deserialized off-heap memory entry

2021-06-07 Thread GitBox
SparkQA commented on pull request #32800: URL: https://github.com/apache/spark/pull/32800#issuecomment-855675388 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43918/ -- This is an automated message from the A

[GitHub] [spark] HyukjinKwon commented on pull request #32799: [SPARK-35646][PYTHON][DOCS] Relocate pandas-on-Spark API references in documentation

2021-06-07 Thread GitBox
HyukjinKwon commented on pull request #32799: URL: https://github.com/apache/spark/pull/32799#issuecomment-855677925 Thanks @viirya!!! I will just merge this in to master. I am pretty confident of this change too -- This is an automated message from the Apache Git Service. To respo

[GitHub] [spark] HyukjinKwon commented on pull request #32799: [SPARK-35646][PYTHON][DOCS] Relocate pandas-on-Spark API references in documentation

2021-06-07 Thread GitBox
HyukjinKwon commented on pull request #32799: URL: https://github.com/apache/spark/pull/32799#issuecomment-855678080 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] SparkQA commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855678164 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43919/ -- This is an automated message from the A

[GitHub] [spark] HyukjinKwon closed pull request #32799: [SPARK-35646][PYTHON][DOCS] Relocate pandas-on-Spark API references in documentation

2021-06-07 Thread GitBox
HyukjinKwon closed pull request #32799: URL: https://github.com/apache/spark/pull/32799 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pl

[GitHub] [spark] gengliangwang opened a new pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
gengliangwang opened a new pull request #32802: URL: https://github.com/apache/spark/pull/32802 ### What changes were proposed in this pull request? Extend Catalyst's type system by a new type that conforms to the SQL standard (see SQL:2016, section 4.6.2): TimestampNTZType r

[GitHub] [spark] SparkQA commented on pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
SparkQA commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-855681148 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43921/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
SparkQA commented on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855682806 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43922/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855683358 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43923/ -- This is an automated message from the A

[GitHub] [spark] attilapiros commented on pull request #32790: [SPARK-35543][CORE] Fix memory leak in BlockManagerMasterEndpoint removeRdd

2021-06-07 Thread GitBox
attilapiros commented on pull request #32790: URL: https://github.com/apache/spark/pull/32790#issuecomment-855688142 > However, looking at removeBlock, it shows it only cleans rdd blocks indeed. So it makes me think whether we missed other block types. Fortunately, we only have broadcast b

[GitHub] [spark] xuechendi commented on a change in pull request #32717: [SPARK-35396] Manual close for CachedBatch in InMemoryRelation

2021-06-07 Thread GitBox
xuechendi commented on a change in pull request #32717: URL: https://github.com/apache/spark/pull/32717#discussion_r646351018 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/CachedBatchSerializerSuite.scala ## @@ -143,3 +143,109 @@ class CachedBat

[GitHub] [spark] viirya commented on pull request #32799: [SPARK-35646][PYTHON][DOCS] Relocate pandas-on-Spark API references in documentation

2021-06-07 Thread GitBox
viirya commented on pull request #32799: URL: https://github.com/apache/spark/pull/32799#issuecomment-855690566 yea, lgtm. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] cloud-fan commented on a change in pull request #32563: [SPARK-35415][SQL] Change `information` to map type for SHOW TABLE EXTENDED command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32563: URL: https://github.com/apache/spark/pull/32563#discussion_r646352358 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala ## @@ -419,7 +419,7 @@ case class CatalogTable( ma

[GitHub] [spark] SparkQA commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855694000 **[Test build #139403 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139403/testReport)** for PR 32797 at commit [`984ab81`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA removed a comment on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855662889 **[Test build #139403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139403/testReport)** for PR 32797 at commit [`984ab81`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
SparkQA commented on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855695504 **[Test build #139400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139400/testReport)** for PR 32801 at commit [`eb7d86f`](https://github.co

[GitHub] [spark] ulysses-you commented on pull request #32084: [SPARK-34980][SQL] Support coalesce partition through union in AQE

2021-06-07 Thread GitBox
ulysses-you commented on pull request #32084: URL: https://github.com/apache/spark/pull/32084#issuecomment-855695488 refactor the PR, do you have time to take a look ? cc @maropu @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] SparkQA removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
SparkQA removed a comment on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855623854 **[Test build #139400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139400/testReport)** for PR 32801 at commit [`eb7d86f`](https://gi

[GitHub] [spark] cloud-fan commented on a change in pull request #32563: [SPARK-35415][SQL] Change `information` to map type for SHOW TABLE EXTENDED command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32563: URL: https://github.com/apache/spark/pull/32563#discussion_r646356610 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala ## @@ -419,7 +419,7 @@ case class CatalogTable( ma

[GitHub] [spark] cloud-fan commented on a change in pull request #32563: [SPARK-35415][SQL] Change `information` to map type for SHOW TABLE EXTENDED command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32563: URL: https://github.com/apache/spark/pull/32563#discussion_r646357632 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -612,7 +612,7 @@ object ShowTableExtended

[GitHub] [spark] attilapiros commented on pull request #32790: [SPARK-35543][CORE] Fix memory leak in BlockManagerMasterEndpoint removeRdd

2021-06-07 Thread GitBox
attilapiros commented on pull request #32790: URL: https://github.com/apache/spark/pull/32790#issuecomment-855698939 K8s failures are unrelated: https://github.com/apache/spark/pull/32790#issuecomment-855590776 -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [spark] cloud-fan commented on a change in pull request #32563: [SPARK-35415][SQL] Change `information` to map type for SHOW TABLE EXTENDED command

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32563: URL: https://github.com/apache/spark/pull/32563#discussion_r646360555 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -612,7 +612,7 @@ object ShowTableExtended

[GitHub] [spark] attilapiros edited a comment on pull request #32790: [SPARK-35543][CORE] Fix memory leak in BlockManagerMasterEndpoint removeRdd

2021-06-07 Thread GitBox
attilapiros edited a comment on pull request #32790: URL: https://github.com/apache/spark/pull/32790#issuecomment-855688142 > However, looking at removeBlock, it shows it only cleans rdd blocks indeed. So it makes me think whether we missed other block types. Fortunately, we only have broa

[GitHub] [spark] cloud-fan commented on a change in pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32802: URL: https://github.com/apache/spark/pull/32802#discussion_r646363331 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimestampNTZType.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] xuechendi commented on pull request #32717: [SPARK-35396] Add a UT to check if a user-defined cachedBatch is completely released

2021-06-07 Thread GitBox
xuechendi commented on pull request #32717: URL: https://github.com/apache/spark/pull/32717#issuecomment-855704942 > LGTM. Please update PR title and description. Updated, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [spark] SparkQA commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855705008 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43925/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
cloud-fan commented on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855706794 cc @viirya @maropu @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] tanelk commented on pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

2021-06-07 Thread GitBox
tanelk commented on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-855709131 @ulysses-you I ran git-bisect and it seems, that this change introduced a regression: ``` test("SPARK-X: observe") { spark.range(5).observe("idSum", sum(

[GitHub] [spark] Ngone51 commented on pull request #32790: [SPARK-35543][CORE] Fix memory leak in BlockManagerMasterEndpoint removeRdd

2021-06-07 Thread GitBox
Ngone51 commented on pull request #32790: URL: https://github.com/apache/spark/pull/32790#issuecomment-855709345 > Why are you sure about that? Just looking only these lines? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [spark] AmplabJenkins commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855710275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For q

[GitHub] [spark] AmplabJenkins commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855710276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For q

[GitHub] [spark] AmplabJenkins commented on pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-855710284 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43921/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855710275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855710276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-855710284 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43921/

[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
SparkQA commented on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855713227 **[Test build #139405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139405/testReport)** for PR 32801 at commit [`b65326c`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32788: [SPARK-35602][SS] Update state schema to be able to accept long length JSON

2021-06-07 Thread GitBox
SparkQA commented on pull request #32788: URL: https://github.com/apache/spark/pull/32788#issuecomment-855713199 **[Test build #139406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139406/testReport)** for PR 32788 at commit [`e1f4b68`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
SparkQA commented on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855713145 **[Test build #139404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139404/testReport)** for PR 32802 at commit [`f6fedd0`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32084: [SPARK-34980][SQL] Support coalesce partition through union in AQE

2021-06-07 Thread GitBox
SparkQA commented on pull request #32084: URL: https://github.com/apache/spark/pull/32084#issuecomment-855714028 **[Test build #139407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139407/testReport)** for PR 32084 at commit [`797ea59`](https://github.com

[GitHub] [spark] Ngone51 edited a comment on pull request #32790: [SPARK-35543][CORE] Fix memory leak in BlockManagerMasterEndpoint removeRdd

2021-06-07 Thread GitBox
Ngone51 edited a comment on pull request #32790: URL: https://github.com/apache/spark/pull/32790#issuecomment-855709345 > Why are you sure about that? Just looking only these lines? It's only called by `BlockManagerMasterEndpoint.removeRdd() -> removeBlock()` in production code path,

[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
SparkQA commented on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855716549 **[Test build #139408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139408/testReport)** for PR 32801 at commit [`748`](https://github.com

[GitHub] [spark] Ngone51 edited a comment on pull request #32790: [SPARK-35543][CORE] Fix memory leak in BlockManagerMasterEndpoint removeRdd

2021-06-07 Thread GitBox
Ngone51 edited a comment on pull request #32790: URL: https://github.com/apache/spark/pull/32790#issuecomment-855709345 > Why are you sure about that? Just looking only these lines? It's only called by `BlockManagerMasterEndpoint.removeRdd() -> removeBlock()` in production code path,

[GitHub] [spark] SparkQA commented on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
SparkQA commented on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855718703 **[Test build #139404 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139404/testReport)** for PR 32802 at commit [`f6fedd0`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855718752 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139404/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855718752 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139404/ -

[GitHub] [spark] SparkQA removed a comment on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
SparkQA removed a comment on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855713145 **[Test build #139404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139404/testReport)** for PR 32802 at commit [`f6fedd0`](https://gi

[GitHub] [spark] Ngone51 edited a comment on pull request #32790: [SPARK-35543][CORE] Fix memory leak in BlockManagerMasterEndpoint removeRdd

2021-06-07 Thread GitBox
Ngone51 edited a comment on pull request #32790: URL: https://github.com/apache/spark/pull/32790#issuecomment-855709345 > Why are you sure about that? Just looking only these lines? It's only called by `BlockManagerMasterEndpoint.removeRdd() -> removeBlock()` in production code path,

[GitHub] [spark] gengliangwang commented on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
gengliangwang commented on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855722367 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [spark] SparkQA commented on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
SparkQA commented on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855723723 **[Test build #139409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139409/testReport)** for PR 32802 at commit [`f6fedd0`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32796: [SPARK-35659][SS] Avoid write null to StateStore

2021-06-07 Thread GitBox
AmplabJenkins commented on pull request #32796: URL: https://github.com/apache/spark/pull/32796#issuecomment-855724121 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43920/ -- T

[GitHub] [spark] SparkQA commented on pull request #32796: [SPARK-35659][SS] Avoid write null to StateStore

2021-06-07 Thread GitBox
SparkQA commented on pull request #32796: URL: https://github.com/apache/spark/pull/32796#issuecomment-855724081 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43920/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32796: [SPARK-35659][SS] Avoid write null to StateStore

2021-06-07 Thread GitBox
AmplabJenkins removed a comment on pull request #32796: URL: https://github.com/apache/spark/pull/32796#issuecomment-855724121 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43920/

[GitHub] [spark] MaxGekk commented on a change in pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
MaxGekk commented on a change in pull request #32802: URL: https://github.com/apache/spark/pull/32802#discussion_r646381329 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimestampNTZType.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software F

[GitHub] [spark] MaxGekk commented on a change in pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
MaxGekk commented on a change in pull request #32802: URL: https://github.com/apache/spark/pull/32802#discussion_r646381329 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimestampNTZType.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software F

[GitHub] [spark] Ngone51 commented on pull request #32790: [SPARK-35543][CORE] Fix memory leak in BlockManagerMasterEndpoint removeRdd

2021-06-07 Thread GitBox
Ngone51 commented on pull request #32790: URL: https://github.com/apache/spark/pull/32790#issuecomment-855730291 Ok, l see there's another type of block - `TaskResultBlockId`. So it looks good!! -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] SparkQA commented on pull request #32797: [SPARK-35603][R][DOCS] Add data source options link for R API documentation.

2021-06-07 Thread GitBox
SparkQA commented on pull request #32797: URL: https://github.com/apache/spark/pull/32797#issuecomment-855736902 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43925/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32802: URL: https://github.com/apache/spark/pull/32802#discussion_r646390414 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimestampNTZType.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] SparkQA commented on pull request #32800: [SPARK-35661][SQL] Allow deserialized off-heap memory entry

2021-06-07 Thread GitBox
SparkQA commented on pull request #32800: URL: https://github.com/apache/spark/pull/32800#issuecomment-855740358 **[Test build #139396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139396/testReport)** for PR 32800 at commit [`55a51be`](https://github.co

[GitHub] [spark] cloud-fan commented on a change in pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32802: URL: https://github.com/apache/spark/pull/32802#discussion_r646392917 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimestampNTZType.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] SparkQA removed a comment on pull request #32800: [SPARK-35661][SQL] Allow deserialized off-heap memory entry

2021-06-07 Thread GitBox
SparkQA removed a comment on pull request #32800: URL: https://github.com/apache/spark/pull/32800#issuecomment-855618471 **[Test build #139396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139396/testReport)** for PR 32800 at commit [`55a51be`](https://gi

[GitHub] [spark] gengliangwang commented on a change in pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
gengliangwang commented on a change in pull request #32802: URL: https://github.com/apache/spark/pull/32802#discussion_r646396900 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimestampNTZType.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Soft

[GitHub] [spark] SparkQA commented on pull request #32788: [SPARK-35602][SS] Update state schema to be able to accept long length JSON

2021-06-07 Thread GitBox
SparkQA commented on pull request #32788: URL: https://github.com/apache/spark/pull/32788#issuecomment-855747193 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43928/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
SparkQA commented on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855748503 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43927/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32802: [SPARK-35663][SQL] Add Timestamp without time zone type

2021-06-07 Thread GitBox
SparkQA commented on pull request #32802: URL: https://github.com/apache/spark/pull/32802#issuecomment-855748925 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43926/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan opened a new pull request #32803: [SPARK-35665][SQL] Resolve UnresolvedAlias in CollectMetrics

2021-06-07 Thread GitBox
cloud-fan opened a new pull request #32803: URL: https://github.com/apache/spark/pull/32803 ### What changes were proposed in this pull request? It's a long-standing bug that we forgot to resolve `UnresolvedAlias` in `CollectMetrics`. It's a bit hard to trigger this bug befor

[GitHub] [spark] cloud-fan commented on pull request #32803: [SPARK-35665][SQL] Resolve UnresolvedAlias in CollectMetrics

2021-06-07 Thread GitBox
cloud-fan commented on pull request #32803: URL: https://github.com/apache/spark/pull/32803#issuecomment-855749560 cc @ulysses-you @yaooqinn @tanelk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [spark] cloud-fan commented on pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

2021-06-07 Thread GitBox
cloud-fan commented on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-855749664 fixing in https://github.com/apache/spark/pull/32803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] SparkQA commented on pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
SparkQA commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-855751621 **[Test build #139399 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139399/testReport)** for PR 32513 at commit [`35ea747`](https://github.co

[GitHub] [spark] cloud-fan commented on a change in pull request #32781: [SPARK-35650][SQL] Coalesce partitions through AQE

2021-06-07 Thread GitBox
cloud-fan commented on a change in pull request #32781: URL: https://github.com/apache/spark/pull/32781#discussion_r646403528 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -1351,6 +1351,45 @@ object Repa

[GitHub] [spark] SparkQA commented on pull request #32801: [SPARK-12567][SQL] Add aes_encrypt and aes_decrypt builtin functions

2021-06-07 Thread GitBox
SparkQA commented on pull request #32801: URL: https://github.com/apache/spark/pull/32801#issuecomment-855751758 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43930/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command

2021-06-07 Thread GitBox
SparkQA removed a comment on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-855618807 **[Test build #139399 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139399/testReport)** for PR 32513 at commit [`35ea747`](https://gi

  1   2   3   4   5   6   7   8   >