[GitHub] [spark] HyukjinKwon commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

HyukjinKwon commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754802449 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32,

[GitHub] [spark] SparkQA commented on pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-976165361 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50003/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

SparkQA commented on pull request #34687: URL: https://github.com/apache/spark/pull/34687#issuecomment-976164822 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50001/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34677: [SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976164799 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50002/ -- This is an automated message from the Apache

[GitHub] [spark] imback82 commented on pull request #34686: [SPARK-37444][SQL] ALTER NAMESPACE ... SET LOCATION should handle empty location consistently across v1 and v2 command

imback82 commented on pull request #34686: URL: https://github.com/apache/spark/pull/34686#issuecomment-976163956 cc: @cloud-fan (GA test failure is from `org.apache.spark.sql.jdbc.DB2KrbIntegrationSuite`) -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976163822 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49998/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

SparkQA removed a comment on pull request #34687: URL: https://github.com/apache/spark/pull/34687#issuecomment-976151540 **[Test build #145532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145532/testReport)** for PR 34687 at commit

[GitHub] [spark] SparkQA commented on pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

SparkQA commented on pull request #34687: URL: https://github.com/apache/spark/pull/34687#issuecomment-976162628 **[Test build #145532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145532/testReport)** for PR 34687 at commit

[GitHub] [spark] sadikovi commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

sadikovi commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r754797714 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java ## @@ -53,19 +53,47 @@ public

[GitHub] [spark] sadikovi commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

sadikovi commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r754796220 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala ## @@ -130,6 +133,89 @@ class

[GitHub] [spark] SparkQA removed a comment on pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

SparkQA removed a comment on pull request #34687: URL: https://github.com/apache/spark/pull/34687#issuecomment-976149259 **[Test build #145529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145529/testReport)** for PR 34687 at commit

[GitHub] [spark] SparkQA commented on pull request #34686: [SPARK-37444][SQL] ALTER NAMESPACE ... SET LOCATION should handle empty location consistently across v1 and v2 command

SparkQA commented on pull request #34686: URL: https://github.com/apache/spark/pull/34686#issuecomment-976158536 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49997/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

SparkQA commented on pull request #34687: URL: https://github.com/apache/spark/pull/34687#issuecomment-976158555 **[Test build #145529 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145529/testReport)** for PR 34687 at commit

[GitHub] [spark] sadikovi commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

sadikovi commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r754795330 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java ## @@ -53,19 +53,47 @@ public

[GitHub] [spark] sadikovi commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

sadikovi commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r754794034 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java ## @@ -53,19 +53,47 @@ public

[GitHub] [spark] sadikovi commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

sadikovi commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r754794034 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java ## @@ -53,19 +53,47 @@ public

[GitHub] [spark] Yikun commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754789113 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] Yikun commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754789113 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] SparkQA commented on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

SparkQA commented on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-976151920 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/5/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

SparkQA commented on pull request #34687: URL: https://github.com/apache/spark/pull/34687#issuecomment-976151540 **[Test build #145532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145532/testReport)** for PR 34687 at commit

[GitHub] [spark] sadikovi commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

sadikovi commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r754789534 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala ## @@ -130,6 +133,89 @@ class

[GitHub] [spark] Yikun commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754789113 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] Yikun commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754789113 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] Yikun commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754789113 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] SparkQA commented on pull request #34677: [SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976149296 **[Test build #145530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145530/testReport)** for PR 34677 at commit

[GitHub] [spark] SparkQA commented on pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-976149315 **[Test build #145531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145531/testReport)** for PR 34676 at commit

[GitHub] [spark] SparkQA commented on pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

SparkQA commented on pull request #34687: URL: https://github.com/apache/spark/pull/34687#issuecomment-976149259 **[Test build #145529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145529/testReport)** for PR 34687 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-976149043 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145521/

[GitHub] [spark] LuciferYang commented on a change in pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

LuciferYang commented on a change in pull request #34676: URL: https://github.com/apache/spark/pull/34676#discussion_r754788216 ## File path: pom.xml ## @@ -3580,6 +3580,18 @@ + + mac-on-apple-silicon + +

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

AmplabJenkins removed a comment on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-976149044 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49996/

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-976149043 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145521/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

AmplabJenkins commented on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-976149044 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49996/ --

[GitHub] [spark] SparkQA commented on pull request #34677: [SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976148717 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/4/ -- This is an automated message from the Apache

[GitHub] [spark] LuciferYang edited a comment on pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

LuciferYang edited a comment on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-976148375 @srowen you are right, f87467b adds code corresponding to sbt. There are 3 modules with `Extended{LevelDB,RocksDB}Test` annotation: `core`,`sql/core`, `yarn`

[GitHub] [spark] LuciferYang commented on pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

LuciferYang commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-976148375 @srowen you are right, f87467b adds code corresponding to sbt. There are 3 modules with `Extended{LevelDB,RocksDB}Test` annotation: `core`,`sql/core`, `yarn`

[GitHub] [spark] Yikun commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754782336 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] SparkQA removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975988587 **[Test build #145521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145521/testReport)** for PR 34596 at commit

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-976142458 **[Test build #145521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145521/testReport)** for PR 34596 at commit

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976141333 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49998/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34686: [SPARK-37444][SQL] ALTER NAMESPACE ... SET LOCATION should handle empty location consistently across v1 and v2 command

SparkQA commented on pull request #34686: URL: https://github.com/apache/spark/pull/34686#issuecomment-976141215 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49997/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

SparkQA commented on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-97610 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49996/ -- This is an automated message from the

[GitHub] [spark] Yikun commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754782336 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] pralabhkumar commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

pralabhkumar commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-976132094 Thx @HyukjinKwon for helping out during the process -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

HyukjinKwon commented on a change in pull request #34687: URL: https://github.com/apache/spark/pull/34687#discussion_r754779946 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,15 @@ def numeric_pdf(self): dtypes = [np.int32,

[GitHub] [spark] Yikun commented on a change in pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34314: URL: https://github.com/apache/spark/pull/34314#discussion_r754778929 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,14 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] HyukjinKwon commented on pull request #34677: [SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

HyukjinKwon commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976126088 This is ready for a review. cc @ueshin @xinrong-databricks @itholic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] Yikun opened a new pull request #34687: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun opened a new pull request #34687: URL: https://github.com/apache/spark/pull/34687 ### What changes were proposed in this pull request? This patch has changes as below to follow the pandas behavior: - **Add nan value process in _non_fractional_astype**: Follow the pandas

[GitHub] [spark] SparkQA commented on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

SparkQA commented on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-976120474 **[Test build #145528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145528/testReport)** for PR 34681 at commit

[GitHub] [spark] Yikun commented on a change in pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on a change in pull request #34314: URL: https://github.com/apache/spark/pull/34314#discussion_r754774559 ## File path: python/pyspark/pandas/tests/data_type_ops/testing_utils.py ## @@ -49,8 +49,14 @@ def numeric_pdf(self): dtypes = [np.int32, int,

[GitHub] [spark] SparkQA removed a comment on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA removed a comment on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976116467 **[Test build #145527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145527/testReport)** for PR 34677 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

AmplabJenkins removed a comment on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976116782 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145527/

[GitHub] [spark] AmplabJenkins commented on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

AmplabJenkins commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976116782 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145527/ -- This

[GitHub] [spark] SparkQA commented on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976116771 **[Test build #145527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145527/testReport)** for PR 34677 at commit

[GitHub] [spark] SparkQA commented on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-976116467 **[Test build #145527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145527/testReport)** for PR 34677 at commit

[GitHub] [spark] SparkQA commented on pull request #34686: [SPARK-37444][SQL] ALTER NAMESPACE ... SET LOCATION should handle empty location consistently across v1 and v2 command

SparkQA commented on pull request #34686: URL: https://github.com/apache/spark/pull/34686#issuecomment-976116428 **[Test build #145525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145525/testReport)** for PR 34686 at commit

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976116444 **[Test build #145526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145526/testReport)** for PR 34685 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

AmplabJenkins removed a comment on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976115852 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49995/

[GitHub] [spark] AmplabJenkins commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

AmplabJenkins commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976115852 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49995/ --

[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

SparkQA commented on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-976111313 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49996/ -- This is an automated message from the Apache

[GitHub] [spark] imback82 opened a new pull request #34686: [SPARK-37444][SQL] ALTER NAMESPACE ... SET LOCATION should handle empty location consistently across v1 and v2 command

imback82 opened a new pull request #34686: URL: https://github.com/apache/spark/pull/34686 ### What changes were proposed in this pull request? Currently, there is an inconsistency when handling an empty location for `ALTER NAMESPACE .. SET LOCATION` between v1 and v2

[GitHub] [spark] imback82 commented on a change in pull request #34610: [SPARK-34332][SQL][TEST] Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

imback82 commented on a change in pull request #34610: URL: https://github.com/apache/spark/pull/34610#discussion_r754764231 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/AlterNamespaceSetLocationSuiteBase.scala ## @@ -0,0 +1,74 @@ +/* + *

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976105603 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49995/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

AmplabJenkins removed a comment on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976100517 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145522/

[GitHub] [spark] AmplabJenkins commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

AmplabJenkins commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976100517 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145522/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA removed a comment on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976015951 **[Test build #145522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145522/testReport)** for PR 34685 at commit

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976100418 **[Test build #145522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145522/testReport)** for PR 34685 at commit

[GitHub] [spark] xuechendi commented on pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

xuechendi commented on pull request #34396: URL: https://github.com/apache/spark/pull/34396#issuecomment-976099176 @BryanCutler and @HyukjinKwon, Please help me take a review, I updated this PR per our discussion last week Here is my last update I moved all arrow related conversions

[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

SparkQA commented on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-976098359 **[Test build #145524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145524/testReport)** for PR 34647 at commit

[GitHub] [spark] beliefer commented on a change in pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

beliefer commented on a change in pull request #34683: URL: https://github.com/apache/spark/pull/34683#discussion_r754756800 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ## @@ -1411,6 +1411,7 @@ object HiveExternalCatalog {

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

AmplabJenkins removed a comment on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976097279 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49994/

[GitHub] [spark] AmplabJenkins commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

AmplabJenkins commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976097279 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49994/ --

[GitHub] [spark] beliefer commented on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

beliefer commented on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-976097094 > cc @beliefer Thank you for ping me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] beliefer commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

beliefer commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754756036 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala ## @@ -1002,15 +1002,19 @@ private[hive] class

[GitHub] [spark] HyukjinKwon commented on pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

HyukjinKwon commented on pull request #34314: URL: https://github.com/apache/spark/pull/34314#issuecomment-976093250 yeah, that is fine because it already doesn't work with 1.2 and 1.1 and no regression. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976092443 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49995/ -- This is an automated message from the Apache

[GitHub] [spark] Yikun edited a comment on pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun edited a comment on pull request #34314: URL: https://github.com/apache/spark/pull/34314#issuecomment-976092247 > if you're stuck to support this with old pandas versions @HyukjinKwon I do some simple test yesterday, there are many test case failed with decimal("Nan") in v1.2

[GitHub] [spark] Yikun commented on pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on pull request #34314: URL: https://github.com/apache/spark/pull/34314#issuecomment-976092247 > if you're stuck to support this with old pandas versions @HyukjinKwon I do some simple test yesterday, there are many test case failed with decimal("Nan") > we can

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976085378 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49994/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

AmplabJenkins removed a comment on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976061043 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145523/

[GitHub] [spark] SparkQA removed a comment on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA removed a comment on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976053975 **[Test build #145523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145523/testReport)** for PR 34685 at commit

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976060943 **[Test build #145523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145523/testReport)** for PR 34685 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

AmplabJenkins commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976061043 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145523/ -- This

[GitHub] [spark] zero323 commented on a change in pull request #34439: [SPARK-37095][PYTHON] Inline type hints for files in python/pyspark/broadcast.py

zero323 commented on a change in pull request #34439: URL: https://github.com/apache/spark/pull/34439#discussion_r754106603 ## File path: python/pyspark/broadcast.py ## @@ -62,35 +81,44 @@ class Broadcast(object): >>> large_broadcast = sc.broadcast(range(1)) """

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976053975 **[Test build #145523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145523/testReport)** for PR 34685 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-976051183 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49993/

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-976051183 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49993/ --

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976040182 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49994/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

HyukjinKwon commented on pull request #34314: URL: https://github.com/apache/spark/pull/34314#issuecomment-976037376 @Yikun, if you're stuck to support this with old pandas versions, we can just conditionally run the tests with only pandas 1.3+ for now -- This is an automated message

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-976033680 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49993/ -- This is an automated message from the

[GitHub] [spark] github-actions[bot] closed pull request #33544: [SPARK-34927][INFRA] Support TPCDSQueryBenchmark in Benchmarks

github-actions[bot] closed pull request #33544: URL: https://github.com/apache/spark/pull/33544 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

HyukjinKwon commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976026130 cc @JoshRosen FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon closed pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

HyukjinKwon closed pull request #34401: URL: https://github.com/apache/spark/pull/34401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

HyukjinKwon commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-976019690 Thanks for working on this @pralabhkumar. Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #34622: [SPARK-37340][UI] Display StageIds in Operators for SQL UI

HyukjinKwon commented on pull request #34622: URL: https://github.com/apache/spark/pull/34622#issuecomment-976018777 cc @sarutak and @gengliangwang FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #34685: [SPARK-37443][PYTHON] Provide a profiler for Python/Pandas UDFs

SparkQA commented on pull request #34685: URL: https://github.com/apache/spark/pull/34685#issuecomment-976015951 **[Test build #145522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145522/testReport)** for PR 34685 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-976014942 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145517/

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-976014942 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145517/ -- This

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-976009339 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49993/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon closed pull request #34608: [SPARK-37337][PYTHON] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion

HyukjinKwon closed pull request #34608: URL: https://github.com/apache/spark/pull/34608 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #34608: [SPARK-37337][PYTHON] Improve the API of Spark DataFrame to pandas-on-Spark DataFrame conversion

HyukjinKwon commented on pull request #34608: URL: https://github.com/apache/spark/pull/34608#issuecomment-976007989 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

<    1   2   3   4   5   6   >