[GitHub] [spark] beliefer commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

beliefer commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-982203092 ping @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [spark] SparkQA commented on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

SparkQA commented on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982205742 **[Test build #145736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145736/testReport)** for PR 34737 at commit [`7e36783`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

SparkQA removed a comment on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982193147 **[Test build #145736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145736/testReport)** for PR 34737 at commit [`7e36783`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982210668 **[Test build #145737 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145737/testReport)** for PR 34746 at commit [`5885d42`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

SparkQA commented on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-982211056 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50204/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982196543 **[Test build #145737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145737/testReport)** for PR 34746 at commit [`5885d42`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-982211525 **[Test build #145731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145731/testReport)** for PR 34596 at commit [`1edef2d`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-982034660 **[Test build #145731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145731/testReport)** for PR 34596 at commit [`1edef2d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

SparkQA commented on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982214328 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50206/ -- This is an automated message from the Apache

[GitHub] [spark] allisonwang-db commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

allisonwang-db commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758876854 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override v

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-982214629 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50205/ -- This is an automated message from the Apache

[GitHub] [spark] HeartSaVioR commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

HeartSaVioR commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-982215454 Ideally saying, we only have three options instead of four. It doesn't make sense both support RDD and support columnar are false. That said, there should be "a" function wh

[GitHub] [spark] HeartSaVioR edited a comment on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

HeartSaVioR edited a comment on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-982215454 Ideally saying, we only have three options instead of four. It doesn't make sense both support RDD and support columnar are false. That said, there should be "a" func

[GitHub] [spark] AmplabJenkins commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982218145 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145737/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

AmplabJenkins commented on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982218147 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145736/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-982218146 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145731/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-982218146 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145731/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

AmplabJenkins removed a comment on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982218147 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145736/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982218145 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145737/ -

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982218435 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50207/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982219652 **[Test build #145738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145738/testReport)** for PR 34723 at commit [`96bb4f5`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

SparkQA commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982219830 **[Test build #145739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145739/testReport)** for PR 34578 at commit [`c30d443`](https://github.com

[GitHub] [spark] summaryzb opened a new pull request #34748: Spark 37493

summaryzb opened a new pull request #34748: URL: https://github.com/apache/spark/pull/34748 ### What changes were proposed in this pull request? show driver's gc time & duration time(equivalent to application time) of driver in both driver side and history side UI ### Why

[GitHub] [spark] AmplabJenkins commented on pull request #34748: Spark 37493

AmplabJenkins commented on pull request #34748: URL: https://github.com/apache/spark/pull/34748#issuecomment-98755 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] LuciferYang commented on pull request #34739: [SPARK-37484][CORE][SQL] Replace `get` and `getOrElse` with `getOrElse`

LuciferYang commented on pull request #34739: URL: https://github.com/apache/spark/pull/34739#issuecomment-98900 Yes, that should be all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] LuciferYang commented on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

LuciferYang commented on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-982223372 This should be all changeable in the current code base -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] dongjoon-hyun commented on pull request #34744: [SPARK-37454][SQL][FOLLOWUP] Time travel timestamp expression should support RuntimeReplaceable

dongjoon-hyun commented on pull request #34744: URL: https://github.com/apache/spark/pull/34744#issuecomment-982223716 +1, LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [spark] ulysses-you commented on a change in pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

ulysses-you commented on a change in pull request #34568: URL: https://github.com/apache/spark/pull/34568#discussion_r758885634 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SortExec.scala ## @@ -206,3 +179,43 @@ case class SortExec( override protected

[GitHub] [spark] summaryzb closed pull request #34748: [Spark 37493][CORE] show driver's gc time and duration time in executors page

summaryzb closed pull request #34748: URL: https://github.com/apache/spark/pull/34748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] LuciferYang commented on pull request #34748: [Spark 37493][CORE] show driver's gc time and duration time in executors page

LuciferYang commented on pull request #34748: URL: https://github.com/apache/spark/pull/34748#issuecomment-982225210 Looks like you need to rebase? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] HyukjinKwon commented on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

HyukjinKwon commented on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-982225373 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] HyukjinKwon commented on pull request #34739: [SPARK-37484][CORE][SQL] Replace `get` and `getOrElse` with `getOrElse`

HyukjinKwon commented on pull request #34739: URL: https://github.com/apache/spark/pull/34739#issuecomment-982225351 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] HyukjinKwon closed pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

HyukjinKwon closed pull request #34740: URL: https://github.com/apache/spark/pull/34740 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] LuciferYang commented on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

LuciferYang commented on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-982225641 thanks all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [spark] HyukjinKwon closed pull request #34739: [SPARK-37484][CORE][SQL] Replace `get` and `getOrElse` with `getOrElse`

HyukjinKwon closed pull request #34739: URL: https://github.com/apache/spark/pull/34739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] HyukjinKwon commented on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

HyukjinKwon commented on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982225971 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] HyukjinKwon closed pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

HyukjinKwon closed pull request #34737: URL: https://github.com/apache/spark/pull/34737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] SparkQA commented on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

SparkQA commented on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-982226440 **[Test build #145740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145740/testReport)** for PR 34568 at commit [`7fddb62`](https://github.com

[GitHub] [spark] ulysses-you commented on a change in pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

ulysses-you commented on a change in pull request #34568: URL: https://github.com/apache/spark/pull/34568#discussion_r758887620 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala ## @@ -37,7 +36,8 @@ class SparkOptimizer( override de

[GitHub] [spark] AngersZhuuuu commented on pull request #34710: [SPARK-37461][YARN] YARN-CLIENT mode client.appId is always null

AngersZh commented on pull request #34710: URL: https://github.com/apache/spark/pull/34710#issuecomment-982229402 > The function submitApplication returns the appId and now that isn't used in this case which seems a bit odd. All we did here was move the assignment to be a little bit so

[GitHub] [spark] SparkQA commented on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

SparkQA commented on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-982230090 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50204/ -- This is an automated message from the A

[GitHub] [spark] summaryzb opened a new pull request #34749: [Spark 37493][CORE] show driver's gc time and duration time in executors page

summaryzb opened a new pull request #34749: URL: https://github.com/apache/spark/pull/34749 …ors page ### What changes were proposed in this pull request? show driver's gc time & duration time(equivalent to application time) of driver in both driver side and history side U

[GitHub] [spark] LuciferYang commented on pull request #34749: [Spark 37493][CORE] show driver's gc time and duration time in executors page

LuciferYang commented on pull request #34749: URL: https://github.com/apache/spark/pull/34749#issuecomment-982234283 should be `[SPARK-37493][CORE] show driver's gc time and duration time in executors page` -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [spark] LuciferYang commented on pull request #34739: [SPARK-37484][CORE][SQL] Replace `get` and `getOrElse` with `getOrElse`

LuciferYang commented on pull request #34739: URL: https://github.com/apache/spark/pull/34739#issuecomment-982235007 thanks all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [spark] Yikun commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

Yikun commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758894595 ## File path: python/pyspark/pandas/tests/test_series.py ## @@ -2209,12 +2209,12 @@ def test_mad(self): pser = pd.Series([1, 2, 3, 4], name="Koalas

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982237262 **[Test build #145732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145732/testReport)** for PR 34611 at commit [`c8680d0`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982077845 **[Test build #145732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145732/testReport)** for PR 34611 at commit [`c8680d0`](https://gi

[GitHub] [spark] Yikun commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

Yikun commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758895254 ## File path: python/pyspark/pandas/tests/test_series.py ## @@ -2209,12 +2209,12 @@ def test_mad(self): pser = pd.Series([1, 2, 3, 4], name="Koalas

[GitHub] [spark] Yikun commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

Yikun commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758894595 ## File path: python/pyspark/pandas/tests/test_series.py ## @@ -2209,12 +2209,12 @@ def test_mad(self): pser = pd.Series([1, 2, 3, 4], name="Koalas

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982239105 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50207/ -- This is an automated message from the A

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34732: [SPARK-37291][PYSPARK][FOLLOWUP] PySpark create SparkSession should pass initialSessionOptions

AngersZh commented on a change in pull request #34732: URL: https://github.com/apache/spark/pull/34732#discussion_r758896993 ## File path: python/pyspark/sql/session.py ## @@ -305,10 +305,9 @@ def __init__( ): jsparkSession = self._jvm.SparkSe

[GitHub] [spark] SparkQA commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982241191 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50208/ -- This is an automated message from the Apache

[GitHub] [spark] Yikun commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

Yikun commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758898358 ## File path: python/docs/source/user_guide/sql/arrow_pandas.rst ## @@ -387,7 +387,7 @@ working with timestamps in ``pandas_udf``\s to get the best perform

[GitHub] [spark] Yikun commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

Yikun commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758894595 ## File path: python/pyspark/pandas/tests/test_series.py ## @@ -2209,12 +2209,12 @@ def test_mad(self): pser = pd.Series([1, 2, 3, 4], name="Koalas

[GitHub] [spark] SparkQA commented on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

SparkQA commented on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-982243298 **[Test build #145733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145733/testReport)** for PR 34060 at commit [`8fdf059`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-982243816 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50205/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

SparkQA commented on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982243751 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50206/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

SparkQA removed a comment on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-982078391 **[Test build #145733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145733/testReport)** for PR 34060 at commit [`8fdf059`](https://gi

[GitHub] [spark] prakharjain09 commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

prakharjain09 commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r758897464 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala ## @@ -57,11 +66,15 @@ case class PartitionedFil

[GitHub] [spark] SparkQA commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

SparkQA commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982245056 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50209/ -- This is an automated message from the Apache

[GitHub] [spark] sunchao closed pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

sunchao closed pull request #34611: URL: https://github.com/apache/spark/pull/34611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] AmplabJenkins commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-982246779 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50205/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982246785 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145732/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

AmplabJenkins commented on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-982246781 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145733/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

AmplabJenkins removed a comment on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982246778 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50206/

[GitHub] [spark] AmplabJenkins commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982246777 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50207/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

AmplabJenkins commented on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982246778 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50206/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

AmplabJenkins commented on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-982246780 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50204/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

AmplabJenkins removed a comment on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-982246781 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145733/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982246785 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145732/ -

[GitHub] [spark] sunchao commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

sunchao commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982246841 Merged to master. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

AmplabJenkins removed a comment on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-982246780 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50204/

[GitHub] [spark] AmplabJenkins commented on pull request #34749: [SPARK-37493][CORE] show driver's gc time and duration time in executors page

AmplabJenkins commented on pull request #34749: URL: https://github.com/apache/spark/pull/34749#issuecomment-982247053 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins removed a comment on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-982246779 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50205/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982246777 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50207/

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982248323 **[Test build #145741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145741/testReport)** for PR 34746 at commit [`bcc326d`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34732: [SPARK-37291][PYSPARK][FOLLOWUP] PySpark create SparkSession should pass initialSessionOptions

SparkQA commented on pull request #34732: URL: https://github.com/apache/spark/pull/34732#issuecomment-982248372 **[Test build #145742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145742/testReport)** for PR 34732 at commit [`21ab18f`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982248373 **[Test build #145743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145743/testReport)** for PR 34731 at commit [`48e355e`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982248418 **[Test build #145744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145744/testReport)** for PR 34717 at commit [`054905f`](https://github.com

[GitHub] [spark] Yikun commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

Yikun commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758898358 ## File path: python/docs/source/user_guide/sql/arrow_pandas.rst ## @@ -387,7 +387,7 @@ working with timestamps in ``pandas_udf``\s to get the best perform

[GitHub] [spark] SparkQA commented on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

SparkQA commented on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-982248728 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50210/ -- This is an automated message from the Apache

[GitHub] [spark] Yikun commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

Yikun commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758902796 ## File path: python/pyspark/pandas/tests/test_series.py ## @@ -2209,12 +2209,12 @@ def test_mad(self): pser = pd.Series([1, 2, 3, 4], name="Koalas

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982251919 **[Test build #145745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145745/testReport)** for PR 34731 at commit [`27207aa`](https://github.com

[GitHub] [spark] kevincmchen commented on pull request #34742: [SPARK-37486][SQL][HIVE] set the ContextClassLoader before using the `addJars` in `HiveClient`

kevincmchen commented on pull request #34742: URL: https://github.com/apache/spark/pull/34742#issuecomment-982253151 > filing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] kevincmchen closed pull request #34742: [SPARK-37486][SQL][HIVE] set the ContextClassLoader before using the `addJars` in `HiveClient`

kevincmchen closed pull request #34742: URL: https://github.com/apache/spark/pull/34742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] gengliangwang commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758906045 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override va

[GitHub] [spark] kevincmchen removed a comment on pull request #34742: [SPARK-37486][SQL][HIVE] set the ContextClassLoader before using the `addJars` in `HiveClient`

kevincmchen removed a comment on pull request #34742: URL: https://github.com/apache/spark/pull/34742#issuecomment-982253151 > filing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] gengliangwang commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758907096 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override va

[GitHub] [spark] Peng-Lei commented on a change in pull request #34719: [SPARK-37381][SQL] Unify v1 and v2 SHOW CREATE TABLE tests

Peng-Lei commented on a change in pull request #34719: URL: https://github.com/apache/spark/pull/34719#discussion_r758910060 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala ## @@ -0,0 +1,141 @@ +/* + * Licensed to the

[GitHub] [spark] kevincmchen commented on pull request #34742: [SPARK-37486][SQL][HIVE] set the ContextClassLoader before using the `addJars` in `HiveClient`

kevincmchen commented on pull request #34742: URL: https://github.com/apache/spark/pull/34742#issuecomment-982257963 > @kevincmchen mind filing a JIRA? see also https://spark.apache.org/contributing.html @HyukjinKwon ok, i have created a [issue](https://issues.apache.org/jira/brow

[GitHub] [spark] cloud-fan commented on a change in pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

cloud-fan commented on a change in pull request #34741: URL: https://github.com/apache/spark/pull/34741#discussion_r758910943 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala ## @@ -531,13 +533,16 @@ object OrcUtils extends Lo

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982259667 **[Test build #145743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145743/testReport)** for PR 34731 at commit [`48e355e`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982259989 **[Test build #145741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145741/testReport)** for PR 34746 at commit [`bcc326d`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982260719 **[Test build #145744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145744/testReport)** for PR 34717 at commit [`054905f`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA removed a comment on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982248373 **[Test build #145743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145743/testReport)** for PR 34731 at commit [`48e355e`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982248323 **[Test build #145741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145741/testReport)** for PR 34746 at commit [`bcc326d`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

SparkQA removed a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982248418 **[Test build #145744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145744/testReport)** for PR 34717 at commit [`054905f`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

SparkQA commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982261198 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50209/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982262729 **[Test build #145745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145745/testReport)** for PR 34731 at commit [`27207aa`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA removed a comment on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982251919 **[Test build #145745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145745/testReport)** for PR 34731 at commit [`27207aa`](https://gi

<    1   2   3   4   5   6   7   8   9   >