[GitHub] [spark] SparkQA commented on pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

SparkQA commented on pull request #34750: URL: https://github.com/apache/spark/pull/34750#issuecomment-982337997 **[Test build #145750 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145750/testReport)** for PR 34750 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-982122968 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50201/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34701: [SPARK-37450][SQL] Prune unnecessary fiels from Generate under count-only Aggregate

AmplabJenkins commented on pull request #34701: URL: https://github.com/apache/spark/pull/34701#issuecomment-982122967 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145729/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982122966 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50200/ --

[GitHub] [spark] HyukjinKwon commented on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

HyukjinKwon commented on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-982170951 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

AmplabJenkins removed a comment on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-981792830 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] itholic commented on a change in pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

itholic commented on a change in pull request #34737: URL: https://github.com/apache/spark/pull/34737#discussion_r758859791 ## File path: python/pyspark/pandas/series.py ## @@ -5179,7 +5179,9 @@ def asof(self, where: Union[Any, List]) -> Union[Scalar, "Series"]:

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-982193103 **[Test build #145735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145735/testReport)** for PR 34741 at commit

[GitHub] [spark] SparkQA commented on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

SparkQA commented on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982193147 **[Test build #145736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145736/testReport)** for PR 34737 at commit

[GitHub] [spark] beliefer commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

beliefer commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982200981 > BTW, is this PR include all instances like this, @beliefer ? Could you check other places too? Yes. I checked every places. -- This is an automated message from the

[GitHub] [spark] HeartSaVioR edited a comment on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

HeartSaVioR edited a comment on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-982215454 Ideally saying, we only have three options instead of four. It doesn't make sense both support RDD and support columnar are false. That said, there should be "a"

[GitHub] [spark] HyukjinKwon closed pull request #34739: [SPARK-37484][CORE][SQL] Replace `get` and `getOrElse` with `getOrElse`

HyukjinKwon closed pull request #34739: URL: https://github.com/apache/spark/pull/34739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] LuciferYang commented on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

LuciferYang commented on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-982225641 thanks all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] HyukjinKwon closed pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

HyukjinKwon closed pull request #34740: URL: https://github.com/apache/spark/pull/34740 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] gengliangwang commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758906045 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override

[GitHub] [spark] kevincmchen removed a comment on pull request #34742: [SPARK-37486][SQL][HIVE] set the ContextClassLoader before using the `addJars` in `HiveClient`

kevincmchen removed a comment on pull request #34742: URL: https://github.com/apache/spark/pull/34742#issuecomment-982253151 > filing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cloud-fan commented on a change in pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

cloud-fan commented on a change in pull request #34738: URL: https://github.com/apache/spark/pull/34738#discussion_r75892 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownLimit.java ## @@ -33,4 +34,9 @@ * Pushes down LIMIT

[GitHub] [spark] dongjoon-hyun closed pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

dongjoon-hyun closed pull request #34723: URL: https://github.com/apache/spark/pull/34723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982284411 **[Test build #145747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145747/testReport)** for PR 34657 at commit

[GitHub] [spark] SparkQA commented on pull request #34622: [SPARK-37340][UI] Display StageIds in Operators for SQL UI

SparkQA commented on pull request #34622: URL: https://github.com/apache/spark/pull/34622#issuecomment-982291693 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50214/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982291748 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50213/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982291656 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50216/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on a change in pull request #34264: URL: https://github.com/apache/spark/pull/34264#discussion_r758938108 ## File path: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSourceSuite.scala

[GitHub] [spark] SparkQA commented on pull request #34732: [SPARK-37291][PYSPARK][FOLLOWUP] PySpark create SparkSession should pass initialSessionOptions

SparkQA commented on pull request #34732: URL: https://github.com/apache/spark/pull/34732#issuecomment-982295028 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50212/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-982324154 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145735/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982324060 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50217/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins removed a comment on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-982324154 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145735/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

AmplabJenkins removed a comment on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982324057 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50218/

[GitHub] [spark] Peng-Lei commented on a change in pull request #34719: [SPARK-37381][SQL] Unify v1 and v2 SHOW CREATE TABLE tests

Peng-Lei commented on a change in pull request #34719: URL: https://github.com/apache/spark/pull/34719#discussion_r758966706 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowCreateTableSuite.scala ## @@ -0,0 +1,140 @@ +/* + * Licensed to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

AmplabJenkins removed a comment on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982324056 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50216/

[GitHub] [spark] AmplabJenkins commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

AmplabJenkins commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982324056 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50216/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982324060 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50217/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

AmplabJenkins commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982324057 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50218/ --

[GitHub] [spark] sunchao commented on a change in pull request #34635: [SPARK-37205][YARN] Introduce a new config 'spark.yarn.am.tokenConfRegex' to support renewing delegation tokens in a multi-cluste

sunchao commented on a change in pull request #34635: URL: https://github.com/apache/spark/pull/34635#discussion_r758970173 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ## @@ -340,6 +344,40 @@ private[spark] class Client(

[GitHub] [spark] cloud-fan commented on a change in pull request #34719: [SPARK-37381][SQL] Unify v1 and v2 SHOW CREATE TABLE tests

cloud-fan commented on a change in pull request #34719: URL: https://github.com/apache/spark/pull/34719#discussion_r758970183 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowCreateTableSuite.scala ## @@ -0,0 +1,141 @@ +/* + * Licensed to

[GitHub] [spark] dongjoon-hyun commented on pull request #34751: [SPARK-37497][K8S] Promote `ExecutorPods[PollingSnapshot|WatchSnapshot]Source` to DeveloperApi

dongjoon-hyun commented on pull request #34751: URL: https://github.com/apache/spark/pull/34751#issuecomment-982332947 Thank you for review, @viirya . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

HyukjinKwon commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r758979898 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala ## @@ -160,6 +169,17 @@ class CSVInferSchema(val

[GitHub] [spark] SparkQA commented on pull request #34751: [SPARK-37497][K8S] Promote `ExecutorPods[PollingSnapshot|WatchSnapshot]Source` to DeveloperApi

SparkQA commented on pull request #34751: URL: https://github.com/apache/spark/pull/34751#issuecomment-982338326 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50222/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34752: [SPARK][STREAMING] minRatePerPartition should be multiplied with secsPerBatch

AmplabJenkins commented on pull request #34752: URL: https://github.com/apache/spark/pull/34752#issuecomment-982353991 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA commented on pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

SparkQA commented on pull request #34750: URL: https://github.com/apache/spark/pull/34750#issuecomment-982354345 **[Test build #145756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145756/testReport)** for PR 34750 at commit

[GitHub] [spark] SparkQA commented on pull request #34751: [SPARK-37497][K8S] Promote `ExecutorPods[PollingSnapshot|WatchSnapshot]Source` to DeveloperApi

SparkQA commented on pull request #34751: URL: https://github.com/apache/spark/pull/34751#issuecomment-982354267 **[Test build #145755 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145755/testReport)** for PR 34751 at commit

[GitHub] [spark] SparkQA commented on pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

SparkQA commented on pull request #34753: URL: https://github.com/apache/spark/pull/34753#issuecomment-982354219 **[Test build #145754 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145754/testReport)** for PR 34753 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

SparkQA removed a comment on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-982226440 **[Test build #145740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145740/testReport)** for PR 34568 at commit

[GitHub] [spark] huaxingao commented on pull request #34744: [SPARK-37454][SQL][FOLLOWUP] Time travel timestamp expression should support RuntimeReplaceable

huaxingao commented on pull request #34744: URL: https://github.com/apache/spark/pull/34744#issuecomment-982087103 Thanks for pinging me. There is a test failure, but doesn't seem to be related to the changes. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-982106720 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50201/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982034664 **[Test build #145730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145730/testReport)** for PR 34611 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982124129 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145730/

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982124129 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145730/ -- This

[GitHub] [spark] HyukjinKwon commented on pull request #34742: set the setContextClassLoader before using the `addJars` in `HiveClient`

HyukjinKwon commented on pull request #34742: URL: https://github.com/apache/spark/pull/34742#issuecomment-982181025 @kevincmchen mind filing a JIRA? see also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

HyukjinKwon commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982180394 Thanks @Yikun !! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] bersprockets commented on a change in pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

bersprockets commented on a change in pull request #34741: URL: https://github.com/apache/spark/pull/34741#discussion_r758854534 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala ## @@ -531,13 +533,16 @@ object OrcUtils

[GitHub] [spark] HeartSaVioR commented on pull request #34642: [SPARK-37369][SQL] Avoid redundant ColumnarToRow transistion on InMemoryTableScan

HeartSaVioR commented on pull request #34642: URL: https://github.com/apache/spark/pull/34642#issuecomment-982215454 Ideally saying, we only have three options instead of four. It doesn't make sense both support RDD and support columnar are false. That said, there should be "a" function

[GitHub] [spark] SparkQA commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982219652 **[Test build #145738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145738/testReport)** for PR 34723 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34748: Spark 37493

AmplabJenkins commented on pull request #34748: URL: https://github.com/apache/spark/pull/34748#issuecomment-98755 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] summaryzb opened a new pull request #34748: Spark 37493

summaryzb opened a new pull request #34748: URL: https://github.com/apache/spark/pull/34748 ### What changes were proposed in this pull request? show driver's gc time & duration time(equivalent to application time) of driver in both driver side and history side UI ### Why

[GitHub] [spark] LuciferYang commented on pull request #34739: [SPARK-37484][CORE][SQL] Replace `get` and `getOrElse` with `getOrElse`

LuciferYang commented on pull request #34739: URL: https://github.com/apache/spark/pull/34739#issuecomment-98900 Yes, that should be all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] LuciferYang commented on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

LuciferYang commented on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-982223372 This should be all changeable in the current code base -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] summaryzb opened a new pull request #34749: [Spark 37493][CORE] show driver's gc time and duration time in executors page

summaryzb opened a new pull request #34749: URL: https://github.com/apache/spark/pull/34749 …ors page ### What changes were proposed in this pull request? show driver's gc time & duration time(equivalent to application time) of driver in both driver side and history side

[GitHub] [spark] SparkQA commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982241191 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50208/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA removed a comment on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982251919 **[Test build #145745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145745/testReport)** for PR 34731 at commit

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982262729 **[Test build #145745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145745/testReport)** for PR 34731 at commit

[GitHub] [spark] LuciferYang commented on a change in pull request #34749: [SPARK-37493][CORE] show driver's gc time and duration time in executors page

LuciferYang commented on a change in pull request #34749: URL: https://github.com/apache/spark/pull/34749#discussion_r758917691 ## File path: core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ## @@ -137,7 +138,9 @@ case object GarbageCollectionMetrics

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

dongjoon-hyun commented on a change in pull request #34723: URL: https://github.com/apache/spark/pull/34723#discussion_r758929816 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcQuerySuite.scala ## @@ -288,10 +288,9 @@ class HiveOrcQuerySuite

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

dongjoon-hyun commented on a change in pull request #34723: URL: https://github.com/apache/spark/pull/34723#discussion_r758929249 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcQuerySuite.scala ## @@ -288,10 +288,9 @@ class HiveOrcQuerySuite

[GitHub] [spark] cloud-fan commented on a change in pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

cloud-fan commented on a change in pull request #34738: URL: https://github.com/apache/spark/pull/34738#discussion_r758929705 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownLimit.java ## @@ -18,6 +18,7 @@ package

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982288646 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50215/ -- This is an automated message from the

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on a change in pull request #34264: URL: https://github.com/apache/spark/pull/34264#discussion_r758938725 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -395,6 +395,23 @@ private[spark]

[GitHub] [spark] gengliangwang commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758938379 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override

[GitHub] [spark] LuciferYang commented on a change in pull request #34719: [SPARK-37381][SQL] Unify v1 and v2 SHOW CREATE TABLE tests

LuciferYang commented on a change in pull request #34719: URL: https://github.com/apache/spark/pull/34719#discussion_r758953291 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowCreateTableSuite.scala ## @@ -0,0 +1,140 @@ +/* + * Licensed to

[GitHub] [spark] cloud-fan commented on a change in pull request #34684: [SPARK-37442][SQL] InMemoryRelation statistics bug causing broadcast join failures with AQE enabled

cloud-fan commented on a change in pull request #34684: URL: https://github.com/apache/spark/pull/34684#discussion_r758962344 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala ## @@ -237,7 +238,23 @@ case class

[GitHub] [spark] HyukjinKwon commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

HyukjinKwon commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982330232 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon commented on pull request #34732: [SPARK-37291][PYSPARK][FOLLOWUP] PySpark create SparkSession should pass initialSessionOptions

HyukjinKwon commented on pull request #34732: URL: https://github.com/apache/spark/pull/34732#issuecomment-982330696 Merged to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon closed pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

HyukjinKwon closed pull request #34746: URL: https://github.com/apache/spark/pull/34746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

HyukjinKwon commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r758980984 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala ## @@ -38,6 +39,13 @@ class CSVInferSchema(val

[GitHub] [spark] SparkQA removed a comment on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

SparkQA removed a comment on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982219652 **[Test build #145738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145738/testReport)** for PR 34723 at commit

[GitHub] [spark] ueshin commented on a change in pull request #34509: [SPARK-34521][PYTHON][SQL] Fix spark.createDataFrame when using pandas with StringDtype

ueshin commented on a change in pull request #34509: URL: https://github.com/apache/spark/pull/34509#discussion_r758804602 ## File path: python/pyspark/sql/pandas/serializers.py ## @@ -169,6 +169,8 @@ def create_array(s, t): elif is_categorical_dtype(s.dtype):

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982121042 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50202/ -- This is an automated message from the Apache

[GitHub] [spark] itholic commented on pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

itholic commented on pull request #34737: URL: https://github.com/apache/spark/pull/34737#issuecomment-982186263 LGTM. Would you mind file a JIRA for fixing the bug mentioned in https://github.com/apache/spark/pull/34737#discussion_r758223279 ? -- This is an automated message

[GitHub] [spark] dchvn commented on a change in pull request #34737: [SPARK-37482][PYTHON] Skip check monotonic increasing for Series.asof with 'compute.eager_check'

dchvn commented on a change in pull request #34737: URL: https://github.com/apache/spark/pull/34737#discussion_r758854154 ## File path: python/pyspark/pandas/tests/test_series.py ## @@ -2115,6 +2115,23 @@ def test_asof(self): self.assert_eq(psser.asof("2014-01-02"),

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982196543 **[Test build #145737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145737/testReport)** for PR 34746 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-982034660 **[Test build #145731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145731/testReport)** for PR 34596 at commit

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-982211525 **[Test build #145731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145731/testReport)** for PR 34596 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #34744: [SPARK-37454][SQL][FOLLOWUP] Time travel timestamp expression should support RuntimeReplaceable

dongjoon-hyun commented on pull request #34744: URL: https://github.com/apache/spark/pull/34744#issuecomment-982223716 +1, LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] ulysses-you commented on a change in pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

ulysses-you commented on a change in pull request #34568: URL: https://github.com/apache/spark/pull/34568#discussion_r758887620 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala ## @@ -37,7 +36,8 @@ class SparkOptimizer( override

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-982237262 **[Test build #145732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145732/testReport)** for PR 34611 at commit

[GitHub] [spark] Yikun commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

Yikun commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758894595 ## File path: python/pyspark/pandas/tests/test_series.py ## @@ -2209,12 +2209,12 @@ def test_mad(self): pser = pd.Series([1, 2, 3, 4],

[GitHub] [spark] SparkQA commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

SparkQA commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982245056 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50209/ -- This is an automated message from the Apache

[GitHub] [spark] prakharjain09 commented on a change in pull request #34575: [SPARK-37273][SQL] Support hidden file metadata columns in Spark SQL

prakharjain09 commented on a change in pull request #34575: URL: https://github.com/apache/spark/pull/34575#discussion_r758897464 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala ## @@ -57,11 +66,15 @@ case class

[GitHub] [spark] gengliangwang commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758907096 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982259989 **[Test build #145741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145741/testReport)** for PR 34746 at commit

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982260719 **[Test build #145744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145744/testReport)** for PR 34717 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

AmplabJenkins removed a comment on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982271490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50209/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

AmplabJenkins removed a comment on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982271495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50208/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

AmplabJenkins removed a comment on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982271492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982271493 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145741/

[GitHub] [spark] AmplabJenkins commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

AmplabJenkins commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982271495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50208/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982271493 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145741/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

AmplabJenkins commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982271492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

AmplabJenkins commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982271491 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145744/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

AmplabJenkins commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982271490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50209/ --

<    1   2   3   4   5   6   7   8   9   >