[GitHub] [spark] yaooqinn commented on a diff in pull request #36053: [SPARK-38778][INFRA][BUILD] Replace http with https for project url in pom

yaooqinn commented on code in PR #36053: URL: https://github.com/apache/spark/pull/36053#discussion_r841373575 ## pom.xml: ## @@ -29,7 +29,7 @@ 3.4.0-SNAPSHOT pom Spark Project Parent POM - http://spark.apache.org/ Review Comment: updated, thanks. -- This is an

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #36038: [SPARK-38759][PYTHON][SS] Add StreamingQueryListener support in PySpark

HyukjinKwon commented on code in PR #36038: URL: https://github.com/apache/spark/pull/36038#discussion_r841351936 ## python/docs/source/reference/pyspark.ss.rst: ## @@ -30,10 +30,10 @@ Core Classes DataStreamReader DataStreamWriter -ForeachBatchFunction Review C

[GitHub] [spark] wangyum commented on a diff in pull request #36053: [SPARK-38778][INFRA][BUILD] Replace http with https for project url in pom

wangyum commented on code in PR #36053: URL: https://github.com/apache/spark/pull/36053#discussion_r841345386 ## pom.xml: ## @@ -29,7 +29,7 @@ 3.4.0-SNAPSHOT pom Spark Project Parent POM - http://spark.apache.org/ Review Comment: Line 53? https://github.com/apa

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #36038: [SPARK-38759][PYTHON][SS] Add StreamingQueryListener support in PySpark

HyukjinKwon commented on code in PR #36038: URL: https://github.com/apache/spark/pull/36038#discussion_r84177 ## python/pyspark/sql/streaming/listener.py: ## @@ -0,0 +1,666 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agr

[GitHub] [spark] monkeyboy123 commented on a diff in pull request #35984: [MINOR] Log AnalysisException output for debug and tracing

monkeyboy123 commented on code in PR #35984: URL: https://github.com/apache/spark/pull/35984#discussion_r841321778 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala: ## @@ -1761,7 +1761,9 @@ class Analyzer(override val catalogManager: CatalogM

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #35979: [SPARK-38664][CORE] Support compact EventLog when there are illegal characters in the path

HyukjinKwon commented on code in PR #35979: URL: https://github.com/apache/spark/pull/35979#discussion_r841300801 ## core/src/main/scala/org/apache/spark/deploy/history/EventLogFileCompactor.scala: ## @@ -221,5 +221,5 @@ private class CompactedEventLogFileWriter( hadoopConf

[GitHub] [spark] zhengruifeng commented on a diff in pull request #36048: [SPARK-38774][PYTHON] Implement Series.autocorr

zhengruifeng commented on code in PR #36048: URL: https://github.com/apache/spark/pull/36048#discussion_r841305123 ## python/pyspark/pandas/series.py: ## @@ -2937,6 +2937,73 @@ def add_suffix(self, suffix: str) -> "Series": DataFrame(internal.with_new_sdf(sdf, index

[GitHub] [spark] zhengruifeng commented on a diff in pull request #36049: [SPARK-38775][ML] cleanup validation functions

zhengruifeng commented on code in PR #36049: URL: https://github.com/apache/spark/pull/36049#discussion_r841303261 ## mllib/src/main/scala/org/apache/spark/ml/util/DatasetUtils.scala: ## @@ -138,4 +140,61 @@ private[spark] object DatasetUtils { case Row(point: Vector) =>

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #35979: [SPARK-38664][CORE] Support compact EventLog when there are illegal characters in the path

HyukjinKwon commented on code in PR #35979: URL: https://github.com/apache/spark/pull/35979#discussion_r841300801 ## core/src/main/scala/org/apache/spark/deploy/history/EventLogFileCompactor.scala: ## @@ -221,5 +221,5 @@ private class CompactedEventLogFileWriter( hadoopConf

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #35979: [SPARK-38664][CORE] Support compact EventLog when there are illegal characters in the path

HyukjinKwon commented on code in PR #35979: URL: https://github.com/apache/spark/pull/35979#discussion_r841300801 ## core/src/main/scala/org/apache/spark/deploy/history/EventLogFileCompactor.scala: ## @@ -221,5 +221,5 @@ private class CompactedEventLogFileWriter( hadoopConf

[GitHub] [spark] huaxingao commented on a diff in pull request #36043: [SPARK-38768][SQL] Remove `Limit` from plan if complete push down limit to data source.

huaxingao commented on code in PR #36043: URL: https://github.com/apache/spark/pull/36043#discussion_r841285841 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -380,27 +380,32 @@ object V2ScanRelationPushDown extends R