[GitHub] [spark] viirya commented on a change in pull request #35395: [SPARK-38085][SQL] DataSource V2: Handle DELETE commands for group-based sources

viirya commented on a change in pull request #35395: URL: https://github.com/apache/spark/pull/35395#discussion_r823436246 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -446,17 +501,52 @@ object DescribeColumn {

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

dongjoon-hyun commented on a change in pull request #35786: URL: https://github.com/apache/spark/pull/35786#discussion_r823435866 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/VolcanoFeatureStep.scala ## @@ -59,6 +55,7 @@ p

[GitHub] [spark] wangyum commented on pull request #35795: [SPARK-38489][SQL] Aggregate.groupOnly support foldable expressions

wangyum commented on pull request #35795: URL: https://github.com/apache/spark/pull/35795#issuecomment-1063761217 Fixed it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] LuciferYang commented on a change in pull request #35694: [SPARK-38360][SQL][SS][PYTHON] Introduce a `exists` function for `TreeNode` to eliminate duplicate code patterns

LuciferYang commented on a change in pull request #35694: URL: https://github.com/apache/spark/pull/35694#discussion_r823431908 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1690,11 +1690,10 @@ object PushPredicateThro

[GitHub] [spark] LuciferYang commented on a change in pull request #35694: [SPARK-38360][SQL][SS][PYTHON] Introduce a `exists` function for `TreeNode` to eliminate duplicate code patterns

LuciferYang commented on a change in pull request #35694: URL: https://github.com/apache/spark/pull/35694#discussion_r823431908 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1690,11 +1690,10 @@ object PushPredicateThro

[GitHub] [spark] beliefer commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

beliefer commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823430706 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -153,9 +153,15 @@ case class RowDataSourceScanExec(

[GitHub] [spark] beliefer commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

beliefer commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823428661 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -153,9 +153,15 @@ case class RowDataSourceScanExec(

[GitHub] [spark] beliefer commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

beliefer commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823428136 ## File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala ## @@ -41,29 +44,63 @@ class V2ExpressionBuilder(e: E

[GitHub] [spark] cloud-fan commented on a change in pull request #35787: [SPARK-38481][SQL] Substitute Java overflow exception from `TIMESTAMPADD` by Spark exception

cloud-fan commented on a change in pull request #35787: URL: https://github.com/apache/spark/pull/35787#discussion_r823425901 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -1174,29 +1174,36 @@ object DateTimeUtils {

[GitHub] [spark] cloud-fan commented on pull request #35795: [SPARK-38489][SQL] Aggregate.groupOnly support foldable expressions

cloud-fan commented on pull request #35795: URL: https://github.com/apache/spark/pull/35795#issuecomment-1063752551 `AggregateOptimizeSuite.SPARK-34808: aggregateExpressions only contains groupingExpressions` fails -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] cloud-fan commented on pull request #35694: [SPARK-38360][SQL][SS][PYTHON] Introduce a `exists` function for `TreeNode` to eliminate duplicate code patterns

cloud-fan commented on pull request #35694: URL: https://github.com/apache/spark/pull/35694#issuecomment-1063752062 looks fine as it matches the scala collection API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] cloud-fan commented on a change in pull request #35694: [SPARK-38360][SQL][SS][PYTHON] Introduce a `exists` function for `TreeNode` to eliminate duplicate code patterns

cloud-fan commented on a change in pull request #35694: URL: https://github.com/apache/spark/pull/35694#discussion_r823422170 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1690,11 +1690,10 @@ object PushPredicateThroug

[GitHub] [spark] beliefer commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

beliefer commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823421900 ## File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala ## @@ -41,29 +44,63 @@ class V2ExpressionBuilder(e: E

[GitHub] [spark] beliefer commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

beliefer commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823421399 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/expressions/Predicate.java ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache S

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823419937 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -153,9 +153,15 @@ case class RowDataSourceScanExec(

[GitHub] [spark] beliefer commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

beliefer commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823419288 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/util/V2ExpressionSQLBuilder.java ## @@ -38,6 +39,11 @@ public String build(Expre

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823417715 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -609,20 +543,67 @@ private[sql] ob

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823417370 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -609,20 +543,67 @@ private[sql] ob

[GitHub] [spark] AngersZhuuuu opened a new pull request #35799: [SPARK-38498][SQL] Support customized StreamingListener by configuration

AngersZh opened a new pull request #35799: URL: https://github.com/apache/spark/pull/35799 ### What changes were proposed in this pull request? Currently, if user want to add an customized StreamingListener to StreamingContext, we need to add this listener in customized code. ```

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823414344 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala ## @@ -175,6 +175,27 @@ object JDBCRDD extends Logg

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823413671 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -153,9 +153,15 @@ case class RowDataSourceScanExec(

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823413184 ## File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala ## @@ -41,29 +44,63 @@ class V2ExpressionBuilder(e:

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823412606 ## File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/util/V2ExpressionBuilder.scala ## @@ -41,29 +44,63 @@ class V2ExpressionBuilder(e:

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823411410 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/expressions/Predicate.java ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

cloud-fan commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823410924 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/util/V2ExpressionSQLBuilder.java ## @@ -38,6 +39,11 @@ public String build(Expr

[GitHub] [spark] cloud-fan commented on a change in pull request #35395: [SPARK-38085][SQL] DataSource V2: Handle DELETE commands for group-based sources

cloud-fan commented on a change in pull request #35395: URL: https://github.com/apache/spark/pull/35395#discussion_r823407137 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryRowLevelOperationTable.scala ## @@ -0,0 +1,96 @@ +/* + * Licen

[GitHub] [spark] Yikun commented on a change in pull request #35797: [SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

Yikun commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823385061 ## File path: .github/workflows/ansi_sql_mode_test.yml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contr

[GitHub] [spark] Yikun commented on pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

Yikun commented on pull request #35786: URL: https://github.com/apache/spark/pull/35786#issuecomment-1063719636 @dongjoon-hyun Yes for sure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [spark] Yikun commented on a change in pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

Yikun commented on a change in pull request #35786: URL: https://github.com/apache/spark/pull/35786#discussion_r823394892 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/VolcanoFeatureStep.scala ## @@ -44,12 +44,8 @@ private[

[GitHub] [spark] Yikun commented on a change in pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

Yikun commented on a change in pull request #35786: URL: https://github.com/apache/spark/pull/35786#discussion_r823392560 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/VolcanoFeatureStep.scala ## @@ -59,6 +55,7 @@ private[s

[GitHub] [spark] gengliangwang commented on a change in pull request #35797: [SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

gengliangwang commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823391542 ## File path: .github/workflows/ansi_sql_mode_test.yml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or mo

[GitHub] [spark] HyukjinKwon edited a comment on pull request #35793: [SPARK-38487][PYTHON][DOC] Fix docstrings of nlargest/nsmallest of DataFrame

HyukjinKwon edited a comment on pull request #35793: URL: https://github.com/apache/spark/pull/35793#issuecomment-1063711824 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] HyukjinKwon closed pull request #35793: [SPARK-38487][PYTHON][DOC] Fix docstrings of nlargest/nsmallest of DataFrame

HyukjinKwon closed pull request #35793: URL: https://github.com/apache/spark/pull/35793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] HyukjinKwon commented on pull request #35793: [SPARK-38487][PYTHON][DOC] Fix docstrings of nlargest/nsmallest of DataFrame

HyukjinKwon commented on pull request #35793: URL: https://github.com/apache/spark/pull/35793#issuecomment-1063711824 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] itholic commented on a change in pull request #35794: [SPARK-38491][PYTHON] Support `ignore_index` of `Series.sort_values`

itholic commented on a change in pull request #35794: URL: https://github.com/apache/spark/pull/35794#discussion_r823386245 ## File path: python/pyspark/pandas/frame.py ## @@ -6901,6 +6904,16 @@ def sort_values( 3 D 7 2 2 None 8 4 +

[GitHub] [spark] Yikun commented on a change in pull request #35797: [SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

Yikun commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823385061 ## File path: .github/workflows/ansi_sql_mode_test.yml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contr

[GitHub] [spark] xinrong-databricks commented on a change in pull request #35794: [SPARK-38491][PYTHON] Support `ignore_index` of `Series.sort_values`

xinrong-databricks commented on a change in pull request #35794: URL: https://github.com/apache/spark/pull/35794#discussion_r823384190 ## File path: python/pyspark/pandas/frame.py ## @@ -6901,6 +6904,16 @@ def sort_values( 3 D 7 2 2 None 8

[GitHub] [spark] HyukjinKwon commented on pull request #35798: [SPARK-38451][R][TESTS] Fix `make_date` test case to pass with ANSI mode

HyukjinKwon commented on pull request #35798: URL: https://github.com/apache/spark/pull/35798#issuecomment-1063707136 I honestly didn't test it it my local because the change looked straightforward. Should wait for the results in the CI. -- This is an automated message from the Apache Gi

[GitHub] [spark] HyukjinKwon opened a new pull request #35798: [SPARK-38451][R][TESTS] Fix `make_date` test case to pass with ANSI mode

HyukjinKwon opened a new pull request #35798: URL: https://github.com/apache/spark/pull/35798 ### What changes were proposed in this pull request? This PR proposes to fix `expose make_date expression in R` test case to pass with ANSI mode enabled by excluding data that throws an exce

[GitHub] [spark] itholic commented on a change in pull request #35794: [SPARK-38491][PYTHON] Support `ignore_index` of `Series.sort_values`

itholic commented on a change in pull request #35794: URL: https://github.com/apache/spark/pull/35794#discussion_r823383277 ## File path: python/pyspark/pandas/frame.py ## @@ -6901,6 +6904,16 @@ def sort_values( 3 D 7 2 2 None 8 4 +

[GitHub] [spark] MaxGekk commented on a change in pull request #35787: [SPARK-38481][SQL] Substitute Java overflow exception from `TIMESTAMPADD` by Spark exception

MaxGekk commented on a change in pull request #35787: URL: https://github.com/apache/spark/pull/35787#discussion_r823379716 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -1174,29 +1174,36 @@ object DateTimeUtils { *

[GitHub] [spark] MaxGekk commented on a change in pull request #35787: [SPARK-38481][SQL] Substitute Java overflow exception from `TIMESTAMPADD` by Spark exception

MaxGekk commented on a change in pull request #35787: URL: https://github.com/apache/spark/pull/35787#discussion_r823379716 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -1174,29 +1174,36 @@ object DateTimeUtils { *

[GitHub] [spark] beliefer commented on a change in pull request #35768: [SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way

beliefer commented on a change in pull request #35768: URL: https://github.com/apache/spark/pull/35768#discussion_r823379308 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/expressions/Predicate.java ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache So

[GitHub] [spark] xinrong-databricks commented on a change in pull request #35794: [SPARK-38491][PYTHON] Support `ignore_index` of `Series.sort_values`

xinrong-databricks commented on a change in pull request #35794: URL: https://github.com/apache/spark/pull/35794#discussion_r823379193 ## File path: python/pyspark/pandas/frame.py ## @@ -6901,6 +6904,16 @@ def sort_values( 3 D 7 2 2 None 8

[GitHub] [spark] senthh commented on pull request #35785: [SPARK-38213][CORE] Adding KafkaSink Metrics feature

senthh commented on pull request #35785: URL: https://github.com/apache/spark/pull/35785#issuecomment-1063696991 @wzhfy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] MaxGekk commented on a change in pull request #35787: [SPARK-38481][SQL] Substitute Java overflow exception from `TIMESTAMPADD` by Spark exception

MaxGekk commented on a change in pull request #35787: URL: https://github.com/apache/spark/pull/35787#discussion_r823374788 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -1174,29 +1174,36 @@ object DateTimeUtils { *

[GitHub] [spark] MaxGekk commented on a change in pull request #35787: [SPARK-38481][SQL] Substitute Java overflow exception from `TIMESTAMPADD` by Spark exception

MaxGekk commented on a change in pull request #35787: URL: https://github.com/apache/spark/pull/35787#discussion_r823371845 ## File path: core/src/main/resources/error/error-classes.json ## @@ -25,6 +25,10 @@ "CONCURRENT_QUERY" : { "message" : [ "Another instance of thi

[GitHub] [spark] dongjoon-hyun commented on pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

dongjoon-hyun commented on pull request #35786: URL: https://github.com/apache/spark/pull/35786#issuecomment-1063690925 We had better have another JIRA for this PR because this removes a conf, @Yikun . -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

dongjoon-hyun commented on a change in pull request #35786: URL: https://github.com/apache/spark/pull/35786#discussion_r823369126 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -299,13 +299,6 @@ private[spark] obje

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

dongjoon-hyun commented on a change in pull request #35786: URL: https://github.com/apache/spark/pull/35786#discussion_r823368694 ## File path: docs/running-on-kubernetes.md ## @@ -1538,6 +1538,14 @@ See the [configuration page](configuration.html) for information on Spark con

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

dongjoon-hyun commented on a change in pull request #35786: URL: https://github.com/apache/spark/pull/35786#discussion_r823368111 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/VolcanoFeatureStep.scala ## @@ -59,6 +55,7 @@ p

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35786: [SPARK-38455][SPARK-38187][K8S][FOLLOWUP] Add driver `PodGroup` templates doc and cleanup executor `PodGroup` templates

dongjoon-hyun commented on a change in pull request #35786: URL: https://github.com/apache/spark/pull/35786#discussion_r823366653 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/VolcanoFeatureStep.scala ## @@ -44,12 +44,8 @@

[GitHub] [spark] Yikf commented on a change in pull request #35788: [SPARK-38482][SQL] Migrate legacy.keepCommandOutputSchema related to KeepLegacyOutputs

Yikf commented on a change in pull request #35788: URL: https://github.com/apache/spark/pull/35788#discussion_r823366174 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTablesSuite.scala ## @@ -136,6 +136,24 @@ trait ShowTablesSuiteBase exte

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35797: [SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

dongjoon-hyun commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823364760 ## File path: .github/workflows/ansi_sql_mode_test.yml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or mo

[GitHub] [spark] dongjoon-hyun commented on pull request #35792: [SPARK-38379][K8S]Fix Kubernetes Client mode when mounting persistent volume with storage class

dongjoon-hyun commented on pull request #35792: URL: https://github.com/apache/spark/pull/35792#issuecomment-1063684865 Could you make a backport PR to branch-3.2, @tgravescs ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] HyukjinKwon commented on a change in pull request #35797: [WIP][SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

HyukjinKwon commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823362634 ## File path: .github/workflows/ansi_sql_mode_test.yml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [spark] aokolnychyi commented on a change in pull request #35395: [SPARK-38085][SQL] DataSource V2: Handle DELETE commands for group-based sources

aokolnychyi commented on a change in pull request #35395: URL: https://github.com/apache/spark/pull/35395#discussion_r823362685 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryRowLevelOperationTable.scala ## @@ -0,0 +1,96 @@ +/* + * Lic

[GitHub] [spark] dongjoon-hyun closed pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun closed pull request #35773: URL: https://github.com/apache/spark/pull/35773 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] gengliangwang commented on a change in pull request #35797: [WIP][SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

gengliangwang commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823362395 ## File path: .github/workflows/ansi_sql_mode_test.yml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or mo

[GitHub] [spark] HyukjinKwon commented on a change in pull request #35797: [WIP][SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

HyukjinKwon commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823362301 ## File path: .github/workflows/ansi_sql_mode_test.yml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823355970 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] aokolnychyi commented on a change in pull request #35395: [SPARK-38085][SQL] DataSource V2: Handle DELETE commands for group-based sources

aokolnychyi commented on a change in pull request #35395: URL: https://github.com/apache/spark/pull/35395#discussion_r823361733 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryRowLevelOperationTable.scala ## @@ -0,0 +1,96 @@ +/* + * Lic

[GitHub] [spark] HyukjinKwon commented on a change in pull request #35797: [WIP][SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

HyukjinKwon commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823360784 ## File path: .github/workflows/ansi_sql_mode_test.yml ## @@ -0,0 +1,34 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [spark] aokolnychyi commented on a change in pull request #35395: [SPARK-38085][SQL] DataSource V2: Handle DELETE commands for group-based sources

aokolnychyi commented on a change in pull request #35395: URL: https://github.com/apache/spark/pull/35395#discussion_r823360477 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryRowLevelOperationTable.scala ## @@ -0,0 +1,96 @@ +/* + * Lic

[GitHub] [spark] aokolnychyi commented on a change in pull request #35395: [SPARK-38085][SQL] DataSource V2: Handle DELETE commands for group-based sources

aokolnychyi commented on a change in pull request #35395: URL: https://github.com/apache/spark/pull/35395#discussion_r823359726 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryRowLevelOperationTable.scala ## @@ -0,0 +1,96 @@ +/* + * Lic

[GitHub] [spark] itholic commented on a change in pull request #35794: [SPARK-38491][PYTHON] Support `ignore_index` of `Series.sort_values`

itholic commented on a change in pull request #35794: URL: https://github.com/apache/spark/pull/35794#discussion_r823359195 ## File path: python/pyspark/pandas/frame.py ## @@ -6901,6 +6904,16 @@ def sort_values( 3 D 7 2 2 None 8 4 +

[GitHub] [spark] ulysses-you commented on pull request #35789: [SPARK-32268][SQL] Row-level Runtime Filtering

ulysses-you commented on pull request #35789: URL: https://github.com/apache/spark/pull/35789#issuecomment-1063664600 > I have a question: why do we need Semi-Join if we have Bloom Filter? I guess it is a trade-off between benifits and costs. BloomFilter has false positives issue and

[GitHub] [spark] gengliangwang commented on a change in pull request #35797: [WIP][SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

gengliangwang commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823356873 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2706,7 +2706,7 @@ object SQLConf { "standard di

[GitHub] [spark] itholic commented on pull request #35774: fix minor typos

itholic commented on pull request #35774: URL: https://github.com/apache/spark/pull/35774#issuecomment-1063673051 Can you create a related JIRA and fix the title ?? Also I think we can move "I'm just learning Spark but am assuming that these code examples refer to the RESPECT NULLS s

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][TESTS] Add K8S IT doc for volcano test

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823354093 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] itholic edited a comment on pull request #35774: fix minor typos

itholic edited a comment on pull request #35774: URL: https://github.com/apache/spark/pull/35774#issuecomment-1063673051 Can you create a related [Apache Spark JIRA](https://issues.apache.org/jira/projects/SPARK/summary) and fix the title ?? Also I think we can move "I'm just learni

[GitHub] [spark] sigmod edited a comment on pull request #35789: [SPARK-32268][SQL] Row-level Runtime Filtering

sigmod edited a comment on pull request #35789: URL: https://github.com/apache/spark/pull/35789#issuecomment-1063664882 > I have a question: why do we need Semi-Join if we have Bloom Filter? When Bloom filter is enabled and kicks in, Semi-Join is be needed. It's under a config tha

[GitHub] [spark] sigmod edited a comment on pull request #35789: [SPARK-32268][SQL] Row-level Runtime Filtering

sigmod edited a comment on pull request #35789: URL: https://github.com/apache/spark/pull/35789#issuecomment-1063664882 > I have a question: why do we need Semi-Join if we have Bloom Filter? When Bloom filter is enabled and kicks in, Semi-Join is not needed. It's under a config th

[GitHub] [spark] itholic edited a comment on pull request #35774: fix minor typos

itholic edited a comment on pull request #35774: URL: https://github.com/apache/spark/pull/35774#issuecomment-1063673051 Great! BTW, also we should follow the guide from community. Can you create a related [Apache Spark JIRA](https://issues.apache.org/jira/projects/SPARK/summary) and

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823355970 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] cloud-fan commented on a change in pull request #35758: [SPARK-38439][CORE][SQL] add curly braces with if, for and while statements

cloud-fan commented on a change in pull request #35758: URL: https://github.com/apache/spark/pull/35758#discussion_r823353187 ## File path: common/network-common/src/main/java/org/apache/spark/network/util/LevelDBProvider.java ## @@ -134,8 +134,12 @@ public StoreVersion(@JsonP

[GitHub] [spark] itholic edited a comment on pull request #35774: fix minor typos

itholic edited a comment on pull request #35774: URL: https://github.com/apache/spark/pull/35774#issuecomment-1063673051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823353462 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] cloud-fan commented on pull request #35451: [SPARK-38148][SQL] Do not add dynamic partition pruning if there exists static partition pruning

cloud-fan commented on pull request #35451: URL: https://github.com/apache/spark/pull/35451#issuecomment-1063669513 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823354093 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] sigmod commented on pull request #35789: [SPARK-32268][SQL] Row-level Runtime Filtering

sigmod commented on pull request #35789: URL: https://github.com/apache/spark/pull/35789#issuecomment-1063664882 > I have a question: why do we need Semi-Join if we have Bloom Filter? When Bloom filter is enabled, Semi-Join might not be needed. It's under a config that is disabled

[GitHub] [spark] itholic edited a comment on pull request #35774: fix minor typos

itholic edited a comment on pull request #35774: URL: https://github.com/apache/spark/pull/35774#issuecomment-1063673051 Great! BTW, also we should follow the [contribution guideline ](https://spark.apache.org/contributing.html)from community. Can you create a related [Apache Spark

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823353344 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] dcoliversun commented on a change in pull request #35758: [SPARK-38439][CORE][SQL] add curly braces with if, for and while statements

dcoliversun commented on a change in pull request #35758: URL: https://github.com/apache/spark/pull/35758#discussion_r823354449 ## File path: common/network-common/src/main/java/org/apache/spark/network/util/LevelDBProvider.java ## @@ -134,8 +134,12 @@ public StoreVersion(@Jso

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823353188 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][TESTS] Add K8S IT doc for volcano test

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823353344 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] cloud-fan closed pull request #35451: [SPARK-38148][SQL] Do not add dynamic partition pruning if there exists static partition pruning

cloud-fan closed pull request #35451: URL: https://github.com/apache/spark/pull/35451 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] cloud-fan commented on a change in pull request #35395: [SPARK-38085][SQL] DataSource V2: Handle DELETE commands for group-based sources

cloud-fan commented on a change in pull request #35395: URL: https://github.com/apache/spark/pull/35395#discussion_r823350778 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryRowLevelOperationTable.scala ## @@ -0,0 +1,96 @@ +/* + * Licen

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823353462 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] cloud-fan commented on a change in pull request #35797: [WIP][SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

cloud-fan commented on a change in pull request #35797: URL: https://github.com/apache/spark/pull/35797#discussion_r823352634 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2706,7 +2706,7 @@ object SQLConf { "standard direct

[GitHub] [spark] xinrong-databricks commented on pull request #35747: [SPARK-38441][PYTHON] Support string and bool `regex` in `Series.replace`

xinrong-databricks commented on pull request #35747: URL: https://github.com/apache/spark/pull/35747#issuecomment-1063665449 Thanks @bjornjorgensen! Sorry, I may not fully get your point, why is `bait` confusing? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][DOCS] Add `volcano` section to K8s IT `README.md`

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823355970 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] itholic edited a comment on pull request #35774: fix minor typos

itholic edited a comment on pull request #35774: URL: https://github.com/apache/spark/pull/35774#issuecomment-1063673051 Can you create a related [Apache Spark JIRA](https://issues.apache.org/jira/projects/SPARK/summary) and fix the title ?? Also I think we can move "I'm just learni

[GitHub] [spark] dongjoon-hyun closed pull request #35792: [SPARK-38379][K8S]Fix Kubernetes Client mode when mounting persistent volume with storage class

dongjoon-hyun closed pull request #35792: URL: https://github.com/apache/spark/pull/35792 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][TESTS] Add K8S IT doc for volcano test

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823353462 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] xinrong-databricks commented on pull request #35794: [SPARK-38491][PYTHON] Support `ignore_index` of `Series.sort_values`

xinrong-databricks commented on pull request #35794: URL: https://github.com/apache/spark/pull/35794#issuecomment-1063663793 Also CC @ueshin @itholic Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #35773: [SPARK-38453][K8S][TESTS] Add K8S IT doc for volcano test

dongjoon-hyun commented on a change in pull request #35773: URL: https://github.com/apache/spark/pull/35773#discussion_r823353188 ## File path: resource-managers/kubernetes/integration-tests/README.md ## @@ -305,3 +305,24 @@ You can also specify your specific dockerfile to buil

[GitHub] [spark] dcoliversun commented on pull request #35758: [SPARK-38439][CORE][SQL] add curly braces with if, for and while statements

dcoliversun commented on pull request #35758: URL: https://github.com/apache/spark/pull/35758#issuecomment-1063660761 Sorry to bother you @HyukjinKwon @dongjoon-hyun @martin-g And thanks for your work @HyukjinKwon @yeikel This PR is useless and close it. -- This is an automated me

[GitHub] [spark] dcoliversun closed pull request #35758: [SPARK-38439][CORE][SQL] add curly braces with if, for and while statements

dcoliversun closed pull request #35758: URL: https://github.com/apache/spark/pull/35758 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] gengliangwang opened a new pull request #35797: [WIP][SPARK-38490][SQL][INFRA] Add Github action test job for ANSI SQL mode

gengliangwang opened a new pull request #35797: URL: https://github.com/apache/spark/pull/35797 ### What changes were proposed in this pull request? Add Github action test job for ANSI SQL mode. It will be triggered after each commit push on the master branch of apache/spark.

  1   2   3   4   >