pan3793 commented on code in PR #45834:
URL: https://github.com/apache/spark/pull/45834#discussion_r1549005195
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -21,17 +21,57 @@ package org.apache.spark.internal
* All structured logging keys should b
pan3793 commented on code in PR #45834:
URL: https://github.com/apache/spark/pull/45834#discussion_r1549003961
##
common/utils/src/main/scala/org/apache/spark/internal/LogKey.scala:
##
@@ -21,17 +21,57 @@ package org.apache.spark.internal
* All structured logging keys should b
HyukjinKwon commented on PR #45829:
URL: https://github.com/apache/spark/pull/45829#issuecomment-2033674679
what's diff between before and after?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t
beryllw opened a new pull request, #45835:
URL: https://github.com/apache/spark/pull/45835
### What changes were proposed in this pull request?
Some MySQL JDBC drivers converts JSON type into Types.CHAR with a precision
of Int.Max.
When receiving CHAR with Int.Max precision
HyukjinKwon commented on PR #45833:
URL: https://github.com/apache/spark/pull/45833#issuecomment-2033673584
cc @MaxGekk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
yaooqinn commented on code in PR #45751:
URL: https://github.com/apache/spark/pull/45751#discussion_r1548995350
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala:
##
@@ -70,21 +70,21 @@ private case class PostgresDialect() extends JdbcDialect
with SQLCo
HyukjinKwon commented on PR #45776:
URL: https://github.com/apache/spark/pull/45776#issuecomment-2033671882
In addition, the errors I get are:
```
scala> spark.read.json().show()
24/04/03 15:37:24 WARN DataSource: All paths were ignored:
org.apache.spark.sql.Analys
anishshri-db commented on code in PR #45758:
URL: https://github.com/apache/spark/pull/45758#discussion_r1548984946
##
sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala:
##
@@ -676,6 +677,33 @@ class KeyValueGroupedDataset[K, V] private[sql](
)
}
itholic commented on code in PR #45377:
URL: https://github.com/apache/spark/pull/45377#discussion_r1548983997
##
python/pyspark/sql/tests/test_dataframe.py:
##
@@ -825,6 +828,172 @@ def test_duplicate_field_names(self):
self.assertEqual(df.schema, schema)
self
itholic commented on code in PR #45377:
URL: https://github.com/apache/spark/pull/45377#discussion_r1548983997
##
python/pyspark/sql/tests/test_dataframe.py:
##
@@ -825,6 +828,172 @@ def test_duplicate_field_names(self):
self.assertEqual(df.schema, schema)
self
gengliangwang commented on code in PR #45808:
URL: https://github.com/apache/spark/pull/45808#discussion_r1548975304
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala:
##
@@ -532,7 +533,8 @@ class ExecutorPodsA
gengliangwang commented on code in PR #45808:
URL: https://github.com/apache/spark/pull/45808#discussion_r1548975304
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala:
##
@@ -532,7 +533,8 @@ class ExecutorPodsA
gengliangwang commented on PR #45834:
URL: https://github.com/apache/spark/pull/45834#issuecomment-2033641746
cc @panbingkun @pan3793 @itholic as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
dongjoon-hyun closed pull request #45832: [SPARK-47452][INFRA][FOLLOWUP]
Enforce to install `six` to `Python 3.10`
URL: https://github.com/apache/spark/pull/45832
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
dongjoon-hyun commented on PR #45832:
URL: https://github.com/apache/spark/pull/45832#issuecomment-2033641613
Thank you, @HyukjinKwon !
Let me merge this since the image building is already tested.
--
This is an automated message from the Apache Git Service.
To respond to the message,
gengliangwang opened a new pull request, #45834:
URL: https://github.com/apache/spark/pull/45834
### What changes were proposed in this pull request?
Migrate logError with variables to structured logging framework. This is
part1 for the logError entries of the following API in
sadikovi opened a new pull request, #45833:
URL: https://github.com/apache/spark/pull/45833
### What changes were proposed in this pull request?
This PR fixes a bug that was introduced by
[SPARK-47704](https://issues.apache.org/jira/browse/SPARK-47704). To be
precise, SPA
cloud-fan commented on code in PR #45751:
URL: https://github.com/apache/spark/pull/45751#discussion_r1548965732
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala:
##
@@ -70,21 +70,21 @@ private case class PostgresDialect() extends JdbcDialect
with SQLC
dongjoon-hyun commented on code in PR #45751:
URL: https://github.com/apache/spark/pull/45751#discussion_r1548963207
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala:
##
@@ -70,21 +70,21 @@ private case class PostgresDialect() extends JdbcDialect
with
pan3793 commented on code in PR #45808:
URL: https://github.com/apache/spark/pull/45808#discussion_r1548962525
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala:
##
@@ -532,7 +533,8 @@ class ExecutorPodsAllocat
anishshri-db commented on code in PR #45778:
URL: https://github.com/apache/spark/pull/45778#discussion_r1548956564
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreSuite.scala:
##
@@ -294,6 +295,60 @@ class RocksDBStateStoreSuite extend
gengliangwang commented on code in PR #45808:
URL: https://github.com/apache/spark/pull/45808#discussion_r1548956155
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala:
##
@@ -532,7 +533,8 @@ class ExecutorPodsA
pan3793 commented on code in PR #45808:
URL: https://github.com/apache/spark/pull/45808#discussion_r1548950710
##
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala:
##
@@ -532,7 +533,8 @@ class ExecutorPodsAllocat
HeartSaVioR commented on code in PR #45778:
URL: https://github.com/apache/spark/pull/45778#discussion_r1548946531
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateEncoder.scala:
##
@@ -276,53 +284,111 @@ class RangeKeyScanStateEncoder(
dongjoon-hyun commented on PR #45827:
URL: https://github.com/apache/spark/pull/45827#issuecomment-2033576501
Merged to master for Apache Spark 4.0.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
dongjoon-hyun closed pull request #45827: [SPARK-47701][SQL][TESTS] Postgres:
Add test for Composite and Range types
URL: https://github.com/apache/spark/pull/45827
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
dongjoon-hyun commented on PR #45832:
URL: https://github.com/apache/spark/pull/45832#issuecomment-2033569304
Could you review this PR, @HyukjinKwon ? This PR will recover the `Python
3.10` CI pipeline.
--
This is an automated message from the Apache Git Service.
To respond to the message
dongjoon-hyun closed pull request #45830:
[SPARK-45733][PYTHON][TESTS][FOLLOWUP] Skip
`pyspark.sql.tests.connect.client.test_client` if not should_test_connect
URL: https://github.com/apache/spark/pull/45830
--
This is an automated message from the Apache Git Service.
To respond to the messa
dongjoon-hyun commented on PR #45830:
URL: https://github.com/apache/spark/pull/45830#issuecomment-2033561359
Thank you, @HyukjinKwon . Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
HeartSaVioR commented on code in PR #45758:
URL: https://github.com/apache/spark/pull/45758#discussion_r1548917934
##
sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala:
##
@@ -676,6 +677,33 @@ class KeyValueGroupedDataset[K, V] private[sql](
)
}
dongjoon-hyun opened a new pull request, #45832:
URL: https://github.com/apache/spark/pull/45832
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### H
gengliangwang commented on code in PR #45822:
URL: https://github.com/apache/spark/pull/45822#discussion_r1548927521
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala:
##
@@ -1235,11 +1235,13 @@ object TryToTimestampExpressionBuil
cloud-fan commented on code in PR #45822:
URL: https://github.com/apache/spark/pull/45822#discussion_r1548924394
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala:
##
@@ -1235,11 +1235,13 @@ object TryToTimestampExpressionBuilder
cloud-fan commented on code in PR #45822:
URL: https://github.com/apache/spark/pull/45822#discussion_r1548924928
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala:
##
@@ -1235,11 +1235,13 @@ object TryToTimestampExpressionBuilder
zhengruifeng opened a new pull request, #45831:
URL: https://github.com/apache/spark/pull/45831
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### Ho
yaooqinn commented on PR #45731:
URL: https://github.com/apache/spark/pull/45731#issuecomment-2033504024
@vadim can you revise the PR description?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
vadim commented on PR #45731:
URL: https://github.com/apache/spark/pull/45731#issuecomment-2033499194
first example:
https://github.com/apache/spark/assets/86705/8a604c35-b067-4ad3-9670-9f3a00d01a20";>
second example:
https://github.com/apache/spark/assets/86705/097cbb82-cc95-496
dongjoon-hyun commented on PR #45776:
URL: https://github.com/apache/spark/pull/45776#issuecomment-2033496439
To be sure, gentle ping once more, @HyukjinKwon , @cloud-fan , @LuciferYang
, @koedlt
- https://github.com/apache/spark/pull/45776#discussion_r1545520694
- https://github.com/
HeartSaVioR commented on PR #45768:
URL: https://github.com/apache/spark/pull/45768#issuecomment-2033495338
Late +1. If we can add more advanced test cases e.g. multiple partitions
even better, which we could revisit in e2e test for python streaming source to
python streaming sink.
--
Th
gengliangwang commented on code in PR #45822:
URL: https://github.com/apache/spark/pull/45822#discussion_r1548887170
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala:
##
@@ -1235,11 +1235,13 @@ object TryToTimestampExpressionBuil
dongjoon-hyun commented on PR #45830:
URL: https://github.com/apache/spark/pull/45830#issuecomment-2033491877
cc @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment
HeartSaVioR commented on code in PR #45778:
URL: https://github.com/apache/spark/pull/45778#discussion_r154888
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateEncoder.scala:
##
@@ -276,53 +284,111 @@ class RangeKeyScanStateEncoder(
dongjoon-hyun closed pull request #45828:
[SPARK-47454][PYTHON][TESTS][FOLLOWUP] Skip
`test_create_dataframe_from_pandas_with_day_time_interval` if pandas is not
avaiable
URL: https://github.com/apache/spark/pull/45828
--
This is an automated message from the Apache Git Service.
To respond
dongjoon-hyun commented on PR #45828:
URL: https://github.com/apache/spark/pull/45828#issuecomment-2033488061
Thank you, @yaooqinn ! Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
dongjoon-hyun commented on code in PR #45830:
URL: https://github.com/apache/spark/pull/45830#discussion_r1548879653
##
python/pyspark/sql/tests/connect/client/test_client.py:
##
@@ -134,18 +245,6 @@ def test_channel_builder_with_session(self):
self.assertEqual(client._
yaooqinn commented on PR #45827:
URL: https://github.com/apache/spark/pull/45827#issuecomment-2033481150
Thank you @dongjoon-hyun, I further added tests for Range types to reduce #
of small PRs
--
This is an automated message from the Apache Git Service.
To respond to the message, please
dongjoon-hyun opened a new pull request, #45830:
URL: https://github.com/apache/spark/pull/45830
…
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
yaooqinn commented on code in PR #45751:
URL: https://github.com/apache/spark/pull/45751#discussion_r1548873575
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala:
##
@@ -70,21 +70,21 @@ private case class PostgresDialect() extends JdbcDialect
with SQLCo
Zouxxyy opened a new pull request, #45829:
URL: https://github.com/apache/spark/pull/45829
### What changes were proposed in this pull request?
Modify the simpleString of DataSourceV2ScanRelation to distinguish it from
DataSourceV2Relation
### Why are the changes needed
cloud-fan commented on code in PR #45751:
URL: https://github.com/apache/spark/pull/45751#discussion_r1548869846
##
sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala:
##
@@ -70,21 +70,21 @@ private case class PostgresDialect() extends JdbcDialect
with SQLC
cloud-fan commented on code in PR #45822:
URL: https://github.com/apache/spark/pull/45822#discussion_r1548867885
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala:
##
@@ -1235,11 +1235,13 @@ object TryToTimestampExpressionBuilder
sahnib commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548800286
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateTTLSuite.scala:
##
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
sahnib commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548864182
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TTLState.scala:
##
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under o
dongjoon-hyun opened a new pull request, #45828:
URL: https://github.com/apache/spark/pull/45828
…
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
HeartSaVioR closed pull request #45780: [SPARK-47655][SS] Integrate timer with
Initial State handling for state-v2
URL: https://github.com/apache/spark/pull/45780
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
HeartSaVioR commented on PR #45780:
URL: https://github.com/apache/spark/pull/45780#issuecomment-2033458353
Thanks! Merging to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
yaooqinn opened a new pull request, #45827:
URL: https://github.com/apache/spark/pull/45827
### What changes were proposed in this pull request?
Add tests for complex types of postgres.
### Why are the changes needed?
test improvments
### Does this
yaooqinn commented on PR #45815:
URL: https://github.com/apache/spark/pull/45815#issuecomment-2033450239
Thank you @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific com
panbingkun commented on PR #45813:
URL: https://github.com/apache/spark/pull/45813#issuecomment-2033440717
> > Do we need to support the second(Sting + MDC) and third(MDC + Sting)
methods besides the first one?
>
> So by the end of this project, all the log entries containing variable
dongjoon-hyun closed pull request #45815: [SPARK-47691][SQL] Postgres: Support
multi dimensional array on the write side
URL: https://github.com/apache/spark/pull/45815
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
dongjoon-hyun commented on code in PR #45425:
URL: https://github.com/apache/spark/pull/45425#discussion_r1548842531
##
common/network-common/src/main/java/org/apache/spark/network/crypto/AuthEngine.java:
##
@@ -224,7 +236,7 @@ private TransportCipher generateTransportCipher(
dongjoon-hyun commented on code in PR #45425:
URL: https://github.com/apache/spark/pull/45425#discussion_r1548843062
##
common/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java:
##
@@ -213,6 +213,11 @@ public boolean encryptionEnabled() {
return
gengliangwang commented on PR #45813:
URL: https://github.com/apache/spark/pull/45813#issuecomment-2033407009
> Do we need to support the second(Sting + MDC) and third(MDC + Sting)
methods besides the first one?
So by the end of this project, all the log entries containing variables w
mridulm commented on code in PR #45425:
URL: https://github.com/apache/spark/pull/45425#discussion_r1548815203
##
common/network-common/src/main/java/org/apache/spark/network/crypto/README.md:
##
@@ -99,3 +103,13 @@ sessions. It would, however, allow impersonation of future
ses
HyukjinKwon commented on PR #45377:
URL: https://github.com/apache/spark/pull/45377#issuecomment-2033393150
cc @cloud-fan too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
HyukjinKwon commented on code in PR #45377:
URL: https://github.com/apache/spark/pull/45377#discussion_r1548814091
##
python/pyspark/testing/utils.py:
##
@@ -280,7 +282,14 @@ def check_error(
exception: PySparkException,
error_class: str,
message_param
yaooqinn commented on PR #45815:
URL: https://github.com/apache/spark/pull/45815#issuecomment-2033390778
cc @cloud-fan @dongjoon-hyun thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
HyukjinKwon commented on code in PR #45377:
URL: https://github.com/apache/spark/pull/45377#discussion_r1548812966
##
python/pyspark/errors/exceptions/captured.py:
##
@@ -379,5 +379,17 @@ def fragment(self) -> str:
def callSite(self) -> str:
return str(self._q.call
HyukjinKwon commented on code in PR #45806:
URL: https://github.com/apache/spark/pull/45806#discussion_r1548811808
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/variantExpressions.scala:
##
@@ -403,3 +405,134 @@ object VariantGetExpressionBuild
anishshri-db commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548810035
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala:
##
@@ -103,22 +113,35 @@ class StatefulProcessorHandleImpl(
anishshri-db commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548810035
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala:
##
@@ -103,22 +113,35 @@ class StatefulProcessorHandleImpl(
anishshri-db commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548807708
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TTLState.scala:
##
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) u
HyukjinKwon commented on code in PR #45826:
URL: https://github.com/apache/spark/pull/45826#discussion_r1548802161
##
python/pyspark/sql/types.py:
##
@@ -1468,6 +1475,36 @@ def __eq__(self, other: Any) -> bool:
return type(self) == type(other)
+class VariantVal:
Re
anishshri-db commented on code in PR #45778:
URL: https://github.com/apache/spark/pull/45778#discussion_r1548801165
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateEncoder.scala:
##
@@ -276,53 +283,113 @@ class RangeKeyScanStateEncoder(
anishshri-db commented on code in PR #45778:
URL: https://github.com/apache/spark/pull/45778#discussion_r1548800791
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreSuite.scala:
##
@@ -294,6 +295,55 @@ class RocksDBStateStoreSuite extend
anishshri-db commented on code in PR #45778:
URL: https://github.com/apache/spark/pull/45778#discussion_r1548800930
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreSuite.scala:
##
@@ -294,6 +295,55 @@ class RocksDBStateStoreSuite extend
sahnib commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548800286
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateTTLSuite.scala:
##
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
sahnib commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548795527
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateTTLSuite.scala:
##
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
sahnib commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548791096
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateTTLSuite.scala:
##
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
sahnib commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548791295
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateTTLSuite.scala:
##
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
cloud-fan commented on code in PR #45383:
URL: https://github.com/apache/spark/pull/45383#discussion_r1548791139
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CollationTypeCasts.scala:
##
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundatio
cloud-fan commented on code in PR #45383:
URL: https://github.com/apache/spark/pull/45383#discussion_r1548790383
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CollationTypeCasts.scala:
##
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundatio
sahnib commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548788984
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateTTLSuite.scala:
##
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
anishshri-db commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548788722
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateTTLSuite.scala:
##
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache Software Foundation
HyukjinKwon commented on PR #45053:
URL: https://github.com/apache/spark/pull/45053#issuecomment-2033351917
cc @zhengruifeng @grundprinzip @ueshin @hvanhovell @itholic @WeichenXu123
@mengxr @allisonwang-db @xinrong-meng @gatorsmile @cloud-fan This is ready for
a look (before merging, should
sahnib commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548787953
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/ValueStateSuite.scala:
##
@@ -303,6 +311,244 @@ class ValueStateSuite extends StateVariableSuit
panbingkun commented on PR #45813:
URL: https://github.com/apache/spark/pull/45813#issuecomment-2033348771
@gengliangwang
Do we need to support the `second`(Sting + MDC) and `third`(MDC + Sting)
methods besides the `first` one?
--
This is an automated message from the Apache Git Servi
chenhao-db commented on PR #45807:
URL: https://github.com/apache/spark/pull/45807#issuecomment-2033348791
@HyukjinKwon Sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
anishshri-db commented on code in PR #45674:
URL: https://github.com/apache/spark/pull/45674#discussion_r1548785634
##
sql/core/src/test/scala/org/apache/spark/sql/streaming/TransformWithStateTTLSuite.scala:
##
@@ -0,0 +1,579 @@
+/*
+ * Licensed to the Apache Software Foundation
neilramaswamy commented on code in PR #45778:
URL: https://github.com/apache/spark/pull/45778#discussion_r1548751462
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateEncoder.scala:
##
@@ -276,53 +283,113 @@ class RangeKeyScanStateEncoder(
panbingkun commented on code in PR #45808:
URL: https://github.com/apache/spark/pull/45808#discussion_r1548785661
##
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala:
##
@@ -854,7 +856,8 @@ private[spark] class ApplicationMaster(
zhengruifeng commented on code in PR #45614:
URL: https://github.com/apache/spark/pull/45614#discussion_r1548784881
##
connector/connect/common/src/main/protobuf/spark/connect/relations.proto:
##
@@ -131,6 +132,23 @@ message SQL {
repeated Expression pos_arguments = 5;
}
+
HyukjinKwon commented on PR #45807:
URL: https://github.com/apache/spark/pull/45807#issuecomment-2033343284
Can you keep the PR template please?
https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE
--
This is an automated message from the Apache Git Service.
To respon
HyukjinKwon commented on code in PR #45816:
URL: https://github.com/apache/spark/pull/45816#discussion_r1548780039
##
common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java:
##
@@ -107,6 +107,28 @@ public void binaryCompareTo() {
assertTrue(fromStrin
HyukjinKwon commented on code in PR #45816:
URL: https://github.com/apache/spark/pull/45816#discussion_r1548779756
##
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java:
##
@@ -447,6 +447,37 @@ private UTF8String toUpperCaseSlow() {
return fromString(
HyukjinKwon commented on PR #45820:
URL: https://github.com/apache/spark/pull/45820#issuecomment-207001
Let's follow https://github.com/databricks/scala-style-guide, and remove
unrelated changes, e.g., adding newlines which makes
cherry-pick/backporting/reverting difficult.
--
This i
HyukjinKwon commented on PR #45819:
URL: https://github.com/apache/spark/pull/45819#issuecomment-206686
Let's follow https://github.com/databricks/scala-style-guide, and remove
unrelated changes, e.g., adding newlines which makes
cherry-pick/backporting/reverting difficult.
--
This i
HyukjinKwon commented on code in PR #45819:
URL: https://github.com/apache/spark/pull/45819#discussion_r1548778640
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala:
##
@@ -509,6 +497,209 @@ class CollationSuite extends DatasourceV2SQLBase with
AdaptiveSparkP
HyukjinKwon commented on code in PR #45819:
URL: https://github.com/apache/spark/pull/45819#discussion_r1548778507
##
sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala:
##
@@ -20,57 +20,60 @@ package org.apache.spark.sql
import scala.collection.
HyukjinKwon commented on code in PR #45819:
URL: https://github.com/apache/spark/pull/45819#discussion_r1548778237
##
sql/core/src/test/scala/org/apache/spark/sql/CollationRegexpExpressionsSuite.scala:
##
@@ -20,421 +20,406 @@ package org.apache.spark.sql
import scala.collectio
1 - 100 of 293 matches
Mail list logo
