HyukjinKwon closed pull request #46570: [SPARK-48258][PYTHON][CONNECT]
Checkpoint and localCheckpoint in Spark Connect
URL: https://github.com/apache/spark/pull/46570
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
HyukjinKwon commented on PR #46570:
URL: https://github.com/apache/spark/pull/46570#issuecomment-2121464274
Merged to master.
I will followup the discussion if there are more to address since we're
releasing preview soon.
--
This is an automated message from the Apache Git Service.
HyukjinKwon commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606684213
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala:
##
@@ -3507,6 +3513,47 @@ class SparkConnectPlanner(
HyukjinKwon commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606678743
##
connector/connect/common/src/main/protobuf/spark/connect/commands.proto:
##
@@ -484,3 +486,26 @@ message CreateResourceProfileCommandResult {
// (Required) Se
HyukjinKwon commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606673393
##
python/pyspark/sql/connect/dataframe.py:
##
@@ -138,6 +138,41 @@ def __init__(
# by __repr__ and _repr_html_ while eager evaluation opens.
self
HyukjinKwon commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606672819
##
python/pyspark/sql/connect/dataframe.py:
##
@@ -138,6 +138,41 @@ def __init__(
# by __repr__ and _repr_html_ while eager evaluation opens.
self
hvanhovell commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606497789
##
python/pyspark/sql/connect/dataframe.py:
##
@@ -138,6 +138,41 @@ def __init__(
# by __repr__ and _repr_html_ while eager evaluation opens.
self.
hvanhovell commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606484039
##
connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala:
##
@@ -3507,6 +3513,47 @@ class SparkConnectPlanner(
hvanhovell commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606482770
##
connector/connect/common/src/main/protobuf/spark/connect/commands.proto:
##
@@ -484,3 +486,26 @@ message CreateResourceProfileCommandResult {
// (Required) Ser
hvanhovell commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1606482118
##
connector/connect/common/src/main/protobuf/spark/connect/commands.proto:
##
@@ -484,3 +486,26 @@ message CreateResourceProfileCommandResult {
// (Required) Ser
hvanhovell commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1605509972
##
connector/connect/common/src/main/protobuf/spark/connect/base.proto:
##
@@ -199,6 +200,17 @@ message AnalyzePlanRequest {
// (Required) The logical plan to g
HyukjinKwon commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1604289643
##
python/pyspark/sql/connect/dataframe.py:
##
@@ -104,6 +107,8 @@
class DataFrame(ParentDataFrame):
+_release_thread_pool: Optional[ThreadPool] = ThreadPo
zhengruifeng commented on code in PR #46570:
URL: https://github.com/apache/spark/pull/46570#discussion_r1604267324
##
python/pyspark/sql/connect/dataframe.py:
##
@@ -104,6 +107,8 @@
class DataFrame(ParentDataFrame):
+_release_thread_pool: Optional[ThreadPool] = ThreadP
HyukjinKwon opened a new pull request, #46570:
URL: https://github.com/apache/spark/pull/46570
### What changes were proposed in this pull request?
TBD
### Why are the changes needed?
TBD
### Does this PR introduce _any_ user-facing change?
TBD
### Ho
14 matches
Mail list logo
