[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1157597665 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,126 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1157385141 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,118 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156583318 ## connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala: ## @@ -1969,6 +2017,142 @@ class SparkConnectPlanner(val

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156569175 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,118 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156271993 ## python/pyspark/sql/dataframe.py: ## @@ -529,14 +529,10 @@ def writeStream(self) -> DataStreamWriter: >>> import tempfile >>> df

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156265721 ## connector/connect/common/src/main/protobuf/spark/connect/relations.proto: ## @@ -154,6 +158,9 @@ message Read { // // This is only supported by the JDBC

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156239889 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,118 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156216828 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,118 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156216828 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,118 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156213953 ## connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala: ## @@ -816,6 +836,27 @@ class SparkConnectPlanner(val

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1156212372 ## connector/connect/common/src/main/protobuf/spark/connect/relations.proto: ## @@ -154,6 +158,9 @@ message Read { // // This is only supported by the JDBC

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1153908962 ## python/pyspark/sql/connect/session.py: ## @@ -14,6 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License.

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1153908437 ## python/pyspark/sql/connect/session.py: ## @@ -489,10 +495,6 @@ def sparkContext(self) -> Any: def streams(self) -> Any: raise

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1153904387 ## python/pyspark/sql/connect/readwriter.py: ## @@ -37,7 +37,7 @@ from pyspark.sql.connect._typing import ColumnOrName, OptionalPrimitiveType from

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1153813344 ## connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala: ## @@ -1969,6 +2014,136 @@ class SparkConnectPlanner(val

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1153629770 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,97 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1152453603 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,97 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1152452932 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,97 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1152451343 ## python/pyspark/sql/connect/streaming/query.py: ## @@ -0,0 +1,173 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1152436270 ## connector/connect/common/src/main/protobuf/spark/connect/commands.proto: ## @@ -177,3 +179,97 @@ message WriteOperationV2 { // (Optional) A condition for

[GitHub] [spark] rangadi commented on a diff in pull request #40586: [SPARK-42939][SS][CONNECT] Core streaming Python API for Spark Connect

rangadi commented on code in PR #40586: URL: https://github.com/apache/spark/pull/40586#discussion_r1152315438 ## connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala: ## @@ -1969,6 +2014,136 @@ class SparkConnectPlanner(val