Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18107
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119172385
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
---
@@ -47,50 +44,54 @@ trait StateStore {
/**
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119172218
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -828,6 +837,8 @@ class SQLConf extends Serializable with Logging {
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119172029
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -165,54 +189,88 @@ case class StateStoreSaveExec(
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118608662
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -828,6 +837,8 @@ class SQLConf extends Serializable with Logging {
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119014976
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -273,27 +333,34 @@ case class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119016141
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -165,54 +189,88 @@ case class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118802674
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -552,6 +552,15 @@ object SQLConf {
.booleanConf
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119014965
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -253,6 +311,8 @@ case class StateStoreSaveExec(
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119014982
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -304,8 +371,9 @@ case class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118804254
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -719,3 +745,23 @@ object ThrowingInterruptedIOException {
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119014511
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
---
@@ -47,50 +44,54 @@ trait StateStore {
/**
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118615627
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -61,11 +60,24 @@ trait StateStoreReader extends
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119014924
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -165,54 +189,88 @@ case class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119014948
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -165,54 +189,88 @@ case class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119013976
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
---
@@ -102,28 +103,100 @@ trait StateStore {
}
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r119014486
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
---
@@ -47,50 +44,54 @@ trait StateStore {
/**
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118595928
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala
---
@@ -508,22 +508,6 @@ class FlatMapGroupsWithStateSuite
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118466139
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
---
@@ -165,54 +189,87 @@ case class StateStoreSaveExec(
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118465937
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
---
@@ -29,12 +29,25 @@ import org.apache.spark.SparkEnv
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118465998
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
---
@@ -105,25 +102,42 @@ trait StateStore {
/** Trait
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118465966
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
---
@@ -47,31 +60,25 @@ trait StateStore {
/**
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/18107#discussion_r118465084
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -552,6 +552,15 @@ object SQLConf {
.booleanConf
GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/18107
[SPARK-20883][SPARK-20376][SS] Refactored StateStore APIs and added conf to
choose implementation
## What changes were proposed in this pull request?
A bunch of changes to the
24 matches
Mail list logo
