HeartSaVioR commented on code in PR #47978:
URL: https://github.com/apache/spark/pull/47978#discussion_r1746661492
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/state/StatePartitionReader.scala:
##########
@@ -166,16 +184,22 @@ class StatePartitionReader(
stateVariableInfoOpt match {
case Some(stateVarInfo) =>
val stateVarType = stateVarInfo.stateVariableType
- val hasTTLEnabled = stateVarInfo.ttlEnabled
stateVarType match {
case StateVariableType.ValueState =>
- if (hasTTLEnabled) {
- SchemaUtil.unifyStateRowPairWithTTL((pair.key, pair.value),
valueSchema,
- partition.partition)
- } else {
- SchemaUtil.unifyStateRowPair((pair.key, pair.value),
partition.partition)
+ SchemaUtil.unifyStateRowPair((pair.key, pair.value),
partition.partition)
+
+ case StateVariableType.ListState =>
+ val key = pair.key
+ val result = store.valuesIterator(key, stateVarName)
+ var unsafeRowArr: Seq[UnsafeRow] = Seq.empty
Review Comment:
Discussed with @cloud-fan . We came up with conclusion that there is no good
solution.
I'm OK with this - we have more time to revisit this (addition of TTL, etc.)
and change our mind later if we think there is a better way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
