sunchao commented on code in PR #57219:
URL: https://github.com/apache/spark/pull/57219#discussion_r3641748848
##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/ui/SparkConnectServerAppStatusStore.scala:
##########
@@ -74,11 +74,16 @@ class SparkConnectServerAppStatusStore(store: KVStore) {
}
private[connect] class SessionInfo(
- @KVIndexParam val sessionId: String,
+ val sessionId: String,
val startTimestamp: Long,
val userId: String,
val finishTimestamp: Long,
val totalExecution: Long) {
+ // Natural key. A session is identified by (userId, sessionId), since two
users may share the
+ // same session UUID; keying on sessionId alone would merge them into one
record.
+ @KVIndexParam
Review Comment:
[P2 remains at `854d578500e4f0633aa3d8d11dcf8a81eed09dff`] The fallback
recovers an uncollided legacy row, but it does not handle the collision case
called out here. Lines 58–59 now explicitly acknowledge that rows already
merged by the old `sessionId` key cannot be recovered: the cached store
contains only one user's row/merged counters, so there is no second `(userId,
sessionId)` value for the scan to find. Because
`AppStatusStore.CURRENT_VERSION` is still 2, `FsHistoryProvider.loadDiskStore`
reopens that cache without replay, leaving the exact same-UUID History Server
bug until cache eviction. The new test writes a current-schema composite-key
`SessionInfo` and only forces `read` to miss; it does not exercise a
pre-upgrade on-disk key or a collapsed pair. Please invalidate/replay affected
stores or add a migration that can reconstruct them from the event log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
