viirya commented on code in PR #56102:
URL: https://github.com/apache/spark/pull/56102#discussion_r3301226851
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala:
##########
@@ -50,7 +53,50 @@ class InMemoryTable(
override val id: String = UUID.randomUUID().toString)
extends InMemoryBaseTable(name, columns, partitioning, properties,
constraints, distribution,
ordering, numPartitions, advisoryPartitionSize,
isDistributionStrictlyRequired,
- numRowsPerSplit) with SupportsDelete {
+ numRowsPerSplit) with SupportsDelete with SupportsBranching {
+
+ private val branches = new ConcurrentHashMap[String, TableBranch]()
+
+ override def createBranch(branchName: String,
+ sourceSnapshotId: OptionalLong): TableBranch = {
+ val snapshot = if (sourceSnapshotId.isPresent) sourceSnapshotId.getAsLong
+ else version().toLong
+ val branch = new TableBranch(branchName, snapshot,
System.currentTimeMillis())
+ val existing = branches.putIfAbsent(branchName, branch)
+ if (existing != null) {
+ throw new SupportsBranching.BranchAlreadyExistsException(branchName)
+ }
+ branch
+ }
+
+ override def dropBranch(branchName: String): Boolean = {
+ branches.remove(branchName) != null
+ }
+
+ override def fastForward(branchName: String, targetBranchName: String):
TableBranch = {
+ val current = branches.get(branchName)
+ if (current == null) {
+ throw new java.util.NoSuchElementException(s"Branch not found:
$branchName")
Review Comment:
Agreed — switched to the typed `BranchNotFoundException` introduced in
`SupportsBranching` (per your comment on the interface).
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryTable.scala:
##########
@@ -50,7 +53,50 @@ class InMemoryTable(
override val id: String = UUID.randomUUID().toString)
extends InMemoryBaseTable(name, columns, partitioning, properties,
constraints, distribution,
ordering, numPartitions, advisoryPartitionSize,
isDistributionStrictlyRequired,
- numRowsPerSplit) with SupportsDelete {
+ numRowsPerSplit) with SupportsDelete with SupportsBranching {
+
+ private val branches = new ConcurrentHashMap[String, TableBranch]()
+
+ override def createBranch(branchName: String,
+ sourceSnapshotId: OptionalLong): TableBranch = {
+ val snapshot = if (sourceSnapshotId.isPresent) sourceSnapshotId.getAsLong
+ else version().toLong
+ val branch = new TableBranch(branchName, snapshot,
System.currentTimeMillis())
+ val existing = branches.putIfAbsent(branchName, branch)
+ if (existing != null) {
+ throw new SupportsBranching.BranchAlreadyExistsException(branchName)
+ }
+ branch
+ }
+
+ override def dropBranch(branchName: String): Boolean = {
+ branches.remove(branchName) != null
+ }
+
+ override def fastForward(branchName: String, targetBranchName: String):
TableBranch = {
+ val current = branches.get(branchName)
+ if (current == null) {
+ throw new java.util.NoSuchElementException(s"Branch not found:
$branchName")
+ }
+ val target = branches.get(targetBranchName)
+ if (target == null) {
+ throw new java.util.NoSuchElementException(s"Branch not found:
$targetBranchName")
Review Comment:
Same fix — switching to `BranchNotFoundException`. The
`IllegalArgumentException` for the fast-forward direction check is also now
`InvalidFastForwardException` for the same reason.
On the DSv2 / Iceberg convention question:
- Spark's DSv2 surface uses typed `NoSuch*Exception` classes
(`NoSuchTableException`, `NoSuchPartitionException`,
`NoSuchNamespaceException`) for missing-entity errors; the plain JDK
`NoSuchElementException` is not used.
- Iceberg doesn't have a standardized DSv2 interface for branching at all —
internally it uses `ValidationException` / `IllegalArgumentException` in
`ManageSnapshots` / branching procedures, but that's an Iceberg-internal choice
rather than a DSv2 convention. Since this PR is proposing the DSv2 interface, I
think we should follow Spark's DSv2 naming here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
