LuciferYang commented on code in PR #40610:
URL: https://github.com/apache/spark/pull/40610#discussion_r1168134446
##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/connect/client/SparkResult.scala:
##########
@@ -46,7 +46,13 @@ private[sql] class SparkResult[T](
private[this] var numRecords: Int = 0
private[this] var structType: StructType = _
private[this] var boundEncoder: ExpressionEncoder[T] = _
- private[this] val batches = mutable.Buffer.empty[ColumnarBatch]
+ private[this] var nextBatchIndex: Int = 0
+ private[this] val idxToBatches = mutable.Map.empty[Int, ColumnarBatch]
+
+ // Exposed only for UT.
+ private[sql] def existingBatches(): Seq[ColumnarBatch] = {
+ idxToBatches.values.toSeq
Review Comment:
The order of `idxToBatches.values.toSeq` may be different from
`idxToBatches.put`, and this is related to the Scala version.
For example
```
val idxToBatches = scala.collection.mutable.Map.empty[Int, Long]
idxToBatches.put(0, 1)
idxToBatches.put(1, 2)
idxToBatches.put(2, 3)
idxToBatches.put(3, 4)
idxToBatches.put(4, 5)
idxToBatches.put(5, 6)
```
the result of `idxToBatches.values.toSeq` is `3, 6, 5, 2, 4, 1` with Scala
2.12 and `1,
2, 3, 4, 5, 6 ` with Scala 2.13. I think a sorting should be added to make
it a stable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
