queimadus commented on code in PR #1622:
URL: https://github.com/apache/pekko/pull/1622#discussion_r1894903082
##########
stream/src/main/scala/org/apache/pekko/stream/impl/fusing/FlatMapPrefix.scala:
##########
@@ -128,8 +129,8 @@ import pekko.util.OptionVal
def materializeFlow(): Unit =
try {
- val prefix = accumulated.toVector
- accumulated.clear()
+ val prefix = builder.result()
+ builder = null // free for GC
Review Comment:
My original code before I simplified it with what looked to be the MRE, was
more of a:
```scala
def myLogic(prefix: Seq[ByteString]): Flow[ByteString, ByteString,
NotUsed] =
Flow[ByteString]
val s = Source
.repeat(())
.map(_ => ByteString('a' * 400000))
.take(1000000)
.prefixAndTail(50000)
.flatMapConcat { case (prefix, tail) =>
Source(prefix).concatLazy(tail).via(myLogic(prefix))
}
Source.empty
.concatAllLazy(List.tabulate(30000)(_ => s): _*)
.runWith(Sink.ignore).onComplete(println(_))
```
Where I would need to use the prefix to generate the remaining flow logic
and then emit the prefix back again. It still runs out of heap in this case,
but it may be a different problem?
Nonetheless, I think this change makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
