None of the out of the box Unmarshallers for json (like SprayJsonSupport) support parsing in a lazy and streaming way. I did find this repo which looks promising:
https://github.com/knutwalker/akka-stream-json Is this the best kind of approach? It's certainly nice in that you don't have to read all of the data into memory, but large responses are not the norm and are more of the exception for us. On Friday, April 8, 2016 at 8:59:10 AM UTC-4, Chris Baxter wrote: > > Thanks for responding Viktor. > > 1) I see the flaw in this design (flatMapConcat) now. If the response is > chunked you only read the first chunk > 2) I want to be able to parse the body as json and have the final result > of the flow be a Future for some object that I have mapped the response > json to. Any suggestions for doing that w/o reading the entire byte string > into memory? Are you maybe suggesting that instead of feeding something > complete into my json parser (String, Array[Byte]) that I should instead > try and feed in something that is more stream oriented, such as an > InputStream and then find a way to plumb that together with the response > stream? > > Any suggestions for a Flow that can deal with chunk and feed the data into > a parsing stage w/o having to read it all into memory would be greatly > appreciated? I don't need perfect code, just an approach so I can take it > from there. > > On Friday, April 8, 2016 at 7:13:51 AM UTC-4, √ wrote: >> >> >> >> On Fri, Apr 8, 2016 at 1:06 PM, Chris Baxter <cba...@gmail.com> wrote: >> >>> If I want to consume a Http service and then do something with the >>> response body, there are a couple of ways to go about doing that. The two >>> ones that I am trying to decide between are: >>> >>> val f:Future[ByteString] = >>> >>> Source.single(req). >>> >>> via(outgoingConn). >>> >>> flatMapConcat(_.entity.dataBytes). >>> >>> completionTimeout(timeout). >>> >>> runWith(Sink.head) >>> >> >> This does not return the entire body as a ByteString. >> >> >>> and >>> >>> >>> val f:Future[ByteString] = >>> >>> Source.single(req). >>> >>> via(pool). >>> >>> mapAsync(1){ resp => >>> >>> resp.entity.toStrict(timeout).map(_.data ) >>> >>> }. >>> >>> completionTimeout(timeout). >>> >>> runWith(Sink.head) >>> >>> >>> I'm thinking the first approach is the better one. Up until now, my >>> common code for making outbound request has been using the second >>> approach. I'm about to refactor that code into using the first approach as >>> it seems cleaner and requires less use of Futures. Just wanted to see what >>> the consensus from the Akka team and others was on this. >>> >> My question is: why do you need to eagerly read everything into memory to >> "do something with the repsonse body"? >> >> >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Cheers, >> √ >> > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
