I saw, sometimes, I get into below issue -
I use : where *emptyResult* of type : Map[String, List[(String, JValue)]]
Note that, Maps *acc* and *curr *are getting bigger and bigger. It might
have some GB of data while combining at last step.
Sink.fold(Future{emptyResult}){
case (acc, curr) => {
acc zip curr map Function.tupled(_ ++ _)}
}
At ultimate step, it fails. These list has
On Thursday, November 9, 2017 at 5:13:45 PM UTC-5, bord...@gmail.com wrote:
>
> Hello,
>
> I have API response coming as a JSON string where Server won't support
> chunked response. I'm using below sample code to parse the JSON using
> *Jackson
> parser*.
>
> type Result = Map[String, List[(String, JValue)]]
>
> private def apiResult(resp : HttpResponse): Future[Result] = {
> Unmarshal(resp.entity.withoutSizeLimit).to[String] map {
> jsonResp =>
> getResultMap(jsonResp) // JSON parsing using Jackson
> }
> }
>
>
>
> I'm hitting API for for 10k different results. Some of the JSON results
> are very large, I'm finding below error ultimately (same error encounters
> even after removing *withoutSizeLimit *in response entity) :
>
> [ERROR] [11/09/2017 11:33:58.337] [myactor-threadpool-dispatcher-7]
> [TaskInvocation] HTTP chunk size exceeds the configured limit of 1048576
> bytes
> akka.http.scaladsl.model.EntityStreamException: HTTP chunk size exceeds
> the configured limit of 1048576 bytes
> at
> akka.http.impl.engine.parsing.HttpMessageParser$$anonfun$2.applyOrElse(HttpMessageParser.scala:329)
> at
> akka.http.impl.engine.parsing.HttpMessageParser$$anonfun$2.applyOrElse(HttpMessageParser.scala:327)
> at
> akka.stream.impl.fusing.Collect$$anon$2.$anonfun$wrappedPf$1(Ops.scala:212)
> at
> akka.stream.impl.fusing.SupervisedGraphStageLogic.withSupervision(Ops.scala:173)
> at akka.stream.impl.fusing.Collect$$anon$2.onPush(Ops.scala:214)
> at
> akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:499)
> at
> akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:462)
> at
> akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:368)
> at
> akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:571)
> at
> akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:457)
> at
> akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:546)
> at
> akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:728)
> at
> akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:743)
> at akka.actor.Actor.aroundReceive(Actor.scala:517)
> at akka.actor.Actor.aroundReceive$(Actor.scala:515)
> at
> akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:653)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
> at akka.actor.ActorCell.invoke(ActorCell.scala:496)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>
> How can i solve my problem?
> Do you think, JSON Streaming can help here for me (Entity size might again
> be an issue as size is restricted to certain limit) ?
>
>
> Regards,
> Aditya Borde
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
