Is the JSON well formed and "normal" or something weird or maybe huge
objects or something in there etc?
Try to debug at which point it gets stuck.

A minimized reproducer would help.

-- 
Konrad `ktoso` Malawski
Akka <http://akka.io> @ Lightbend <http://lightbend.com>

On 6 December 2016 at 20:26:43, subo...@gmail.com (subo...@gmail.com) wrote:

I working on a Akka Streams project that reads gzipped files from S3 and
parses json. The issue I'm running into the stream stalls at about 24523530
bytes and then times-out after a 1 minute
(java.util.concurrent.TimeoutException: No elements passed in the last 1
minute), but there is no error otherwise. If I remove the JsonFraming.
objectScanner from the flow, it does not exhibit this behavior. The stream
looks like:

val s3FileSource: Source[ByteString, NotUsed] =
s3FileNamesSource.via(Flows.downloadObjectsFlow(s3Client, bucket))


  val result = s3FileSource
    .via(Compression.gunzip(100))
    .via(JsonFraming.objectScanner(Int.MaxValue))
    .runWith(FileIO.toPath(Paths.get("stream.out"), Set(CREATE, WRITE)))


The json that is does write to the file looks just fine. I'm on Akka
2.4.14. Any ideas on what might be going on or how to troubleshoot?
Thanks,


Andrew


--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>>>>>>>>>      Read the docs: http://akka.io/docs/
>>>>>>>>>>      Check the FAQ: 
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>      Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Reply via email to