Thanks Martynas I just wanted to check what is the difference between
val source = Source.synchronousFile(file) and val source = scala.io.Source.fromFile(file).getLines()) (as this also returns Iterator) as I can use it as source.map(_.utf8String).runForeach(println) Are there any performance issues with the seconds approach (scala.io.Source.fromFile(file).getLines()))? Thanks On Sun, Oct 11, 2015 at 9:36 AM, Martynas Mickevičius < martynas.mickevic...@typesafe.com> wrote: > Hi Harit, > > according to the docs > <http://doc.akka.io/api/akka-stream-and-http-experimental/1.0/?_ga=1.121420744.1150407535.1394739016#akka.stream.io.Framing$> > allowTruncation > controls whether to fail the stream if the last upstream line does not > contain a delimiter. > > The built-in delimiter framing > <https://github.com/akka/akka/blob/97622c22b151eeb230302aff6479780690b42b8d/akka-stream/src/main/scala/akka/stream/io/Framing.scala#L34> > stage does not handle all use cases. However it is a good example of such > functionality. I would recommend taking a look at the source code and > adapting it to your needs. > > On Tue, Oct 6, 2015 at 10:28 PM, Harit Himanshu < > harit.subscripti...@gmail.com> wrote: > >> Hello there, >> >> I am new to Akka-Stream and working on a use case where I need to parse >> log files. These log files have lines separated by new line. >> >> I was looked at GroupLogFile.scala >> <https://github.com/typesafehub/activator-akka-stream-scala/blob/master/src/main/scala/sample/stream/GroupLogFile.scala> >> and >> my code now looks like >> >> class LogFile(file: File, implicit val system: ActorSystem) { >> Predef.assert(file.exists(), "log file must exists") >> >> implicit val materializer = ActorMaterializer() >> val logger = Logger(LoggerFactory.getLogger(getClass)) >> >> val source: Source[ByteString, Future[Long]] = >> Source.synchronousFile(file) >> >> // todo (harit): what should be maximumFrameLength >> val flow: Flow[ByteString, String, Unit] = Flow[ByteString] >> .via(Framing.delimiter(ByteString(System.lineSeparator), >> maximumFrameLength = 1500, allowTruncation = true)) >> .map(_.utf8String) >> >> def process() = { >> logger.debug(s"processing $file") >> source.via(flow).runForeach(println) >> } >> } >> >> The problem arises when a log line is more than 1500 characers (1500 >> bytes), the stream fails. I asserted by running it against a log file and >> it stopped on a line which had 1932 characters (1932 bytes) >> I set allowTruncation = true, so I thought the line would truncate >> everything after 1500 bytes and move on, but its not the case. >> >> *Question* >> - How can I truncate over 1500 bytes, process (or print) the line and >> move on with next line? without failing the stream? >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+unsubscr...@googlegroups.com. >> To post to this group, send email to akka-user@googlegroups.com. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Martynas Mickevičius > Typesafe <http://typesafe.com/> – Reactive > <http://www.reactivemanifesto.org/> Apps on the JVM > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to a topic in the > Google Groups "Akka User List" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/akka-user/sbHgCbcGYiI/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
