Just a thought (I've never had this exact problem): I would try an experiment with cascading Source.unfoldResource[Async] stages.
1. The first unfold stage would extract the CSV headers. This would use CSV resource description that's little more than a FileReader. 2. The second unfold stage would extract individual CSV records, using the data description extracted in the first stage. The resource description in this stage would include both the open file resource (FileReader) from the first stage, plus the CSV data description extracted in the first stage If you are clever, you wouldn't have re-open the file between stages -- with the caveat that if there's any stage-breaking error in the first stage the file would never be closed, so you'd need to come up with some clever magic for closing the file if there is some error during the "read" operation of the first stage. To avoid the need for cleverness, you close the file at the end of the first stage and then re-open at the beginning of the second stage. Since I assume you have a relatively small number of header lines in the CSV file relative to the number of data rows, I wouldn't think this inefficiency would be much of a burden. Brian Maso On Thu, Dec 21, 2017 at 1:15 AM, <dvdgr...@gmail.com> wrote: > Hi all, > I have the following problem, I need to parse CSV files where the heade > lines are not a fixed number, i.e. a file could have an header with three > lines and another with five lines for example. > At the same time I need to parse the header to extract the information I > need to parse following lines successfully. > So, I would need to implement a aka stream flow capable to read the lines > till it detects the end of the header, it extracts the metadata and it > pushes those metadata together with the remaining rows to the rest of the > flow. > Any idea how to solve this problem in a idiomatic aka-streams way? I have > an obvious solution using some mutable state and marking the rows as part > of the header and using then dropWhile. > Any hints would be greatly appreciated. > David > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
