This can easily modeled as:
val source = Source(ids)
source
.mapAsync(4)(fetchDocumentById)
.map(_.profile)
.flatMapConcat{prof =>
sourceOfRelatedDocs
.mapAsync(4)(persistDoc)
.fold(0)(_ ++ _ )
.map(count => (count, prof))
}
.mapAsync(4){case (count, prof) => updateProfile(count, prof)}
.to(Sink.ignore)
On Thursday, November 3, 2016 at 11:57:59 PM UTC-3, Eugene Dzhurinsky wrote:
>
> Hello, I want to implement the following workflow:
>
> - a source has the sequence of IDs to process
> - initial flow *INIT* fetches the document by ID and extracts the
> *profile*
> - another flow *FETCH* is spawned, it fetches one or more of the
> associated documents and store them in some sink *DATA (file)*
> - the flow must also calculate the number of records saved to the sink
> *DATA*
> - once the flow *FETCH *is complete - the *profile* is updated with the
> count of fetched documents
> - then the *profile* is written into the sink *METADATA (file)*
> - optionally if the number of records in *FETCH* phase doesn't match the
> number of the actual number set in *metadata - *then it should write some
> key into yet another sink *ERROR*
>
>
> <https://lh3.googleusercontent.com/-oRI9CM1wJzU/WBv4EHfPJ6I/AAAAAAAAGvE/7INfXncW_sAJsL7fsKx_LhUYshaefQU_ACLcB/s1600/flow-test.png>
>
> So far it's not clear how would I
>
> - aggregate the records produced by certain flow to calculate the number
> of the records processed for certain input
> - keep the intermediate profile object somewhere until the records are not
> fetched and saved in another flow
>
> Please advice.
>
> Thanks!
>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
