In this case, persisting to Cassandra is for future analytics and Visualization.
I want to notify that the app of the event, so it makes the app interactive. Thanks On Mon, Nov 28, 2016 at 2:24 PM, vincent gromakowski < vincent.gromakow...@gmail.com> wrote: > Sorry I don't understand... > Is it a cassandra acknowledge to actors that you want ? Why do you want to > ack after writing to cassandra ? Your pipeline kafka=>spark=>cassandra is > supposed to be exactly once, so you don't need to wait for cassandra ack, > you can just write to kafka from actors and then notify the user ? > > 2016-11-28 23:15 GMT+01:00 shyla deshpande <deshpandesh...@gmail.com>: > >> Thanks Vincent for the input. Not sure I understand your suggestion. >> Please clarify. >> >> Few words about my use case : >> When the user watches a video, I get the position events from the user >> that indicates how much they have completed viewing and at a certain point, >> I mark that Video as complete and persist that info to cassandra. >> >> How do I notify the user that it was marked complete? >> >> Are you suggesting I write the completed events to kafka(different topic) >> and the akka consumer could read from this? There could be many completed >> events from different users in this topic. So the akka consumer should >> pretty much do what a spark streaming does to process this without the >> knowledge of the kafka offset. >> >> So not sure what you mean by kafka offsets will do the job, how will the >> akka consumer know the kafka offset? >> >> On Mon, Nov 28, 2016 at 12:52 PM, vincent gromakowski < >> vincent.gromakow...@gmail.com> wrote: >> >>> You don't need actors to do kafka=>spark processing=>kafka >>> Why do you need to notify the akka producer ? If you need to get back >>> the processed message in your producer, then implement an akka consummer in >>> your akka app and kafka offsets will do the job >>> >>> 2016-11-28 21:46 GMT+01:00 shyla deshpande <deshpandesh...@gmail.com>: >>> >>>> Thanks Daniel for the response. >>>> >>>> I am planning to use Spark streaming to do Event Processing. I will >>>> have akka actors sending messages to kafka. I process them using Spark >>>> streaming and as a result a new events will be generated. How do I notify >>>> the akka actor(Message producer) that a new event has been generated? >>>> >>>> >>>> >>>> On Mon, Nov 28, 2016 at 9:51 AM, Daniel van der Ende < >>>> daniel.vandere...@gmail.com> wrote: >>>> >>>>> Well, I would say it depends on what you're trying to achieve. Right >>>>> now I don't know why you are considering using Akka. Could you please >>>>> explain your use case a bit? >>>>> >>>>> In general, there is no single correct answer to your current question >>>>> as it's quite broad. >>>>> >>>>> Daniel >>>>> >>>>> On Mon, Nov 28, 2016 at 9:11 AM, shyla deshpande < >>>>> deshpandesh...@gmail.com> wrote: >>>>> >>>>>> My data pipeline is Kafka --> Spark Streaming --> Cassandra. >>>>>> >>>>>> Can someone please explain me when would I need to wrap akka around >>>>>> the spark streaming app. My knowledge of akka and the actor system is >>>>>> poor. >>>>>> Please help! >>>>>> >>>>>> Thanks >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Daniel >>>>> >>>> >>>> >>> >> >
