Hi Kewei, Thanks for your clarifications.
Gianmarco (one of the project leaders) was trying to post the following response for you to try a couple of options but was running into issues with his email. So I am posting it here for you and the rest of the list to try: <<<< Hi Kewei, We haven't tried building embedded SAMOA topologies yet, but I think it's something we will have to look at in the future. One option is to encapsulate your logic in SAMOA by writing it in a Processor. You can then build the topology in SAMOA (as a new task), and reuse components such as the classifiers. Then SAMOA will take care to execute it on Storm. The other option would involve exposing the partially built Storm topology to the programmer via some API. I think this would be the preferred option from the point of view of the developer. However, I haven't thought of how to expose it cleanly, while still having multi-platform support. One easy way to do what you describe is to break down the three parts in three topologies, and use something like Kafka to connect them. Re you question about the moa package, they are mostly utility components, and it does not include MOA algorithms. Cheers, Gianmarco <<<<<<< On Wed, Dec 16, 2015 at 4:35 AM, Sun, Kewei K <[email protected]> wrote: > Hi Nicolas, > > Thank you for your prompt reply! > > I think what I want is the second one. I would like to explain it in > detail for you. > > For example, I have a Storm topology with three parts, the first part > produces data or get data from somewhere else and send the data to the > second part. In the second part we want to use some ML algorithm in SAMOA > (i.e., VHT) as a sub-topology in this storm topology to train the data, and > then in the third part can receive the trained model or other output from > the previous part so that we can do prediction or something else based on > it. > > So is it possible to do this? > > And I have another question. Does the moa package in samoa-api include > some models can be used in MOA? Or does it implement algorithms in MOA and > adapts them to SAMOA? > > Thanks a lot! > > Thanks, > Kewei > > > -----Original Message----- > From: Nicolas Kourtellis [mailto:[email protected]] > Sent: Tuesday, December 15, 2015 5:13 PM > To: [email protected] > Subject: Re: Using Apache SAMOA in Storm topology > > Hi Kewei, > > Thanks for your interest in SAMOA! > > I would like to ask you a clarifying question for us to understand better > what you need. > > You want to be able to define for the same topology, part of it to execute > a classification (or other ML) tasks from SAMOA, and the other part of it > to execute other things? > > Can SAMOA have its own dedicated topology, instead? If yes, I think > running the VHT (or other ML tasks) should be straight forward. > > However, if this is not the case (i.e., if SAMOA needs to share the > topology), I think it will be more difficult to do it. > > Cheers, > > Nicolas > > > On Mon, Dec 14, 2015 at 3:01 AM, Sun, Kewei K <[email protected]> > wrote: > > > Hi SAMOA team, > > > > I am very interested in using SAMOA for Storm and I > > would like to build a storm topology in which one part of the Storm > > topology does classification (or other ML algorithms implemented in > > SAMOA), but I am not able to find out any example about how I can do > > this. Could you tell me how to do that or is it possible? > > > > Thanks, > > Kewei > > > > > > > -- > Nicolas Kourtellis > -- Nicolas Kourtellis
