Hi Mahesh, On Thu, Jul 21, 2016 at 2:10 PM, Mahesh Dananjaya <dananjayamah...@gmail.com > wrote:
> Hi All, > I am onto connecting cep streams with samoa streams to data analysis using > samoa framework. To connect samoa with cep siddhi event streams what i we > can do is that try to convert cep streams into samoa streams or else > writing wrpper for samoa for cep streasm to be used. In both cases i have > to covert siddhi cep streasm into samoa streams. Samoa is using MOA to > analyse data. Moo contains ML framework to analyse stream data. Samoa is > wrapping MOA withsome of its classes. > > Samoa streams is based on MOA, Instance and InstanceStreams. Samoa see > streams as a stream of instances [1]. So if we are going to convert cep > events into samoa instances , it will take time. But if we have some > similarity between cep siddhi streams and samoa streasm we can reduce the > time. > 1. What is the underlying infrastructure for cep siddhi streasm.? > 2. Are there anything as Instances or InstanceStreams kind of > implmentation underlying cep streams? > 3. How can i get more underestanding on CEP siddhi streams. > > On the other hand i can use my cep siddhi extension and put those events > into event queue and convert them into samoa instances and feed them into > samoa streaming ml topologies. > I think this is OK. I assume this is much easier. Let's do this and check the performance. > There is another option. In Samoa what they are basically doing is that > wrapping MOA ML framework and write some classes for build streaming ml > topologies. So as the other option i can wrap samoa moa with my design and > use moa ml framework directly. (No need for Samoa extension). I have > building some topologies to streaming data analysis [2]. Main problem is > that lack of documentation. Anyway i had go through their whole samoa > design.thank you. > If we use MOA directly, would we loose the distributed support in SAOMA. Let's do a call when you can, so we can dsicuss this in detail. --Srinath > regards, > Mahesh. > > [1] > https://github.com/apache/incubator-samoa/blob/master/samoa-api/src/main/java/org/apache/samoa/streams/clustering/ClusteringStream.java > [2] > https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming > > On Mon, Jul 18, 2016 at 11:40 AM, Mahesh Dananjaya < > dananjayamah...@gmail.com> wrote: > >> Hi Maheshakya, >> Samoa modules built as topologies that connect streams with the internal >> processors. I have already written some examples to test the ML algorithms >> and samoa analysis topologies. What we need to done is mostly developing a >> wrapper around samoa topologies to connect their input and output streams >> with our cep streams. So i am currently going through their stream >> architecture to connect our streams with their streams. Couple of examples >> exapaining samoa ml topologies and streaming can be found in my git hub >> repo [1]. Samoa using MOA ml algorithms by wrapping them with their >> classes. Initailly i am trying to develop a KMeansClustering analysis with >> cep streams with samoa ml topologies. >> And also i could not find a maven repo for samoa 0.4.0 incubating. So i >> am currently using my local m2 repo's samoa 0..4.0 incubating for my >> dependencies to work. The local one is built by original samoa source.thank >> you. >> >> regards, >> Mahesh. >> >> [1] >> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >> >> >> On Mon, Jul 18, 2016 at 8:32 AM, Maheshakya Wijewardena < >> mahesha...@wso2.com> wrote: >> >>> Hi Mahesh, >>> >>> Can you please share your samoa project? >>> >>> On Sun, Jul 17, 2016 at 11:19 AM, Mahesh Dananjaya < >>> dananjayamah...@gmail.com> wrote: >>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: Mahesh Dananjaya <dananjayamah...@gmail.com> >>>> Date: Sun, Jul 17, 2016 at 11:18 AM >>>> Subject: Re: GSOC2016: [ML][CEP] [SAMOA]Predictive analytic with online >>>> data for WSO2 Machine Learner-Samoa Integration >>>> To: Maheshakya Wijewardena <mahesha...@wso2.com> >>>> >>>> >>>> Hi Maheshakaya, >>>> just need a little help. In Samoa when we want to run a class what is >>>> does it used this commands [1], >>>> 1. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar >>>> "ClusteringEvaluation" >>>> 2. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar >>>> "PrequentialEvaluation -d /tmp/dump.csv -i 1000000 -f 100000 -l >>>> (classifiers.trees.VerticalHoeffdingTree -p 4) -s >>>> (generators.RandomTreeGenerator -c 2 -o 10 -u 10)" >>>> >>>> what is does is call a class named LocalDoTask [4] and pass this string >>>> as argument.After that that LocalDoTask call the relevent Tasks such as >>>> ClusteringEvaluation or PrequentialEvaluation. [2]. >>>> >>>> Now i have add samoa dependencies to my new maven project, where i used >>>> original samoa source to write examples and test then earlier.Now i want to >>>> push them into my new java project with samoa dependencies. I added >>>> dependency and it was built fine. Now i am calling my local DoTask.java [3] >>>> file as same as i did with samoa with, >>>> java -cp target/streaming-1.0-SNAPSHOT.jar >>>> org.gsoc.samoa.streaming.DoTask >>>> "org.gsoc.samoa.streaming.ClusteringEvaluation" >>>> But seems to be i am incorrect in some place. >>>> Error: A JNI error has occurred, please check your installation and try >>>> again >>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>> org/apache/samoa/topology/ComponentFactory >>>> at java.lang.Class.getDeclaredMethods0(Native Method) >>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) >>>> at java.lang.Class.privateGetMethodRecursive(Class.java:3048) >>>> at java.lang.Class.getMethod0(Class.java:3018) >>>> at java.lang.Class.getMethod(Class.java:1784) >>>> at >>>> sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) >>>> at >>>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) >>>> Caused by: java.lang.ClassNotFoundException: >>>> org.apache.samoa.topology.ComponentFactory >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> ... 7 more >>>> >>>> >>>> can i actually call the Task like this. >>>> >>>> BR, >>>> Mahesh. >>>> >>>> [1] >>>> https://samoa.incubator.apache.org/documentation/Prequential-Evaluation-Task.html >>>> [2] >>>> https://github.com/apache/incubator-samoa/blob/releases/0.4.0-incubating-RC0/samoa-api/src/main/java/org/apache/samoa/tasks/ClusteringEvaluation.java >>>> [3] >>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming >>>> [4] >>>> https://github.com/apache/incubator-samoa/tree/releases/0.4.0-incubating-RC0/samoa-local/src/main/java/org/apache/samoa >>>> >>>> >>>> On Thu, Jul 14, 2016 at 3:47 PM, Mahesh Dananjaya < >>>> dananjayamah...@gmail.com> wrote: >>>> >>>>> Hi srinath, >>>>> sure.i am working on it.thank you. >>>>> regards, >>>>> Mahesh. >>>>> >>>>> On Thu, Jul 14, 2016 at 11:12 AM, Srinath Perera <srin...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi Mahesh, >>>>>> >>>>>> Let's focus on getting SAOMA work with CEP. It is OK to be limited to >>>>>> few algorithms. >>>>>> >>>>>> --Srinath >>>>>> >>>>>> On Thu, Jul 14, 2016 at 10:49 AM, Mahesh Dananjaya < >>>>>> dananjayamah...@gmail.com> wrote: >>>>>> >>>>>>> Hi Maheshakya, >>>>>>> I think we can build new tasks [1] like the one in execution plan in >>>>>>> cep with samoa. I will try to build a one. >>>>>>> regards, >>>>>>> Mahesh. >>>>>>> [1] >>>>>>> https://samoa.incubator.apache.org/documentation/Developing-New-Tasks-in-SAMOA.html >>>>>>> >>>>>>> >>>>>>> On Thu, Jul 14, 2016 at 10:35 AM, Mahesh Dananjaya < >>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi Maheshakya, >>>>>>>> I am building and running samoa to see its functionality. In samoa >>>>>>>> still we have limited supports in algorithms. Samoa supports only >>>>>>>> classification and clustering with streams. It also use kind of >>>>>>>> StreamProcessor, like the one we use in StreamProcessor extension. I >>>>>>>> was >>>>>>>> getting started with Samoa referring to this page [1]. Then i ran >>>>>>>> couple of >>>>>>>> examples to identified the flow. Samoa use hadoop framework instead >>>>>>>> spark >>>>>>>> for distribution. But i am using it in a local mode. When i see the >>>>>>>> Samoa >>>>>>>> core there is only limited algorithms. IMO if we are going to use >>>>>>>> Samoa we >>>>>>>> have to limit the functionality and algorithms [2]. When i go to >>>>>>>> developer >>>>>>>> corner in [3], it seems to be something like CEP extension that we are >>>>>>>> using currenlty. SO in Samoa though the algorihtms are limited, they >>>>>>>> have >>>>>>>> implemented streaming support for them. Therefore if we integrate it >>>>>>>> into >>>>>>>> CEP we have to look for how to handle streams and algorithms in Samoa >>>>>>>> side. >>>>>>>> Is it good for your side to have both hadoop and spark running >>>>>>>> background.thank you. >>>>>>>> regards, >>>>>>>> Mahesh. >>>>>>>> >>>>>>>> [1] https://samoa.incubator.apache.org/documentation/Home.html >>>>>>>> [2] >>>>>>>> https://samoa.incubator.apache.org/documentation/api/current/index.html >>>>>>>> [3] >>>>>>>> https://samoa.incubator.apache.org/documentation/SAMOA-Topology.html >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> ============================ >>>>>> Srinath Perera, Ph.D. >>>>>> http://people.apache.org/~hemapani/ >>>>>> http://srinathsview.blogspot.com/ >>>>>> >>>>> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> Dev@wso2.org >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> Pruthuvi Maheshakya Wijewardena >>> >>> mahesha...@wso2.com >>> +94711228855 >>> >>> >>> >> > -- ============================ Srinath Perera, Ph.D. http://people.apache.org/~hemapani/ http://srinathsview.blogspot.com/
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
