Hi All, Here is the status of the integration, since the last update:
- Launch process cleaned up. Launch still happening through DTCli calls. Once APEXCORE-405 <https://issues.apache.org/jira/browse/APEXCORE-405> is implemented, this will become a lot better. - Some issues still causing problems in running Samoa apps on Apex. Containers getting killed. This may be due to low memory. This is work in progress. ~Bhupesh On Wed, Mar 2, 2016 at 10:14 AM, Bhupesh Chawda <[email protected]> wrote: > Hi David, > > Here is the working branch you can look at: > https://github.com/bhupeshchawda/incubator-samoa/tree/samoa-apex > > As I mentioned, the launch part needs to be worked on. Currently I have a > few hacks in my local environment. > You can use the test cases though to get an idea. > > -Bhupesh > > On Wed, Mar 2, 2016 at 1:01 AM, David Yan <[email protected]> wrote: > >> Hi Bhupesh, >> >> That's good progress. Can you send us a link to the code you did for >> this? Or maybe a review-only PR? >> >> David >> >> On Mon, Feb 29, 2016 at 10:15 PM, Bhupesh Chawda <[email protected] >> > >> wrote: >> >> > Hi All, >> > >> > Here is the status of integration of Apache Apex into Apache Samoa. >> > >> > - Samoa API implemented and able to convert Samoa topology into Apex >> > Dag. >> > - Implemented partitioning support using parallelism hints from Samoa >> > API. >> > - Implemented stream multiplexing: >> > - Added All-based partitioner. Upstream tuples go to all >> downstream >> > partitions >> > - Stream codec for Key based partitioning >> > - Stream codec for Random partitioning >> > - Able to launch a Samoa task on the cluster. This has to be worked >> on. >> > Currently some hacks are used by calling DTCli explicitly from the >> main >> > entry point in Samoa code. Also jars are needed to be manually >> bundled. >> > This will be worked on in this sprint. >> > - Tested the following algorithms on local cluster: >> > - Prequential Evaluation using Vertical Hoeffding Tree classifier. >> > This is a decision tree based classifier. >> > - Clustering using CluStream algorithm. >> > - I have asked clarifications on some more details of these >> algorithms >> > as well as serialization issues with Samoa classes. I am waiting for >> > some >> > response from the Samoa community. >> > - Although Samoa does not have many algorithms currently (it is a >> > framework for developing algorithms), more algorithms are expected >> as a >> > part of their roadmap: >> > https://cwiki.apache.org/confluence/display/SAMOA/Roadmap >> > >> > Thanks, >> > >> > Bhupesh >> > >> > >
