On Tue, Jul 7, 2015 at 9:13 PM, Niranda Perera <[email protected]> wrote:
> Quick question... > Does ML go as a different Carbon server or does it go as a profile of DAS > (ex: we have data receiver, data analyzer etc) where ML features are > installed in? > In both modes. > > if ML is a separate carbon server, DAS should start in the cluster only > mode and ML should point to this DAS spark cluster :-) > May be it makes sense to ask Users to use ML profile of DAS, if they are willing to do predictive analytics with data stored by DAS and have ML standalone carbon server which will be used in all non-DAS scenarios. > On Tue, Jul 7, 2015 at 9:04 PM, Nirmal Fernando <[email protected]> wrote: > >> Ok, thanks. I think client mode is the mode we have to use when DAL >> features run in ML. >> >> On Tue, Jul 7, 2015 at 8:47 PM, Niranda Perera <[email protected]> wrote: >> >>> Hi Nirmal, >>> >>> Actually, default is the local mode. (it is checked in the line 197) :-) >>> >>> clarification. >>> there are two states in DAS Spark at the moment. >>> 1. Local mode (Default) - DAS sets up Spark in 'local' mode, a single >>> node all-in-one server >>> 2. cluster mode - DAS sets up its own Spark cluster and the client app >>> in a cluster using Carbon Clustering >>> >>> two more states are yet to be merged to the current code >>> 3. Client mode - DAS acts only as a client (Spark terms - only the app >>> (driver) is instantiated in the server) where it points to an external >>> spark cluster >>> 4. cluster only mode - DAS sets up Spark Cluster only using Carbon >>> Clustering so that other clients (ex. ML) can submit applications form the >>> outside of the cluster >>> >>> rgds >>> >>> >>> On Tue, Jul 7, 2015 at 7:50 PM, Nirmal Fernando <[email protected]> wrote: >>> >>>> Hi All, >>>> >>>> If I read it correctly, by default DAS starts a Spark Server [1] since >>>> the default value of client mode is false. Shouldn't we be using Spark >>>> Local mode instead as the default config? >>>> >>>> [1] >>>> https://github.com/wso2/carbon-analytics/blob/master/components/analytics-processors/org.wso2.carbon.analytics.spark.core/src/main/java/org/wso2/carbon/analytics/spark/core/internal/SparkAnalyticsExecutor.java#L197 >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >>> >>> -- >>> *Niranda Perera* >>> Software Engineer, WSO2 Inc. >>> Mobile: +94-71-554-8430 >>> Twitter: @n1r44 <https://twitter.com/N1R44> >>> https://pythagoreanscript.wordpress.com/ >>> >> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> > > > -- > *Niranda Perera* > Software Engineer, WSO2 Inc. > Mobile: +94-71-554-8430 > Twitter: @n1r44 <https://twitter.com/N1R44> > https://pythagoreanscript.wordpress.com/ > -- Thanks & regards, Nirmal Associate Technical Lead - Data Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
