Quick question... Does ML go as a different Carbon server or does it go as a profile of DAS (ex: we have data receiver, data analyzer etc) where ML features are installed in?
if ML is a separate carbon server, DAS should start in the cluster only mode and ML should point to this DAS spark cluster :-) On Tue, Jul 7, 2015 at 9:04 PM, Nirmal Fernando <[email protected]> wrote: > Ok, thanks. I think client mode is the mode we have to use when DAL > features run in ML. > > On Tue, Jul 7, 2015 at 8:47 PM, Niranda Perera <[email protected]> wrote: > >> Hi Nirmal, >> >> Actually, default is the local mode. (it is checked in the line 197) :-) >> >> clarification. >> there are two states in DAS Spark at the moment. >> 1. Local mode (Default) - DAS sets up Spark in 'local' mode, a single >> node all-in-one server >> 2. cluster mode - DAS sets up its own Spark cluster and the client app in >> a cluster using Carbon Clustering >> >> two more states are yet to be merged to the current code >> 3. Client mode - DAS acts only as a client (Spark terms - only the app >> (driver) is instantiated in the server) where it points to an external >> spark cluster >> 4. cluster only mode - DAS sets up Spark Cluster only using Carbon >> Clustering so that other clients (ex. ML) can submit applications form the >> outside of the cluster >> >> rgds >> >> >> On Tue, Jul 7, 2015 at 7:50 PM, Nirmal Fernando <[email protected]> wrote: >> >>> Hi All, >>> >>> If I read it correctly, by default DAS starts a Spark Server [1] since >>> the default value of client mode is false. Shouldn't we be using Spark >>> Local mode instead as the default config? >>> >>> [1] >>> https://github.com/wso2/carbon-analytics/blob/master/components/analytics-processors/org.wso2.carbon.analytics.spark.core/src/main/java/org/wso2/carbon/analytics/spark/core/internal/SparkAnalyticsExecutor.java#L197 >>> >>> >>> -- >>> >>> Thanks & regards, >>> Nirmal >>> >>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>> Mobile: +94715779733 >>> Blog: http://nirmalfdo.blogspot.com/ >>> >>> >>> >> >> >> -- >> *Niranda Perera* >> Software Engineer, WSO2 Inc. >> Mobile: +94-71-554-8430 >> Twitter: @n1r44 <https://twitter.com/N1R44> >> https://pythagoreanscript.wordpress.com/ >> > > > > -- > > Thanks & regards, > Nirmal > > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > -- *Niranda Perera* Software Engineer, WSO2 Inc. Mobile: +94-71-554-8430 Twitter: @n1r44 <https://twitter.com/N1R44> https://pythagoreanscript.wordpress.com/
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
