[-strategy@, +architecture@] On Thu, Apr 30, 2015 at 5:58 PM, Srinath Perera <srin...@wso2.com> wrote:
> should go to arch@ > > On Thu, Apr 30, 2015 at 6:28 AM, Srinath Perera <srin...@wso2.com> wrote: > >> Thanks Supun!! this looks good. >> >> --Srinath >> >> On Thu, Apr 30, 2015 at 6:25 AM, Supun Sethunga <sup...@wso2.com> wrote: >> >>> Hi all, >>> >>> Following is the break down of the Model Summary illustrations that can >>> be supported by ML at the moment. Initiating this thread to finalize on >>> what we can support and what cannot, with the initial release. Blue colored >>> ones are yet to implement. >>> >>> - Numerical Prediction >>> - Standard Error [1] >>> - Residual Plot [2] >>> - Feature Importance (*Graph containing weights assigned to each >>> of the feature in the model*) >>> >>> >>> - Classification: >>> - Binary >>> - ROC [3] >>> - AUC >>> - Confusion Matrix (*Available on spark as a static metric. >>> But if this was calculated manually, it can be made interactive, >>> so that >>> user can find the optimal threshold*) >>> - Accuracy >>> - Feature Importance >>> - Multi-Class >>> - Confusion Matrix (*Available on spark*) >>> - Accuracy >>> - Feature Importance >>> >>> >>> - Clustering >>> - Scatter plot with clustered points >>> >>> >>> *Cross-comparing Models* >>> >>> As you can see, major limitation we have when cross comparing models >>> within a project is, different categories have different summary >>> statistics/plots, and hence we cannot compare two models in two categories. >>> >>> Following are the possibilities: >>> >>> - ROC can be used to compare Binary classification models. >>> - Cobweb (a radar chart) can be used to compare Multi-Class >>> classification models (This is the possible alternative for ROC in >>> multi-class case. But the drawback is, the graph will be very unclear >>> when >>> there are excess amounts of features in the models). [4] [5] >>> - Accuracy can be used to compare all classification models. >>> >>> Please add if I've missed anything. >>> >>> *Ref:* >>> [1] http://onlinestatbook.com/2/regression/accuracy.html >>> [2] http://stattrek.com/regression/residual-analysis.aspx >>> [3] http://www.sciencedirect.com/science/article/pii/S016786550500303X >>> [4] >>> http://www.academia.edu/2519022/Visualization_and_analysis_of_classifiers_performance_in_multi-class_medical_data >>> [5] >>> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.107.8450&rep=rep1&type=pdf >>> >>> >>> Thanks, >>> Supun >>> >>> -- >>> *Supun Sethunga* >>> Software Engineer >>> WSO2, Inc. >>> http://wso2.com/ >>> lean | enterprise | middleware >>> Mobile : +94 716546324 >>> >> >> >> >> -- >> ============================ >> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >> Site: http://people.apache.org/~hemapani/ >> Photos: http://www.flickr.com/photos/hemapani/ >> Phone: 0772360902 >> > > > > -- > ============================ > Blog: http://srinathsview.blogspot.com twitter:@srinath_perera > Site: http://people.apache.org/~hemapani/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > -- *Supun Sethunga* Software Engineer WSO2, Inc. http://wso2.com/ lean | enterprise | middleware Mobile : +94 716546324
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
