[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265118#comment-15265118 ]
Saikat Kanjilal commented on SPARK-14302: ----------------------------------------- And here's the duplication inside the mllib directory: mllib Tasks in common initialize sparkContext MLUtils.loadLibSVMFile correlations && correlations example—share Statistics.corr, should be generalized like correlations_example gaussian mixture model and example kmeans and kmeans_example word2vec and word2vec_example We really should think about combining the example python files and the actual algorithms, what do you think (for example kmeans and kmeans_example) Let me know your thoughts on my duplication removal ideas above before I make any code changes > Python examples code merge and clean up > --------------------------------------- > > Key: SPARK-14302 > URL: https://issues.apache.org/jira/browse/SPARK-14302 > Project: Spark > Issue Type: Sub-task > Components: Examples > Reporter: Xusen Yin > Priority: Minor > Labels: starter > > Duplicated code that I found in python/examples/mllib and python/examples/ml: > * python/ml > ** None > * Unsure duplications, double check > ** dataframe_example.py > ** kmeans_example.py > ** simple_params_example.py > ** simple_text_classification_pipeline.py > * python/mllib > ** gaussian_mixture_model.py > ** kmeans.py > ** logistic_regression.py > * Unsure duplications, double check > ** correlations.py > ** random_rdd_generation.py > ** sampled_rdds.py > ** word2vec.py > When merging and cleaning those code, be sure not disturb the previous > example on and off blocks. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org