50gb of data is not much. besides master local[4] what else do you have for other parameters?
${SPARK_HOME}/bin/spark-submit \ --driver-memory 4G \ --num-executors 1 \ --executor-memory 4G \ --master local[4] \ Try running it and check web GUI 4040 for resource usage HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 15 June 2016 at 17:03, spR <data.smar...@gmail.com> wrote: > Hi, > > can I use spark in local mode using 4 cores to process 50gb data > effeciently? > > Thank you > > misha >