If it is just for test purpose, why not use a smaller size of data and test it on your notebook. When you go for the cluster, you can go for 50GB (I am a newbie so my thought would be very naive)
Hey Namaskara~Nalama~Guten Tag~Bonjour -- Keigu Deepak 73500 12833 www.simtree.net, dee...@simtree.net deic...@gmail.com LinkedIn: www.linkedin.com/in/deicool Skype: thumsupdeicool Google talk: deicool Blog: http://loveandfearless.wordpress.com Facebook: http://www.facebook.com/deicool "Contribute to the world, environment and more : http://www.gridrepublic.org " On Wed, Jun 15, 2016 at 10:40 PM, Sergio Fernández <wik...@apache.org> wrote: > In theory yes... the common sense say that: > > volume / resources = time > > So more volume on the same processing resources would just take more time. > On Jun 15, 2016 6:43 PM, "spR" <data.smar...@gmail.com> wrote: > >> I have 16 gb ram, i7 >> >> Will this config be able to handle the processing without my ipythin >> notebook dying? >> >> The local mode is for testing purpose. But, I do not have any cluster at >> my disposal. So can I make this work with the configuration that I have? >> Thank you. >> On Jun 15, 2016 9:40 AM, "Deepak Goel" <deic...@gmail.com> wrote: >> >>> What do you mean by "EFFECIENTLY"? >>> >>> Hey >>> >>> Namaskara~Nalama~Guten Tag~Bonjour >>> >>> >>> -- >>> Keigu >>> >>> Deepak >>> 73500 12833 >>> www.simtree.net, dee...@simtree.net >>> deic...@gmail.com >>> >>> LinkedIn: www.linkedin.com/in/deicool >>> Skype: thumsupdeicool >>> Google talk: deicool >>> Blog: http://loveandfearless.wordpress.com >>> Facebook: http://www.facebook.com/deicool >>> >>> "Contribute to the world, environment and more : >>> http://www.gridrepublic.org >>> " >>> >>> On Wed, Jun 15, 2016 at 9:33 PM, spR <data.smar...@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> can I use spark in local mode using 4 cores to process 50gb data >>>> effeciently? >>>> >>>> Thank you >>>> >>>> misha >>>> >>> >>>
