Re: EMR for spark job - instance type suggestion

2016-08-26 Thread Gavin Yue
I tried both M4 and R3. R3 is slightly more expensive, but has larger memory. If you doing a lot of in-memory staff, like Join. I recommend R3. Otherwise M4 is fine. Also I remember M4 is EBS instance, so you have to pay for additional EBS cost as well. On Fri, Aug 26, 2016 at 10:29 AM,

EMR for spark job - instance type suggestion

2016-08-26 Thread Saurabh Malviya (samalviy)
We are going to use EMR cluster for spark jobs in aws. Any suggestion for instance type to be used. M3.xlarge or r3.xlarge. Details: 1) We are going to run couple of streaming jobs so we need on demand instance type. 2) There is no data on hdfs/s3 all data pull from kafka or