Re: How does AWS know how many map/reduce slot should be configured to each EC2 instance?

2013-07-19 Thread TianYi Zhu
1. Yes, it's depends on instance type. Generally, number of map slots + number of reduce slots = number of ECU, number of map slots / number of reduce slots = 3. You can customize these numbers. 2. Yes, Number of nodes * Running hours * Price per EMR node per hour (EMR node is a little bit more

Re: Pig vs hive performance

2012-10-03 Thread TianYi Zhu
from amazon web site: http://aws.amazon.com/elasticmapreduce/faqs/#hive-8 Q: When should I use Hive vs. PIG? Hive and PIG both provide high level data-processing languages with support for complex data types for operating on large datasets. The Hive language is a variant of SQL and so is more

Re: Pig vs hive performance

2012-10-03 Thread TianYi Zhu
was also thinking that pig optimizer is better than hive optimizer. Regards Abhi Sent from my iPhone On Oct 3, 2012, at 7:15 PM, TianYi Zhu tianyi@facilitatedigital.com wrote: from amazon web site: http://aws.amazon.com/elasticmapreduce/faqs/#hive-8 Q: When should I use Hive

Re: Why cannot I start namenode or localhost:50070 ?

2012-08-27 Thread TianYi Zhu
Hi Charles, map/reduce(jobtracker/tasktrackers, localhost:50030) is based on hdfs(namenode/datanodes, localhost:50070) or local file system. It seems there is something wrong with the hdfs, so the map/reduce is blocked and shows INITIALIZING, please check the log of namenode(