Hi. Wonder how one should improve the startup times of a hadoop job. Some of my jobs which have a lot of dependencies in terms of many jar files take a long time to start in hadoop up to 2 minutes some times. The data input amounts in these cases are neglible so it seems that Hadoop have a really high setup cost, which I can live with but this seems to much.
Let's say a job takes 10 minutes to complete then it is bad if it takes 2 mins to set it up... 20-30 sec max would be a lot more reasonable. Hints ? //Marcus -- Marcus Herou CTO and co-founder Tailsweep AB +46702561312 marcus.he...@tailsweep.com http://www.tailsweep.com/
