On Mon, Jan 16, 2012 at 10:25 PM, Devaraj Das <[email protected]> wrote:
> Don't understand this. Every component knows what components it depends on, > right? MapRed component developer would know that it depends on hdfs. If you > depend on hbase, you should ensure that the list of dependent components > includes hbase... Mapreduce can depend on either HDFS only, or HDFS+HBase+ZooKeeper. (For people who runs mapreduce on top of HBase). Therefore, the developer for implementing mapreduce plugin needs to handle both cases in the plugin. It is also possible to run mapreduce on other systems like cassandra, or mongodb. Hence, the permutation for conditions in mapreduce plugin becomes too complex, and there could be different ways to handle mapreduce 1.0 or YARN. The current plugin design does not provide good visibility on how to handle the conditions. > Actually, Ambari could get out of the business of orchestration. Instead, the > components should be updated to tolerate their dependencies being down.. I > have heard this from some people, and generally concur with this thinking. It > makes the ecosystem better (if you ignore Ambari for the moment). It seems unsafe to make this assumption. The eventual consistent model might not be a good idea. Remember the example that I suggested a while back, a old datanode might be doing hand shake with a newer version of namenode, and accepted data for writing. This might be causing data instability because the system is in transition of upgrade. There could be unexpected bugs in software that breaks eventual consistent model. I would recommend to think twice with going this approach. regards, Eric
