On 17/02/2011 18:36, Eric Yang wrote:
I think the bigger concern is that Hadoop ecosystem does not have a standard
method in linking dependencies. Hbase depends on Zookeeper, and Pig depends
on Hadoop and Hbase. Then pig decided to put hadoop-core jar in it's own
jar file. Chukwa depends on pig + hbase + hadoop and zookeeper. The
version incompatibility is probably what driving people nuts. Hence, there
is a new proposal on how to integrate among hadoop ecosystem. I urge
project owners to review the proposal and provide feedbacks.
The proposal is located at:
https://issues.apache.org/jira/secure/attachment/12470823/deployment.pdf
The related jiras are:
https://issues.apache.org/jira/browse/HADOOP-6255
https://issues.apache.org/jira/browse/PIG-1857
There are plans to file more jiras for related projects.
I think a focus on RPMs/debs and then say "local VM for testing",
because that gets people used to deploying on Linux from day one, rather
than developing on windows and being surprised that things work
differently in production. You still have the problem of debugging
remotely to a VM in the VLAN, but that's tractable.
The integration
would also be a lot easier if all related projects are using maven for
dependency management.
Do you mean Maven the tool or the maven repo format?
