Re: The IBM Distribution of Apache Hadoop now available

2010-06-28 Thread Steve Loughran
Stephen Watt wrote: Hi Folks ??IBM has made available a preview version of the IBM Distribution of Apache Hadoop. You can access it here - http://www.alphaworks.ibm.com/tech/idah/ . This distribution contains Apache Hadoop 0.20.2, a 32-bit Linux version of the IBM SDK for Java 6 SR 8, and

Does hadoop need a zookeeper installation to work?

2010-06-28 Thread Sarah kho
Hi, I am wondering whether hadoop depends on zookeeper or zookeeper need hadoop to work. When I download hadoop distribution from the website, does it have some zookeeper libraries included? thanks.

Re: Does hadoop need a zookeeper installation to work?

2010-06-28 Thread Arun C Murthy
Currently, Hadoop (HDFS/Map-Reduce) doesn't depend on ZooKeeper. Arun On Jun 28, 2010, at 7:20 AM, Sarah kho wrote: Hi, I am wondering whether hadoop depends on zookeeper or zookeeper need hadoop to work. When I download hadoop distribution from the website, does it have some zookeeper

Re: Does hadoop need a zookeeper installation to work?

2010-06-28 Thread Sarah kho
Thanks for your reply. What about ZooKeeper being depended on Hadoop? is there something like that or Hadoop and ZooKeeper are two independent projects? On Mon, Jun 28, 2010 at 7:23 PM, Arun C Murthy a...@yahoo-inc.com wrote: Currently, Hadoop (HDFS/Map-Reduce) doesn't depend on ZooKeeper.

Re: Does hadoop need a zookeeper installation to work?

2010-06-28 Thread Patrick Hunt
Hbase depends on ZK but it recently moved to TLP status. As of today there are no cross dependencies. Patrick On 06/28/2010 08:28 AM, Sarah kho wrote: Thanks for your reply. What about ZooKeeper being depended on Hadoop? is there something like that or Hadoop and ZooKeeper are two

Can we modify files in HDFS?

2010-06-28 Thread elton sky
hello everyone, After some research I found HDFS only support create new file and append to exiting file. What if I want to modify some parts of a, say 2 Petabyte, file. Do I have to remove it and create it again or we have some alternative way?

Re: [DISCUSSION] Proposal for making core Hadoop changes

2010-06-28 Thread Konstantin Shvachko
Eli, Just checking on the status of this proposal. In the past I was hesitant about introducing more formalities. I now think we really need some mechanism for new feature and project proposals, also tracking decisions. For the reasons exactly as you describe in your email. Whether it is going

Re: [DISCUSSION] Proposal for making core Hadoop changes

2010-06-28 Thread Eli Collins
Hey Konstantin, Apologies for the delay, busy with stuff for the summit. I'll get a concrete proposal to general based on our discussion at the contributor's meeting out this week. Thanks, Eli On Mon, Jun 28, 2010 at 5:50 PM, Konstantin Shvachko s...@yahoo-inc.com wrote: Eli, Just checking

Re: Can we modify files in HDFS?

2010-06-28 Thread elton sky
thanx Jeff, So...it is a significant drawback. As a matter of fact, there are many cases we need to modify. I dont understand why Yahoo didn't provoid that functionality. And as I know no one else is working on this. Why is that?

Re: Can we modify files in HDFS?

2010-06-28 Thread Todd Lipcon
Hi Elton, Typically, large data sets are of the sort that continuously grow, and are not edited or amended. For example, a common Hadoop use case is the analysis of log data or other instrumentation from web or application servers. In these cases, files are simply added, but there is no need to