Re: Hodoop namenode startup problem

2010-08-04 Thread edward choi
I've fixed the problem. The reason namenode won't start was that I accidentally started the cluster with root account. This somehow changed the ownership of some hadoop-related files(ex: log files, and hadoop.tmp.dir/dfs/name/current/edits) from hadoop:hadoop to root:root. After I fixed the owners

Re: Hodoop namenode startup problem

2010-08-04 Thread Harsh J
Could you check the NameNode/SecondaryNameNode logs and try to find the exact issue? Post the errors (if) it contains here, so we can try to help you better. On Thu, Aug 5, 2010 at 9:39 AM, edward choi wrote: > I am currently stuck with hadoop namenode that won't start. > > When I type "start-all

Hodoop namenode startup problem

2010-08-04 Thread edward choi
I am currently stuck with hadoop namenode that won't start. When I type "start-all.sh", everything prints out fine. But when I type "jps", only JobTracker is activated. When this happens, I usually format the namenode. But the problem is that there are 500gigs of date in HDFS. So I really want t

Re: error:Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzopCodec

2010-08-04 Thread Sudhir Vallamkondu
Couple of things you can try. Did you restart the task trackers after installing the lzo libs? You can check if the lzo lib is in the classpath or not by doing ps auxw | grep tasktracker | grep lzo | wc –l The above returns 1 if classpath contains the lzo lib There is a local LZO indexer that y

Re: Partitioner in Hadoop 0.20

2010-08-04 Thread Owen O'Malley
On Aug 4, 2010, at 10:58 AM, David Rosenstrauch wrote: So my partitioner needs to implement Configurable, then not JobConfigurable. Tnx much! ReflectionUtils.newInstance will use either Configurable or JobConfigurable (or both!). So implementing either one will work fine. -- Owen

Re: Partitioner in Hadoop 0.20

2010-08-04 Thread David Rosenstrauch
On 08/04/2010 01:55 PM, Wilkes, Chris wrote: On Aug 4, 2010, at 10:50 AM, David Rosenstrauch wrote: On 08/04/2010 12:30 PM, Owen O'Malley wrote: On Aug 4, 2010, at 8:38 AM, David Rosenstrauch wrote: Anyone know if there's any particular reason why the new Partitioner class doesn't implement

Re: Partitioner in Hadoop 0.20

2010-08-04 Thread David Rosenstrauch
On 08/04/2010 12:30 PM, Owen O'Malley wrote: On Aug 4, 2010, at 8:38 AM, David Rosenstrauch wrote: Anyone know if there's any particular reason why the new Partitioner class doesn't implement JobConfigurable? (And, if not, whether there's any plans to fix this omission?) We're working on a som

Re: Partitioner in Hadoop 0.20

2010-08-04 Thread Owen O'Malley
On Aug 4, 2010, at 8:38 AM, David Rosenstrauch wrote: Anyone know if there's any particular reason why the new Partitioner class doesn't implement JobConfigurable? (And, if not, whether there's any plans to fix this omission?) We're working on a somewhat complex partitioner, and it would

Fwd: Partitioner in Hadoop 0.20

2010-08-04 Thread David Rosenstrauch
Someone sent this email to the commons-user list a while back, but it seems like it slipped through the cracks. We're starting to dig into some hard-core Hadoop development and just came upon this same issue, though. Anyone know if there's any particular reason why the new Partitioner class

Problem after turning hadoop machines power-off without shutting down

2010-08-04 Thread Hieu Le
Hello, We have a cluster with ~1.5TB of data on it. After the power is off unexpectedly, I restart the cluster again but the administrative webpage is not available. When I ran $hadoop fsck /, I got bellow error message: had...@b1:~/hadoop/bin$ ./hadoop fsck / Exception in thread "main" java.io.I