Documentation suggestion: Quick start

2010-08-25 Thread Oleg V. Zhylin
Hi All, The Quick Start page http://hadoop.apache.org/common/docs/current/quickstart.html contains a suggestion on how to ssh to localhost w/o a passphrase Now check that you can ssh to the localhost without a passphrase: $ ssh localhost If you cannot ssh to localhost without a

Re: apache commons configuration

2010-08-25 Thread Jeff Hammerbacher
For some recent discussion, see https://issues.apache.org/jira/browse/HADOOP-6910 On Tue, Aug 24, 2010 at 3:24 AM, Steve Loughran ste...@apache.org wrote: On 24/08/10 10:43, Thomas Koch wrote: Hi, just out of curiosity: Is there any particular reason, why Hadoop projects or ZooKeeper do

Searching Hadoop project(s) content

2010-08-25 Thread Alex Baranau
Hello guys, Over at http://search-hadoop.com we index Hadoop sub-project's mailing lists, wiki, web site, source code, javadoc, jira... Including general MLs for Hadoop TLP. Would the community be interested in a patch that replaces the Google-powered search with that from search-hadoop.com by

Re: [VOTE] Pig to become a TLP

2010-08-25 Thread Hemanth Yamijala
+1 On Mon, Aug 23, 2010 at 11:08 PM, Alan Gates ga...@yahoo-inc.com wrote: I propose that Pig become a top level Apache project. The Pig development community has already voted on and approved this proposal.  In summary, the community voted that all current active committers (listed at

Re: [DISCUSS] Hadoop Security Release off Yahoo! patchset

2010-08-25 Thread Hemanth Yamijala
Arun, How much time do you think it would take to have a version of 0.20 with the security features in it ready ? In a different thread, Owen has started discussing plans around 0.22. Do you think this effort would affect 0.22 release ? I do agree that this would be very useful for folks who

Re: Searching Hadoop project(s) content

2010-08-25 Thread Nigel Daley
Big +1. This is cool and gr8 that it uses Solr. Cheers, Nige On Aug 25, 2010, at 10:01 AM, Alex Baranau alex.barano...@gmail.com wrote: Hello guys, Over at http://search-hadoop.com we index Hadoop sub-project's mailing lists, wiki, web site, source code, javadoc, jira... Including

Child processes on datanodes/task trackers

2010-08-25 Thread C J
Hi, I wanted to know why I see running Child processes on my datanodes even though there is no job running at that time. Are these left over from failed attempts? Is there anything I can do to keep these clean? Thanks, Deepika

Re: [DISCUSS] Hadoop Security Release off Yahoo! patchset

2010-08-25 Thread Allen Wittenauer
On Aug 25, 2010, at 10:46 AM, Hemanth Yamijala wrote: I do agree that this would be very useful for folks who want security sooner. And the fact that Yahoo! have been running it at scale for a good while now is also assuring. As has been mentioned a few times, part of the security features

Re: [DISCUSS] Hadoop Security Release off Yahoo! patchset

2010-08-25 Thread Devaraj Das
As has been mentioned a few times, part of the security features are dependent upon Yahoo!-type operations. Allen, could you please enlist them here again (for the benefit of the community)? Or, are you referring to only the cluster-wide start scripts? On 8/25/10 1:25 PM, Allen Wittenauer

Re: Child processes on datanodes/task trackers

2010-08-25 Thread Ted Yu
Use jps to find out pid of the Child. Then use this to find out which job the Child belongs to: ps aux | grep pid On Wed, Aug 25, 2010 at 12:20 PM, C J c.josh...@yahoo.com wrote: Hi, I wanted to know why I see running Child processes on my datanodes even though there is no job running at

Re: Child processes on datanodes/task trackers

2010-08-25 Thread C J
Thanks for your reply. Some of these child tasks belong to successful jobs. I am wondering why they are still hanging there for long finished jobs. From: Ted Yu yuzhih...@gmail.com To: general@hadoop.apache.org Sent: Wed, August 25, 2010 4:17:38 PM Subject:

Re: Child processes on datanodes/task trackers

2010-08-25 Thread Ted Yu
After you obtain pid, you can use jstack to see what the Child process was doing. What hadoop version are you using ? On Wed, Aug 25, 2010 at 7:28 PM, C J c.josh...@yahoo.com wrote: Thanks for your reply. Some of these child tasks belong to successful jobs. I am wondering why they are

Re: Child processes on datanodes/task trackers

2010-08-25 Thread C J
Thanks Ted! I did a jstack and it seems there is an issue with ehcache that I am using in the mapper task. net.sf.ehcache.cachemana...@57ac3379 daemon prio=10 tid=0x59180800 nid=0x379e in Object.wait() [0x41506000] java.lang.Thread.State: TIMED_WAITING (on object monitor)

Re: Child processes on datanodes/task trackers

2010-08-25 Thread Ted Yu
I don't use ehcache. Did you forget to close CacheManager at the end of your job by any chance ? On Wed, Aug 25, 2010 at 7:59 PM, C J c.josh...@yahoo.com wrote: Thanks Ted! I did a jstack and it seems there is an issue with ehcache that I am using in the mapper task.