Documentation suggestion: Quick start

2010-08-25 Thread Oleg V. Zhylin
Hi All, The Quick Start page http://hadoop.apache.org/common/docs/current/quickstart.html contains a suggestion on how to ssh to localhost w/o a passphrase Now check that you can ssh to the localhost without a passphrase: $ ssh localhost If you cannot ssh to localhost without a passphrase

Re: apache commons configuration

2010-08-25 Thread Jeff Hammerbacher
For some recent discussion, see https://issues.apache.org/jira/browse/HADOOP-6910 On Tue, Aug 24, 2010 at 3:24 AM, Steve Loughran wrote: > On 24/08/10 10:43, Thomas Koch wrote: > >> Hi, >> >> just out of curiosity: Is there any particular reason, why Hadoop projects >> or >> ZooKeeper do not use

Re: Branching and testing strategy for 0.22

2010-08-25 Thread Tom White
+1 Thanks for driving this, Owen. Tom On Mon, Aug 23, 2010 at 3:19 PM, Owen O'Malley wrote: > I'd like to get started testing 0.22. > > I plan to start making mini-branches for QA. These branches will be > snapshots that QA can use for testing with an expected lifetime of two weeks > each. Only

Searching Hadoop project(s) content

2010-08-25 Thread Alex Baranau
Hello guys, Over at http://search-hadoop.com we index Hadoop sub-project's mailing lists, wiki, web site, source code, javadoc, jira... Including general MLs for Hadoop TLP. Would the community be interested in a patch that replaces the Google-powered search with that from search-hadoop.com by de

Re: [DISCUSS] Hadoop Security Release off Yahoo! patchset

2010-08-25 Thread Tom White
Hi Arun, I think it would be good to have a shared 0.20 Apache security branch. Since security isn't in 0.21, and the 0.22 release is a some way off as you mention, this would be useful for folks who want the security features sooner (and want to use an Apache release). Thanks, Tom On Mon, Aug 2

Re: [VOTE] Pig to become a TLP

2010-08-25 Thread Hemanth Yamijala
+1 On Mon, Aug 23, 2010 at 11:08 PM, Alan Gates wrote: > I propose that Pig become a top level Apache project. > > The Pig development community has already voted on and approved this > proposal.  In summary, the community voted that all current active > committers (listed at http://hadoop.apache

Re: [DISCUSS] Hadoop Security Release off Yahoo! patchset

2010-08-25 Thread Hemanth Yamijala
Arun, How much time do you think it would take to have a version of 0.20 with the security features in it ready ? In a different thread, Owen has started discussing plans around 0.22. Do you think this effort would affect 0.22 release ? I do agree that this would be very useful for folks who want

Re: [DISCUSS] Hadoop Security Release off Yahoo! patchset

2010-08-25 Thread Arun C Murthy
On Aug 25, 2010, at 10:46 AM, Hemanth Yamijala wrote: Arun, How much time do you think it would take to have a version of 0.20 with the security features in it ready ? In a different thread, Owen has started discussing plans around 0.22. Do you think this effort would affect 0.22 release ? I

Re: Searching Hadoop project(s) content

2010-08-25 Thread Nigel Daley
Big +1. This is cool and gr8 that it uses Solr. Cheers, Nige On Aug 25, 2010, at 10:01 AM, Alex Baranau wrote: > Hello guys, > > Over at http://search-hadoop.com we index Hadoop sub-project's mailing > lists, wiki, web site, > source code, javadoc, jira... Including general MLs for Hadoop TLP

Child processes on datanodes/task trackers

2010-08-25 Thread C J
Hi, I wanted to know why I see running Child processes on my datanodes even though there is no job running at that time. Are these left over from failed attempts? Is there anything I can do to keep these clean? Thanks, Deepika

Re: [DISCUSS] Hadoop Security Release off Yahoo! patchset

2010-08-25 Thread Allen Wittenauer
On Aug 25, 2010, at 10:46 AM, Hemanth Yamijala wrote: > I do agree that this would be very useful for folks who want security > sooner. And the fact that Yahoo! have been running it at scale for a > good while now is also assuring. As has been mentioned a few times, part of the security features

Re: [DISCUSS] Hadoop Security Release off Yahoo! patchset

2010-08-25 Thread Devaraj Das
>As has been mentioned a few times, part of the security features are dependent >upon Yahoo!-type operations. Allen, could you please enlist them here again (for the benefit of the community)? Or, are you referring to only the cluster-wide start scripts? On 8/25/10 1:25 PM, "Allen Wittenauer

Re: Child processes on datanodes/task trackers

2010-08-25 Thread Ted Yu
Use jps to find out pid of the Child. Then use this to find out which job the Child belongs to: ps aux | grep On Wed, Aug 25, 2010 at 12:20 PM, C J wrote: > Hi, > > I wanted to know why I see running Child processes on my datanodes even > though > there is no job running at that time. Are these

Re: Child processes on datanodes/task trackers

2010-08-25 Thread C J
Thanks for your reply. Some of these child tasks belong to successful jobs. I am wondering why they are still hanging there for long finished jobs. From: Ted Yu To: general@hadoop.apache.org Sent: Wed, August 25, 2010 4:17:38 PM Subject: Re: Child processes

Re: Child processes on datanodes/task trackers

2010-08-25 Thread Ted Yu
After you obtain pid, you can use jstack to see what the Child process was doing. What hadoop version are you using ? On Wed, Aug 25, 2010 at 7:28 PM, C J wrote: > Thanks for your reply. > > Some of these child tasks belong to successful jobs. I am wondering why > they are > still hanging there

Re: Child processes on datanodes/task trackers

2010-08-25 Thread C J
Thanks Ted! I did a jstack and it seems there is an issue with ehcache that I am using in the mapper task. "net.sf.ehcache.cachemana...@57ac3379" daemon prio=10 tid=0x59180800 nid=0x379e in Object.wait() [0x41506000] java.lang.Thread.State: TIMED_WAITING (on object monitor)

Re: Child processes on datanodes/task trackers

2010-08-25 Thread Ted Yu
I don't use ehcache. Did you forget to close CacheManager at the end of your job by any chance ? On Wed, Aug 25, 2010 at 7:59 PM, C J wrote: > Thanks Ted! > > I did a jstack and it seems there is an issue with ehcache that I am using > in > the mapper task. > > > "net.sf.ehcache.cachemana...@57a