yeah, tats what we do. But its again an extra process, if hadoop had an ability, then it would be great. it uses log4j, i tired to tweak it, but it is throwing error.
Regards, Jagaran ________________________________ From: Michael Segel <michael_se...@hotmail.com> To: common-user@hadoop.apache.org Sent: Sat, 25 June, 2011 3:58:19 AM Subject: RE: Any reason Hadoop logs cant be directed to a separate filesystem? Yes, and its called using cron and writing a simple ksh script to clear out any files that are older than 15 days. There may be another way, but that's really the easiest. > Date: Thu, 23 Jun 2011 02:44:48 +0530 > From: jagaran_...@yahoo.co.in > Subject: Re: Any reason Hadoop logs cant be directed to a separate filesystem? > To: common-user@hadoop.apache.org > > Hi, > > Can I limit the log file duration ? > I want to keep files for last 15 days only. > > Regards, > Jagaran > > > > ________________________________ > From: Jack Craig <jcr...@carrieriq.com> > To: "common-user@hadoop.apache.org" <common-user@hadoop.apache.org> > Sent: Wed, 22 June, 2011 2:00:23 PM > Subject: Re: Any reason Hadoop logs cant be directed to a separate filesystem? > > Thx to both respondents. > > Note i've not tried this redirection as I have only production grids available. > > Our grids are growing and with them, log volume. > > As until now that log location has been in the same fs as the grid data, > so running out of space due log bloat is a growing problem. > > From your replies, sounds like I can relocate my logs, Cool! > > But now the tough question, if i set up a too small partition and it runs out >of > > space, > will my grid become unstable if hadoop can no longer write to its logs? > > Thx again, jackc... > > ------------------------------------ > Jack Craig, Operations > CarrierIQ.com<http://CarrierIQ.com> > 1200 Villa Ct, Suite 200 > Mountain View, CA. 94041 > 650-625-5456 > > On Jun 22, 2011, at 1:09 PM, Harsh J wrote: > > Jack, > > I believe the location can definitely be set to any desired path. > Could you tell us the issues you face when you change it? > > P.s. The env var is used to set the config property hadoop.log.dir > internally. So as long as you use the regular scripts (bin/ or init.d/ > ones) to start daemons, it would apply fine. > > On Thu, Jun 23, 2011 at 1:32 AM, Jack Craig > <jcr...@carrieriq.com<mailto:jcr...@carrieriq.com>> wrote: > Hi Folks, > > In the hadoop-env.sh, we find, ... > > # Where log files are stored. $HADOOP_HOME/logs by default. > # export HADOOP_LOG_DIR=${HADOOP_HOME}/logs > > is there any reason this location could not be a separate filesystem on the >name > > node? > > Thx, jackc... > ------------------------------------ > Jack Craig, Operations > CarrierIQ.com<http://CarrierIQ.com> > 1200 Villa Ct, Suite 200 > Mountain View, CA. 94041 > 650-625-5456 > > > > > > -- > Harsh J
