unsubscribe
On Thu, Aug 28, 2014 at 6:42 PM, Eric Payne <eric.payne1...@yahoo.com> wrote: > Or, maybe have a look at Apache Falcon: > Falcon - Apache Falcon - Data management and processing platform > <http://falcon.incubator.apache.org/> > > > > > > > Falcon - Apache Falcon - Data management and processing platform > <http://falcon.incubator.apache.org/> > Apache Falcon - Data management and processing platform > View on falcon.incubator.apache.org <http://falcon.incubator.apache.org/> > Preview by Yahoo > > > *From:* Stanley Shi <s...@pivotal.io> > *To:* "user@hadoop.apache.org" <user@hadoop.apache.org> > *Sent:* Thursday, August 28, 2014 1:15 AM > *Subject:* Re: What happens when .....? > > Normally MR job is used for batch processing. So I don't think this is a > good use case here for MR. > Since you need to run the program periodically, you cannot submit a single > mapreduce job for this. > An possible way is to create a cron job to scan the folder size and submit > a MR job if necessary; > > > > On Wed, Aug 27, 2014 at 7:38 PM, Kandoi, Nikhil <nikhil.kan...@emc.com> > wrote: > > Hi All, > > I have a system where files are coming in hdfs at regular intervals and I > perform an operation everytime the directory size goes above a particular > point. > My Question is that when I submit a map reduce job, would it only work on > the files present at that point ?? > > Regards, > Nikhil Kandoi > > > > > > > -- > Regards, > *Stanley Shi,* > > > >
