What Flume Dev's meant was, Flume is not intended to do File transfers as a whole. A File shall be read and an Event created out of the content based on how an Event needs to be created (each line as an Event by default).
To answer you question (to the best of my knowledge) Q1. I would like to know how much file has been processed in parallel by SpoolDir ? Nope Q2. Is it possible to how much* size* of file has been moved by SpoolDir to sink at any point using built-in API? Nope If you can share what you want to achieve, we may be able to provide some pointers. thanks ashish On Fri, Feb 13, 2015 at 5:43 PM, mahendran m <[email protected]> wrote: > I need to move IIS logs to hdfs , then which is suitable for this purpose ? > > Thanks , > > ------------------------------ > From: [email protected] > To: [email protected] > Subject: RE: No threads used in Spooling Directory > Date: Fri, 13 Feb 2015 11:00:48 +0000 > > > I had posted for similar requirement few days back , but from core team > members got to know that spoolDir is not suitable for fileBased handling, > its more of event based content transfer that is being done. > > > > *From:* mahendran m [mailto:[email protected]] > *Sent:* Friday, February 13, 2015 11:02 AM > *To:* FlumeMaling List > *Subject:* No threads used in Spooling Directory > > > > HI , > > > > Q1. I would like to know how much file has been processed in parallel by > SpoolDir ? > > > > Q2. Is it possible to how much* size* of file has been moved by SpoolDir > to sink at any point using built-in API? > > > > Thanks , > > > > Please consider the environment before printing this e-mail > > > Disclaimer: This communication is for the exclusive use of the intended > recipient(s) and shall not attach any liability on the originator or ITC > Infotech India Ltd./its Holding company/ its Subsidiaries/ its Group > Companies. If you are the addressee, the contents of this e-mail are intended > for your use only and it shall not be forwarded to any third party, without > first obtaining written authorization from the originator or ITC Infotech > India Ltd./ its Holding company/its Subsidiaries/ its Group Companies. It > may contain information which is confidential and legally privileged and the > same shall not be used or dealt with by any third party in any manner > whatsoever without the specific consent of ITC Infotech India Ltd./ its > Holding company/ its Subsidiaries/ its Group Companies. > -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
