Looking at .COMPLETED is not an indication that the data has been written out to HDFS. As of now, unfortunately there is no way to tag an event as coming from a specific file. I can’t think of a way to do this in a fool-proof way off the top of my mind. What is your use-case, there might be another way to do the same thing?
Thanks, Hari On Fri, Dec 5, 2014 at 4:19 AM, Manohar CS <[email protected]> wrote: > Hi All, > I wanted to know if there is a way of notification mechanism or some way of > finding out if flume has finished transfer of certain file from spoolDir to > HDFS sink? We know by looking at .COMPLETED files in spoolDir we can assume > its completed but wanted to know if there is more reliable way of call back > mechanism ? > Thanks, > Manohar. > Please consider the environment before printing this e-mail > Disclaimer: This communication is for the exclusive use of the intended > recipient(s) and shall not attach any liability on the originator or ITC > Infotech India Ltd./its Holding company/ its Subsidiaries/ its Group > Companies. If you are the addressee, the contents of this e-mail are intended > for your use only and it shall not be forwarded to any third party, without > first obtaining written authorization from the originator or ITC Infotech > India Ltd./ its Holding company/its Subsidiaries/ its Group Companies. It > may contain information which is confidential and legally privileged and the > same shall not be used or dealt with by any third party in any manner > whatsoever without the specific consent of ITC Infotech India Ltd./ its > Holding company/ its Subsidiaries/ its Group Companies.
