What is the use case? Where in hadoop you want to store? Where is the source coming from is it in files?
What is the issue with BlobDeserializer? Ehsan On 11 Apr 2014 12:07, "R W" <[email protected]> wrote: > Hi Ehsan > > Thanks for your quick responding, is it BlobDeserializer the only way to > go? Thanks. > > Cheers > aij > > > On Fri, Apr 11, 2014 at 4:27 PM, Ehsan ul Haq <[email protected]>wrote: > >> Yes it is possible to transfer binary files into Hadoop (HDFS/Hbase etc) >> using flume. >> If your binary files are already being stored in some local file system >> in some folders then you can use SpoolingDirectory source " >> https://flume.apache.org/FlumeUserGuide.html#spooling-directory-source";. >> The default deserializer is a LINE based text deserializer which is not for >> binary data. You can use BlobDeserializer " >> https://flume.apache.org/FlumeUserGuide.html#blobdeserializer"; which can >> handle binary data. >> >> Cheers >> Ehsan >> >> >> On Fri, Apr 11, 2014 at 9:58 AM, R W <[email protected]> wrote: >> >>> Hi All >>> >>> We have an App will generate huge amount of binary based file, it's not >>> like text file which we may have new line separator for it, we want to >>> collect these binary files and store them into a Hadoop cluster, I'm new to >>> Flume, so could anyone share some ideas here on how to do it with Flume, >>> thanks in advance. >>> >>> Cheers >>> aij >>> >> >> >
