Hi, I don't believe sub directories are supported at this time. Perhaps you should file a JIRA for this improvement?
Brock On Mon, Feb 4, 2013 at 10:01 AM, Tzur Turkenitz <[email protected]> wrote: > Hi, > I couldn't find any information about this in the Wiki / User guide. > Are sub directories for the Flume SpoolDir supported in 1.3x? > Thanks! > > > On Thu, Jan 31, 2013 at 11:56 AM, Tzur Turkenitz <[email protected]> wrote: > >> Hi All, >> >> I am experimenting with Flumes` 1.3 new SpoolDir adapter. >> I was wondering is there an option to tell the adapter to walk down >> directories and search for files. >> >> Currently, if I have a directory within the source location I get java >> exceptions. >> 13/01/30 17:41:30 WARN avro.SpoolingFileLineReader: Could not find file: >> /home/fl/source/dir >> java.io.FileNotFoundException: /home/fl/source/dir (Is a directory) >> at java.io.FileInputStream.open(Native Method) >> at java.io.FileInputStream.<init>(FileInputStream.java:120) >> at java.io.FileReader.<init>(FileReader.java:55) >> at >> org.apache.flume.client.avro.SpoolingFileLineReader.getNextFile(SpoolingFileLineReader.java:322) >> at >> org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:172) >> at >> org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:135) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >> at >> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >> at java.lang.Thread.run(Thread.java:662) >> Thanks! >> >> >> >> -- >> Regards, >> Tzur Turkenitz >> Vision.BI >> http://www.vision.bi/ >> >> "*Facts are stubborn things, but statistics are more pliable*" >> -Mark Twain >> > > > > -- > Regards, > Tzur Turkenitz > Vision.BI > http://www.vision.bi/ > > "*Facts are stubborn things, but statistics are more pliable*" > -Mark Twain > -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
