hi all, I checked the code just now. Spark scan the dir and filter files by timestamp of file. I copied files to the dir but not changed the timestamp. I know the reason now.
thanks ---Original--- From: "??????????"<[email protected]> Date: 2017/2/23 17:07:33 To: "dev"<[email protected]>; Subject: Filestream can not recognize the copied file Hi all, I tested filestream today, my code looks like: val fs = ssc.textFileStream(*) erroelines = fs.filter( _.contains("erroe")) erroelines.print ssc.start() when I edit a file and save it to the dir, it works well. If i copy a file to the dir, it does work. my issues are: 1, is it OK please? I means whether the code was designed to work as this. 2,which class monitor the dir pls?
