Hi, Is there a way to read a text file from inside a spark executor? I need to do this for an streaming application where we need to read a file(whose contents would change) from a closure.
I cannot use the "sc.textFile" method since spark context is not serializable. I also cannot read a file using the Hadoop Api since the "FileSystem" class is not serializable as well. Does anyone have any idea on how I can go about this? Thanks, Udit
