I am porting a Hadoop job to Spark - One issue is that the workers need to read files from hdfs reading a different file based on the key or in some cases reading an object that is expensive to serialize. This is easy if the worker has access to the JavaSparkContext (I am working in Java) but this cannot be serialized - how can a worker read from a Path - assume hdfs
- Stupid Spark question Steve Lewis
- Re: Stupid Spark question Sean Owen
