Spark and Stanford CoreNLP

2014-11-24 Thread tvas
Hello, I was wondering if anyone has gotten the Stanford CoreNLP Java library to work with Spark. My attempts to use the parser/annotator fail because of task serialization errors since the class StanfordCoreNLP cannot be serialized. I've tried the remedies of registering StanfordCoreNLP

Re: Optimizing text file parsing, many small files versus few big files

2014-11-20 Thread tvas
Hello Rzykov, I tried you approach but unfortunately I'm getting an error. What I get is: [info] Caused by: java.lang.reflect.InvocationTargetException [info] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [info] at