Re: Running spark from Eclipse and then Jar

Oh thanks! I'll take a look On Sat, Dec 10, 2016 at 11:37 AM Md. Rezaul Karim < rezaul.ka...@insight-centre.org> wrote: > Hello Iman, > > Finally, I managed to solve the problem. I had been experiencing the > problem because of the locking issue in the "*metastore_db*" under the > project tree

Re: Running spark from Eclipse and then Jar

Hello Iman, Finally, I managed to solve the problem. I had been experiencing the problem because of the locking issue in the "*metastore_db*" under the project tree on Eclipse. If you see the project tree, under the "*metastore_db*" folder you should see a file named "*db.lck*" file which was

Re: Running spark from Eclipse and then Jar

yes exactly. I run mine fine in Eclipse but when I run it from a corresponding jar I get the same error! On Wed, Dec 7, 2016 at 5:04 PM Md. Rezaul Karim < rezaul.ka...@insight-centre.org> wrote: > I believe, it's not about the location (i.e., local machine or HDFS) but > it's all about the

Re: Running spark from Eclipse and then Jar

I believe, it's not about the location (i.e., local machine or HDFS) but it's all about the format of the input file. For example, I am getting the following error while trying to read an input file in libsvm format: *Exception in thread "main" java.lang.ClassNotFoundException: Failed to find

Re: Running spark from Eclipse and then Jar

No but I tried that too and still didn't work. Where are the files being read from? From the local machine or HDFS? Do I need to get the files to HDFS first? In Eclipse I just point to the location of the directory? On Wed, Dec 7, 2016 at 3:34 PM Md. Rezaul Karim <

Re: Running spark from Eclipse and then Jar

Hi, You should prepare your jar file (from your Spark application written in Java) with all the necessary dependencies. You can create a Maven project on Eclipse by specifying the dependencies in a Maven friendly pom.xml file. For building the jar with the dependencies and *main class (since you

Re: Running spark from Eclipse and then Jar

Don't you need to provide your class name "JavaWordCount"? Thanks, Vasu. > On Dec 7, 2016, at 3:18 PM, im281 wrote: > > Hello, > I have a simple word count example in Java and I can run this in Eclipse > (code at the bottom) > > I then create a jar file from it and