Re: Building Spark 2.X in Intellij

I just checked out completely fresh directory and created new IJ project. Then followed your tip for adding the avro source. Here is an additional set of errors Error:(31, 12) object razorvine is not a member of package net import net.razorvine.pickle.{IObjectPickler, Opcodes, Pickler}

Re: Building Spark 2.X in Intellij

Thanks Jeff - I remember that now from long time ago. After making that change the next errors are: Error:scalac: missing or invalid dependency detected while loading class file 'RDDOperationScope.class'. Could not access term fasterxml in package com, because it (or its dependencies) are

Re: Building Spark 2.X in Intellij

You need to spark/external/flume-sink/target/scala-2.11/src_managed/main/compiled_avro under build path, this is the only thing you need to do manually if I remember correctly. On Thu, Jun 23, 2016 at 2:30 PM, Stephen Boesch wrote: > Hi Jeff, > I'd like to understand what

Re: Building Spark 2.X in Intellij

Hi Jeff, I'd like to understand what may be different. I have rebuilt and reimported many times. Just now I blew away the .idea/* and *.iml to start from scratch. I just opened the $SPARK_HOME directory from intellij File | Open . After it finished the initial import I tried to run one of

Re: Building Spark 2.X in Intellij

I had some errors like SqlBaseParser class missing, and figured out I needed to get these classes from SqlBase.g4 using antlr4. It works fine now. On Thu, Jun 23, 2016 at 9:20 AM, Jeff Zhang wrote: > It works well with me. You can try reimport it into intellij. > > On Thu, Jun

Re: Building Spark 2.X in Intellij

It works well with me. You can try reimport it into intellij. On Thu, Jun 23, 2016 at 10:25 AM, Stephen Boesch wrote: > > Building inside intellij is an ever moving target. Anyone have the magical > procedures to get it going for 2.X? > > There are numerous library references

Building Spark 2.X in Intellij

Building inside intellij is an ever moving target. Anyone have the magical procedures to get it going for 2.X? There are numerous library references that - although included in the pom.xml build - are for some reason not found when processed within Intellij.