Problems running examples in IDEA

2014-08-24 Thread Ron Gonzalez
Hi, After getting the code base to compile, I tried running some of the scala examples. They all fail since it can't find classes like SparkConf. If I change the iml file to convert provided scope from PROVIDED to COMPILE, I am able to run them. It's simple by doing the following in the

Re: Problems running examples in IDEA

2014-08-24 Thread Ron Gonzalez
included Spark on the classpath. You would never modify the .iml files anyway. You can change the Maven pom.xml files if you were to need to modify a dependency scope. On Mon, Aug 25, 2014 at 12:12 AM, Ron Gonzalez zlgonza...@yahoo.com.invalid wrote: Hi, After getting the code base to compile, I

Re: Spark Avro Generation

2014-08-11 Thread Ron Gonzalez
If you don't want to build the entire thing, you can also do mvn generate-sources in externals/flume-sink Thanks, Ron Sent from my iPhone On Aug 11, 2014, at 8:32 AM, Hari Shreedharan hshreedha...@cloudera.com wrote: Jay running sbt compile or assembly should generate the sources. On

1.1.0-SNAPSHOT possible regression

2014-08-08 Thread Ron Gonzalez
Hi, I have a running spark app against the released version of 1.0.1. I recently decided to try and upgrade to the trunk version. Interestingly enough, after building the 1.1.0-SNAPSHOT assembly, replacing it as my assembly in my app caused errors. In particular, it seems Kryo serialization

Re: 1.1.0-SNAPSHOT possible regression

2014-08-08 Thread Ron Gonzalez
, Ron Gonzalez zlgonza...@yahoo.com.invalid wrote: Hi, I have a running spark app against the released version of 1.0.1. I recently decided to try and upgrade to the trunk version. Interestingly enough, after building the 1.1.0-SNAPSHOT assembly, replacing it as my assembly in my app caused

Re: Buidling spark in Eclipse Kepler

2014-08-07 Thread Ron Gonzalez
it directly as a Maven project in IntelliJ.) On Thu, Aug 7, 2014 at 4:53 AM, Ron Gonzalez zlgonza...@yahoo.com.invalid wrote: So I downloaded community edition of IntelliJ, and ran sbt/sbt gen-idea. I then imported the pom.xml file. I'm still getting all sorts of errors from IntelliJ about

Buidling spark in Eclipse Kepler

2014-08-06 Thread Ron Gonzalez
Hi,   I'm trying to get the apache spark trunk compiling in my Eclipse, but I can't seem to get it going. In particular, I've tried sbt/sbt eclipse, but it doesn't seem to create the eclipse pieces for yarn and other projects. Doing mvn eclipse:eclipse on yarn seems to fail as well as sbt/sbt

Re: Buidling spark in Eclipse Kepler

2014-08-06 Thread Ron Gonzalez
recommend IntelliJ over Eclipse for working with Maven and Scala. On Wed, Aug 6, 2014 at 8:29 PM, Ron Gonzalez zlgonza...@yahoo.com.invalid wrote: Hi, I'm trying to get the apache spark trunk compiling in my Eclipse, but I can't seem to get it going. In particular, I've tried sbt/sbt

Re: Buidling spark in Eclipse Kepler

2014-08-06 Thread Ron Gonzalez
So I downloaded community edition of IntelliJ, and ran sbt/sbt gen-idea. I then imported the pom.xml file. I'm still getting all sorts of errors from IntelliJ about unresolved dependencies. Any suggestions? Thanks, Ron On Wednesday, August 6, 2014 12:29 PM, Ron Gonzalez zlgonza