Yep, sure enough the jar I need is not in my classpath (as per the YARN container's stdout).
To fix this issue, and for future reference, what is the best way of adding external libraries to the classpath for Samza? Going by how the samza/bin/run-class.sh script is written, it seems like it would probably work if I place the jars I need in samza/lib directory. This seems rather hacky though, and I would have to do it everytime I run `mvn clean package`. So is there something I'm missing in the Maven pom.xml that is placing all the needed libraries, apart from the new ones I've added as dependencies, into the samza/lib directory of the tarball? (sorry if this is something obvious...) Thanks, Jonathan ------------------------------------------------------ From: Chris Riccomini [email protected] Reply: [email protected] [email protected] Date: 10 February 2014 at 14:16:04 To: [email protected] [email protected] Subject: Re: Unable to call external library classes from within Samza > > Hey Jonathan, > > Can you post your classpath? You can usually find this in your > YARN > container's stdout file, if you're running with YARN. If you're > running > with LocalJobFactory, it should print to SDTOUT. > > Cheers, > Chris > > On 2/8/14 11:54 AM, "Jonathan Poltak Samosir" > wrote: > > >Hello, > > > >This is a bit of a follow on on a previous thread of mine, > >"org.apache.hadoop.util.Shell$ExitCodeException whenever > Samza container > >launches". > > > >So I am not sure whether this is a bug, or if I am not doing something > >correct, or it is intended, but whenever I attempt to call > >Class.forName() on an external library class from within Samza, > I am > >running into a ClassNotFoundException. My Maven deps are set > up > >correctly, and the exact same code works without any issues > if invoked > >manually through a main() method, for example, as opposed to > running it > >through Samza. > > > >The reason I want to do this, is to test that classes for JDBC drivers > >can be found and used. I fully understand that this method is > no longer > >needed as of JDBC 4.0, and that the drivers should be automatically > >loaded if found in the class path, although this isn't happening > either > >(an SQLException is thrown with "No suitable driver found" > if I leave out > >the Class.forName() call, which leads to the same problem). > Just so you > >know, this same code also works fine and loads the appropriate > JDBC > >driver if invoked directly through a main() method. > > > >I have also tried this with a number of different external libraries, > >both pulled in using Maven and manually linking .jar files on > my disk, > >and the same result; those classes can be found fine from a standard > Java > >main() method call, but cannot be found running in a Samza container. > > > >Has anyone encountered the same issue, or know of why this could > be > >happening? > > > >I hope all that made sense, let me know if it didn't and I'll try > to > >rephrase. > > > >Here is the class I am trying to do this in, in case anyone wants > to see > >what I'm talking about: > >https://github.com/poltak/hello-samza/blob/database-reader/samza-wikipedia > >/src/main/java/samza/examples/databasereader/system/DatabaseReaderConsumer > >.java > > > >Anyway, thanks for your time, > >Jonathan > >
