Dear All, I'm using - => Spark 1.2.0 => Hive 0.13.1 => Mesos 0.18.1 => Spring => JDK 1.7
I've written a scala program which => instantiates a spark and hive context => parses an XML file which provides the where clauses for queries => generates full fledged hive queries to be run on hive tables => registers obtained SchemaRDD as temp tables to get reduced data sets to be queried further => prints the count of finally obtained data set I'm running this scala programatically through java command (command invokes a controller program to create some useful value objects using input parameters and properties files and then calls the above scala program). *I'm getting PermGen Space error when it hits the last line to print the count.* I'm printing to console the generated hive queries from the scala program. When I run the same from a spark shell it works fine. As mentioned in some posts and blogs I tried using the option spark.driver.extraJavaOptions to increase the size, tried with 256 and 512 but still no luck. Please help me in resolving the space issue Thanks & Regards, Sarath.
