Re: using hivecontext with sparksql on cdh 5.3

2015-02-20 Thread chirag lakhani
the CDH configs for hive and hadoop. Hopefully this works for you. On Fri, Feb 20, 2015 at 3:41 PM, chirag lakhani chirag.lakh...@gmail.com wrote: Thanks! I am able to login to Spark now but I am still getting the same error scala sqlContext.sql(FROM analytics.trainingdatafinal SELECT

Re: using hivecontext with sparksql on cdh 5.3

2015-02-20 Thread chirag lakhani
) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) On Fri, Feb 20, 2015 at 3:28 PM, Sourigna Phetsarath gna.phetsar...@teamaol.com wrote: Try it without --master yarn-cluster if you are trying to run a spark-shell. :) On Fri, Feb 20, 2015 at 3:18 PM, chirag lakhani chirag.lakh

using hivecontext with sparksql on cdh 5.3

2015-02-20 Thread chirag lakhani
I am trying to access a hive table using spark sql but I am having trouble. I followed the instructions in a cloudera community board which stated 1) Import hive jars into the class path export SPARK_CLASSPATH=$(find /data/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/lib/hive/lib/ -name

Re: using hivecontext with sparksql on cdh 5.3

2015-02-20 Thread chirag lakhani
/cloudera/parcels/CDH/lib/hive/lib/*' --driver-java-options '-Dspark.executor.extraClassPath=/opt/cloudera/parcels/CDH/lib/hive/lib/*' ... Let me know, if you have any issues. On Fri, Feb 20, 2015 at 2:43 PM, chirag lakhani chirag.lakh...@gmail.com wrote: I am trying to access a hive table using

Re: deploying a model built in mllib

2014-11-07 Thread chirag lakhani
. The latter one may be useful to you. And you can customize the Algorithm component, too. I have just created a doc: http://docs.prediction.io/0.8.1/templates/ Love to hear your feedback! Regards, Simon On Mon, Oct 27, 2014 at 11:03 AM, chirag lakhani chirag.lakh...@gmail.com wrote: Would

deploying a model built in mllib

2014-10-27 Thread chirag lakhani
Hello, I have been prototyping a text classification model that my company would like to eventually put into production. Our technology stack is currently Java based but we would like to be able to build our models in Spark/MLlib and then export something like a PMML file which can be used for

Re: apache whirr for spark

2014-06-01 Thread chirag lakhani
don’t think Whirr provides support for this, but Spark’s own EC2 scripts also launch a Hadoop cluster: http://spark.apache.org/docs/latest/ec2-scripts.html. Matei On May 30, 2014, at 12:59 PM, chirag lakhani chirag.lakh...@gmail.com wrote: Does anyone know if it is possible to use Whirr