Spark YARN Shuffle service wire compatibility

2015-10-22 Thread Jong Wook Kim
Hi, I’d like to know if there is a guarantee that Spark YARN shuffle service has wire compatibility between 1.x versions. I could run Spark 1.5 job with YARN nodemanagers having shuffle service 1.4, but it might’ve been just a coincidence. Now we’re upgrading CDH to 5.3 to 5.4, whose

Re: Spark as a service

2015-03-25 Thread Irfan Ahmad
You're welcome. How did it go? *Irfan Ahmad* CTO | Co-Founder | *CloudPhysics* http://www.cloudphysics.com Best of VMworld Finalist Best Cloud Management Award NetworkWorld 10 Startups to Watch EMA Most Notable Vendor On Wed, Mar 25, 2015 at 7:53 AM, Ashish Mukherjee ashish.mukher...@gmail.com

Spark as a service

2015-03-24 Thread Ashish Mukherjee
Hello, As of now, if I have to execute a Spark job, I need to create a jar and deploy it. If I need to run a dynamically formed SQL from a Web application, is there any way of using SparkSQL in this manner? Perhaps, through a Web Service or something similar. Regards, Ashish

Re: Spark as a service

2015-03-24 Thread Jeffrey Jedele
Hi Ashish, this might be what you're looking for: https://spark.apache.org/docs/latest/sql-programming-guide.html#running-the-thrift-jdbcodbc-server Regards, Jeff 2015-03-24 11:28 GMT+01:00 Ashish Mukherjee ashish.mukher...@gmail.com: Hello, As of now, if I have to execute a Spark job, I

Re: Spark as a service

2015-03-24 Thread Jeffrey Jedele
I don't think there's are general approach to that - the usecases are just to different. If you really need it, you probably will have to implement yourself in the driver of your application. PS: Make sure to use the reply to all button so that the mailing list is included in your reply.

Re: Spark as a service

2015-03-24 Thread Todd Nist
Perhaps this project, https://github.com/calrissian/spark-jetty-server, could help with your requirements. On Tue, Mar 24, 2015 at 7:12 AM, Jeffrey Jedele jeffrey.jed...@gmail.com wrote: I don't think there's are general approach to that - the usecases are just to different. If you really need

Re: Spark as a service

2015-03-24 Thread Irfan Ahmad
Also look at the spark-kernel and spark job server projects. Irfan On Mar 24, 2015 5:03 AM, Todd Nist tsind...@gmail.com wrote: Perhaps this project, https://github.com/calrissian/spark-jetty-server, could help with your requirements. On Tue, Mar 24, 2015 at 7:12 AM, Jeffrey Jedele

Re: Hive on Spark with Spark as a service on CDH5.2

2015-03-17 Thread Arush Kharbanda
to import hive context in spark shell as well as to be able to do a spark-submit for the job to be run on the entire cluster. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hive-on-Spark-with-Spark-as-a-service-on-CDH5-2-tp22091.html Sent from the Apache

Hive on Spark with Spark as a service on CDH5.2

2015-03-17 Thread anu
to be run on the entire cluster. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Hive-on-Spark-with-Spark-as-a-service-on-CDH5-2-tp22091.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Creating Apache Spark-powered “As Service” applications

2015-01-16 Thread olegshirokikh
this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Creating-Apache-Spark-powered-As-Service-applications-tp21193.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe

Re: Creating Apache Spark-powered “As Service” applications

2015-01-16 Thread Corey Nolet
.nabble.com/Creating-Apache-Spark-powered-As-Service-applications-tp21193.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional

Re: Creating Apache Spark-powered “As Service” applications

2015-01-16 Thread Robert C Senkbeil
Emerging Technology Software Engineer From: olegshirokikh o...@solver.com To: user@spark.apache.org Date: 01/16/2015 01:32 PM Subject:Creating Apache Spark-powered “As Service” applications The question is about the ways to create a Windows desktop-based and/or web-based application

RE: Creating Apache Spark-powered “As Service” applications

2015-01-16 Thread Oleg Shirokikh
? Thanks, Oleg From: Robert C Senkbeil [mailto:rcsen...@us.ibm.com] Sent: Friday, January 16, 2015 12:21 PM To: Oleg Shirokikh Cc: user@spark.apache.org Subject: Re: Creating Apache Spark-powered “As Service” applications Hi, You can take a look at the Spark Kernel project: https://github.com/ibm