RE: Re: Spark assembly in Maven repo?

Thanks for the info! Xiaoyong From: Sean Owen [mailto:so...@cloudera.com] Sent: Monday, December 14, 2015 12:20 AM To: Xiaoyong Zhu <xiaoy...@microsoft.com> Cc: user <user@spark.apache.org> Subject: Re: Re: Spark assembly in Maven repo? Yes, though I think the Maven Central reposi

RE: Re: Spark assembly in Maven repo?

Thanks! do you mean something here (for example for 1.5.1 using scala 2.10)? https://repository.apache.org/content/repositories/releases/org/apache/spark/spark-core_2.10/1.5.1/ Xiaoyong From: Sean Owen [mailto:so...@cloudera.com] Sent: Saturday, December 12, 2015 12:45 AM To: Xiaoyong Zhu <xi

RE: Re: Spark assembly in Maven repo?

To: Mark Hamstra <m...@clearstorydata.com> Cc: Xiaoyong Zhu <xiaoy...@microsoft.com>; Jeff Zhang <zjf...@gmail.com>; user <user@spark.apache.org>; Zhaomin Xu <z...@microsoft.com>; Joe Zhang (SDE) <gui...@microsoft.com> Subject: Re: Re: Spark assembly in Maven

RE: Spark assembly in Maven repo?

it via Maven. In this case what’s the recommended way? Xiaoyong From: Jeff Zhang [mailto:zjf...@gmail.com] Sent: Thursday, December 10, 2015 11:03 PM To: Xiaoyong Zhu <xiaoy...@microsoft.com> Cc: user@spark.apache.org Subject: Re: Spark assembly in Maven repo? I don't think make the as

Spark assembly in Maven repo?

Hi Experts, We have a project which has a dependency for the following jar spark-assembly--hadoop.jar for example: spark-assembly-1.4.1.2.3.3.0-2983-hadoop2.7.1.2.3.3.0-2983.jar since this assembly might be updated in the future, I am not sure if there is a Maven repo that has the above spark

how to get Spark stage DAGs thru the REST APIs?

Hi experts It seems that for the below Spark Stage DAGs, they are available for the Spark UI/Spark History, however they are not available from any of the Spark REST APIs. Not sure if I missed anything if we want to get such kind of data from

is there a way to interact with Spark clusters remotely?

Hi experts I am wondering if there is a way to interactive with Spark remotely? i.e. no access to clusters required but submit Python/Scala scripts to cluster and get result based on (REST) APIs. That will facilitate the development process a lot.. Xiaoyong

RE: is there a way to interact with Spark clusters remotely?

...@gmail.com] Sent: Monday, December 15, 2014 10:33 PM To: Xiaoyong Zhu Subject: R: is there a way to interact with Spark clusters remotely? Hi, try this https://github.com/spark-jobserver/spark-jobserver . Best Regards, Pietro Gentile Da: Xiaoyong Zhu [mailto:xiaoy...@microsoft.com] Inviato: lunedì 15

Spark Streaming Python APIs?

Hi spark experts Are there any Python APIs for Spark Streaming? I didn't find the Python APIs in Spark Streaming programming guide.. http://spark.apache.org/docs/latest/streaming-programming-guide.html Xiaoyong

RE: Spark Streaming Python APIs?

Cool thanks! Xiaoyong From: Shao, Saisai [mailto:saisai.s...@intel.com] Sent: Monday, December 15, 2014 10:57 AM To: Xiaoyong Zhu Cc: user@spark.apache.org Subject: RE: Spark Streaming Python APIs? AFAIK, this will be a new feature in version 1.2, you can check out the master branch or 1.2

RE: Spark Streaming Python APIs?

Btw I have seen the python related docs in the 1.2 doc here: http://people.apache.org/~pwendell/spark-1.2.0-rc2-docs/streaming-programming-guide.html Xiaoyong From: Xiaoyong Zhu [mailto:xiaoy...@microsoft.com] Sent: Monday, December 15, 2014 10:58 AM To: Shao, Saisai Cc: user@spark.apache.org

Spark SQL Roadmap?

Dear spark experts, I am very interested in Spark SQL availability in the future - could someone share with me the information about the following questions? 1. Is there some ETAs for the Spark SQL release? 2. I heard there is a Hive on Spark program also - what's the difference

RE: Spark SQL Roadmap?

: Saturday, December 13, 2014 10:59 PM To: Xiaoyong Zhu; user@spark.apache.org Subject: Re: Spark SQL Roadmap? Hi Xiaoyong, SparkSQL has already been released and has been part of the Spark code-base since Spark 1.0. The latest stable release is Spark 1.1 (here's the Spark SQL Programming