Re: Python Spark on YARN

2014-04-29 Thread Guanhua Yan
Thanks, Matei. Will take a look at it.

Best regards,
Guanhua

From:  Matei Zaharia 
Reply-To:  
Date:  Tue, 29 Apr 2014 14:19:30 -0700
To:  
Subject:  Re: Python Spark on YARN

This will be possible in 1.0 after this pull request:
https://github.com/apache/spark/pull/30

Matei

On Apr 29, 2014, at 9:51 AM, Guanhua Yan  wrote:

> Hi all:
> 
> Is it possible to develop Spark programs in Python and run them on YARN? From
> the Python SparkContext class, it doesn't seem to have such an option.
> 
> Thank you,
> - Guanhua
> 
> ===
> Guanhua Yan, Ph.D.
> Information Sciences Group (CCS-3)
> Los Alamos National Laboratory
> Tel: +1-505-667-0176
> Email: gh...@lanl.gov
> Web: http://ghyan.weebly.com <http://ghyan.weebly.com/> /
> ===





Re: Python Spark on YARN

2014-04-29 Thread Matei Zaharia
This will be possible in 1.0 after this pull request: 
https://github.com/apache/spark/pull/30

Matei

On Apr 29, 2014, at 9:51 AM, Guanhua Yan  wrote:

> Hi all:
> 
> Is it possible to develop Spark programs in Python and run them on YARN? From 
> the Python SparkContext class, it doesn't seem to have such an option.
> 
> Thank you,
> - Guanhua
> 
> ===
> Guanhua Yan, Ph.D.
> Information Sciences Group (CCS-3)
> Los Alamos National Laboratory
> Tel: +1-505-667-0176
> Email: gh...@lanl.gov
> Web: http://ghyan.weebly.com/
> ===