On Wed, Jan 28, 2015 at 1:54 PM, Ken Williams <ke...@theasi.co> wrote:

>
> Hi Sam/Amir,
>
>     I've been able to 'juju ssh spark-master/0' and I successfully ran the
> two
> simple examples for pyspark and spark-shell,
>
>     ./bin/pyspark
>     >>> sc.parallelize(range(1000)).count()
>     1000
>
>     ./bin/spark-shell
>      scala> sc.parallelize(1 to 1000).count()
>     1000
>
>
> Now I want to run some of the spark examples in the spark-exampes*.jar
> file, which I have on my local machine. How do I copy the jar file from
> my local machine to the AWS machine ?
>
> I have tried 'scp' and 'juju scp' from the local command-line but both
> fail (below),
>
> root@adminuser:~# scp /tmp/spark-examples-1.2.0-hadoop2.4.0.jar
> ubuntu@ip-172-31-59:/tmp
> ssh: Could not resolve hostname ip-172-31-59: Name or service not known
> lost connection
> root@adminuser:~# juju scp /tmp/spark-examples-1.2.0-hadoop2.4.0.jar
> ubuntu@ip-172-31-59:/tmp
> ERROR exit status 1 (nc: getaddrinfo: Name or service not known)
>
> Any ideas ?
>

juju scp /tmp/spark-examples-1.2.0-hadoop2.4.0.jar spark-master/0:/tmp

>
> Ken
>
>
>
>
>
>
>
>
>
>
>
> On 28 January 2015 at 17:29, Samuel Cozannet <
> samuel.cozan...@canonical.com> wrote:
>
>> Glad it worked!
>>
>> I'll make a merge request to the upstream so that it works natively from
>> the store asap.
>>
>> Thanks for catching that!
>> Samuel
>>
>> Best,
>> Samuel
>>
>> --
>> Samuel Cozannet
>> Cloud, Big Data and IoT Strategy Team
>> Business Development - Cloud and ISV Ecosystem
>> Changing the Future of Cloud
>> Ubuntu <http://ubuntu.com>  / Canonical UK LTD <http://canonical.com> /
>> Juju <https://jujucharms.com>
>> samuel.cozan...@canonical.com
>> mob: +33 616 702 389
>> skype: samnco
>> Twitter: @SaMnCo_23
>>
>> On Wed, Jan 28, 2015 at 6:15 PM, Ken Williams <ke...@theasi.co> wrote:
>>
>>>
>>> Hi Sam (and Maarten),
>>>
>>>     Cloning Spark 1.2.0 from github seems to have worked!
>>>     I can install the Spark examples afterwards.
>>>
>>>     Thanks for all your help!
>>>
>>>     Yes - Andrew and Angie both say 'hi'  :-)
>>>
>>>     Best Regards,
>>>
>>> Ken
>>>
>>>
>>> On 28 January 2015 at 16:43, Samuel Cozannet <
>>> samuel.cozan...@canonical.com> wrote:
>>>
>>>> Hey Ken,
>>>>
>>>> So I had a closer look to your Spark problem and found out what went
>>>> wrong.
>>>>
>>>> The charm available on the charmstore is trying to download Spark
>>>> 1.0.2, and the versions available on the Apache website are 1.1.0, 1.1.1
>>>> and 1.2.0.
>>>>
>>>> There is another version of the charm available on GitHub that actually
>>>> will deploy 1.2.0
>>>>
>>>> 1. On your computer, the below folders & get there:
>>>>
>>>> cd ~
>>>> mkdir charms
>>>> mkdir charms/trusty
>>>> cd charms/trusty
>>>>
>>>> 2. Branch the Spark charm.
>>>>
>>>> git clone https://github.com/Archethought/spark-charm spark
>>>>
>>>> 3. Deploy Spark from local repository
>>>>
>>>> juju deploy --repository=~/charms local:trusty/spark spark-master
>>>> juju deploy --repository=~/charms local:trusty/spark spark-slave
>>>> juju add-relation spark-master:master spark-slave:slave
>>>>
>>>> Worked on AWS for me just minutes ago. Let me know how it goes for you.
>>>> Note that this version of the charm does NOT install the Spark examples.
>>>> The files are present though, so you'll find them in
>>>> /var/lib/juju/agents/unit-spark-master-0/charm/files/archive
>>>>
>>>> Hope that helps...
>>>> Let me know if it works for you!
>>>>
>>>> Best,
>>>> Sam
>>>>
>>>>
>>>> Best,
>>>> Samuel
>>>>
>>>> --
>>>> Samuel Cozannet
>>>> Cloud, Big Data and IoT Strategy Team
>>>> Business Development - Cloud and ISV Ecosystem
>>>> Changing the Future of Cloud
>>>> Ubuntu <http://ubuntu.com>  / Canonical UK LTD <http://canonical.com> /
>>>> Juju <https://jujucharms.com>
>>>> samuel.cozan...@canonical.com
>>>> mob: +33 616 702 389
>>>> skype: samnco
>>>> Twitter: @SaMnCo_23
>>>>
>>>> On Wed, Jan 28, 2015 at 4:44 PM, Ken Williams <ke...@theasi.co> wrote:
>>>>
>>>>>
>>>>> Hi folks,
>>>>>
>>>>> I'm completely new to juju so any help is appreciated.
>>>>>
>>>>> I'm trying to create a hadoop/analytics-type platform.
>>>>>
>>>>> I've managed to install the 'data-analytics-with-sql-like' bundle
>>>>> (using this command)
>>>>>
>>>>>     juju quickstart
>>>>> bundle:data-analytics-with-sql-like/data-analytics-with-sql-like
>>>>>
>>>>> This is very impressive, and gives me virtually everything that I want
>>>>> (hadoop, hive, etc) - but I also need Spark.
>>>>>
>>>>> The Spark charm (http://manage.jujucharms.com/~asanjar/trusty/spark)
>>>>> and bundle (
>>>>> http://manage.jujucharms.com/bundle/~asanjar/spark/spark-cluster)
>>>>> however do not seem stable or available and I can't figure out how to
>>>>> install them.
>>>>>
>>>>> Should I just download and install the Spark tar-ball on the nodes
>>>>> in my AWS cluster, or is there a better way to do this ?
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> Ken
>>>>>
>>>>>
>>>>> --
>>>>> Juju mailing list
>>>>> Juju@lists.ubuntu.com
>>>>> Modify settings or unsubscribe at:
>>>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>>>
>>>>>
>>>>
>>>
>>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Reply via email to