Re: Presentation in London: Running Spark on Hive or Hive on Spark

[mylisttech]

Hi Mich,

Would it be on YouTube , post session ?

- Harmeet



On Jul 7, 2016, at 3:07, Mich Talebzadeh  wrote:

> Dear forum members
> 
> I will be presenting on the topic of "Running Spark on Hive or Hive on Spark, 
> your mileage varies" in Future of Data: London 
> 
> Details
> 
> Organized by: Hortonworks
> 
> Date: Wednesday, July 20, 2016, 6:00 PM to 8:30 PM 
> 
> Place: London
> 
> Location: One Canada Square, Canary Wharf,  London E14 5AB.
> 
> Nearest Underground:  Canary Warf (map)
> 
> If you are interested please register here
> 
> Looking forward to seeing those who can make it to have an interesting 
> discussion and leverage your experience.
> 
> Regards,
> 
> Dr Mich Talebzadeh
>  
> LinkedIn  
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>  
> http://talebzadehmich.wordpress.com
> 
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, 
> damage or destruction of data or any other property which may arise from 
> relying on this email's technical content is explicitly disclaimed. The 
> author will in no case be liable for any monetary damages arising from such 
> loss, damage or destruction.
>  

Re: Book for Machine Learning (MLIB and other libraries on Spark)

[mylisttech]

Machine learning - I would suggest that you pick up a fine book that explains 
machine learning. That's the way I went about - pick up each type of machine 
learning concept - say Linear regression then understand the why/when/how etc 
and infer results etc. 

Then apply the learning to a small data set using python or R or scala without 
Spark. This is to familiarize the learning.

Then run the same with MLlib and see it with a big data set on Spark. I would 
call this consolidation. 

Few things to remember - all Machine learning algorithms are not available On 
spark. There is a list of machine learning supported in spark. Kindly look at 
that. Also look at how to integrate mahout / h20 with spark and see how you can 
run the machine learning stuff supported by mahout with spark.

And then your journey begins :-).

Regards,
Harmeet




On Jun 12, 2016, at 0:31, Mich Talebzadeh  wrote:

> yes absolutely Ted.
> 
> Thanks for highlighting it
> 
> 
> 
> Dr Mich Talebzadeh
>  
> LinkedIn  
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>  
> http://talebzadehmich.wordpress.com
>  
> 
> On 11 June 2016 at 19:00, Ted Yu  wrote:
> Another source is the presentation on various ocnferences.
> e.g.
> http://www.slideshare.net/databricks/apache-spark-mllib-20-preview-data-science-and-production
> 
> FYI
> 
> On Sat, Jun 11, 2016 at 8:47 AM, Mich Talebzadeh  
> wrote:
> Interesting.
> 
> The pace of development in this field is such that practically every single 
> book in Big Data landscape gets out of data before the ink dries on it  :)
> 
> I concur that they serve as good reference for starters but in my opinion the 
> best way to learn is to start from on-line docs (and these are pretty 
> respectful when it comes to Spark) and progress from there.
> 
> If you have a certain problem then put to this group and I am sure someone 
> somewhere in this forum has come across it. Also most of these books' authors 
> actively contribute to this mailing list.
> 
> 
> HTH
> 
> 
> Dr Mich Talebzadeh
>  
> LinkedIn  
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>  
> http://talebzadehmich.wordpress.com
>  
> 
> On 11 June 2016 at 16:10, Ted Yu  wrote:
> https://www.amazon.com/Machine-Learning-Spark-Powerful-Algorithms/dp/1783288515/ref=sr_1_1?ie=UTF8=1465657706=8-1=spark+mllib
> 
> https://www.amazon.com/Spark-Practical-Machine-Learning-Chinese/dp/7302420424/ref=sr_1_3?ie=UTF8=1465657706=8-3=spark+mllib
> 
> https://www.amazon.com/Advanced-Analytics-Spark-Patterns-Learning/dp/1491912766/ref=sr_1_2?ie=UTF8=1465657706=8-2=spark+mllib
> 
> 
> On Sat, Jun 11, 2016 at 8:04 AM, Deepak Goel  wrote:
> 
> Hey
> 
> Namaskara~Nalama~Guten Tag~Bonjour
> 
> I am a newbie to Machine Learning (MLIB and other libraries on Spark)
> 
> Which would be the best book to learn up?
> 
> Thanks
> Deepak
>-- 
> Keigu
> 
> Deepak
> 73500 12833
> www.simtree.net, dee...@simtree.net
> deic...@gmail.com
> 
> LinkedIn: www.linkedin.com/in/deicool
> Skype: thumsupdeicool
> Google talk: deicool
> Blog: http://loveandfearless.wordpress.com
> Facebook: http://www.facebook.com/deicool
> 
> "Contribute to the world, environment and more : http://www.gridrepublic.org
> "
> 
> 
> 
> 

Re: Running R codes in sparkR

[mylisttech]

Hi Arunkumar ,

Yes , R can be integrated with Spark to give you SparkR. There are a couple of 
blogs on the net. The Spark dev page has it too.

https://spark.apache.org/docs/latest/sparkr.html



Just remember that all packages of R that you may have worked on in R are not 
supported in SparkR. There are a good set of R packages in SparkR. 

As I have understood you cannot run sapply etc for example. The constraint 
being these packages need to be ported/coded for RDD's. The R community as I 
understand is not very deeply involved with the Spark community. - this I have 
understood by seeing you tube videos. 





On May 31, 2016, at 18:16, Arunkumar Pillai  wrote:

> Hi
> 
> I have some basic doubt regarding spark R.
> 
> 1. can we run R codes in spark using sparkR or some spark functionalities  
> that are executed in spark through R.
> 
> 
> 
> -- 
> Thanks and Regards
>Arun


-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: ANOVA test in Spark

[mylisttech]

Mayank,

Assuming Anova not present in MLIB can you not exploit the Anova from SparkR? I 
am enquiring not making a factual statement.

Thanks 



On May 13, 2016, at 15:54, mayankshete  wrote:

> Is ANOVA present in Spark Mllib if not then, when will be this feature be
> available in Spark ?
> 
> 
> 
> --
> View this message in context: 
> http://apache-spark-user-list.1001560.n3.nabble.com/ANOVA-test-in-Spark-tp26949.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
> 

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: Hello !

[mylisttech]

Thank you ! 



On Apr 12, 2016, at 1:41, Ted Yu  wrote:

> For SparkR, please refer to https://spark.apache.org/docs/latest/sparkr.html
> 
> bq. on Ubuntu or CentOS
> 
> Both platforms are supported.
> 
> On Mon, Apr 11, 2016 at 1:08 PM,  wrote:
> Dear Experts ,
> 
> I am posting this for your information. I am a newbie to spark.
> I am interested in understanding Spark at the internal level.
> 
> I need your opinion, which unix flavor should I install spark on Ubuntu or 
> CentOS. I have had enough trouble with the windows version (1.6.1 with Hadoop 
> 2.6 pre built binaries , keeps giving me exceptions ).
> 
> I have worked on R on windows till date . Is there an R for unix? I have not 
> googled this either. Sorry about that.Just want to make sure SparkR has a 
> smooth run.
> 
> Thanks in advance.
> Harry
> 
> 
> 
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
> 
> 

Hello !

[mylisttech]

Dear Experts ,

I am posting this for your information. I am a newbie to spark.
I am interested in understanding Spark at the internal level.

I need your opinion, which unix flavor should I install spark on Ubuntu or 
CentOS. I have had enough trouble with the windows version (1.6.1 with Hadoop 
2.6 pre built binaries , keeps giving me exceptions ). 

I have worked on R on windows till date . Is there an R for unix? I have not 
googled this either. Sorry about that.Just want to make sure SparkR has a 
smooth run.

Thanks in advance.
Harry




-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org