Re: Spark replacing Hadoop

2016-04-14 Thread Mich Talebzadeh
One can see from the responses that Big Data landscape is getting very crowded with tools and there are dozens of alternatives offered. However, as usual the laws of selection will gravitate towards solutions that are scalable, reliable and more importantly cost effective. To this end any

Re: Spark replacing Hadoop

2016-04-14 Thread Peyman Mohajerian
Cloud adds another dimension: The fact that in cloud compute and storage is decoupled, s3-emr or blob-hdisight, means in cloud Hadoop ends up being more of a compute engine and a lot of the governance, security features are irrelevant or less important because data at rest is out of Hadoop.

Re: Spark replacing Hadoop

2016-04-14 Thread Cody Koeninger
I've been using spark for years and have (thankfully) been able to avoid needing HDFS, aside from one contract where it was already in use. At this point, many of the people I know would consider Kafka to be more important than HDFS. On Thu, Apr 14, 2016 at 3:11 PM, Jörn Franke

Re: Spark replacing Hadoop

2016-04-14 Thread Jörn Franke
I do not think so. Hadoop provides an ecosystem in which you can deploy different engines, such as MR, HBase, TEZ, Spark, Flink, titandb, hive, solr... I observe also that commercial analytical tools use one or more of these engines to execute their code in a distributed fashion. You need this

Re: Spark replacing Hadoop

2016-04-14 Thread Sean Owen
jects implemented on hadoop, …) I would probably start using >> spark. Its faster and easier to use >> >> Your mileage may vary >> >> Andy >> >> From: Ashok Kumar <ashok34...@yahoo.com.INVALID> >> Reply-To: Ashok Kumar <ashok34...@yah

Re: Spark replacing Hadoop

2016-04-14 Thread Arunkumar Chandrasekar
e your question is “Will Spark replace Hadoop MapReduce?” or do > you literally mean replacing the whole of Hadoop? > > David > > From: Ashok Kumar [mailto:ashok34...@yahoo.com.INVALID] > Sent: Thursday, April 14, 2016 2:13 PM > To: User > Subject: Spark replacing Ha

Re: Spark replacing Hadoop

2016-04-14 Thread Ashok Kumar
y mean replacing the whole of Hadoop?   David   From: Ashok Kumar [mailto:ashok34...@yahoo.com.INVALID] Sent: Thursday, April 14, 2016 2:13 PM To: User Subject: Spark replacing Hadoop   Hi,   I hear that some saying that Hadoop is getting old and out of date and will be replaced by Spark!

Re: Spark replacing Hadoop

2016-04-14 Thread Felipe Gustavo
> To: "user @spark" <user@spark.apache.org> > Subject: Spark replacing Hadoop > > Hi, > > I hear that some saying that Hadoop is getting old and out of date and > will be replaced by Spark! > > Does this make sense and if so how accurate is it? > > Best > >

RE: Spark replacing Hadoop

2016-04-14 Thread David Newberger
Can we assume your question is “Will Spark replace Hadoop MapReduce?” or do you literally mean replacing the whole of Hadoop? David From: Ashok Kumar [mailto:ashok34...@yahoo.com.INVALID] Sent: Thursday, April 14, 2016 2:13 PM To: User Subject: Spark replacing Hadoop Hi, I hear that some

Re: Spark replacing Hadoop

2016-04-14 Thread Mich Talebzadeh
yahoo.com.INVALID> > Reply-To: Ashok Kumar <ashok34...@yahoo.com> > Date: Thursday, April 14, 2016 at 12:13 PM > To: "user @spark" <user@spark.apache.org> > Subject: Spark replacing Hadoop > > Hi, > > I hear that some saying that Hadoop is getting

Re: Spark replacing Hadoop

2016-04-14 Thread Andy Davidson
From: Ashok Kumar <ashok34...@yahoo.com.INVALID> Reply-To: Ashok Kumar <ashok34...@yahoo.com> Date: Thursday, April 14, 2016 at 12:13 PM To: "user @spark" <user@spark.apache.org> Subject: Spark replacing Hadoop > Hi, > > I hear that some saying that

Spark replacing Hadoop

2016-04-14 Thread Ashok Kumar
Hi, I hear that some saying that Hadoop is getting old and out of date and will be replaced by Spark! Does this make sense and if so how accurate is it? Best