Re: Secondary Indexing?

2016-05-30 Thread Mich Talebzadeh
mmutable existing RDDs, and > Hive/HBase/MapRDB/ which are mutable. > So we have to keep this in mind while we are talking about secondary > indexing. (Its not just RDDs) > > > I think the only advantage to being immutable is that once you generate > and index the RDD, its not go

Re: Secondary Indexing?

2016-05-30 Thread Gourav Sengupta
ms of design and vision for spark. > > If we look at SparkSQL and performance… where does Secondary indexing fit > in? > > The reason this is a bit awkward is that if you view Spark as querying > RDDs which are temporary, indexing doesn’t make sense until you consider > your us

Re: Secondary Indexing?

2016-05-30 Thread Mich Talebzadeh
a philosophical > question in terms of design and vision for spark. > > If we look at SparkSQL and performance… where does Secondary indexing fit > in? > > The reason this is a bit awkward is that if you view Spark as querying > RDDs which are temporary, indexing doesn’t make sens

Secondary Indexing?

2016-05-30 Thread Michael Segel
I’m not sure where to post this since its a bit of a philosophical question in terms of design and vision for spark. If we look at SparkSQL and performance… where does Secondary indexing fit in? The reason this is a bit awkward is that if you view Spark as querying RDDs which are temporary