Hi, Like anything else your mileage varies using any tool.
To start what is your use case here (fit for your needs)? You stated that you want to perform OLAP on large datasets. OLAP is normally performed on large data sets anyway so I assume you already have some form of Data Warehouse commercial or otherwise. Do you also need to do Big Data analytics containing a variety of data formats including un-structured data? HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 2 April 2016 at 21:34, Eris Lawrence <vortexi...@gmail.com> wrote: > Hi Spark devs, > > I was recently into a tech session about data processing with spark vs > redshift which concluded with metrics and datapoint that for 2 Billion > data, Select queries on data based on filters on attributes were faster and > cheaper on AWS Redshift as compared to an AWS Spark cluster. > > I have researched around a bit, and both Redshift and Spark seem to > processing softwares where we want to do OLAP queries on a large dataset. I > was wondering in which usecases does Spark has an edge over Redshift? Are > there certain kind of Complex queries where Spark can outperform Redshift? > Or does Redshift only work well with schema defined data? > > Please share your experience with either of the technologies. Thanks. > > Cheers, > Eris. >
