Re: Design patterns involving Spark

BTW, we now support OLAP functionality natively in spark w/o the need for Druid, through our Spark native BI platform(SNAP): https://www.linkedin.com/pulse/integrated-business-intelligence-big-data-stacks-harish-butani - we provide SQL commands to: create star schema, create olap index, and

Re: Design patterns involving Spark

Have not tried this, but looks quite useful if one is using Druid: https://github.com/implydata/pivot - An interactive data exploration UI for Druid On Tue, Aug 30, 2016 at 4:10 AM, Alonso Isidoro Roman wrote: > Thanks Mitch, i will check it. > > Cheers > > > Alonso

Re: Design patterns involving Spark

Thanks Mitch, i will check it. Cheers Alonso Isidoro Roman [image: https://]about.me/alonso.isidoro.roman 2016-08-30 9:52 GMT+02:00 Mich Talebzadeh : >

Re: Design patterns involving Spark

You can use Hbase for building real time dashboards Check this link HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Design patterns involving Spark

HBase for real time queries? HBase was designed with the batch in mind. Impala should be a best choice, but i do not know what Druid can do Cheers Alonso Isidoro Roman [image: https://]about.me/alonso.isidoro.roman

Re: Design patterns involving Spark

Hi Chanh, Druid sounds like a good choice. But again the point being is that what else Druid brings on top of Hbase. Unless one decides to use Druid for both historical data and real time data in place of Hbase! It is easier to write API against Druid that Hbase? You still want a UI dashboard?

Re: Design patterns involving Spark

Hi everyone, Seems a lot people using Druid for realtime Dashboard. I’m just wondering of using Druid for main storage engine because Druid can store the raw data and can integrate with Spark also (theoretical). In that case do we need to store 2 separate storage Druid (store segment in HDFS)

Re: Design patterns involving Spark

Spark best fits for processing. But depending on the use case, you could expand the scope of Spark to moving data using the native connectors. The only that Spark is not, is Storage. Connectors are available for most storage options though. Regards, Sivakumaran S > On 28-Aug-2016, at 6:04

Design patterns involving Spark

Hi, There are design patterns that use Spark extensively. I am new to this area so I would appreciate if someone explains where Spark fits in especially within faster or streaming use case. What are the best practices involving Spark. Is it always best to deploy it for processing engine,  For