Re: Does Spark SQL support indexes?

2016-08-15 Thread Mich Talebzadeh
ot use >>> idexes? >>> There might be some very interesting new concepts I've missed? >>> >>> Could you be more precise? >>> >>> ;-) >>> >>> Regards, >>> Uwe >>> >>> >>> >>> Am 15.08.

Re: Does Spark SQL support indexes?

2016-08-15 Thread Gourav Sengupta
ndexes, materialized views, and other single >> processor non-distributed system algorithms. Nice that you are not asking >> questions regarding hierarchical file systems. >> >> >> Regards, >> Gourav >> >> On Sun, Aug 14, 2016 at 4:03 AM, Taotao.Li

Re: Does Spark SQL support indexes?

2016-08-15 Thread Mich Talebzadeh
algorithms. Nice that you are not asking > questions regarding hierarchical file systems. > > > Regards, > Gourav > > On Sun, Aug 14, 2016 at 4:03 AM, Taotao.Li <charles.up...@gmail.com> > wrote: > >> >> hi, guys, does Spark SQL support indexes? if so, how

Re: Does Spark SQL support indexes?

2016-08-15 Thread u...@moosheimer.com
mail.com>> wrote: > > > hi, guys, does Spark SQL support indexes? if so, how can I create > an index on my temp table? if not, how can I handle some specific > queries on a very large table? it would iterate all the table even > though all I wa

RE: Does Spark SQL support indexes?

2016-08-15 Thread Ashic Mahtab
Guess the good people in the Cassandra world are stuck in the past making indexes, materialized views, etc. better with every release :) From: mich.talebza...@gmail.com Date: Mon, 15 Aug 2016 11:11:03 +0100 Subject: Re: Does Spark SQL support indexes? To: gourav.sengu...@gmail.com CC: charles.up

Re: Does Spark SQL support indexes?

2016-08-15 Thread Mich Talebzadeh
archical file systems. > > > Regards, > Gourav > > On Sun, Aug 14, 2016 at 4:03 AM, Taotao.Li <charles.up...@gmail.com> > wrote: > >> >> hi, guys, does Spark SQL support indexes? if so, how can I create an >> index on my temp table? if not, how can I handle

Re: Does Spark SQL support indexes?

2016-08-15 Thread Gourav Sengupta
> > hi, guys, does Spark SQL support indexes? if so, how can I create an > index on my temp table? if not, how can I handle some specific queries on a > very large table? it would iterate all the table even though all I want is > just a small piece of that ta

Re: Does Spark SQL support indexes?

2016-08-14 Thread Michael Armbrust
wrote: > Use a format that has built-in indexes, such as Parquet or Orc. Do not > forget to sort the data on the columns that your filter on. > > On 14 Aug 2016, at 05:03, Taotao.Li <charles.up...@gmail.com> wrote: > > > hi, guys, does Spark SQL support indexes? if so, how c

Re: Does Spark SQL support indexes?

2016-08-14 Thread Jörn Franke
Use a format that has built-in indexes, such as Parquet or Orc. Do not forget to sort the data on the columns that your filter on. > On 14 Aug 2016, at 05:03, Taotao.Li <charles.up...@gmail.com> wrote: > > > hi, guys, does Spark SQL support indexes? if so, how can I create

Re: Does Spark SQL support indexes?

2016-08-13 Thread Chanh Le
Hi Taotao, Spark SQL doesn’t support index :). > On Aug 14, 2016, at 10:03 AM, Taotao.Li <charles.up...@gmail.com> wrote: > > > hi, guys, does Spark SQL support indexes? if so, how can I create an index > on my temp table? if not, how can I handle some specific queri

Does Spark SQL support indexes?

2016-08-13 Thread Taotao.Li
hi, guys, does Spark SQL support indexes? if so, how can I create an index on my temp table? if not, how can I handle some specific queries on a very large table? it would iterate all the table even though all I want is just a small piece of that table. great thanks