Re: Accessing Hbase tables through Spark, this seems to work

Careful Hbase with Phoenix is only in certain scenarios faster. When it is about processing small amounts out of a bigger amount of data (depends on node memory, the operation etc). Hive+tez+orc can be rather competitive, llap makes sense for interactive ad-hoc queries that are rather similar.

Re: Accessing Hbase tables through Spark, this seems to work

The design really needs to look at other stack as well. If the visualisation layer is going to use Tableau then you cannot use Spark functional programming. Only Spark SQL or anything that works with SQL like Hive or Phoenix. Tableau is not a real time dashboard so for analytics it maps tables in

Re: Accessing Hbase tables through Spark, this seems to work

yes Hive external table is partitioned on a daily basis (datestamp below) CREATE EXTERNAL TABLE IF NOT EXISTS ${DATABASE}.externalMarketData ( KEY string , SECURITY string , TIMECREATED string , PRICE float ) COMMENT 'From prices Kakfa delivered by Flume location by day' ROW FORMAT s

Re: Accessing Hbase tables through Spark, this seems to work

I do not see a rationale to have hbase in this scheme of thingsmay be I am missing something? If data is delivered in HDFS, why not just add partition to an existing Hive table? On Tue, Oct 18, 2016 at 8:23 AM, Mich Talebzadeh wrote: > Thanks Mike, > > My test csv data comes as > > UUID,

Re: Accessing Hbase tables through Spark, this seems to work

Thanks Mike, My test csv data comes as UUID, ticker, timecreated, price a2c844ed-137f-4820-aa6e-c49739e46fa6, S01, 2016-10-17T22:02:09, 53.36665625650533484995 a912b65e-b6bc-41d4-9e10-d6a44ea1a2b0, S02, 2016-10-17T22:02:09, 86.31917515824627016510

Re: Accessing Hbase tables through Spark, this seems to work

Thanks Ted. I have seen that before, but sounds like breaking a nut with sledgehammer. It should be a simpler than that. Regards Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Accessing Hbase tables through Spark, this seems to work

Please take a look at http://hbase.apache.org/book.html#_language_integrated_query The above is based on hbase-spark module which is part of the upcoming hbase 2.0 release. Cheers On Sun, Oct 16, 2016 at 11:37 AM, Mich Talebzadeh wrote: > Hi, > > I have trade data stored in Hbase table. Data a

Accessing Hbase tables through Spark, this seems to work

Hi, I have trade data stored in Hbase table. Data arrives in csv format to HDFS and then loaded into Hbase via periodic load with org.apache.hadoop.hbase.mapreduce.ImportTsv. The Hbase table has one Column family "trade_info" and three columns: ticker, timecreated, price. The RowKey is UUID. So