I was just looking at that…
Out of curiosity… if you make it a Hive Temp Table… who has access to the data?
Just your app, or anyone with access to the same database? (Would you be able
to share data across different JVMs? )
(E.G - I have a reader who reads from source A that needs to publish the data
to a bunch of minions (B) )
Would this be an option?
Thx
-Mike
> On Feb 15, 2016, at 7:54 AM, Mich Talebzadeh
> <mich.talebza...@cloudtechnologypartners.co.uk> wrote:
>
>> Hi,
>>
>>
>> It is my understanding that the registered temporary tables created by
>> registerTempTable() used in Spark shell built on ORC files?
>>
>> For example the following Data Frame just creates a logical abstraction
>>
>> scala> var s = HiveContext.sql("SELECT AMOUNT_SOLD, TIME_ID, CHANNEL_ID FROM
>> oraclehadoop.sales")
>> s: org.apache.spark.sql.DataFrame = [AMOUNT_SOLD: decimal(10,0), TIME_ID:
>> timestamp, CHANNEL_ID: bigint]
>>
>> Then I registar this data frame as temporary table using registerTempTable()
>> call
>>
>> s.registerTempTable("t_s")
>>
>> Also I believe that s.registerTempTable("t_s") creates an in-memory table
>> that is scoped to the cluster in which it was created. The data is stored
>> using Hive's ORC format and this tempTable is stored in memory on all nodes
>> of the cluster? In other words every node in the cluster has a copy of
>> tempTable in its memory?
>>
>> Thanks,
>>
>>
>> --
>> Dr Mich Talebzadeh
>>
>> LinkedIn
>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>
>> http://talebzadehmich.wordpress.com
>>
>> NOTE: The information in this email is proprietary and confidential. This
>> message is for the designated recipient only, if you are not the intended
>> recipient, you should destroy it immediately. Any information in this
>> message shall not be understood as given or endorsed by Cloud Technology
>> Partners Ltd, its subsidiaries or their employees, unless expressly so
>> stated. It is the responsibility of the recipient to ensure that this email
>> is virus free, therefore neither Cloud Technology partners Ltd, its
>> subsidiaries nor their employees accept any responsibility.
>>
>
>
> --
> Dr Mich Talebzadeh
>
> LinkedIn
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>
> http://talebzadehmich.wordpress.com
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this message
> shall not be understood as given or endorsed by Cloud Technology Partners
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus free,
> therefore neither Cloud Technology partners Ltd, its subsidiaries nor their
> employees accept any responsibility.
>
