Re: SPARK SQL READING FROM HIVE

[Mich Talebzadeh]

Unfortunately it is the case for now

spark-sql> show create table payees;
Error in query: Failed to execute SHOW CREATE TABLE against table
`accounts`.`payees`, which is created by Hive and uses the following
unsupported feature(s)
 - bucketing;

HTH

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.



On 8 August 2016 at 20:12, manish jaiswal  wrote:

> correct its creating delta file in hdfs.but after compaction it merge all
> data and create extra directory where all bucketed data present.( i am able
> to read data from hive but not from sparksql).
>

Re: SPARK SQL READING FROM HIVE

[manish jaiswal]

correct its creating delta file in hdfs.but after compaction it merge all
data and create extra directory where all bucketed data present.( i am able
to read data from hive but not from sparksql).

Re: SPARK SQL READING FROM HIVE

[manish jaiswal]

i am using spark 1.6.0 and hive 1.2.1.

reading from hive transactional table is not supported yet by sparl sql?

On Tue, Aug 9, 2016 at 12:18 AM, manish jaiswal 
wrote:

> Hi,
>
> I am not able to read data from hive transactional table using sparksql.
> (i don't want read via hive jdbc)
>
>
>
> Please help.
>

Re: SPARK SQL READING FROM HIVE

[Mich Talebzadeh]

I suspect this is happening because the underlying table has got delta
files in it due to updates etc and spark cannot read it and requires
compaction

Can you do

hdfs dfs -ls  


Also can you query a normal table in hive (meaning non transactional)

HTH

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.



On 8 August 2016 at 19:51, Deepak Sharma  wrote:

> Can you please post the code snippet and the error you are getting ?
>
> -Deepak
>
> On 9 Aug 2016 12:18 am, "manish jaiswal"  wrote:
>
>> Hi,
>>
>> I am not able to read data from hive transactional table using sparksql.
>> (i don't want read via hive jdbc)
>>
>>
>>
>> Please help.
>>
>

Re: SPARK SQL READING FROM HIVE

[Deepak Sharma]

Can you please post the code snippet and the error you are getting ?

-Deepak

On 9 Aug 2016 12:18 am, "manish jaiswal"  wrote:

> Hi,
>
> I am not able to read data from hive transactional table using sparksql.
> (i don't want read via hive jdbc)
>
>
>
> Please help.
>

Re: SPARK SQL READING FROM HIVE

[Mich Talebzadeh]

Which version of Spark and Hive are you using?

Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.



On 8 August 2016 at 19:48, manish jaiswal  wrote:

> Hi,
>
> I am not able to read data from hive transactional table using sparksql.
> (i don't want read via hive jdbc)
>
>
>
> Please help.
>