Have you updated partition statistics by any chance?
I assume you can access the table and data though Hive itself?
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Do the permissions on the hive table files on HDFS correspond with what the
spark user is able to read? This might arise from spark being run as
different users.
On Wed, Aug 7, 2019 at 3:15 PM Rishikesh Gawade
wrote:
> Hi,
> I did not explicitly create a Hive Context. I have been using the
>
Hi,
I did not explicitly create a Hive Context. I have been using the
spark.sqlContext that gets created upon launching the spark-shell.
Isn't this sqlContext same as the hiveContext?
Thanks,
Rishikesh
On Wed, Aug 7, 2019 at 12:43 PM Jörn Franke wrote:
> Do you use the HiveContext in Spark? Do
Do you use the HiveContext in Spark? Do you configure the same options there?
Can you share some code?
> Am 07.08.2019 um 08:50 schrieb Rishikesh Gawade :
>
> Hi.
> I am using Spark 2.3.2 and Hive 3.1.0.
> Even if i use parquet files the result would be same, because after all
> sparkSQL
Hi.
I am using Spark 2.3.2 and Hive 3.1.0.
Even if i use parquet files the result would be same, because after all
sparkSQL isn't able to descend into the subdirectories over which the table
is created. Could there be any other way?
Thanks,
Rishikesh
On Tue, Aug 6, 2019, 1:03 PM Mich Talebzadeh
which versions of Spark and Hive are you using.
what will happen if you use parquet tables instead?
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Hi.
I have built a Hive external table on top of a directory 'A' which has data
stored in ORC format. This directory has several subdirectories inside it,
each of which contains the actual ORC files.
These subdirectories are actually created by spark jobs which ingest data
from other sources and
