Re: From Hive to Spark, what is the default database/table

[Mich Talebzadeh]

flight201601 is the name of database or schema it is not a TABLE!

In Hive you can do

show databases

to see the list of databases. By Default Hive has a default database called
"default" out of box

For example to see list of tables in database flight201601 do the following:

use flight201601;
show tables;

HTH




Dr Mich Talebzadeh



LinkedIn * 
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.



On 31 December 2016 at 20:55, Raymond Xie  wrote:

> Hello,
>
>
> It is indicated in https://spark.apache.org/docs/1.6.1/sql-programming-
> guide.html#dataframes when Running SQL Queries Programmatically you can
> do:
>
> from pyspark.sql import SQLContextsqlContext = SQLContext(sc)df = 
> sqlContext.sql("SELECT * FROM table")
>
> However, it did not indicate what should be put there as "table", for
> example in my case I do have couple datawarehouses and tables and one of it
> is:
> hdfs dfs -ls hdfs://localhost:9000/user/hive/warehouse/flight201601
>
> if I use:
> >>> df = sqlContext.sql("SELECT * FROM flight201601")
>
> it will prompt:
> pyspark.sql.utils.AnalysisException: u'Table not found: flight201601;'
>
>
>
> How do I write the sql query if I want to select from flight201601?
>
> Thank you.
>
>
>
> **
> *Sincerely yours,*
>
>
> *Raymond*
>

From Hive to Spark, what is the default database/table

[Raymond Xie]

Hello,


It is indicated in
https://spark.apache.org/docs/1.6.1/sql-programming-guide.html#dataframes
 when Running SQL Queries Programmatically you can do:

from pyspark.sql import SQLContextsqlContext = SQLContext(sc)df =
sqlContext.sql("SELECT * FROM table")

However, it did not indicate what should be put there as "table", for
example in my case I do have couple datawarehouses and tables and one of it
is:
hdfs dfs -ls hdfs://localhost:9000/user/hive/warehouse/flight201601

if I use:
>>> df = sqlContext.sql("SELECT * FROM flight201601")

it will prompt:
pyspark.sql.utils.AnalysisException: u'Table not found: flight201601;'



How do I write the sql query if I want to select from flight201601?

Thank you.



**
*Sincerely yours,*


*Raymond*