[jira] [Commented] (SPARK-21063) Spark return an empty result from remote hadoop cluster

[Peter Bykov (JIRA)]

    [ 
https://issues.apache.org/jira/browse/SPARK-21063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16046596#comment-16046596
 ] 

Peter Bykov commented on SPARK-21063:
-------------------------------------

[~sowen] data available in table, also i have empty result if i will execute:
{code:java}
val df = spark.read
  .option("url", "jdbc:hive2://remote.hive.local:10000/default")
  .option("user", "user")
  .option("password", "pass")
  .option("dbtable", "(SELECT 1 AS col) AS dft")
  .option("driver", "org.apache.hive.jdbc.HiveDriver")
  .format("jdbc")
  .load()
{code}
As you can see in my description, i receive table schema, but no data. This 
means that connection ok.
I'm pointing right cluster, i checked link thru simple JDBC connection.
Also, if i will execute query like: 
{code:java}
val spark = SparkSession.builder
      .appName("RemoteSparkTest")
      .master("local")
      .config("hive.metastore.uris", "thrift://remote.hive.local:9083/default")
      .enableHiveSupport()
      .getOrCreate()

    val df = spark.sql("SELECT * FROM test_table")
    df.show()
{code}

Result:
{noformat}
+--------+
|test_col|
+--------+
|test row|
+--------+
{noformat}

So, data available.
 

> Spark return an empty result from remote hadoop cluster
> -------------------------------------------------------
>
>                 Key: SPARK-21063
>                 URL: https://issues.apache.org/jira/browse/SPARK-21063
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core, SQL
>    Affects Versions: 2.1.1
>            Reporter: Peter Bykov
>
> Spark returning empty result from when querying remote hadoop cluster.
> All firewall settings removed.
> Querying using JDBC working properly using hive-jdbc driver from version 1.1.1
> Code snippet is:
> {code:java}
> val spark = SparkSession.builder
>     .appName("RemoteSparkTest")
>     .master("local")
>     .getOrCreate()
> val df = spark.read
>   .option("url", "jdbc:hive2://remote.hive.local:10000/default")
>   .option("user", "user")
>   .option("password", "pass")
>   .option("dbtable", "test_table")
>   .option("driver", "org.apache.hive.jdbc.HiveDriver")
>   .format("jdbc")
>   .load()
>  
> df.show()
> {code}
> Result:
> {noformat}
> +-------------------+
> |test_table.test_col|
> +-------------------+
> +-------------------+
> {noformat}
> All manipulations like: 
> {code:java}
> df.select(*).show()
> {code}
> returns empty result too.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org