[
https://issues.apache.org/jira/browse/SPARK-32358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
philipse updated SPARK-32358:
-----------------------------
Description:
After upgrading from 2.3.3 to spark 2.4.5. the temp view seems not working .
Please correct me if i miss sth. Thanks!
Steps to reproduce:
```
from pyspark.sql import SparkSession
from pyspark.sql import Row
spark=SparkSession\
.builder \
.appName('scenary_address_1') \
.enableHiveSupport() \
.getOrCreate()
address_tok_result_df=spark.createDataFrame([Row(a=1,b='难',c=80),Row(a=2,b='v',c=81)])
print("create dataframe finished")
address_tok_result_df.createOrReplaceTempView("scenery_address_test1")
print(spark.read.table('scenery_address_test1').dtypes)
spark.sql("select * from scenery_address_test1").show()
```
In spark2.3.3 i can easily gey the following result:
```
create dataframe finished
[('a', 'bigint'), ('b', 'string'), ('c', 'bigint')]
+----+--++---
|a|b|c|
+----+--++---
|1|难|80|
|2|v|81|
+----+--+—+
```
But in 2.4.5. i can only get:
create dataframe finished
[('a', 'bigint'), ('b', 'string'), ('c', 'bigint')]
was:
After upgrading from 2.3.3 to spark 2.4.5. the temp view seems not working . i
am not sure if if missing sth
Steps to reproduce:
```
from pyspark.sql import SparkSession
from pyspark.sql import Row
spark=SparkSession\
.builder \
.appName('scenary_address_1') \
.enableHiveSupport() \
.getOrCreate()
address_tok_result_df=spark.createDataFrame([Row(a=1,b='难',c=80),Row(a=2,b='v',c=81)])
print("create dataframe finished")
address_tok_result_df.createOrReplaceTempView("scenery_address_test1")
print(spark.read.table('scenery_address_test1').dtypes)
spark.sql("select * from scenery_address_test1").show()
```
In spark2.3.3 i can easily gey the following result:
```
create dataframe finished
[('a', 'bigint'), ('b', 'string'), ('c', 'bigint')]
+---+---+---+
| a| b| c|
+---+---+---+
| 1| 难| 80|
| 2| v| 81|
+---+---+—+
```
But in 2.4.5. i can only get:
create dataframe finished
[('a', 'bigint'), ('b', 'string'), ('c', 'bigint')]
> temp view not working after upgrading from 2.3.3 to 2.4.5
> ---------------------------------------------------------
>
> Key: SPARK-32358
> URL: https://issues.apache.org/jira/browse/SPARK-32358
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 2.4.5
> Reporter: philipse
> Priority: Major
>
> After upgrading from 2.3.3 to spark 2.4.5. the temp view seems not working .
> Please correct me if i miss sth. Thanks!
> Steps to reproduce:
> ```
> from pyspark.sql import SparkSession
> from pyspark.sql import Row
> spark=SparkSession\
> .builder \
> .appName('scenary_address_1') \
> .enableHiveSupport() \
> .getOrCreate()
>
> address_tok_result_df=spark.createDataFrame([Row(a=1,b='难',c=80),Row(a=2,b='v',c=81)])
> print("create dataframe finished")
> address_tok_result_df.createOrReplaceTempView("scenery_address_test1")
> print(spark.read.table('scenery_address_test1').dtypes)
> spark.sql("select * from scenery_address_test1").show()
> ```
>
> In spark2.3.3 i can easily gey the following result:
> ```
> create dataframe finished
> [('a', 'bigint'), ('b', 'string'), ('c', 'bigint')]
> +----+--++---
> |a|b|c|
> +----+--++---
> |1|难|80|
> |2|v|81|
> +----+--+—+
> ```
>
> But in 2.4.5. i can only get:
> create dataframe finished
> [('a', 'bigint'), ('b', 'string'), ('c', 'bigint')]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
