[
https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136527#comment-17136527
]
Yuming Wang commented on SPARK-31955:
-------------------------------------
The issue fixed by upgrading the beeline to 2.3.7. How to reproduce this issue:
*Prepare table*
{code:sql}
create table test_beeline using parquet as select id from range(5);
{code}
*Prepare SQL*
{code:sql}
echo -en "select * from test_beeline\n where id=2;" >> test.sql
{code}
*Spark 2.4*:
{noformat}
[root@spark-3267648 spark-2.4.4-bin-hadoop2.7]# bin/beeline -u
"jdbc:hive2://localhost:10000" -f /root/spark-3.0.0-bin-hadoop3.2/test.sql
Connecting to jdbc:hive2://localhost:10000
log4j:WARN No appenders could be found for logger (org.apache.hive.jdbc.Utils).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Connected to: Spark SQL (version 3.0.0)
Driver: Hive JDBC (version 1.2.1.spark2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> select * from test_beeline
0: jdbc:hive2://localhost:10000> where id=2;+-----+--+
| id |
+-----+--+
| 0 |
| 2 |
| 1 |
| 3 |
| 4 |
+-----+--+
5 rows selected (5.622 seconds)
0: jdbc:hive2://localhost:10000> where id=2;
Closing: 0: jdbc:hive2://localhost:10000
{noformat}
*Spark 3.0*:
{noformat}
[root@spark-3267648 spark-3.0.0-bin-hadoop3.2]# bin/beeline -u
"jdbc:hive2://localhost:10000" -f test.sql
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Connecting to jdbc:hive2://localhost:10000
Connected to: Spark SQL (version 3.0.0)
Driver: Hive JDBC (version 2.3.7)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> select * from test_beeline
. . . . . . . . . . . . . . . .> where id=2;
+-----+
| id |
+-----+
| 2 |
+-----+
1 row selected (7.749 seconds)
0: jdbc:hive2://localhost:10000> Closing: 0: jdbc:hive2://localhost:10000
{noformat}
> Beeline discard the last line of the sql file when submited to thriftserver
> via beeline
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-31955
> URL: https://issues.apache.org/jira/browse/SPARK-31955
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.4
> Reporter: Lin Gang Deng
> Priority: Major
> Fix For: 3.0.0
>
>
> I submitted a sql file on beeline and the result returned is wrong. After
> many tests, it was found that the sql executed by Spark would discard the
> last line.This should be beeline's bug parsing sql file.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
