[
https://issues.apache.org/jira/browse/SPARK-23863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chencheng updated SPARK-23863:
--
Description:
We merged multiple files in the Text format. When using the select count(1) to
count the merged results, we found that the result of the merged wholetext mode
was inconsistent with that of the normal mode.
E.g:
Combine 10 text files with a total of 100 lines. When the wholeTextMode
parameter of the buildReader method is set to true, the result of select
count(1) will be 110.
was:
We merged multiple files in the Text format. When using the select count(*) to
count the merged results, we found that the result of the merged wholetext mode
was inconsistent with that of the normal mode.
E.g:
Combine 10 text files with a total of 100 lines. When the wholeTextMode
parameter of the buildReader method is set to true, the result of select
count(*) will be 110.
> Wholetext mode should not add line breaks
> -
>
> Key: SPARK-23863
> URL: https://issues.apache.org/jira/browse/SPARK-23863
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 2.3.0
>Reporter: chencheng
>Priority: Major
> Fix For: 2.3.1
>
>
> We merged multiple files in the Text format. When using the select count(1)
> to count the merged results, we found that the result of the merged wholetext
> mode was inconsistent with that of the normal mode.
> E.g:
> Combine 10 text files with a total of 100 lines. When the wholeTextMode
> parameter of the buildReader method is set to true, the result of select
> count(1) will be 110.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
