[ https://issues.apache.org/jira/browse/HIVE-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675625#comment-16675625 ]
Hive QA commented on HIVE-20079: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12931668/HIVE-20079.2.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14750/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14750/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14750/ Messages: {noformat} **** This message was trimmed, see log for full details **** Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_limit.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_0.q.out:34 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_0.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_1.q.out:60 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_1.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_10.q.out:64 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_10.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_11.q.out:46 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_11.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_12.q.out:83 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_12.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_13.q.out:85 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_13.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_14.q.out:85 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_14.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_15.q.out:81 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_15.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_16.q.out:58 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_16.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_17.q.out:66 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_17.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_2.q.out:64 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_2.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_3.q.out:69 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_3.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_4.q.out:64 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_4.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_5.q.out:58 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_5.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_6.q.out:58 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_6.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_7.q.out:72 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_7.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_8.q.out:68 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_8.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_9.q.out:58 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_9.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_limit.q.out:98 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_limit.q.out' with conflicts. Going to apply patch with: git apply -p1 /data/hiveptest/working/scratch/build.patch:203: trailing whitespace. /data/hiveptest/working/scratch/build.patch:1355: trailing whitespace. rawDataSize 5936 /data/hiveptest/working/scratch/build.patch:2049: trailing whitespace. # col_name data_type comment /data/hiveptest/working/scratch/build.patch:2050: trailing whitespace. id int /data/hiveptest/working/scratch/build.patch:2051: trailing whitespace. str string error: patch failed: ql/src/test/results/clientpositive/parquet_analyze.q.out:93 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_analyze.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/parquet_map_type_vectorization.q.out:125 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_map_type_vectorization.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_13.q.out:80 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_13.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_14.q.out:80 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_14.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_15.q.out:76 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_15.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_16.q.out:53 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_16.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_17.q.out:61 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_17.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_2.q.out:59 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_2.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_3.q.out:64 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_3.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_4.q.out:59 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_4.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_5.q.out:53 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_5.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_6.q.out:55 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_6.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_7.q.out:67 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_7.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_8.q.out:63 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_8.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_9.q.out:53 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_9.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/parquet_vectorization_limit.q.out:90 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/parquet_vectorization_limit.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_0.q.out:34 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_0.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_1.q.out:60 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_1.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_10.q.out:64 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_10.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_11.q.out:46 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_11.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_12.q.out:83 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_12.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_13.q.out:85 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_13.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_14.q.out:85 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_14.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_15.q.out:81 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_15.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_16.q.out:58 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_16.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_17.q.out:66 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_17.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_2.q.out:64 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_2.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_3.q.out:69 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_3.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_4.q.out:64 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_4.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_5.q.out:58 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_5.q.out' cleanly. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_6.q.out:58 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_6.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_7.q.out:72 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_7.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_8.q.out:68 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_8.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_9.q.out:58 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_9.q.out' with conflicts. error: patch failed: ql/src/test/results/clientpositive/spark/parquet_vectorization_limit.q.out:98 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/spark/parquet_vectorization_limit.q.out' with conflicts. U ql/src/test/results/clientpositive/parquet_analyze.q.out U ql/src/test/results/clientpositive/parquet_map_type_vectorization.q.out U ql/src/test/results/clientpositive/parquet_vectorization_13.q.out U ql/src/test/results/clientpositive/parquet_vectorization_6.q.out U ql/src/test/results/clientpositive/parquet_vectorization_7.q.out U ql/src/test/results/clientpositive/parquet_vectorization_8.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_0.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_10.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_12.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_13.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_14.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_15.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_16.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_17.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_6.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_7.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_8.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_9.q.out U ql/src/test/results/clientpositive/spark/parquet_vectorization_limit.q.out warning: squelched 23 whitespace errors warning: 28 lines add whitespace errors. + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-14750 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12931668 - PreCommit-HIVE-Build > Populate more accurate rawDataSize for parquet format > ----------------------------------------------------- > > Key: HIVE-20079 > URL: https://issues.apache.org/jira/browse/HIVE-20079 > Project: Hive > Issue Type: Improvement > Components: File Formats > Affects Versions: 2.0.0 > Reporter: Aihua Xu > Priority: Major > Attachments: HIVE-20079.1.patch, HIVE-20079.2.patch > > > Run the following queries and you will see the raw data for the table is 4 > (that is the number of fields) incorrectly. We need to populate correct data > size so data can be split properly. > {noformat} > SET hive.stats.autogather=true; > CREATE TABLE parquet_stats (id int,str string) STORED AS PARQUET; > INSERT INTO parquet_stats values(0, 'this is string 0'), (1, 'string 1'); > DESC FORMATTED parquet_stats; > {noformat} > {noformat} > Table Parameters: > COLUMN_STATS_ACCURATE true > numFiles 1 > numRows 2 > rawDataSize 4 > totalSize 373 > transient_lastDdlTime 1530660523 > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)