[ https://issues.apache.org/jira/browse/HIVE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880236#comment-13880236 ]
Shuaishuai Nie commented on HIVE-5795: -------------------------------------- Hi [~thejas]. I checked the latest trunk of Hive and the file /data/files/header_footer_table_3 is missing. The reason is there are only two empty file in this folder. The test file_with_header_footer.q is failing because of this. I checked the result of HiveQA here: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/ and lots of new tests in TestMinimrCliDriver are missing. That is also why we didn't notice the missing file before. Thanks, Shuaishuai > Hive should be able to skip header and footer rows when reading data file for > a table > ------------------------------------------------------------------------------------- > > Key: HIVE-5795 > URL: https://issues.apache.org/jira/browse/HIVE-5795 > Project: Hive > Issue Type: New Feature > Reporter: Shuaishuai Nie > Assignee: Shuaishuai Nie > Fix For: 0.13.0 > > Attachments: HIVE-5795.1.patch, HIVE-5795.2.patch, HIVE-5795.3.patch, > HIVE-5795.4.patch, HIVE-5795.5.patch > > > Hive should be able to skip header and footer lines when reading data file > from table. In this way, user don't need to processing data which generated > by other application with a header or footer and directly use the file for > table operations. > To implement this, the idea is adding new properties in table descriptions to > define the number of lines in header and footer and skip them when reading > the record from record reader. An DDL example for creating a table with > header and footer should be like this: > {code} > Create external table testtable (name string, message string) row format > delimited fields terminated by '\t' lines terminated by '\n' location > '/testtable' tblproperties ("skip.header.line.count"="1", > "skip.footer.line.count"="2"); > {code} -- This message was sent by Atlassian JIRA (v6.1.5#6160)