Kim Jin Chul has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8747 )

Change subject: IMPALA-5993: Fix the file offset in value parsing error
......................................................................


Patch Set 1: Code-Review-1

(4 comments)

As I commented, this change does not include a test which is ongoing locally. 
By the way, I would like to get your early review for the kernel code change. 
Please check my comments. Thanks.

http://gerrit.cloudera.org:8080/#/c/8747/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/8747/1//COMMIT_MSG@14
PS1, Line 14: Testing:
A custom test will be introduced at tests/custom_cluster/test_impala_5993.py. 
The test contains some hdfs commands which come from tests/util/hdfs_util.py 
and table will be created by 'create table stored as textfile location' query.


http://gerrit.cloudera.org:8080/#/c/8747/1/be/src/exec/hdfs-scanner.cc
File be/src/exec/hdfs-scanner.cc:

http://gerrit.cloudera.org:8080/#/c/8747/1/be/src/exec/hdfs-scanner.cc@668
PS1, Line 668: stream_->file_offset()
The offset can point to EOF because the data is read ahead of time via 
ScannerContext :: Stream :: GetBuffer in 
impala::HdfsTextScanner::FillByteBuffer. Therefore, file_offset cannot be used 
to print the problematic offset.


http://gerrit.cloudera.org:8080/#/c/8747/1/be/src/exec/hdfs-scanner.cc@674
PS1, Line 674: Error parsing row: file: $0, (line: $1, pos: $2, offset: $3)
I think the refined message format is more user friendly. Let me provide 
another change if you think this improvement cannot be in a bug fix.


http://gerrit.cloudera.org:8080/#/c/8747/1/be/src/exec/hdfs-scanner.cc@683
PS1, Line 683: "Error converting column at the index: "
             :        << desc->col_pos() - scan_node_->num_partition_keys()
             :        << " (type: " << desc->type() << ")";
I think the refined message format is more user friendly. Let me provide 
another change if you think this improvement cannot be in a bug fix.



--
To view, visit http://gerrit.cloudera.org:8080/8747
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a4ac3199ffe12fcce0bf792b3e6ce529b9af61f
Gerrit-Change-Number: 8747
Gerrit-PatchSet: 1
Gerrit-Owner: Kim Jin Chul <jinc...@gmail.com>
Gerrit-Reviewer: Kim Jin Chul <jinc...@gmail.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Dec 2017 15:09:55 +0000
Gerrit-HasComments: Yes

Reply via email to