Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18329
Change subject: IMPALA-11192: Batch uploading files in test_scanner_fuzz.py ...................................................................... IMPALA-11192: Batch uploading files in test_scanner_fuzz.py test_scanner_fuzz.py runs much slower on ORC than other formats. The majority of the time is spent in uploading local files one by one to the hdfs table folder. The local files are copied from hdfs and randomly corrupted by the test. The directory layout remains the same as the table folder. There are no staging dirs that we should skip. So we can upload the whole local folder at once, which saves a lot of the test time. Tested locally and verified profiles of the succeeded queries. They all scan the expected number of rows. Change-Id: I504e160b84b3cc01d3be0b4e242d3c372692d181 --- M tests/query_test/test_scanners_fuzz.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/18329/1 -- To view, visit http://gerrit.cloudera.org:8080/18329 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I504e160b84b3cc01d3be0b4e242d3c372692d181 Gerrit-Change-Number: 18329 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
