Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8056 )
Change subject: IMPALA-5525 Extend TestScannersFuzzing to test uncompressed parquet ...................................................................... Patch Set 6: (4 comments) The test doesn't appear to be creating uncompressed parquet files. Looking at the query profile scanning test_fuzz_uncompressed_parquet_fc4a3734.parquet_uncomp_dst_decimal_tbl I see: HDFS_SCAN_NODE (id=0):(Total: 5.970ms, non-child: 5.970ms, % non-child: 100.00%) Hdfs split stats (<volume id>:<# splits>/<split lengths>): -1:1/1.43 KB ExecOption: PARQUET Codegen Enabled, Codegen enabled: 1 out of 1 Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% File Formats: PARQUET/SNAPPY:6 It looks like compression_codec isn't modified so we're just getting the default behaviour of using snappy compression. http://gerrit.cloudera.org:8080/#/c/8056/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8056/6//COMMIT_MSG@12 PS6, Line 12: Can you mention what testing you did? The fuzz tests are randomised we should run them in a loop for a while to confirm that that they're stable. http://gerrit.cloudera.org:8080/#/c/8056/6/tests/query_test/test_scanners_fuzz.py File tests/query_test/test_scanners_fuzz.py: http://gerrit.cloudera.org:8080/#/c/8056/6/tests/query_test/test_scanners_fuzz.py@101 PS6, Line 101: """Parquet tables in default schema are compressed, so in order It's weird that parquet/none means parquet/snappy. Unsure what the history is here and we don't need to change it but it is confusing. http://gerrit.cloudera.org:8080/#/c/8056/6/tests/query_test/test_scanners_fuzz.py@116 PS6, Line 116: " select * from functional_parquet.{1}".format(fq_tbl_name, orig_tbl_name)) Long lines > 90 chars here and just below. http://gerrit.cloudera.org:8080/#/c/8056/6/tests/query_test/test_scanners_fuzz.py@140 PS6, Line 140: self.execute_query("create table %s.%s like %s.%s" % (fuzz_db, fuzz_table, src_db, src_table)) Long line -- To view, visit http://gerrit.cloudera.org:8080/8056 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I760de7203a51cf82b16016fa8043cadc7c8325bc Gerrit-Change-Number: 8056 Gerrit-PatchSet: 6 Gerrit-Owner: Pranay Singh Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Pranay Singh Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Tue, 03 Oct 2017 18:38:43 +0000 Gerrit-HasComments: Yes