Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/10413 )
Change subject: IMPALA-7019: Schedule EC as remote & disable failed tests ...................................................................... Patch Set 1: (5 comments) http://gerrit.cloudera.org:8080/#/c/10413/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10413/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@439 PS1, Line 439: if (synthesizeFileMd || fileStatus.isErasureCoded()) { Shouldn't this check be in L424 so that synthesizeFileMd is true for EC? Are they EC blocks really completely useless to us and we're better of synthesizing them? I think it's worth adding a comment explaining why we chose this path for EC. http://gerrit.cloudera.org:8080/#/c/10413/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@510 PS1, Line 510: if (synthesizeFileMd || fileStatus.isErasureCoded()) { move check to L488? http://gerrit.cloudera.org:8080/#/c/10413/1/tests/common/skip.py File tests/common/skip.py: http://gerrit.cloudera.org:8080/#/c/10413/1/tests/common/skip.py@149 PS1, Line 149: ec = pytest.mark.skipif(IS_EC, reason="It shouldn't work with erasure coding.") Any more concrete reason, or are there too many to list? http://gerrit.cloudera.org:8080/#/c/10413/1/tests/custom_cluster/test_hdfs_fd_caching.py File tests/custom_cluster/test_hdfs_fd_caching.py: http://gerrit.cloudera.org:8080/#/c/10413/1/tests/custom_cluster/test_hdfs_fd_caching.py@29 PS1, Line 29: @SkipIfEC.fix_later Is this really expected to work? FD caching is only applied to local sc reads, and not used for remote reads. http://gerrit.cloudera.org:8080/#/c/10413/1/tests/query_test/test_mt_dop.py File tests/query_test/test_mt_dop.py: http://gerrit.cloudera.org:8080/#/c/10413/1/tests/query_test/test_mt_dop.py@101 PS1, Line 101: # Impala scans less row group than it should with erasure coding. fewer row groups -- To view, visit http://gerrit.cloudera.org:8080/10413 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I138738d3e28e5daa1718c05c04cd9dd146c4ff84 Gerrit-Change-Number: 10413 Gerrit-PatchSet: 1 Gerrit-Owner: Tianyi Wang <tw...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Comment-Date: Tue, 15 May 2018 20:43:35 +0000 Gerrit-HasComments: Yes
