[
https://issues.apache.org/jira/browse/IMPALA-7098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498560#comment-16498560
]
Dan Hecht edited comment on IMPALA-7098 at 6/1/18 8:59 PM:
-----------------------------------------------------------
What value does mBlockSize on the return of hdfsGetPathInfo() give? Does it
give the block group size?
(And for parquet, the parquet scanner will find the row-groups and assign them
to the scan range that contains the mid-point offset of the row-group.
Ideally, what we'd want is to align scan ranges with row-groups. It would be
worse to create scan ranges larger than a row-group. It doesn't really hurt to
have a scan range smaller than a row-group.)
was (Author: dhecht):
What value does mBlockSize on the return of hdfsGetPathInfo() give? Does it
give the block group size?
> Re-enable blocksize-related tests under EC
> ------------------------------------------
>
> Key: IMPALA-7098
> URL: https://issues.apache.org/jira/browse/IMPALA-7098
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend, Infrastructure
> Affects Versions: Impala 3.1.0
> Reporter: Tianyi Wang
> Assignee: Taras Bobrovytsky
> Priority: Major
>
> In EC the unit of a scan range is a block group. So in our mini cluster a
> scan range is of 3X the size of a regular block. This breaks some tests under
> EC. One way to address this problem is to shrink the blocks to 1/3 of the
> original size in those tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
