[
https://issues.apache.org/jira/browse/DRILL-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Volodymyr Vysotskyi updated DRILL-6852:
---------------------------------------
Description:
According to the design document for DRILL-6552, existing metadata cache API
should be adapted to use generalized API for metastore and parquet metadata
cache will be presented as the implementation of metastore API.
The aim of this Jira is to refactor Parquet Metadata cache implementation and
adapt it to use Drill Metastore API.
Execution plan:
- Refactor AbstractParquetGroupScan and its implementations to use metastore
metadata classes.
- Rework logic in AbstractParquetGroupScan to allow filtering at different
metadata layers: partition, file, row group, etc. The same for pushing the
limit.
- Implement logic to populate metastore metadata classes from existing parquet
metadata
- Rework EasyGroupScan to use the same logic as for refactored
AbstractParquetGroupScan if metadata is available.
- TBA...
was:
According to the design document for DRILL-6552, existing metadata cache API
should be adapted to use generalized API for metastore and parquet metadata
cache will be presented as the implementation of metastore API.
The aim of this Jira is to refactor Parquet Metadata cache implementation and
adapt it to use Drill Metastore API.
Execution plan:
- Refactor AbstractParquetGroupScan and its implementations to use metastore
metadata classes.
- Rework logic in AbstractParquetGroupScan to allow filtering at different
metadata layers: partition, file, row group, etc. The same for pushing the
limit.
- Implement logic to populate metastore metadata classes from existing parquet
metadata.
- Rework EasyGroupScan to use the same logic as for refactored
AbstractParquetGroupScan if metadata is available.
- TBA...
> Adapt current Parquet Metadata cache implementation to use Drill Metastore API
> ------------------------------------------------------------------------------
>
> Key: DRILL-6852
> URL: https://issues.apache.org/jira/browse/DRILL-6852
> Project: Apache Drill
> Issue Type: Sub-task
> Reporter: Volodymyr Vysotskyi
> Assignee: Volodymyr Vysotskyi
> Priority: Major
> Fix For: 1.16.0
>
>
> According to the design document for DRILL-6552, existing metadata cache API
> should be adapted to use generalized API for metastore and parquet metadata
> cache will be presented as the implementation of metastore API.
> The aim of this Jira is to refactor Parquet Metadata cache implementation and
> adapt it to use Drill Metastore API.
> Execution plan:
> - Refactor AbstractParquetGroupScan and its implementations to use metastore
> metadata classes.
> - Rework logic in AbstractParquetGroupScan to allow filtering at different
> metadata layers: partition, file, row group, etc. The same for pushing the
> limit.
> - Implement logic to populate metastore metadata classes from existing
> parquet metadata
> - Rework EasyGroupScan to use the same logic as for refactored
> AbstractParquetGroupScan if metadata is available.
> - TBA...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
