[
https://issues.apache.org/jira/browse/KYLIN-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Long resolved KYLIN-3626.
------------------------------
Resolution: Resolved
> Allow customization for Parquet path
> ------------------------------------
>
> Key: KYLIN-3626
> URL: https://issues.apache.org/jira/browse/KYLIN-3626
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine, Metadata, Storage - Parquet
> Reporter: Shaofeng SHI
> Assignee: Chao Long
> Priority: Major
>
> Due to the difference of HDFS and object store like S3/blob store, Kylin
> needs to allow different path patterns for the files when putting to the
> underlying storage.
>
> For example, to HDFS, this pattern is good:
> /kylin/metadata_name/cube_name/segment_name/file1.parquet
>
> But, the above pattern is not good for S3, since S3 uses the first couple
> bytes of the path to do the hashing, so we'd better add salt ahead of the
> original pathname:
> /salt/kylin/metadata_name/cube_name/segment_name/file1.parquet
> or do a reverse order:
> /segment_name/cube_name/metadata_name/kylin/file1.parquet
>
> To get the flexibility, we should make this extensible.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
