[
https://issues.apache.org/jira/browse/KYLIN-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14899951#comment-14899951
]
Yerui Sun commented on KYLIN-978:
---------------------------------
The cleanup of hbase related files, including rowkey_stats and hfiles, is truly
not the key point of this issue. But with these cleanup, the hdfs dir keeps
very clean after building or merging jobs, only includes cuboid and coprocessor
files.
If you think the simple of code is important than the clean of storage, I would
accept that, to not cleanup hbase related files, until merging job completed.
> GarbageCollectionStep dropped Hive Intermediate Table but didn't drop
> external hdfs path
> ----------------------------------------------------------------------------------------
>
> Key: KYLIN-978
> URL: https://issues.apache.org/jira/browse/KYLIN-978
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v1.0, v0.7.2
> Reporter: Yerui Sun
> Assignee: Shaofeng SHI
> Fix For: v1.1
>
> Attachments: KYLIN-978-1.x-staging-v2.patch,
> KYLIN-978-1.x-staging-v3.patch, KYLIN-978-2.x-staging.patch
>
>
> In GarbageCollectionStep, the hive intermediate table created in step 1 was
> dropped.
> As the table is external table, data was stored in a external hdfs path, like
> '.../kylin-$\{jobId\}/kylin_intermediate_...', which didn't deleted when drop
> hive table.
> Considering the purpose of GarbageCollectionStep, the external data path
> should also be deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
