[
https://issues.apache.org/jira/browse/HIVE-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916329#action_12916329
]
He Yongqiang commented on HIVE-1665:
------------------------------------
If "2 failed and rolling back 1) also failed", then the data is in trash
scratch dir and the table's metadata is there.
But "2 failed and rolling back 1) also failed" will rarely happen. Most concern
here is to deal with hdfs down and housekeeping operations.
For 'mark-then-delete', I think the main problem is there is no administration
daemon process or helper script for it.
> drop operations may cause file leak
> -----------------------------------
>
> Key: HIVE-1665
> URL: https://issues.apache.org/jira/browse/HIVE-1665
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Attachments: hive-1665.1.patch
>
>
> Right now when doing a drop, Hive first drops metadata and then drops the
> actual files. If file system is down at that time, the files will keep not
> deleted.
> Had an offline discussion about this:
> to fix this, add a new conf "scratch dir" into hive conf.
> when doing a drop operation:
> 1) move data to scratch directory
> 2) drop metadata
> 3) if 2) failed, roll back 1) and report error 3.1
> if 2) succeeded, drop data from scratch directory 3.2
> 4) if 3.2 fails, we are ok because we assume the scratch dir will be emptied
> manually.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
