[
https://issues.apache.org/jira/browse/HDFS-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122378#comment-14122378
]
James Thomas commented on HDFS-6981:
------------------------------------
bq. Not an issue. If there was no layout version change then 'previous' will be
absent and doTransition will skip calling doRollback, instead restoring from
trash. An existing test case already verifies it by starting the DN with the
-rollback flag.
Right, I am talking about the case where we roll back to a version of the
software prior to HDFS-6800, in which case the following code will be executed
in {{doTransition}}:
{code}
if (startOpt == StartupOption.ROLLBACK) {
doRollback(sd, nsInfo); // rollback if applicable
} else {
// Restore all the files in the trash. The restored files are retained
// during rolling upgrade rollback. They are deleted during rolling
// upgrade downgrade.
int restored = restoreBlockFilesFromTrash(getTrashRootDir(sd));
LOG.info("Restored " + restored + " block files from trash.");
}
{code}
So the trash won't be restored until the first heartbeat response is received
from the NN, at which point {{signalRollingUpgrade}} will be called with
{{null}}, and we'll call {{restoreTrash}}. So the blocks in trash will
potentially be missing for a short period of time -- not sure if this is an
issue.
> DN upgrade with layout version change should not use trash
> ----------------------------------------------------------
>
> Key: HDFS-6981
> URL: https://issues.apache.org/jira/browse/HDFS-6981
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 3.0.0
> Reporter: James Thomas
> Assignee: Arpit Agarwal
> Attachments: HDFS-6981.01.patch, HDFS-6981.02.patch,
> HDFS-6981.03.patch, HDFS-6981.04.patch
>
>
> Post HDFS-6800, we can encounter the following scenario:
> # We start with DN software version -55 and initiate a rolling upgrade to
> version -56
> # We delete some blocks, and they are moved to trash
> # We roll back to DN software version -55 using the -rollback flag – since we
> are running the old code (prior to this patch), we will restore the previous
> directory but will not delete the trash
> # We append to some of the blocks that were deleted in step 2
> # We then restart a DN that contains blocks that were appended to – since the
> trash still exists, it will be restored at this point, the appended-to blocks
> will be overwritten, and we will lose the appended data
> So I think we need to avoid writing anything to the trash directory if we
> have a previous directory.
> Thanks to [~james.thomas] for reporting this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
