[
https://issues.apache.org/jira/browse/HDFS-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136093#comment-16136093
]
Wei-Chiu Chuang commented on HDFS-10899:
----------------------------------------
ReencryptionUpdater#throttle():
{code}
private void throttle() throws InterruptedException {
if (throttleLimitRatio
>= 1.0) {
return;
}
{code}
the default is 1.0, which means no throttle for updater, and updater would keep
contending for namenode lock even if it has nothing to do?
ReencryptionUpdater#processCheckpoints
{code}
LOG.warn("Failed to update re-encrypted edek to xattr for file {}",
zonePath);
{code}
zonePath is not a file name.
Please log the ioe as well.
ReencryptionUpdater#processCheckpoints()
{code}
LinkedList<Future> tasks = zst.getTasks();
{code}
ReencryptionHandler and ReencryptionUpdater shares ZoneSubmissionTracker#tasks.
Access to zst.getTasks() is protected in processCheckpoints() by a FSDirectory
write lock and a FSNamesystem write lock, whereas it is protected in
ReencryptionHandler#submitCurrentBatch by a FSDirectory read lock and a
FSNameSystem readlock. If there will always be just one ReencryptionHandler,
then this is okay.
{code}
List<XAttr> xAttrs = null;
…
while( … ) {
xAttrs = Lists.newArrayListWithCapacity(1);
final XAttr xattr = FSDirEncryptionZoneOp
.updateReencryptionProgress(dir, zoneNode, status, task.lastFile,
task.numFilesUpdated, task.numFailures);
xAttrs.add(xattr);
}
…
return xAttrs;
{code}
Looks like you allocate a new xAttrs array for each task, but then discard it,
only to return the the xAttrs array for the last task?
I guess you want to instantiate the xAttrs array outside of the while loop?
the edit log is written only when all tasks are successful. If some tasks fail,
the successful tasks would record xattrs in memory, but will not get written
into edit log.
> Add functionality to re-encrypt EDEKs
> -------------------------------------
>
> Key: HDFS-10899
> URL: https://issues.apache.org/jira/browse/HDFS-10899
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: encryption, kms
> Reporter: Xiao Chen
> Assignee: Xiao Chen
> Attachments: editsStored, HDFS-10899.01.patch, HDFS-10899.02.patch,
> HDFS-10899.03.patch, HDFS-10899.04.patch, HDFS-10899.05.patch,
> HDFS-10899.06.patch, HDFS-10899.07.patch, HDFS-10899.08.patch,
> HDFS-10899.09.patch, HDFS-10899.10.patch, HDFS-10899.10.wip.patch,
> HDFS-10899.11.patch, HDFS-10899.12.patch, HDFS-10899.13.patch,
> HDFS-10899.14.patch, HDFS-10899.wip.2.patch, HDFS-10899.wip.patch, Re-encrypt
> edek design doc.pdf, Re-encrypt edek design doc V2.pdf
>
>
> Currently when an encryption zone (EZ) key is rotated, it only takes effect
> on new EDEKs. We should provide a way to re-encrypt EDEKs after the EZ key
> rotation, for improved security.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
