[jira] [Commented] (HDFS-7663) Erasure Coding: Append on striped file

[Vinayakumar B (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HDFS-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776655#comment-16776655
 ] 

Vinayakumar B commented on HDFS-7663:
-------------------------------------

Thanks [~ayushtkn] for taking this up.

Following are the comments from patch.
1.
{code}
   protected final AtomicReference<CachingStrategy> cachingStrategy;
-  private FileEncryptionInfo fileEncryptionInfo;
+  protected FileEncryptionInfo fileEncryptionInfo;
   private int writePacketSize;
{code}
{code}
-  private DFSOutputStream(DFSClient dfsClient, String src,
+  protected DFSOutputStream(DFSClient dfsClient, String src,
       EnumSet<CreateFlag> flag,
{code}
These changes are unnecessary

2.
{code}
+  /** Construct a new output stream for appending to a file. */
+  DFSStripedOutputStream(DFSClient dfsClient, String src,
+      EnumSet<CreateFlag> flags, Progressable progress, LocatedBlock lastBlock,
+      HdfsFileStatus stat, DataChecksum checksum, String[] favoredNodes)
+      throws IOException {
+    super(dfsClient, src, stat, flags, progress, checksum, favoredNodes, 
false);
+    if (LOG.isDebugEnabled()) {
+      LOG.debug("Creating DFSStripedOutputStream for " + src);
+    }
.
.
.
{code}
You can re-use the existing constructor itself. Just add remaining statements.
No need to validate the NEW_BLOCK flag here. It will be already validated 
during namenode call itself.

3.
{code}
       // not support appending file with striped blocks
-      if (file.isStriped()) {
+      if (file.isStriped() && file.isUnderConstruction()) {
         throw new UnsupportedOperationException(
             "Cannot append to files with striped block " + path);
       }
{code}
Remove the entire check itself. For Underconstruction, 
AlreadyBeingCreatedException will be thrown later.

4. Also, add Validation of NEW_BLOCK flag in the namenode-side, until support 
of append to existing blocks.



> Erasure Coding: Append on striped file
> --------------------------------------
>
>                 Key: HDFS-7663
>                 URL: https://issues.apache.org/jira/browse/HDFS-7663
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Jing Zhao
>            Assignee: Ayush Saxena
>            Priority: Major
>         Attachments: HDFS-7663-02.patch, HDFS-7663-03.patch, 
> HDFS-7663-04.patch, HDFS-7663.00.txt, HDFS-7663.01.patch
>
>
> Append should be easy if we have variable length block support from 
> HDFS-3689, i.e., the new data will be appended to a new block. We need to 
> revisit whether and how to support appending data to the original last block.
> 1. Append to a closed striped file, with NEW_BLOCK flag enabled (this)
> 2. Append to a under-construction striped file, with NEW_BLOCK flag enabled 
> (HDFS-9173)
> 3. Append to a striped file, by appending to last block group (follow-on)
> This jira attempts to implement the #1, and also track #2, #3.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org