[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381653#comment-14381653 ] Li Bo commented on HDFS-7729: - hi, Zhe I have just uploaded a patch to HDFS-7889 and add a unit

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347819#comment-14347819 ] Zhe Zhang commented on HDFS-7729: - In my tests I set {{cellSize}} to a smaller value (1024)

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348066#comment-14348066 ] Li Bo commented on HDFS-7729: - hi, Zhe, I will check this problem soon. This JIRA has been

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348077#comment-14348077 ] Kai Zheng commented on HDFS-7729: - bq.how about creating a new JIRA for subclassing

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14338169#comment-14338169 ] Li Bo commented on HDFS-7729: - hi, [~jingzhao] and [~zhz] , I come back today and a new

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336932#comment-14336932 ] Jing Zhao commented on HDFS-7729: - I think we'd better finish HDFS-7793 first. I will also

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336962#comment-14336962 ] Zhe Zhang commented on HDFS-7729: - OK, adding more synthetic tests sounds a good idea to me

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318660#comment-14318660 ] Zhe Zhang commented on HDFS-7729: - Thanks Bo for the quick rev! The overall structure looks

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319136#comment-14319136 ] Zhe Zhang commented on HDFS-7729: - Thanks [~jingzhao] and [~szetszwo] for the in-depth

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319034#comment-14319034 ] Jing Zhao commented on HDFS-7729: - My main concern is that whether we can/should now define

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319517#comment-14319517 ] Uma Maheswara Rao G commented on HDFS-7729: --- Great thoughts! I think here my

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319523#comment-14319523 ] Kai Zheng commented on HDFS-7729: - Hi [~umamaheswararao], Good idea. Let me proceed to

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319537#comment-14319537 ] Li Bo commented on HDFS-7729: - At first my idea is using a block writer to receive bytes and

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319468#comment-14319468 ] Kai Zheng commented on HDFS-7729: - Thanks [~jingzhao] and [~szetszwo] for the great

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319530#comment-14319530 ] Kai Zheng commented on HDFS-7729: - I opened HDFS-7793 for the refactoring. Thanks all !

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319546#comment-14319546 ] Li Bo commented on HDFS-7729: - Thanks [~zhz] , [~drankye], [~jingzhao], [~umamaheswararao]

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319692#comment-14319692 ] Zhe Zhang commented on HDFS-7729: - [~libo-intel] Thank you for taking on the refactor task!

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318157#comment-14318157 ] Li Bo commented on HDFS-7729: - patch 008: use JRSRawEncoder to encode buffers because

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318144#comment-14318144 ] Li Bo commented on HDFS-7729: - Sorry for replying late. 1. Give simple handling of slow

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318149#comment-14318149 ] Li Bo commented on HDFS-7729: - Thanks for your careful review. 1.the variable

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318358#comment-14318358 ] Kai Zheng commented on HDFS-7729: - I just took a quick look at the new patch checking how

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318534#comment-14318534 ] Zhe Zhang commented on HDFS-7729: - Hmm, somehow it passes with 008 patch. Add logic to

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14318519#comment-14318519 ] Zhe Zhang commented on HDFS-7729: - On my local machine {{testCopyOnWrite}} passes on

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317494#comment-14317494 ] Li Bo commented on HDFS-7729: - Thanks [~zhz] for the more review. Thanks [~jingzhao] in advance

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317449#comment-14317449 ] Jing Zhao commented on HDFS-7729: - Thanks for working on this, Bo and Zhe! I will also take

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317651#comment-14317651 ] Zhe Zhang commented on HDFS-7729: - Thanks Bo. If possible please also check the code

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315028#comment-14315028 ] Zhe Zhang commented on HDFS-7729: - The existing read/write unit test, {{TestWriteRead}},

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315231#comment-14315231 ] Kai Zheng commented on HDFS-7729: - bq.this patch looks much better now ! Client striping

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14313318#comment-14313318 ] Zhe Zhang commented on HDFS-7729: - [~libo-intel] {{BlockManager}} has changed quite a bit.

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312114#comment-14312114 ] Li Bo commented on HDFS-7729: - Just upload 003 patch. In order to make the unit tests pass, you

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303827#comment-14303827 ] Zhe Zhang commented on HDFS-7729: - bq. There may exist some problems about client and

[jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303066#comment-14303066 ] Li Bo commented on HDFS-7729: - Because HDFS-7417 and HDFS-7418 are still in progress, so I