Hi, I am worry about KMS and transparent encryption since there are quite many bugs discovered after it got merged to branch-2. It gives us an impression that the feature is not yet well tested. Indeed, transparent encryption is a complicated feature which changes the core part of HDFS. It is not easy to get everything right.
For HDFS-6584: Archival Storage, it is a relatively simple and low risk feature. It introduces a new storage type ARCHIVE and the concept of block storage policy to HDFS. When a cluster is configured with ARCHIVE storage, the blocks will be stored using the appropriate storage types specified by storage policies assigned to the files/directories. Cluster admin could disable the feature by simply not configuring any storage type and not setting any storage policy as before. As Suresh mentioned, HDFS-6584 is in the final stages to be merged to branch-2. Regards, Tsz-Wo On Wednesday, September 24, 2014 7:00 AM, Suresh Srinivas <sur...@hortonworks.com> wrote: > > >I actually would like to see both archival storage and single replica >memory writes to be in 2.6 release. Archival storage is in the final stages >of getting ready for branch-2 merge as Nicholas has already indicated on >the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of these >features are being in development for sometime. > >On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang <andrew.w...@cloudera.com> >wrote: > >> Hey Arun, >> >> Maybe we could do a quick run through of the Roadmap wiki and add/retarget >> things accordingly? >> >> I think the KMS and transparent encryption are ready to go. We've got a >> very few further bug fixes pending, but that's it. >> >> Two HDFS things that I think probably won't make the end of the week are >> archival storage (HDFS-6584) and single replica memory writes (HDFS-6581), >> which I believe are under the HSM banner. HDFS-6484 was just merged to >> trunk and I think needs a little more work before it goes into branch-2. >> HDFS-6581 hasn't even been merged to trunk yet, so seems a bit further off >> yet. >> >> Just my 2c as I did not work directly on these features. I just generally >> shy away from shipping bits quite this fresh. >> >> Thanks, >> Andrew >> >> On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy <a...@hortonworks.com> wrote: >> >> > Looks like most of the content is in and hadoop-2.6 is shaping up nicely. >> > >> > I'll create branch-2.6 by end of the week and we can go from there to >> > stabilize it - hopefully in the next few weeks. >> > >> > Thoughts? >> > >> > thanks, >> > Arun >> > >> > On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy <a...@hortonworks.com> >> > wrote: >> > >> > > Folks, >> > > >> > > With hadoop-2.5 nearly done, it's time to start thinking ahead to >> > > hadoop-2.6. >> > > >> > > Currently, here is the Roadmap per the wiki: >> > > >> > > • HADOOP >> > > • Credential provider HADOOP-10607 >> > > • HDFS >> > > • Heterogeneous storage (Phase 2) - Support APIs for >> > using >> > > storage tiers by the applications HDFS-5682 >> > > • Memory as storage tier HDFS-5851 >> > > • YARN >> > > • Dynamic Resource Configuration YARN-291 >> > > • NodeManager Restart YARN-1336 >> > > • ResourceManager HA Phase 2 YARN-556 >> > > • Support for admin-specified labels in YARN YARN-796 >> > > • Support for automatic, shared cache for YARN >> > application >> > > artifacts YARN-1492 >> > > • Support NodeGroup layer topology on YARN YARN-18 >> > > • Support for Docker containers in YARN YARN-1964 >> > > • YARN service registry YARN-913 >> > > >> > > My suspicion is, as is normal, some will make the cut and some won't. >> > > Please do add/subtract from the list as appropriate. Ideally, it would >> be >> > > good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a >> > cadence. >> > > >> > > More importantly, as we discussed previously, we'd like hadoop-2.6 to >> be >> > > the *last* Apache Hadoop 2.x release which support JDK6. I'll start a >> > > discussion with other communities (HBase, Pig, Hive, Oozie etc.) and >> see >> > > how they feel about this. >> > > >> > > thanks, >> > > Arun >> > > >> > > >> > >> > >> > -- >> > >> > -- >> > Arun C. Murthy >> > Hortonworks Inc. >> > http://hortonworks.com/ >> > >> > -- >> > CONFIDENTIALITY NOTICE >> > NOTICE: This message is intended for the use of the individual or entity >> to >> > which it is addressed and may contain information that is confidential, >> > privileged and exempt from disclosure under applicable law. If the reader >> > of this message is not the intended recipient, you are hereby notified >> that >> > any printing, copying, dissemination, distribution, disclosure or >> > forwarding of this communication is strictly prohibited. If you have >> > received this communication in error, please contact the sender >> immediately >> > and delete it from your system. Thank You. >> > >> > > > >-- >http://hortonworks.com/download/ > >-- >CONFIDENTIALITY NOTICE >NOTICE: This message is intended for the use of the individual or entity to >which it is addressed and may contain information that is confidential, >privileged and exempt from disclosure under applicable law. If the reader >of this message is not the intended recipient, you are hereby notified that >any printing, copying, dissemination, distribution, disclosure or >forwarding of this communication is strictly prohibited. If you have >received this communication in error, please contact the sender immediately >and delete it from your system. Thank You. > >
