I think we all agree on that. :) The sticky point is: When can we remove old code that is *only* needed to migrate even older installations? And should we spent a lot of our resources on supporting upgrades that skip versions?
I think that ties in with time-based released and the point that Ted made earlier that time-based released are hard to swallow until we have cross-version wire-compatibility. If we had that, one could upgrade a cluster from (say) 0.89fb to 0.96, but deploying 0.92, then 0.94, then 0.96 all with controlled rolling restarts. Sure, tedious, but with wire-compatibility it would be transparent to the application. We need to weigh that inconvenience against the resources needed of supporting and testing multi version jumps. -- Lars ________________________________ From: Nicolas Spiegelberg <[email protected]> To: "[email protected]" <[email protected]> Sent: Thursday, February 2, 2012 5:24 PM Subject: Re: hbase 0.94.0 >>I think HFile upgrade in particular is more complicated than you think. >>We currently have production traffic running with HFileV1. It has a >>5-min >>SLA. We can't afford to take the entire downtime to rewrite 100GB (or >>whatever) worth of data. We need to do this while the cluster is live. > >AFAIK that's how it's done, V1 files are being rewritten to V2 when a >compaction happens. You don't have to do some offline processing >before getting the cluster back online. Correct. Sorry for the confusion. I just meant that we shouldn't get rid of the HFileV1 Reader because it allows online upgrades to happen as compactions happen. Trying to remove this functionality or refactor this to some transformation utility would have negative consequences. LSMT is designed with immutability as a core principle, so I think we should incline towards creating online mutation of persistent data as opposed to migration utilities.
