Ok, I think we're on the same page as to what is needed. The tool needs to be in place before any version is EOL.
I do think we should retain some BC tests with the older versions, of the form: - init cluster with 4.1.0 - run 4.7.0 upgrade tool - start latest (4.8.0-SNAPSHOT) bookie and client Another thing to consider is that ledger metadata upgrade will involve downtime for the client. This is fine, they can have a maintenance window, but we should be clear in the documentation that this is needed. -Ivan On Thu, Feb 1, 2018 at 5:58 PM, Sijie Guo <guosi...@gmail.com> wrote: > I think you misunderstood what I was saying. > > > On Thu, Feb 1, 2018 at 4:53 AM, Ivan Kelly <iv...@apache.org> wrote: > >> >> What do we mean by stopping support for BC? >> > >> > It means we will NOT consider BC problems with those old releases. >> >> If EOL means ignoring all BC considerations, then I'm -1 on these >> proposals. But I don't think we need to go to those extremes. >> I think we need to specify what BC guarantees we give on any version, >> and then EOL policies will follow from that. The 3 big BC >> considerations, as I see it, are: >> 1) Data on the bookie >> 2) Client wire protocol >> 3) Data shared between bookies and clients (i.e. zookeeper metadata) >> >> For 1 we can have quite a short timeframe for EOL old software. >> Bookies can be upgraded individually, and this is controlled by the >> administrator. >> For 2 we need a bit longer, because within an organisation there may >> be many different versions of the client knocking around and it's hard >> to get everyone to upgrade. >> For 3 we can pretty much never have a hard break. If a cluster was >> initialized with an old version, they must be able to able to upgrade >> that cluster to the latest version of the software. Otherwise they >> will stick with the old version forever. >> > > when I am saying stop all BC consideration, I mean: > > 1) all the data and metadata will be upgraded to the minimal version we > claim to support. > 2) all the client would have to upgrade to the minimal version we claim to > support for wire protocol. > > Let's take 4.7.0 as the example, when I am saying enforce EOL at 4.7.0. It > means 4.7.0 will still compatible with 4.1.0. but at 4.7.0 we are thinking > about dropping 4.1.x, 4.2.x and 4.3.x. > > so at 4.7.0: > > - we will have to develop a tool to upgrade metadata and data to the > version that will be recognized by 4.4.x and onwards releases. > > so at 4.8.0: > > - we will not consider clients older than 4.4.x. all the legal code can be > potentially removed. > > That means if a cluster (either metadata/data produced by clients older > than 4.4.x or still have old clients) want to upgrade to 4.8.0, it has to > first upgrade the cluster (both metadata and data) to 4.7.0 following the > instructions provided at 4.7.0 > and upgrade all the clients to minimal version that 4.8.0 will support. > After all is done first, the cluster can be upgraded to 4.8.0. > > > Hope this example make things clear. > > Enforcing EOL means dropping the BC considerations between the oldest > version and newest version and limit the BC considerations within a limited > number of versions which is more controllable, maintenanable. Using the > example above, > it means if you want to upgrade from 4.7.x to 4.8.x, please use the tools > provided in 4.7.x to ensure metadata/data are upgraded to latest version > and no more clients older than 4.4.0 talk to the cluster. > > >> >> > for example, the password and digest type was introduced at a very old >> > release. >> > There is code assuming ledgers might not have password or digest. >> However, >> > the releases onwards would already have those fields. >> > If we stop supporting those old version, we can remove that special BC >> > code, which will make code clearer to maintain. >> >> This code is in the client. I'm fine with stopping support for all >> clients older than a certain era. However, yahoo is still on 4.3, so >> we should at least support back that far until they've been migrated >> forward. >> > > the yahoo branch is between 4.3 and 4.4. it is a special case that we need > to take care of. we are not dropping it out of consideration radar. > > >> >> >> The BC >> >> tests should help in this regard, but we need keep testing scenarios >> >> where the original cluster was brought up in the oldest version >> >> (4.1.0). >> > >> > If we stop supporting 4.1.0, we don't need to test those scenarios. We >> > started with from the lowest version we are claiming to support. >> >> Are we supporting the software or data that was created with that >> software? That's two different things. Sure, we're not going to fix >> bugs on pre-4.4.0 releases, but there are pre-4.4.0 clusters running >> in the wild (yahoo being an example), and they have to be usable with >> the latest software, otherwise they will never upgrade. I had this in >> my previous job where they made a big breaking BC change in the >> metadata between two versions. While it did allow us to delete a lot >> of code, we ended up supporting two codebases forever, as customers >> were unable to upgrade to the latest version. >> > > Please check my example above to see if that makes sense to you. > > >> >> > That is the whole point of having EOL. >> It's not clear what the point of having EOL is, since we don't have a >> clearly defined EOL policy nor BC policy. >> > > Please check my example above to see if that makes sense to you. > > >> >> -Ivan >>
