On Mon, Mar 5, 2018 at 11:22 AM, Joe Francis <j...@oath.com.invalid> wrote:
> taking BK dev off this.. > > Pulsar storage has to be rock stable at all times. I will make a few > suggestions to that end. > > Note that there is a direct conflict of interests here. For new > installations, they have no baggage and would like the latest/greatest > features. But for users who are already running it in production, stability > to existing applications is almost always not welcome. No bleeding edge > changes for the latter case. > > - Storage upgrades should be less frequent. They are not cheap > operationally. Rollback is a mitigation action only. Even rollback comes > with a cost, of degradation and disruption to service. And > loss/corruption of data is not an operationally recoverable event. For any > user that runs Pulsar in production, a BK upgrade (any storage upgrade fro > that matter) has to be planned and executed to cover business risks. > > - Decoupling BK and Pulsar. There was a hard-wried dependency in Pulsar to > a specific version of BK till now. A byproduct of that was a kind of > forced > storage stability across many Pulsar releases. Going forward, stability is > not automatic. My suggestion is that any release of Pulsar must be able to > run on a range (min version - max version) of compatible BK releases and > not just one version. Now that Pulsar can run on the Apache BK, maybe > relax the dependency to some extent. That would make storage upgrade > optional, till say some major release that mandated it. > > So while I can understand the need for a storage upgrade in this case now, > the Pulsar community has to make a release plan and set a pace for storage > upgrades. Storage system releases are quite different from stateless > serving software. So far, Pulsar upgrades have been easy, because more or > less the storage was the same and stable. I don't want to be in a place > where I cannot deploy a Pulsar release because of the risks in a storage > upgrade. --- Is every release of Pulsar going to have a BK upgrade? -- > How many times will storage upgrade occur in a year? --- Is the storage > upgrade optional? These are important questions that the community > should discuss. > My take on this: I think bookkeeper release should be coupled from pulsar release. That said not every release of Pulsar should upgrade BK, and should probably make BK upgrade optional for Pulsar users. However that means we might need to be looking into testing mechanisms to ensure Pulsar can work with different BK releases, which can probably achieved with arquillian based integration testing. BK 4.7 and Pulsar 2.0 is kind of special in this picture, where Pulsar is flipping from using yahoo specific release to apache release. I would expect Pulsar would stay with 4.7 for a while after 2.0. - Sijie > > Joe > > > > On Mon, Mar 5, 2018 at 1:41 AM, Ivan Kelly <iv...@apache.org> wrote: > > > > Keep in mind the next Pulsar release needs to be based on a particular > > BookKeeper release. An Apache project cannot release against another > > project's SNAPSHOT since that has not been validated by their release > > process. > > > > The bookkeeper 4.7.0 release process should be starting soon, so it > > should be out before the pulsar 2.0 release process kicks off. > > > > -Ivan > > >
