Re: [DISCUSS] HBase-2.0 SHOULD be rolling upgradable and wire-compatible with 1.x
+1 on both rolling upgradable and wire-compatible with 1.x. Requiring end users to upgrade hbase client means they have to restart their application, which is really miserable, not mentioning the coordination work required with different users... Best Regards, Yu On 22 June 2016 at 10:29, Heng Chen wrote: > bq. We should keep main data paths working between 1.x client and 2.0 > cluster. It is fine if some admin operation does not work with older > client. > +1 > > > > 2016-06-22 2:13 GMT+08:00 Enis Söztutar : > > > Agreed with above. We should keep main data paths working between 1.x > > client and 2.0 cluster. It is fine if some admin operation does not work > > with older client. Agreed on replication as well, it must work or we > should > > have a dedicated replicator implementation. > > > > HBASE-16060 would have been fine to leave unresolved according to above, > > however, accessing the table state is needed from the main data path in > > retry logic. Whenever we cannot find a region between retries, we check > > whether the table is disabled or not (because from region assignment > > perspective, there is no distinction between a region not assigned yet, > or > > a disabled table. So, I think HBASE-16060 is a blocker for 2.0 > > unfortunately. > > > > Enis > > > > On Tue, Jun 21, 2016 at 10:51 AM, Andrew Purtell < > andrew.purt...@gmail.com > > > > > wrote: > > > > > Inline > > > > > > > On Jun 20, 2016, at 11:30 PM, Matteo Bertozzi < > theo.berto...@gmail.com > > > > > > wrote: > > > > > > > > I think everyone wants rolling upgrade. the discussion should > probably > > be > > > > around how much compatibility code do we want to keep around. > > > > > > > > using as example HBASE-16060, we need to decide how much are we > rolling > > > > upgradable and from where. > > > > I'm not too convinced that we should have extra code in master to > > > "simulate > > > > the old states", > > > > I'll rather have cleaner code in 2.0 and force the users to move to > one > > > of > > > > the latest 1.x.y > > > > there are not many changes in the 1.x releases, so we should be able > to > > > say: > > > > if you are on 1.1 move to the latest 1.1.x, if you are on 1.2 move to > > the > > > > latest 1.2.x and so on. > > > > > > > > also there are some operations that may not be needed during rolling > > > > upgrades, > > > > and we can cut on compatibility to have some code removed. > > > > an example here is HBASE-15521 where we are no longer able to > > > clone/restore > > > > snapshot during 1.x -> 2.x rolling upgrade, until the two master are > on > > > > 2.x. but this may be extended to you can't perform some operation > until > > > all > > > > the machines are on 2.x for some future change. > > > > > > > > I think we should aim for something like: > > > > - data path: HTable put/get/scan/... must work during a rolling > upgrade > > > > > > Yes. > > > > > > > - replication: must? work during rolling upgrade > > > > > > This is a must. If anything this is what gave users the most pain at > the > > > "singularity" - so much so that at least one built custom cross version > > > replication endpoints. That should have been on us to provide. > > > > > > > - admin: some operation may not be working during rolling upgrade > > > > > > This would depend on what won't work. > > > > > > I think it would be great if you could start a rolling upgrade, stop at > > > like 10% or 20% of the fleet, see how it goes for a while, and then > > either > > > commit or roll back. How comfortable that mixed version state is will > > > depend on what won't work. I submit this for consideration as a stretch > > > goal. > > > > > > > - upgrade to the latest 1.x.y before the 2.x upgrade (we can add in > 2.x > > > > master and rs the ability to check the client version) > > > > > > This would be fine, I think. > > > > > > > > > > > > > > > Matteo > > > > > > > > > > > >> On Tue, Jun 21, 2016 at 12:05 AM, Dima Spivak > > > > wrote: > > > >> > > > >> If there’s no technical limitation, we should definitely do it. As > you > > > >> note, customers running in production hate when they have to shut > down > > > >> clusters and with some of the testing infrastructure being rolled > out, > > > this > > > >> is definitely something we can set up automated testing for. +1 > > > >> > > > >> -Dima > > > >> > > > >>> On Mon, Jun 20, 2016 at 2:58 PM, Enis Söztutar > > > wrote: > > > >>> > > > >>> Time to formalize 2.0 rolling upgrade scenario? > > > >>> > > > >>> 0.94 -> 0.96 singularity was a real pain for operators and for our > > > users. > > > >>> If possible we should not have the users suffer through the same > > thing > > > >>> unless there is a very compelling reason. For the current stuff in > > > >> master, > > > >>> there is nothing that will prevent us to not have rolling upgrade > > > support > > > >>> for 2.0. So I say, we should decide on the rolling upgrade > > requirement > > > >> now, > > > >>> and start to evaluate incoming patc
Re: [DISCUSS] HBase-2.0 SHOULD be rolling upgradable and wire-compatible with 1.x
bq. We should keep main data paths working between 1.x client and 2.0 cluster. It is fine if some admin operation does not work with older client. +1 2016-06-22 2:13 GMT+08:00 Enis Söztutar : > Agreed with above. We should keep main data paths working between 1.x > client and 2.0 cluster. It is fine if some admin operation does not work > with older client. Agreed on replication as well, it must work or we should > have a dedicated replicator implementation. > > HBASE-16060 would have been fine to leave unresolved according to above, > however, accessing the table state is needed from the main data path in > retry logic. Whenever we cannot find a region between retries, we check > whether the table is disabled or not (because from region assignment > perspective, there is no distinction between a region not assigned yet, or > a disabled table. So, I think HBASE-16060 is a blocker for 2.0 > unfortunately. > > Enis > > On Tue, Jun 21, 2016 at 10:51 AM, Andrew Purtell > > wrote: > > > Inline > > > > > On Jun 20, 2016, at 11:30 PM, Matteo Bertozzi > > > wrote: > > > > > > I think everyone wants rolling upgrade. the discussion should probably > be > > > around how much compatibility code do we want to keep around. > > > > > > using as example HBASE-16060, we need to decide how much are we rolling > > > upgradable and from where. > > > I'm not too convinced that we should have extra code in master to > > "simulate > > > the old states", > > > I'll rather have cleaner code in 2.0 and force the users to move to one > > of > > > the latest 1.x.y > > > there are not many changes in the 1.x releases, so we should be able to > > say: > > > if you are on 1.1 move to the latest 1.1.x, if you are on 1.2 move to > the > > > latest 1.2.x and so on. > > > > > > also there are some operations that may not be needed during rolling > > > upgrades, > > > and we can cut on compatibility to have some code removed. > > > an example here is HBASE-15521 where we are no longer able to > > clone/restore > > > snapshot during 1.x -> 2.x rolling upgrade, until the two master are on > > > 2.x. but this may be extended to you can't perform some operation until > > all > > > the machines are on 2.x for some future change. > > > > > > I think we should aim for something like: > > > - data path: HTable put/get/scan/... must work during a rolling upgrade > > > > Yes. > > > > > - replication: must? work during rolling upgrade > > > > This is a must. If anything this is what gave users the most pain at the > > "singularity" - so much so that at least one built custom cross version > > replication endpoints. That should have been on us to provide. > > > > > - admin: some operation may not be working during rolling upgrade > > > > This would depend on what won't work. > > > > I think it would be great if you could start a rolling upgrade, stop at > > like 10% or 20% of the fleet, see how it goes for a while, and then > either > > commit or roll back. How comfortable that mixed version state is will > > depend on what won't work. I submit this for consideration as a stretch > > goal. > > > > > - upgrade to the latest 1.x.y before the 2.x upgrade (we can add in 2.x > > > master and rs the ability to check the client version) > > > > This would be fine, I think. > > > > > > > > > > > Matteo > > > > > > > > >> On Tue, Jun 21, 2016 at 12:05 AM, Dima Spivak > > wrote: > > >> > > >> If there’s no technical limitation, we should definitely do it. As you > > >> note, customers running in production hate when they have to shut down > > >> clusters and with some of the testing infrastructure being rolled out, > > this > > >> is definitely something we can set up automated testing for. +1 > > >> > > >> -Dima > > >> > > >>> On Mon, Jun 20, 2016 at 2:58 PM, Enis Söztutar > > wrote: > > >>> > > >>> Time to formalize 2.0 rolling upgrade scenario? > > >>> > > >>> 0.94 -> 0.96 singularity was a real pain for operators and for our > > users. > > >>> If possible we should not have the users suffer through the same > thing > > >>> unless there is a very compelling reason. For the current stuff in > > >> master, > > >>> there is nothing that will prevent us to not have rolling upgrade > > support > > >>> for 2.0. So I say, we should decide on the rolling upgrade > requirement > > >> now, > > >>> and start to evaluate incoming patches accordingly. Otherwise, we > risk > > >> the > > >>> option to go deeper down the hole. > > >>> > > >>> What do you guys think. Previous threads [1] and [2] seems to be in > > >> favor. > > >>> Should we vote? > > >>> > > >>> Ref: > > >>> [1] > > >> > > > http://search-hadoop.com/m/YGbbsd4An1aso5E1&subj=HBase+1+x+to+2+0+upgrade+goals+ > > >>> > > >>> [2] > > >> > > > http://search-hadoop.com/m/YGbb1CBXTL8BTI&subj=thinking+about+supporting+upgrades+to+HBase+1+x+and+2+x > > >> > > >
Re: [DISCUSS] HBase-2.0 SHOULD be rolling upgradable and wire-compatible with 1.x
Agreed with above. We should keep main data paths working between 1.x client and 2.0 cluster. It is fine if some admin operation does not work with older client. Agreed on replication as well, it must work or we should have a dedicated replicator implementation. HBASE-16060 would have been fine to leave unresolved according to above, however, accessing the table state is needed from the main data path in retry logic. Whenever we cannot find a region between retries, we check whether the table is disabled or not (because from region assignment perspective, there is no distinction between a region not assigned yet, or a disabled table. So, I think HBASE-16060 is a blocker for 2.0 unfortunately. Enis On Tue, Jun 21, 2016 at 10:51 AM, Andrew Purtell wrote: > Inline > > > On Jun 20, 2016, at 11:30 PM, Matteo Bertozzi > wrote: > > > > I think everyone wants rolling upgrade. the discussion should probably be > > around how much compatibility code do we want to keep around. > > > > using as example HBASE-16060, we need to decide how much are we rolling > > upgradable and from where. > > I'm not too convinced that we should have extra code in master to > "simulate > > the old states", > > I'll rather have cleaner code in 2.0 and force the users to move to one > of > > the latest 1.x.y > > there are not many changes in the 1.x releases, so we should be able to > say: > > if you are on 1.1 move to the latest 1.1.x, if you are on 1.2 move to the > > latest 1.2.x and so on. > > > > also there are some operations that may not be needed during rolling > > upgrades, > > and we can cut on compatibility to have some code removed. > > an example here is HBASE-15521 where we are no longer able to > clone/restore > > snapshot during 1.x -> 2.x rolling upgrade, until the two master are on > > 2.x. but this may be extended to you can't perform some operation until > all > > the machines are on 2.x for some future change. > > > > I think we should aim for something like: > > - data path: HTable put/get/scan/... must work during a rolling upgrade > > Yes. > > > - replication: must? work during rolling upgrade > > This is a must. If anything this is what gave users the most pain at the > "singularity" - so much so that at least one built custom cross version > replication endpoints. That should have been on us to provide. > > > - admin: some operation may not be working during rolling upgrade > > This would depend on what won't work. > > I think it would be great if you could start a rolling upgrade, stop at > like 10% or 20% of the fleet, see how it goes for a while, and then either > commit or roll back. How comfortable that mixed version state is will > depend on what won't work. I submit this for consideration as a stretch > goal. > > > - upgrade to the latest 1.x.y before the 2.x upgrade (we can add in 2.x > > master and rs the ability to check the client version) > > This would be fine, I think. > > > > > > > Matteo > > > > > >> On Tue, Jun 21, 2016 at 12:05 AM, Dima Spivak > wrote: > >> > >> If there’s no technical limitation, we should definitely do it. As you > >> note, customers running in production hate when they have to shut down > >> clusters and with some of the testing infrastructure being rolled out, > this > >> is definitely something we can set up automated testing for. +1 > >> > >> -Dima > >> > >>> On Mon, Jun 20, 2016 at 2:58 PM, Enis Söztutar > wrote: > >>> > >>> Time to formalize 2.0 rolling upgrade scenario? > >>> > >>> 0.94 -> 0.96 singularity was a real pain for operators and for our > users. > >>> If possible we should not have the users suffer through the same thing > >>> unless there is a very compelling reason. For the current stuff in > >> master, > >>> there is nothing that will prevent us to not have rolling upgrade > support > >>> for 2.0. So I say, we should decide on the rolling upgrade requirement > >> now, > >>> and start to evaluate incoming patches accordingly. Otherwise, we risk > >> the > >>> option to go deeper down the hole. > >>> > >>> What do you guys think. Previous threads [1] and [2] seems to be in > >> favor. > >>> Should we vote? > >>> > >>> Ref: > >>> [1] > >> > http://search-hadoop.com/m/YGbbsd4An1aso5E1&subj=HBase+1+x+to+2+0+upgrade+goals+ > >>> > >>> [2] > >> > http://search-hadoop.com/m/YGbb1CBXTL8BTI&subj=thinking+about+supporting+upgrades+to+HBase+1+x+and+2+x > >> >
Re: [DISCUSS] HBase-2.0 SHOULD be rolling upgradable and wire-compatible with 1.x
Inline > On Jun 20, 2016, at 11:30 PM, Matteo Bertozzi wrote: > > I think everyone wants rolling upgrade. the discussion should probably be > around how much compatibility code do we want to keep around. > > using as example HBASE-16060, we need to decide how much are we rolling > upgradable and from where. > I'm not too convinced that we should have extra code in master to "simulate > the old states", > I'll rather have cleaner code in 2.0 and force the users to move to one of > the latest 1.x.y > there are not many changes in the 1.x releases, so we should be able to say: > if you are on 1.1 move to the latest 1.1.x, if you are on 1.2 move to the > latest 1.2.x and so on. > > also there are some operations that may not be needed during rolling > upgrades, > and we can cut on compatibility to have some code removed. > an example here is HBASE-15521 where we are no longer able to clone/restore > snapshot during 1.x -> 2.x rolling upgrade, until the two master are on > 2.x. but this may be extended to you can't perform some operation until all > the machines are on 2.x for some future change. > > I think we should aim for something like: > - data path: HTable put/get/scan/... must work during a rolling upgrade Yes. > - replication: must? work during rolling upgrade This is a must. If anything this is what gave users the most pain at the "singularity" - so much so that at least one built custom cross version replication endpoints. That should have been on us to provide. > - admin: some operation may not be working during rolling upgrade This would depend on what won't work. I think it would be great if you could start a rolling upgrade, stop at like 10% or 20% of the fleet, see how it goes for a while, and then either commit or roll back. How comfortable that mixed version state is will depend on what won't work. I submit this for consideration as a stretch goal. > - upgrade to the latest 1.x.y before the 2.x upgrade (we can add in 2.x > master and rs the ability to check the client version) This would be fine, I think. > > > Matteo > > >> On Tue, Jun 21, 2016 at 12:05 AM, Dima Spivak wrote: >> >> If there’s no technical limitation, we should definitely do it. As you >> note, customers running in production hate when they have to shut down >> clusters and with some of the testing infrastructure being rolled out, this >> is definitely something we can set up automated testing for. +1 >> >> -Dima >> >>> On Mon, Jun 20, 2016 at 2:58 PM, Enis Söztutar wrote: >>> >>> Time to formalize 2.0 rolling upgrade scenario? >>> >>> 0.94 -> 0.96 singularity was a real pain for operators and for our users. >>> If possible we should not have the users suffer through the same thing >>> unless there is a very compelling reason. For the current stuff in >> master, >>> there is nothing that will prevent us to not have rolling upgrade support >>> for 2.0. So I say, we should decide on the rolling upgrade requirement >> now, >>> and start to evaluate incoming patches accordingly. Otherwise, we risk >> the >>> option to go deeper down the hole. >>> >>> What do you guys think. Previous threads [1] and [2] seems to be in >> favor. >>> Should we vote? >>> >>> Ref: >>> [1] >> http://search-hadoop.com/m/YGbbsd4An1aso5E1&subj=HBase+1+x+to+2+0+upgrade+goals+ >>> >>> [2] >> http://search-hadoop.com/m/YGbb1CBXTL8BTI&subj=thinking+about+supporting+upgrades+to+HBase+1+x+and+2+x >>
Re: [DISCUSS] HBase-2.0 SHOULD be rolling upgradable and wire-compatible with 1.x
+1 on requiring upgrading to one of the latest 1.x.y before upgrading to 2.0. 2016-06-21 14:30 GMT+08:00 Matteo Bertozzi : > I think everyone wants rolling upgrade. the discussion should probably be > around how much compatibility code do we want to keep around. > > using as example HBASE-16060, we need to decide how much are we rolling > upgradable and from where. > I'm not too convinced that we should have extra code in master to "simulate > the old states", > I'll rather have cleaner code in 2.0 and force the users to move to one of > the latest 1.x.y > there are not many changes in the 1.x releases, so we should be able to > say: > if you are on 1.1 move to the latest 1.1.x, if you are on 1.2 move to the > latest 1.2.x and so on. > > also there are some operations that may not be needed during rolling > upgrades, > and we can cut on compatibility to have some code removed. > an example here is HBASE-15521 where we are no longer able to clone/restore > snapshot during 1.x -> 2.x rolling upgrade, until the two master are on > 2.x. but this may be extended to you can't perform some operation until all > the machines are on 2.x for some future change. > > I think we should aim for something like: > - data path: HTable put/get/scan/... must work during a rolling upgrade > - replication: must? work during rolling upgrade > - admin: some operation may not be working during rolling upgrade > - upgrade to the latest 1.x.y before the 2.x upgrade (we can add in 2.x > master and rs the ability to check the client version) > > > Matteo > > > On Tue, Jun 21, 2016 at 12:05 AM, Dima Spivak > wrote: > > > If there’s no technical limitation, we should definitely do it. As you > > note, customers running in production hate when they have to shut down > > clusters and with some of the testing infrastructure being rolled out, > this > > is definitely something we can set up automated testing for. +1 > > > > -Dima > > > > On Mon, Jun 20, 2016 at 2:58 PM, Enis Söztutar wrote: > > > > > Time to formalize 2.0 rolling upgrade scenario? > > > > > > 0.94 -> 0.96 singularity was a real pain for operators and for our > users. > > > If possible we should not have the users suffer through the same thing > > > unless there is a very compelling reason. For the current stuff in > > master, > > > there is nothing that will prevent us to not have rolling upgrade > support > > > for 2.0. So I say, we should decide on the rolling upgrade requirement > > now, > > > and start to evaluate incoming patches accordingly. Otherwise, we risk > > the > > > option to go deeper down the hole. > > > > > > What do you guys think. Previous threads [1] and [2] seems to be in > > favor. > > > Should we vote? > > > > > > Ref: > > > [1] > > > > > > > > > http://search-hadoop.com/m/YGbbsd4An1aso5E1&subj=HBase+1+x+to+2+0+upgrade+goals+ > > > > > > [2] > > > > > > > > > http://search-hadoop.com/m/YGbb1CBXTL8BTI&subj=thinking+about+supporting+upgrades+to+HBase+1+x+and+2+x > > > > > >
Re: [DISCUSS] HBase-2.0 SHOULD be rolling upgradable and wire-compatible with 1.x
I think everyone wants rolling upgrade. the discussion should probably be around how much compatibility code do we want to keep around. using as example HBASE-16060, we need to decide how much are we rolling upgradable and from where. I'm not too convinced that we should have extra code in master to "simulate the old states", I'll rather have cleaner code in 2.0 and force the users to move to one of the latest 1.x.y there are not many changes in the 1.x releases, so we should be able to say: if you are on 1.1 move to the latest 1.1.x, if you are on 1.2 move to the latest 1.2.x and so on. also there are some operations that may not be needed during rolling upgrades, and we can cut on compatibility to have some code removed. an example here is HBASE-15521 where we are no longer able to clone/restore snapshot during 1.x -> 2.x rolling upgrade, until the two master are on 2.x. but this may be extended to you can't perform some operation until all the machines are on 2.x for some future change. I think we should aim for something like: - data path: HTable put/get/scan/... must work during a rolling upgrade - replication: must? work during rolling upgrade - admin: some operation may not be working during rolling upgrade - upgrade to the latest 1.x.y before the 2.x upgrade (we can add in 2.x master and rs the ability to check the client version) Matteo On Tue, Jun 21, 2016 at 12:05 AM, Dima Spivak wrote: > If there’s no technical limitation, we should definitely do it. As you > note, customers running in production hate when they have to shut down > clusters and with some of the testing infrastructure being rolled out, this > is definitely something we can set up automated testing for. +1 > > -Dima > > On Mon, Jun 20, 2016 at 2:58 PM, Enis Söztutar wrote: > > > Time to formalize 2.0 rolling upgrade scenario? > > > > 0.94 -> 0.96 singularity was a real pain for operators and for our users. > > If possible we should not have the users suffer through the same thing > > unless there is a very compelling reason. For the current stuff in > master, > > there is nothing that will prevent us to not have rolling upgrade support > > for 2.0. So I say, we should decide on the rolling upgrade requirement > now, > > and start to evaluate incoming patches accordingly. Otherwise, we risk > the > > option to go deeper down the hole. > > > > What do you guys think. Previous threads [1] and [2] seems to be in > favor. > > Should we vote? > > > > Ref: > > [1] > > > > > http://search-hadoop.com/m/YGbbsd4An1aso5E1&subj=HBase+1+x+to+2+0+upgrade+goals+ > > > > [2] > > > > > http://search-hadoop.com/m/YGbb1CBXTL8BTI&subj=thinking+about+supporting+upgrades+to+HBase+1+x+and+2+x > > >
Re: [DISCUSS] HBase-2.0 SHOULD be rolling upgradable and wire-compatible with 1.x
If there’s no technical limitation, we should definitely do it. As you note, customers running in production hate when they have to shut down clusters and with some of the testing infrastructure being rolled out, this is definitely something we can set up automated testing for. +1 -Dima On Mon, Jun 20, 2016 at 2:58 PM, Enis Söztutar wrote: > Time to formalize 2.0 rolling upgrade scenario? > > 0.94 -> 0.96 singularity was a real pain for operators and for our users. > If possible we should not have the users suffer through the same thing > unless there is a very compelling reason. For the current stuff in master, > there is nothing that will prevent us to not have rolling upgrade support > for 2.0. So I say, we should decide on the rolling upgrade requirement now, > and start to evaluate incoming patches accordingly. Otherwise, we risk the > option to go deeper down the hole. > > What do you guys think. Previous threads [1] and [2] seems to be in favor. > Should we vote? > > Ref: > [1] > > http://search-hadoop.com/m/YGbbsd4An1aso5E1&subj=HBase+1+x+to+2+0+upgrade+goals+ > > [2] > > http://search-hadoop.com/m/YGbb1CBXTL8BTI&subj=thinking+about+supporting+upgrades+to+HBase+1+x+and+2+x >
[DISCUSS] HBase-2.0 SHOULD be rolling upgradable and wire-compatible with 1.x
Time to formalize 2.0 rolling upgrade scenario? 0.94 -> 0.96 singularity was a real pain for operators and for our users. If possible we should not have the users suffer through the same thing unless there is a very compelling reason. For the current stuff in master, there is nothing that will prevent us to not have rolling upgrade support for 2.0. So I say, we should decide on the rolling upgrade requirement now, and start to evaluate incoming patches accordingly. Otherwise, we risk the option to go deeper down the hole. What do you guys think. Previous threads [1] and [2] seems to be in favor. Should we vote? Ref: [1] http://search-hadoop.com/m/YGbbsd4An1aso5E1&subj=HBase+1+x+to+2+0+upgrade+goals+ [2] http://search-hadoop.com/m/YGbb1CBXTL8BTI&subj=thinking+about+supporting+upgrades+to+HBase+1+x+and+2+x
