Re: [DISCUSS] Releasing the next Omid version

[Istvan Toth]

Everything on my list for 1.1 is done now.
Thanks to Andrew, Geoffrey, Richard and Aron for their recent commits.

Unless someone identifies further issues that we have to address for 1.1,
I think that we are ready to release.

Unless someone else volunteers to be the RM, I will start the process when
time permits.

regards
Istvan.

On Mon, Aug 29, 2022 at 12:40 PM Istvan Toth  wrote:

> https://issues.apache.org/jira/browse/OMID-231 is the ticket.
>
> On Mon, Aug 29, 2022 at 10:28 AM Istvan Toth  wrote:
>
>> I had a go at this, and it seems to be much easier than I thought.
>>
>> I'm going to open a ticket and PR for this soon.
>>
>> Istvan
>>
>> On Fri, Aug 26, 2022 at 5:15 PM Andrew Purtell 
>> wrote:
>>
>>> I agree with Geoffrey that OMID should align with Phoenix on default
>>> dependency versions but based on this you don’t have an immediate
>>> integration problem. I don’t believe the Configuration API has changed in a
>>> long time.
>>>
>>> The security posture of Hadoop 2 in general is a problem, though. I know
>>> Hadoop released 2.10.2 to address some CVE worthy problems but it is
>>> unclear if 2.10.2 addresses all known issues, unlike 3.3.4. Also as you
>>> know Hadoop 2 has unpatchable dependencies on org.codehaus versions of
>>> Jackson and Jetty, which themselves have high scoring CVEs that will never
>>> be fixed because they are EOL, and other similar issues. Hadoop 3 doesn’t
>>> completely solve such problems but is the only realistic place we can hope
>>> they can be addressed as required. For organizations that implement or
>>> require a top to bottom security audit of their software bill of materials,
>>> it seems possible to avoid some user pain here by aligning build defaults.
>>>
>>> I also see Geoffrey started a discussion on dev@hbase about HBase
>>> producing Hadoop 3 variant builds that could be consumed by downstreamers.
>>> I have responded on that thread today to express my intention to sponsor
>>> that effort, for what it’s worth.
>>>
>>>
>>> > On Aug 25, 2022, at 10:41 PM, Istvan Toth 
>>> wrote:
>>> >
>>> > The only non-hbase Hadoop APIs referred in Omid are
>>> > org.apache.hadoop.conf.Configuration, and a few classes from of
>>> > org.apache.hadoop.security
>>> > (the latter is not referenced directly from the coprocessors) .
>>> >
>>> > Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded
>>> in.
>>> >
>>> >
>>> >
>>> >> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell <
>>> andrew.purt...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> Is any OMID coprocessor code using Hadoop APIs?
>>> >>
>>> >>> On Aug 25, 2022, at 9:41 AM, Istvan Toth >> >
>>> >> wrote:
>>> >>>
>>> >>> We dependencyManage every Hadoop component in Phoenix, so going to
>>> >> Hadoop3
>>> >>> in Omid is not strictly necessary.
>>> >>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
>>> >>> Hadoop2 and Hadoop3 are wire compatible
>>> >>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so
>>> that
>>> >> is
>>> >>> not a show stopper either.
>>> >>>
>>> >>> If we go with the assumption that there is no significant usage of
>>> Omid
>>> >>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to
>>> avoid
>>> >>> juggling
>>> >>> the Hadoop2 and Hadoop3 compiled HBases when developing.
>>> >>> Unfortunately it's quite a bit of work with all the dependency
>>> >> management,
>>> >>> build docs, and updating the CI pipelines to rebuild HBase
>>> >>> like we do in Phoenix.
>>> >>>
>>> >>> Istvan
>>> >>>
>>> >>>
>>>  On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby >> >
>>> >> wrote:
>>> 
>>>  Thanks for volunteering to be RM for the next Omid release.
>>> 
>>>  I agree we should have a release soon (I think we'll want Phoenix
>>> 5.2 to
>>>  use the new Omid). In addition to OMID-226, I suggest we also
>>> upgrade
>>>  Omid's internal Hadoop version to align with whatever default
>>> Hadoop we
>>>  choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
>>>  little) simpler. Right now Omid appears to depend on Hadoop 2.10,
>>> which
>>>  since Phoenix 5 is Hadoop 3-based, seems incorrect.
>>> 
>>>  Geoffrey
>>> 
>>> > On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth 
>>> wrote:
>>> >
>>> > Hi!
>>> >
>>> > Most of the planned OMID changes for Phoenix 5.2 have landed.
>>> > The only outstanding ticket that I'm aware of is OMID-226 which I
>>> also
>>> > expect to land soon.
>>> >
>>> > Unless someone has more changes targeted for the next release, I
>>> >> propose
>>> > that we release the next Omid version soon after OMID-226.
>>> >
>>> > I also propose bumping the version to 1.1.0, though because of the
>>> >> HBase
>>> > 1.x compatibility removal, and maven artifact changes we could also
>>> >> argue
>>> > for 2.0.0
>>> >
>>> > If there are no other volunteers, I also volunteer to be the RM
>>> for the
>>> > release.
>>> 

Re: [DISCUSS] Releasing the next Omid version

[Istvan Toth]

https://issues.apache.org/jira/browse/OMID-231 is the ticket.

On Mon, Aug 29, 2022 at 10:28 AM Istvan Toth  wrote:

> I had a go at this, and it seems to be much easier than I thought.
>
> I'm going to open a ticket and PR for this soon.
>
> Istvan
>
> On Fri, Aug 26, 2022 at 5:15 PM Andrew Purtell 
> wrote:
>
>> I agree with Geoffrey that OMID should align with Phoenix on default
>> dependency versions but based on this you don’t have an immediate
>> integration problem. I don’t believe the Configuration API has changed in a
>> long time.
>>
>> The security posture of Hadoop 2 in general is a problem, though. I know
>> Hadoop released 2.10.2 to address some CVE worthy problems but it is
>> unclear if 2.10.2 addresses all known issues, unlike 3.3.4. Also as you
>> know Hadoop 2 has unpatchable dependencies on org.codehaus versions of
>> Jackson and Jetty, which themselves have high scoring CVEs that will never
>> be fixed because they are EOL, and other similar issues. Hadoop 3 doesn’t
>> completely solve such problems but is the only realistic place we can hope
>> they can be addressed as required. For organizations that implement or
>> require a top to bottom security audit of their software bill of materials,
>> it seems possible to avoid some user pain here by aligning build defaults.
>>
>> I also see Geoffrey started a discussion on dev@hbase about HBase
>> producing Hadoop 3 variant builds that could be consumed by downstreamers.
>> I have responded on that thread today to express my intention to sponsor
>> that effort, for what it’s worth.
>>
>>
>> > On Aug 25, 2022, at 10:41 PM, Istvan Toth 
>> wrote:
>> >
>> > The only non-hbase Hadoop APIs referred in Omid are
>> > org.apache.hadoop.conf.Configuration, and a few classes from of
>> > org.apache.hadoop.security
>> > (the latter is not referenced directly from the coprocessors) .
>> >
>> > Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded
>> in.
>> >
>> >
>> >
>> >> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell <
>> andrew.purt...@gmail.com>
>> >> wrote:
>> >>
>> >> Is any OMID coprocessor code using Hadoop APIs?
>> >>
>> >>> On Aug 25, 2022, at 9:41 AM, Istvan Toth 
>> >> wrote:
>> >>>
>> >>> We dependencyManage every Hadoop component in Phoenix, so going to
>> >> Hadoop3
>> >>> in Omid is not strictly necessary.
>> >>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
>> >>> Hadoop2 and Hadoop3 are wire compatible
>> >>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so
>> that
>> >> is
>> >>> not a show stopper either.
>> >>>
>> >>> If we go with the assumption that there is no significant usage of
>> Omid
>> >>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to
>> avoid
>> >>> juggling
>> >>> the Hadoop2 and Hadoop3 compiled HBases when developing.
>> >>> Unfortunately it's quite a bit of work with all the dependency
>> >> management,
>> >>> build docs, and updating the CI pipelines to rebuild HBase
>> >>> like we do in Phoenix.
>> >>>
>> >>> Istvan
>> >>>
>> >>>
>>  On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby 
>> >> wrote:
>> 
>>  Thanks for volunteering to be RM for the next Omid release.
>> 
>>  I agree we should have a release soon (I think we'll want Phoenix
>> 5.2 to
>>  use the new Omid). In addition to OMID-226, I suggest we also upgrade
>>  Omid's internal Hadoop version to align with whatever default Hadoop
>> we
>>  choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
>>  little) simpler. Right now Omid appears to depend on Hadoop 2.10,
>> which
>>  since Phoenix 5 is Hadoop 3-based, seems incorrect.
>> 
>>  Geoffrey
>> 
>> > On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth 
>> wrote:
>> >
>> > Hi!
>> >
>> > Most of the planned OMID changes for Phoenix 5.2 have landed.
>> > The only outstanding ticket that I'm aware of is OMID-226 which I
>> also
>> > expect to land soon.
>> >
>> > Unless someone has more changes targeted for the next release, I
>> >> propose
>> > that we release the next Omid version soon after OMID-226.
>> >
>> > I also propose bumping the version to 1.1.0, though because of the
>> >> HBase
>> > 1.x compatibility removal, and maven artifact changes we could also
>> >> argue
>> > for 2.0.0
>> >
>> > If there are no other volunteers, I also volunteer to be the RM for
>> the
>> > release.
>> >
>> > regards
>> > Istvan
>> >
>> 
>> >>>
>> >>>
>> >>> --
>> >>> *István Tóth* | Staff Software Engineer
>> >>> st...@cloudera.com 
>> >>> [image: Cloudera] 
>> >>> [image: Cloudera on Twitter]  [image:
>> >>> Cloudera on Facebook]  [image:
>> >> Cloudera
>> >>> on LinkedIn] 
>> >>> 
>> >>> 

Re: [DISCUSS] Releasing the next Omid version

[Istvan Toth]

I had a go at this, and it seems to be much easier than I thought.

I'm going to open a ticket and PR for this soon.

Istvan

On Fri, Aug 26, 2022 at 5:15 PM Andrew Purtell 
wrote:

> I agree with Geoffrey that OMID should align with Phoenix on default
> dependency versions but based on this you don’t have an immediate
> integration problem. I don’t believe the Configuration API has changed in a
> long time.
>
> The security posture of Hadoop 2 in general is a problem, though. I know
> Hadoop released 2.10.2 to address some CVE worthy problems but it is
> unclear if 2.10.2 addresses all known issues, unlike 3.3.4. Also as you
> know Hadoop 2 has unpatchable dependencies on org.codehaus versions of
> Jackson and Jetty, which themselves have high scoring CVEs that will never
> be fixed because they are EOL, and other similar issues. Hadoop 3 doesn’t
> completely solve such problems but is the only realistic place we can hope
> they can be addressed as required. For organizations that implement or
> require a top to bottom security audit of their software bill of materials,
> it seems possible to avoid some user pain here by aligning build defaults.
>
> I also see Geoffrey started a discussion on dev@hbase about HBase
> producing Hadoop 3 variant builds that could be consumed by downstreamers.
> I have responded on that thread today to express my intention to sponsor
> that effort, for what it’s worth.
>
>
> > On Aug 25, 2022, at 10:41 PM, Istvan Toth 
> wrote:
> >
> > The only non-hbase Hadoop APIs referred in Omid are
> > org.apache.hadoop.conf.Configuration, and a few classes from of
> > org.apache.hadoop.security
> > (the latter is not referenced directly from the coprocessors) .
> >
> > Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded in.
> >
> >
> >
> >> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell <
> andrew.purt...@gmail.com>
> >> wrote:
> >>
> >> Is any OMID coprocessor code using Hadoop APIs?
> >>
> >>> On Aug 25, 2022, at 9:41 AM, Istvan Toth 
> >> wrote:
> >>>
> >>> We dependencyManage every Hadoop component in Phoenix, so going to
> >> Hadoop3
> >>> in Omid is not strictly necessary.
> >>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
> >>> Hadoop2 and Hadoop3 are wire compatible
> >>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so
> that
> >> is
> >>> not a show stopper either.
> >>>
> >>> If we go with the assumption that there is no significant usage of Omid
> >>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to
> avoid
> >>> juggling
> >>> the Hadoop2 and Hadoop3 compiled HBases when developing.
> >>> Unfortunately it's quite a bit of work with all the dependency
> >> management,
> >>> build docs, and updating the CI pipelines to rebuild HBase
> >>> like we do in Phoenix.
> >>>
> >>> Istvan
> >>>
> >>>
>  On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby 
> >> wrote:
> 
>  Thanks for volunteering to be RM for the next Omid release.
> 
>  I agree we should have a release soon (I think we'll want Phoenix 5.2
> to
>  use the new Omid). In addition to OMID-226, I suggest we also upgrade
>  Omid's internal Hadoop version to align with whatever default Hadoop
> we
>  choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
>  little) simpler. Right now Omid appears to depend on Hadoop 2.10,
> which
>  since Phoenix 5 is Hadoop 3-based, seems incorrect.
> 
>  Geoffrey
> 
> > On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth 
> wrote:
> >
> > Hi!
> >
> > Most of the planned OMID changes for Phoenix 5.2 have landed.
> > The only outstanding ticket that I'm aware of is OMID-226 which I
> also
> > expect to land soon.
> >
> > Unless someone has more changes targeted for the next release, I
> >> propose
> > that we release the next Omid version soon after OMID-226.
> >
> > I also propose bumping the version to 1.1.0, though because of the
> >> HBase
> > 1.x compatibility removal, and maven artifact changes we could also
> >> argue
> > for 2.0.0
> >
> > If there are no other volunteers, I also volunteer to be the RM for
> the
> > release.
> >
> > regards
> > Istvan
> >
> 
> >>>
> >>>
> >>> --
> >>> *István Tóth* | Staff Software Engineer
> >>> st...@cloudera.com 
> >>> [image: Cloudera] 
> >>> [image: Cloudera on Twitter]  [image:
> >>> Cloudera on Facebook]  [image:
> >> Cloudera
> >>> on LinkedIn] 
> >>> 
> >>> --
> >>
> >
> >
> > --
> > *István Tóth* | Staff Software Engineer
> > st...@cloudera.com 
> > [image: Cloudera] 
> > [image: Cloudera on Twitter]  [image:
> > Cloudera on 

Re: [DISCUSS] Releasing the next Omid version

[Andrew Purtell]

I agree with Geoffrey that OMID should align with Phoenix on default dependency 
versions but based on this you don’t have an immediate integration problem. I 
don’t believe the Configuration API has changed in a long time. 

The security posture of Hadoop 2 in general is a problem, though. I know Hadoop 
released 2.10.2 to address some CVE worthy problems but it is unclear if 2.10.2 
addresses all known issues, unlike 3.3.4. Also as you know Hadoop 2 has 
unpatchable dependencies on org.codehaus versions of Jackson and Jetty, which 
themselves have high scoring CVEs that will never be fixed because they are 
EOL, and other similar issues. Hadoop 3 doesn’t completely solve such problems 
but is the only realistic place we can hope they can be addressed as required. 
For organizations that implement or require a top to bottom security audit of 
their software bill of materials, it seems possible to avoid some user pain 
here by aligning build defaults.

I also see Geoffrey started a discussion on dev@hbase about HBase producing 
Hadoop 3 variant builds that could be consumed by downstreamers. I have 
responded on that thread today to express my intention to sponsor that effort, 
for what it’s worth. 


> On Aug 25, 2022, at 10:41 PM, Istvan Toth  wrote:
> 
> The only non-hbase Hadoop APIs referred in Omid are
> org.apache.hadoop.conf.Configuration, and a few classes from of
> org.apache.hadoop.security
> (the latter is not referenced directly from the coprocessors) .
> 
> Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded in.
> 
> 
> 
>> On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell 
>> wrote:
>> 
>> Is any OMID coprocessor code using Hadoop APIs?
>> 
>>> On Aug 25, 2022, at 9:41 AM, Istvan Toth 
>> wrote:
>>> 
>>> We dependencyManage every Hadoop component in Phoenix, so going to
>> Hadoop3
>>> in Omid is not strictly necessary.
>>> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
>>> Hadoop2 and Hadoop3 are wire compatible
>>> (and I'm not sure if Omid even calls Hadoop endpoints directly), so that
>> is
>>> not a show stopper either.
>>> 
>>> If we go with the assumption that there is no significant usage of Omid
>>> outside HBase, then updating to Hadoop3 is a no-brainer, if only to avoid
>>> juggling
>>> the Hadoop2 and Hadoop3 compiled HBases when developing.
>>> Unfortunately it's quite a bit of work with all the dependency
>> management,
>>> build docs, and updating the CI pipelines to rebuild HBase
>>> like we do in Phoenix.
>>> 
>>> Istvan
>>> 
>>> 
 On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby 
>> wrote:
 
 Thanks for volunteering to be RM for the next Omid release.
 
 I agree we should have a release soon (I think we'll want Phoenix 5.2 to
 use the new Omid). In addition to OMID-226, I suggest we also upgrade
 Omid's internal Hadoop version to align with whatever default Hadoop we
 choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
 little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
 since Phoenix 5 is Hadoop 3-based, seems incorrect.
 
 Geoffrey
 
> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth  wrote:
> 
> Hi!
> 
> Most of the planned OMID changes for Phoenix 5.2 have landed.
> The only outstanding ticket that I'm aware of is OMID-226 which I also
> expect to land soon.
> 
> Unless someone has more changes targeted for the next release, I
>> propose
> that we release the next Omid version soon after OMID-226.
> 
> I also propose bumping the version to 1.1.0, though because of the
>> HBase
> 1.x compatibility removal, and maven artifact changes we could also
>> argue
> for 2.0.0
> 
> If there are no other volunteers, I also volunteer to be the RM for the
> release.
> 
> regards
> Istvan
> 
 
>>> 
>>> 
>>> --
>>> *István Tóth* | Staff Software Engineer
>>> st...@cloudera.com 
>>> [image: Cloudera] 
>>> [image: Cloudera on Twitter]  [image:
>>> Cloudera on Facebook]  [image:
>> Cloudera
>>> on LinkedIn] 
>>> 
>>> --
>> 
> 
> 
> -- 
> *István Tóth* | Staff Software Engineer
> st...@cloudera.com 
> [image: Cloudera] 
> [image: Cloudera on Twitter]  [image:
> Cloudera on Facebook]  [image: Cloudera
> on LinkedIn] 
> 
> --

Re: [DISCUSS] Releasing the next Omid version

[Istvan Toth]

The only non-hbase Hadoop APIs referred in Omid are
org.apache.hadoop.conf.Configuration, and a few classes from of
org.apache.hadoop.security
(the latter is not referenced directly from the coprocessors) .

Also, the coprocessor JARs don't have any Hbase or Hadoop code shaded in.



On Fri, Aug 26, 2022 at 5:45 AM Andrew Purtell 
wrote:

> Is any OMID coprocessor code using Hadoop APIs?
>
> > On Aug 25, 2022, at 9:41 AM, Istvan Toth 
> wrote:
> >
> > We dependencyManage every Hadoop component in Phoenix, so going to
> Hadoop3
> > in Omid is not strictly necessary.
> > Hadoop is also added to the TSO libraries in the assembly, but AFAIK
> > Hadoop2 and Hadoop3 are wire compatible
> > (and I'm not sure if Omid even calls Hadoop endpoints directly), so that
> is
> > not a show stopper either.
> >
> > If we go with the assumption that there is no significant usage of Omid
> > outside HBase, then updating to Hadoop3 is a no-brainer, if only to avoid
> > juggling
> > the Hadoop2 and Hadoop3 compiled HBases when developing.
> > Unfortunately it's quite a bit of work with all the dependency
> management,
> > build docs, and updating the CI pipelines to rebuild HBase
> > like we do in Phoenix.
> >
> > Istvan
> >
> >
> >> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby 
> wrote:
> >>
> >> Thanks for volunteering to be RM for the next Omid release.
> >>
> >> I agree we should have a release soon (I think we'll want Phoenix 5.2 to
> >> use the new Omid). In addition to OMID-226, I suggest we also upgrade
> >> Omid's internal Hadoop version to align with whatever default Hadoop we
> >> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
> >> little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
> >> since Phoenix 5 is Hadoop 3-based, seems incorrect.
> >>
> >> Geoffrey
> >>
> >>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth  wrote:
> >>>
> >>> Hi!
> >>>
> >>> Most of the planned OMID changes for Phoenix 5.2 have landed.
> >>> The only outstanding ticket that I'm aware of is OMID-226 which I also
> >>> expect to land soon.
> >>>
> >>> Unless someone has more changes targeted for the next release, I
> propose
> >>> that we release the next Omid version soon after OMID-226.
> >>>
> >>> I also propose bumping the version to 1.1.0, though because of the
> HBase
> >>> 1.x compatibility removal, and maven artifact changes we could also
> argue
> >>> for 2.0.0
> >>>
> >>> If there are no other volunteers, I also volunteer to be the RM for the
> >>> release.
> >>>
> >>> regards
> >>> Istvan
> >>>
> >>
> >
> >
> > --
> > *István Tóth* | Staff Software Engineer
> > st...@cloudera.com 
> > [image: Cloudera] 
> > [image: Cloudera on Twitter]  [image:
> > Cloudera on Facebook]  [image:
> Cloudera
> > on LinkedIn] 
> > 
> > --
>


-- 
*István Tóth* | Staff Software Engineer
st...@cloudera.com 
[image: Cloudera] 
[image: Cloudera on Twitter]  [image:
Cloudera on Facebook]  [image: Cloudera
on LinkedIn] 

--

Re: [DISCUSS] Releasing the next Omid version

[Andrew Purtell]

Is any OMID coprocessor code using Hadoop APIs? 

> On Aug 25, 2022, at 9:41 AM, Istvan Toth  wrote:
> 
> We dependencyManage every Hadoop component in Phoenix, so going to Hadoop3
> in Omid is not strictly necessary.
> Hadoop is also added to the TSO libraries in the assembly, but AFAIK
> Hadoop2 and Hadoop3 are wire compatible
> (and I'm not sure if Omid even calls Hadoop endpoints directly), so that is
> not a show stopper either.
> 
> If we go with the assumption that there is no significant usage of Omid
> outside HBase, then updating to Hadoop3 is a no-brainer, if only to avoid
> juggling
> the Hadoop2 and Hadoop3 compiled HBases when developing.
> Unfortunately it's quite a bit of work with all the dependency management,
> build docs, and updating the CI pipelines to rebuild HBase
> like we do in Phoenix.
> 
> Istvan
> 
> 
>> On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby  wrote:
>> 
>> Thanks for volunteering to be RM for the next Omid release.
>> 
>> I agree we should have a release soon (I think we'll want Phoenix 5.2 to
>> use the new Omid). In addition to OMID-226, I suggest we also upgrade
>> Omid's internal Hadoop version to align with whatever default Hadoop we
>> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
>> little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
>> since Phoenix 5 is Hadoop 3-based, seems incorrect.
>> 
>> Geoffrey
>> 
>>> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth  wrote:
>>> 
>>> Hi!
>>> 
>>> Most of the planned OMID changes for Phoenix 5.2 have landed.
>>> The only outstanding ticket that I'm aware of is OMID-226 which I also
>>> expect to land soon.
>>> 
>>> Unless someone has more changes targeted for the next release, I propose
>>> that we release the next Omid version soon after OMID-226.
>>> 
>>> I also propose bumping the version to 1.1.0, though because of the HBase
>>> 1.x compatibility removal, and maven artifact changes we could also argue
>>> for 2.0.0
>>> 
>>> If there are no other volunteers, I also volunteer to be the RM for the
>>> release.
>>> 
>>> regards
>>> Istvan
>>> 
>> 
> 
> 
> -- 
> *István Tóth* | Staff Software Engineer
> st...@cloudera.com 
> [image: Cloudera] 
> [image: Cloudera on Twitter]  [image:
> Cloudera on Facebook]  [image: Cloudera
> on LinkedIn] 
> 
> --

Re: [DISCUSS] Releasing the next Omid version

[Istvan Toth]

We dependencyManage every Hadoop component in Phoenix, so going to Hadoop3
in Omid is not strictly necessary.
Hadoop is also added to the TSO libraries in the assembly, but AFAIK
Hadoop2 and Hadoop3 are wire compatible
(and I'm not sure if Omid even calls Hadoop endpoints directly), so that is
not a show stopper either.

If we go with the assumption that there is no significant usage of Omid
outside HBase, then updating to Hadoop3 is a no-brainer, if only to avoid
juggling
the Hadoop2 and Hadoop3 compiled HBases when developing.
Unfortunately it's quite a bit of work with all the dependency management,
build docs, and updating the CI pipelines to rebuild HBase
like we do in Phoenix.

Istvan


On Wed, Aug 24, 2022 at 9:24 PM Geoffrey Jacoby  wrote:

> Thanks for volunteering to be RM for the next Omid release.
>
> I agree we should have a release soon (I think we'll want Phoenix 5.2 to
> use the new Omid). In addition to OMID-226, I suggest we also upgrade
> Omid's internal Hadoop version to align with whatever default Hadoop we
> choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
> little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
> since Phoenix 5 is Hadoop 3-based, seems incorrect.
>
> Geoffrey
>
> On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth  wrote:
>
> > Hi!
> >
> > Most of the planned OMID changes for Phoenix 5.2 have landed.
> > The only outstanding ticket that I'm aware of is OMID-226 which I also
> > expect to land soon.
> >
> > Unless someone has more changes targeted for the next release, I propose
> > that we release the next Omid version soon after OMID-226.
> >
> > I also propose bumping the version to 1.1.0, though because of the HBase
> > 1.x compatibility removal, and maven artifact changes we could also argue
> > for 2.0.0
> >
> > If there are no other volunteers, I also volunteer to be the RM for the
> > release.
> >
> > regards
> > Istvan
> >
>


-- 
*István Tóth* | Staff Software Engineer
st...@cloudera.com 
[image: Cloudera] 
[image: Cloudera on Twitter]  [image:
Cloudera on Facebook]  [image: Cloudera
on LinkedIn] 

--

Re: [DISCUSS] Releasing the next Omid version

[Geoffrey Jacoby]

Thanks for volunteering to be RM for the next Omid release.

I agree we should have a release soon (I think we'll want Phoenix 5.2 to
use the new Omid). In addition to OMID-226, I suggest we also upgrade
Omid's internal Hadoop version to align with whatever default Hadoop we
choose for Phoenix 5.2, to keep Phoenix's transitive dependencies (a
little) simpler. Right now Omid appears to depend on Hadoop 2.10, which
since Phoenix 5 is Hadoop 3-based, seems incorrect.

Geoffrey

On Wed, Aug 24, 2022 at 12:46 AM Istvan Toth  wrote:

> Hi!
>
> Most of the planned OMID changes for Phoenix 5.2 have landed.
> The only outstanding ticket that I'm aware of is OMID-226 which I also
> expect to land soon.
>
> Unless someone has more changes targeted for the next release, I propose
> that we release the next Omid version soon after OMID-226.
>
> I also propose bumping the version to 1.1.0, though because of the HBase
> 1.x compatibility removal, and maven artifact changes we could also argue
> for 2.0.0
>
> If there are no other volunteers, I also volunteer to be the RM for the
> release.
>
> regards
> Istvan
>

[DISCUSS] Releasing the next Omid version

[Istvan Toth]

Hi!

Most of the planned OMID changes for Phoenix 5.2 have landed.
The only outstanding ticket that I'm aware of is OMID-226 which I also
expect to land soon.

Unless someone has more changes targeted for the next release, I propose
that we release the next Omid version soon after OMID-226.

I also propose bumping the version to 1.1.0, though because of the HBase
1.x compatibility removal, and maven artifact changes we could also argue
for 2.0.0

If there are no other volunteers, I also volunteer to be the RM for the
release.

regards
Istvan

Re: [DISCUSS] Releasing the next Omid version

[Istvan Toth]

Java code wise the current shim system is fine.

The packaging up to 1.0.2 is nightmarish, with having to manually exclude
hbase-1 artifacts if you use HBase 2.
However, that is mostly fixed by OMID-188.

I will check if Omid works with the current HBase 3 snapshot without
modifications, and remove the shims if it does.

On Fri, Apr 29, 2022 at 12:19 AM Josh Elser  wrote:

> +1
>
> If we're dropping Phoenix 4.x imminently, that means dropping HBase 1.x
> and we should follow suit in Omid.
>
> A "beta" HBase 3.0 is probably not too, too far away. I would consider
> how "nice" the current shim logic is in Omid (i.e. is it actually
> helpful? nice to work with? effective?), and make the call on that.
> However, HBase 3.0 should not drop any API from HBase 2.x, so we should
> not _have_ to shim anything.
>
> 1.1.0 as a release version makes sense to me for the API reason you gave.
>
> On 4/19/22 4:53 AM, Istvan Toth wrote:
> > Hi!
> >
> > When Geoffrey proposed releasing Phoenix 5.2.0, I asked for time to
> release
> > a new Omid version first, as there are a lot of unreleased fixes in
> master.
> >
> > One of those fixes removes the need to add a lot of explicit excludes for
> > the Omid HBase-1 artifacts when depending on Omid for HBase 2.
> >
> > However, the discussion on dropping HBase 1.x support from Phoenixhas
> been
> > re-opened, and so far there are no objections.
> >
> > We can either release the Omid master as is (perhaps with some dependency
> > version bumps), or we could just drop HBase 1.x support, and simplify the
> > project structure quite a bit for the next version.
> >
> > In case we drop support for Hbase 1, we also need to decide whether to
> keep
> > the maven build infrastructure (shims and flatten-maven-plugin) for
> > supporting different HBase releases for an upcoming HBase 3 release (if
> the
> > API changes will require it), or to remove it altogether ?
> >
> > We'll need to update the dependencies and exclusions in Phoenix either
> way.
> >
> > What do you think ?
> > Can we make an official decision to drop Phoenix 4.x soon, and drop
> HBase 1
> > support from Omid for the next release,
> > or should I just go ahead with the Omid next release process, and worry
> > about removing the HBase 1.x support from Omid later ?
> >
> > Also, as we're making incompatible changes to the way Omid is to be
> > consumed via maven, I think that we should bump the version either to
> > 1.1.0, or 2.0.0. (I prefer 1.1.0, as the API doesn't change.)
> >
> > Looking forward to your input,
> >
> > Istvan
> >
>


-- 
*István Tóth* | Staff Software Engineer
st...@cloudera.com 
[image: Cloudera] 
[image: Cloudera on Twitter]  [image:
Cloudera on Facebook]  [image: Cloudera
on LinkedIn] 

--

Re: [DISCUSS] Releasing the next Omid version

[Josh Elser]

+1

If we're dropping Phoenix 4.x imminently, that means dropping HBase 1.x 
and we should follow suit in Omid.


A "beta" HBase 3.0 is probably not too, too far away. I would consider 
how "nice" the current shim logic is in Omid (i.e. is it actually 
helpful? nice to work with? effective?), and make the call on that. 
However, HBase 3.0 should not drop any API from HBase 2.x, so we should 
not _have_ to shim anything.


1.1.0 as a release version makes sense to me for the API reason you gave.

On 4/19/22 4:53 AM, Istvan Toth wrote:

Hi!

When Geoffrey proposed releasing Phoenix 5.2.0, I asked for time to release
a new Omid version first, as there are a lot of unreleased fixes in  master.

One of those fixes removes the need to add a lot of explicit excludes for
the Omid HBase-1 artifacts when depending on Omid for HBase 2.

However, the discussion on dropping HBase 1.x support from Phoenixhas been
re-opened, and so far there are no objections.

We can either release the Omid master as is (perhaps with some dependency
version bumps), or we could just drop HBase 1.x support, and simplify the
project structure quite a bit for the next version.

In case we drop support for Hbase 1, we also need to decide whether to keep
the maven build infrastructure (shims and flatten-maven-plugin) for
supporting different HBase releases for an upcoming HBase 3 release (if the
API changes will require it), or to remove it altogether ?

We'll need to update the dependencies and exclusions in Phoenix either way.

What do you think ?
Can we make an official decision to drop Phoenix 4.x soon, and drop HBase 1
support from Omid for the next release,
or should I just go ahead with the Omid next release process, and worry
about removing the HBase 1.x support from Omid later ?

Also, as we're making incompatible changes to the way Omid is to be
consumed via maven, I think that we should bump the version either to
1.1.0, or 2.0.0. (I prefer 1.1.0, as the API doesn't change.)

Looking forward to your input,

Istvan

[DISCUSS] Releasing the next Omid version

[Istvan Toth]

Hi!

When Geoffrey proposed releasing Phoenix 5.2.0, I asked for time to release
a new Omid version first, as there are a lot of unreleased fixes in  master.

One of those fixes removes the need to add a lot of explicit excludes for
the Omid HBase-1 artifacts when depending on Omid for HBase 2.

However, the discussion on dropping HBase 1.x support from Phoenixhas been
re-opened, and so far there are no objections.

We can either release the Omid master as is (perhaps with some dependency
version bumps), or we could just drop HBase 1.x support, and simplify the
project structure quite a bit for the next version.

In case we drop support for Hbase 1, we also need to decide whether to keep
the maven build infrastructure (shims and flatten-maven-plugin) for
supporting different HBase releases for an upcoming HBase 3 release (if the
API changes will require it), or to remove it altogether ?

We'll need to update the dependencies and exclusions in Phoenix either way.

What do you think ?
Can we make an official decision to drop Phoenix 4.x soon, and drop HBase 1
support from Omid for the next release,
or should I just go ahead with the Omid next release process, and worry
about removing the HBase 1.x support from Omid later ?

Also, as we're making incompatible changes to the way Omid is to be
consumed via maven, I think that we should bump the version either to
1.1.0, or 2.0.0. (I prefer 1.1.0, as the API doesn't change.)

Looking forward to your input,

Istvan