Re: Hadoop-3.2.3 Release Update

[Masatake Iwasaki]

I cherry-picked some commits and edited JIRA issues
based on the warnigs of git_jira_fix_version_check.py.

The warning about HADOOP-17592 can be ignored since it is caused by
wrong issue id in the commit log (HADOOP-17952 instead of HADOOP-17592).

  $ ./venv/bin/python 
dev-support/git-jira-validation/git_jira_fix_version_check.py
  ...
  Any resolved Jira with fixVersion 3.2.3 but corresponding commit not present
  Starting diff: ##
  HADOOP-17592 is marked resolved with fixVersion 3.2.3 but no corresponding 
commit found
  Completed diff: ##
  
  $ git log --grep 'Fix the wrong CIDR range example in Proxy User documentation'

  commit 35855007f6e18298c849f7830c0d6550ffba9e9a
  Author: kwangsun 
  Date:   Mon Mar 22 11:43:32 2021 +0900
  
  HADOOP-17952. Fix the wrong CIDR range example in Proxy User documentation. (#2780)
  
  Signed-off-by: Akira Ajisaka 

  (cherry picked from commit c8d327a4f1a7f15d6be35051414199d1d3fdc5ef)

I will create RC0 shortly after updating index.md.vm (which is the top page of 
site documentation).

Thanks,
Masatake Iwasaki

On 2022/03/11 22:17, Xiaoqiao He wrote:

Thanks Masatake for the update. Sounds great! Please ping me if I could help 
with something.

Best Regards,
- He Xiaoqiao

On Thu, Mar 10, 2022 at 6:16 PM Masatake Iwasaki mailto:iwasak...@oss.nttdata.co.jp>> wrote:

 > Or Masatake, are you interested in continuing the 3.2.3 release? I would 
like to offer some bandwidth if need any help.

I'm going to cut RC0 on behalf of Brahma on this weekend.

There looks no blocker/critical issues on JIRA.
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.2+Release 


branch-3.2.3 was cut and version was set by Brahma.
https://github.com/apache/hadoop/tree/branch-3.2.3 


I found some TODOs to cut RC from branch-3.2.3.

* Cherry-picking HADOOP-17718 to branch-3.2 to fix error on running 
create-release script.
https://issues.apache.org/jira/browse/HADOOP-17718 


* Checking git commit / Jira fixVersion discrepancies by script added by 
HADOOP-18125.
https://issues.apache.org/jira/browse/HADOOP-18125 


While HADOOP-18088 is targeted to 3.2.3, I think we can release 3.2.3 
without this.
We can release 3.2.4 right after 3.2.3 just for reload4j migration if we 
need.
https://issues.apache.org/jira/browse/HADOOP-18088 


Thanks,
Masatake Iwasaki

On 2022/03/10 12:23, Xiaoqiao He wrote:
 > Thanks Masatake for reaching out.
 >
 > Not sure if Brahma has bandwidth to push this release forward. If there 
is no more bandwidth, I think we should involve other volunteers to make it happen.
 > Or Masatake, are you interested in continuing the 3.2.3 release? I would 
like to offer some bandwidth if need any help.
 > Thanks again.
 >
 > Regards
 > - He Xiaoqiao
 >
 > On Thu, Mar 3, 2022 at 10:11 PM Masatake Iwasaki mailto:iwasak...@oss.nttdata.co.jp> >> wrote:
 >
 >     Hi Brahma,
 >
 >     I would like to use 3.2.3 for next minor release of Bigtop.
 > https://issues.apache.org/jira/browse/BIGTOP-3605 
 
>
 >
 >     Let me know if there is any blocker I can help.
 >
 >     Thanks,
 >     Masatake Iwasaki
 >
 >     On 2022/01/12 4:41, Brahma Reddy Battula wrote:
 >      > Sorry, I was looking for some CVE’s fixes.will make it progress 
on this…
 >      >
 >      >
 >      > On Tue, 11 Jan 2022 at 5:58 PM, Wei-Chiu Chuang mailto:weic...@apache.org> >> 
wrote:
 >      >
 >      >> Is this still making progress?
 >      >>
 >      >> On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula mailto:bra...@apache.org> >>
 >      >> wrote:
 >      >>
 >      >>> Hi Akira,
 >      >>>
 >      >>> Thanks for your email!!
 >      >>>
 >      >>> I am evaluating the CVE’s which needs to go for this release..
 >      >>>
 >      >>> Will update soon!!
 >      >>>
 >      >>>
 >      >>> On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka mailto:aajis...@apache.org> >>
 >      >> wrote:
 >      >>>
 >       Hi Brahma,
 >      
 >       What is the release process going on? Is there any b

Re: Hadoop-3.2.3 Release Update

[Xiaoqiao He]

Thanks Masatake for the update. Sounds great! Please ping me if I could
help with something.

Best Regards,
- He Xiaoqiao

On Thu, Mar 10, 2022 at 6:16 PM Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:

> > Or Masatake, are you interested in continuing the 3.2.3 release? I would
> like to offer some bandwidth if need any help.
>
> I'm going to cut RC0 on behalf of Brahma on this weekend.
>
> There looks no blocker/critical issues on JIRA.
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.2+Release
>
> branch-3.2.3 was cut and version was set by Brahma.
> https://github.com/apache/hadoop/tree/branch-3.2.3
>
> I found some TODOs to cut RC from branch-3.2.3.
>
> * Cherry-picking HADOOP-17718 to branch-3.2 to fix error on running
> create-release script.
>https://issues.apache.org/jira/browse/HADOOP-17718
>
> * Checking git commit / Jira fixVersion discrepancies by script added by
> HADOOP-18125.
>https://issues.apache.org/jira/browse/HADOOP-18125
>
> While HADOOP-18088 is targeted to 3.2.3, I think we can release 3.2.3
> without this.
> We can release 3.2.4 right after 3.2.3 just for reload4j migration if we
> need.
> https://issues.apache.org/jira/browse/HADOOP-18088
>
> Thanks,
> Masatake Iwasaki
>
> On 2022/03/10 12:23, Xiaoqiao He wrote:
> > Thanks Masatake for reaching out.
> >
> > Not sure if Brahma has bandwidth to push this release forward. If there
> is no more bandwidth, I think we should involve other volunteers to make
> it happen.
> > Or Masatake, are you interested in continuing the 3.2.3 release? I would
> like to offer some bandwidth if need any help.
> > Thanks again.
> >
> > Regards
> > - He Xiaoqiao
> >
> > On Thu, Mar 3, 2022 at 10:11 PM Masatake Iwasaki <
> iwasak...@oss.nttdata.co.jp > wrote:
> >
> > Hi Brahma,
> >
> > I would like to use 3.2.3 for next minor release of Bigtop.
> > https://issues.apache.org/jira/browse/BIGTOP-3605 <
> https://issues.apache.org/jira/browse/BIGTOP-3605>
> >
> > Let me know if there is any blocker I can help.
> >
> > Thanks,
> > Masatake Iwasaki
> >
> > On 2022/01/12 4:41, Brahma Reddy Battula wrote:
> >  > Sorry, I was looking for some CVE’s fixes.will make it progress
> on this…
> >  >
> >  >
> >  > On Tue, 11 Jan 2022 at 5:58 PM, Wei-Chiu Chuang <
> weic...@apache.org > wrote:
> >  >
> >  >> Is this still making progress?
> >  >>
> >  >> On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula <
> bra...@apache.org >
> >  >> wrote:
> >  >>
> >  >>> Hi Akira,
> >  >>>
> >  >>> Thanks for your email!!
> >  >>>
> >  >>> I am evaluating the CVE’s which needs to go for this release..
> >  >>>
> >  >>> Will update soon!!
> >  >>>
> >  >>>
> >  >>> On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka <
> aajis...@apache.org >
> >  >> wrote:
> >  >>>
> >   Hi Brahma,
> >  
> >   What is the release process going on? Is there any blocker for
> the RC?
> >  
> >   -Akira
> >  
> >   On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He <
> xq.he2...@gmail.com >
> >  >> wrote:
> >  
> >  > Hi Brahma,
> >  >
> >  > The feature 'BPServiceActor processes commands from NameNode
> >  > asynchronously' has been ready for both branch-3.2 and
> branch-3.2.3.
> >   While
> >  > cherry-picking there is only minor conflict, So I checked in
> >  >> directly.
> >   BTW,
> >  > run some unit tests and build pseudo cluster to verify, it
> seems to
> >  >>> work
> >  > fine.
> >  > FYI.
> >  >
> >  > Regards,
> >  > - He Xiaoqiao
> >  >
> >  > On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula <
> >  >>> bra...@apache.org 
> >  >
> >  > wrote:
> >  >
> >  >> Please go ahead. Let me know any help required on review.
> >  >>
> >  >> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He <
> xq.he2...@gmail.com >
> >   wrote:
> >  >>
> >  >>> Hi Brahma,
> >  >>>
> >  >>> I plan to involve HDFS-14997 and related JIRAs if possible.
> I have
> >  >>> resolved the conflict and verified them locally.
> >  >>> It will include: HDFS-14997 HDFS-15075 HDFS-15651
> HDFS-15113.
> >  >>> I would like to hear some more response that if we have
> enough time
> >  >>> to
> >  >>> wait for it to be ready.
> >  >>> Thanks.
> >  >>>
> >  >>> Best Regards,
> >  >>> - He Xiaoqiao
> >  >>>
> >  >>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He <
> xq.he2...@gmail.com >
> >   wrote:
> >  >>>
>

Re: Hadoop-3.2.3 Release Update

[Masatake Iwasaki]

Or Masatake, are you interested in continuing the 3.2.3 release? I would like 
to offer some bandwidth if need any help.


I'm going to cut RC0 on behalf of Brahma on this weekend.

There looks no blocker/critical issues on JIRA.
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.2+Release

branch-3.2.3 was cut and version was set by Brahma.
https://github.com/apache/hadoop/tree/branch-3.2.3

I found some TODOs to cut RC from branch-3.2.3.

* Cherry-picking HADOOP-17718 to branch-3.2 to fix error on running 
create-release script.
  https://issues.apache.org/jira/browse/HADOOP-17718

* Checking git commit / Jira fixVersion discrepancies by script added by 
HADOOP-18125.
  https://issues.apache.org/jira/browse/HADOOP-18125

While HADOOP-18088 is targeted to 3.2.3, I think we can release 3.2.3 without 
this.
We can release 3.2.4 right after 3.2.3 just for reload4j migration if we need.
https://issues.apache.org/jira/browse/HADOOP-18088

Thanks,
Masatake Iwasaki

On 2022/03/10 12:23, Xiaoqiao He wrote:

Thanks Masatake for reaching out.

Not sure if Brahma has bandwidth to push this release forward. If there is no 
more bandwidth, I think we should involve other volunteers to make it happen.
Or Masatake, are you interested in continuing the 3.2.3 release? I would like 
to offer some bandwidth if need any help.
Thanks again.

Regards
- He Xiaoqiao

On Thu, Mar 3, 2022 at 10:11 PM Masatake Iwasaki mailto:iwasak...@oss.nttdata.co.jp>> wrote:

Hi Brahma,

I would like to use 3.2.3 for next minor release of Bigtop.
https://issues.apache.org/jira/browse/BIGTOP-3605 


Let me know if there is any blocker I can help.

Thanks,
Masatake Iwasaki

On 2022/01/12 4:41, Brahma Reddy Battula wrote:
 > Sorry, I was looking for some CVE’s fixes.will make it progress on this…
 >
 >
 > On Tue, 11 Jan 2022 at 5:58 PM, Wei-Chiu Chuang mailto:weic...@apache.org>> wrote:
 >
 >> Is this still making progress?
 >>
 >> On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula mailto:bra...@apache.org>>
 >> wrote:
 >>
 >>> Hi Akira,
 >>>
 >>> Thanks for your email!!
 >>>
 >>> I am evaluating the CVE’s which needs to go for this release..
 >>>
 >>> Will update soon!!
 >>>
 >>>
 >>> On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka mailto:aajis...@apache.org>>
 >> wrote:
 >>>
  Hi Brahma,
 
  What is the release process going on? Is there any blocker for the RC?
 
  -Akira
 
  On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He mailto:xq.he2...@gmail.com>>
 >> wrote:
 
 > Hi Brahma,
 >
 > The feature 'BPServiceActor processes commands from NameNode
 > asynchronously' has been ready for both branch-3.2 and branch-3.2.3.
  While
 > cherry-picking there is only minor conflict, So I checked in
 >> directly.
  BTW,
 > run some unit tests and build pseudo cluster to verify, it seems to
 >>> work
 > fine.
 > FYI.
 >
 > Regards,
 > - He Xiaoqiao
 >
 > On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula <
 >>> bra...@apache.org 
 >
 > wrote:
 >
 >> Please go ahead. Let me know any help required on review.
 >>
 >> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He mailto:xq.he2...@gmail.com>>
  wrote:
 >>
 >>> Hi Brahma,
 >>>
 >>> I plan to involve HDFS-14997 and related JIRAs if possible. I have
 >>> resolved the conflict and verified them locally.
 >>> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
 >>> I would like to hear some more response that if we have enough time
 >>> to
 >>> wait for it to be ready.
 >>> Thanks.
 >>>
 >>> Best Regards,
 >>> - He Xiaoqiao
 >>>
 >>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He mailto:xq.he2...@gmail.com>>
  wrote:
 >>>
  Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3.
 >> FYI.
 
  On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula <
  bra...@apache.org >
  wrote:
 
 > Hi All,
 >
 > Waiting for the following jira to commit to hadoop-3.2.3 , mostly
  this
 > can
 > be done by this week,then I will try to create the RC next if
 >> there
  is
 > no
 > objection.
 >
 > https://issues.apache.org/jira/browse/HDFS-15160 

 >
 >
 >
 > On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula <
 > bra...@

Re: Hadoop-3.2.3 Release Update

[Xiaoqiao He]

Thanks Masatake for reaching out.

Not sure if Brahma has bandwidth to push this release forward. If there is
no more bandwidth, I think we should involve other volunteers to make
it happen.
Or Masatake, are you interested in continuing the 3.2.3 release? I would
like to offer some bandwidth if need any help.
Thanks again.

Regards
- He Xiaoqiao

On Thu, Mar 3, 2022 at 10:11 PM Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:

> Hi Brahma,
>
> I would like to use 3.2.3 for next minor release of Bigtop.
> https://issues.apache.org/jira/browse/BIGTOP-3605
>
> Let me know if there is any blocker I can help.
>
> Thanks,
> Masatake Iwasaki
>
> On 2022/01/12 4:41, Brahma Reddy Battula wrote:
> > Sorry, I was looking for some CVE’s fixes.will make it progress on this…
> >
> >
> > On Tue, 11 Jan 2022 at 5:58 PM, Wei-Chiu Chuang 
> wrote:
> >
> >> Is this still making progress?
> >>
> >> On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula 
> >> wrote:
> >>
> >>> Hi Akira,
> >>>
> >>> Thanks for your email!!
> >>>
> >>> I am evaluating the CVE’s which needs to go for this release..
> >>>
> >>> Will update soon!!
> >>>
> >>>
> >>> On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka 
> >> wrote:
> >>>
>  Hi Brahma,
> 
>  What is the release process going on? Is there any blocker for the RC?
> 
>  -Akira
> 
>  On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He 
> >> wrote:
> 
> > Hi Brahma,
> >
> > The feature 'BPServiceActor processes commands from NameNode
> > asynchronously' has been ready for both branch-3.2 and branch-3.2.3.
>  While
> > cherry-picking there is only minor conflict, So I checked in
> >> directly.
>  BTW,
> > run some unit tests and build pseudo cluster to verify, it seems to
> >>> work
> > fine.
> > FYI.
> >
> > Regards,
> > - He Xiaoqiao
> >
> > On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula <
> >>> bra...@apache.org
> >
> > wrote:
> >
> >> Please go ahead. Let me know any help required on review.
> >>
> >> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He 
>  wrote:
> >>
> >>> Hi Brahma,
> >>>
> >>> I plan to involve HDFS-14997 and related JIRAs if possible. I have
> >>> resolved the conflict and verified them locally.
> >>> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
> >>> I would like to hear some more response that if we have enough time
> >>> to
> >>> wait for it to be ready.
> >>> Thanks.
> >>>
> >>> Best Regards,
> >>> - He Xiaoqiao
> >>>
> >>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He 
>  wrote:
> >>>
>  Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3.
> >> FYI.
> 
>  On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula <
>  bra...@apache.org>
>  wrote:
> 
> > Hi All,
> >
> > Waiting for the following jira to commit to hadoop-3.2.3 , mostly
>  this
> > can
> > be done by this week,then I will try to create the RC next if
> >> there
>  is
> > no
> > objection.
> >
> > https://issues.apache.org/jira/browse/HDFS-15160
> >
> >
> >
> > On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula <
> > bra...@apache.org>
> > wrote:
> >
> >> @Akira Ajisaka   and @Masatake Iwasaki
> >> 
> >> Looks all are build related issues when you try with bigtop. We
> >>> can
> >> discuss and prioritize this.. Will connect with you guys.
> >>
> >> On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
> >> iwasak...@oss.nttdata.co.jp> wrote:
> >>
> > -
> >>>
> >
> 
> >>>
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> 
>  This is for building hadoop-3.2.2 against zookeeper-3.4.14.
>  we do not see the issue usually since branch-3.2 uses
> > zooekeper-3.4.13,
>  while it would be harmless to add the exclusion even for
> >>> zooekeeper-3.4.13.
> >>>
> >>> I filed HADOOP-17849 for this.
> >>>
> >>> On 2021/08/16 12:02, Masatake Iwasaki wrote:
>  Thanks for bringing this up, Akira. Let me explain some
> > background.
> 
> 
> > -
> >>>
> >
> 
> >>>
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> 
>  This is for building hadoop-3.2.2 against zookeeper-3.4.14.
>  we do not see the issue usually since branch-3.2 uses
> > zooekeper-3.4.13,
>  while it would be harmless to add the exclusion even for
> >>> zooekeeper-3.4.13.
> 
> 
> >>

Re: Hadoop-3.2.3 Release Update

[Masatake Iwasaki]

Hi Brahma,

I would like to use 3.2.3 for next minor release of Bigtop.
https://issues.apache.org/jira/browse/BIGTOP-3605

Let me know if there is any blocker I can help.

Thanks,
Masatake Iwasaki

On 2022/01/12 4:41, Brahma Reddy Battula wrote:

Sorry, I was looking for some CVE’s fixes.will make it progress on this…


On Tue, 11 Jan 2022 at 5:58 PM, Wei-Chiu Chuang  wrote:


Is this still making progress?

On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula 
wrote:


Hi Akira,

Thanks for your email!!

I am evaluating the CVE’s which needs to go for this release..

Will update soon!!


On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka 

wrote:



Hi Brahma,

What is the release process going on? Is there any blocker for the RC?

-Akira

On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He 

wrote:



Hi Brahma,

The feature 'BPServiceActor processes commands from NameNode
asynchronously' has been ready for both branch-3.2 and branch-3.2.3.

While

cherry-picking there is only minor conflict, So I checked in

directly.

BTW,

run some unit tests and build pseudo cluster to verify, it seems to

work

fine.
FYI.

Regards,
- He Xiaoqiao

On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula <

bra...@apache.org


wrote:


Please go ahead. Let me know any help required on review.

On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He 

wrote:



Hi Brahma,

I plan to involve HDFS-14997 and related JIRAs if possible. I have
resolved the conflict and verified them locally.
It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
I would like to hear some more response that if we have enough time

to

wait for it to be ready.
Thanks.

Best Regards,
- He Xiaoqiao

On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He 

wrote:



Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3.

FYI.


On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula <

bra...@apache.org>

wrote:


Hi All,

Waiting for the following jira to commit to hadoop-3.2.3 , mostly

this

can
be done by this week,then I will try to create the RC next if

there

is

no
objection.

https://issues.apache.org/jira/browse/HDFS-15160



On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula <
bra...@apache.org>
wrote:


@Akira Ajisaka   and @Masatake Iwasaki

Looks all are build related issues when you try with bigtop. We

can

discuss and prioritize this.. Will connect with you guys.

On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:


-









https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff


This is for building hadoop-3.2.2 against zookeeper-3.4.14.
we do not see the issue usually since branch-3.2 uses

zooekeper-3.4.13,

while it would be harmless to add the exclusion even for

zooekeeper-3.4.13.

I filed HADOOP-17849 for this.

On 2021/08/16 12:02, Masatake Iwasaki wrote:

Thanks for bringing this up, Akira. Let me explain some

background.




-









https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff


This is for building hadoop-3.2.2 against zookeeper-3.4.14.
we do not see the issue usually since branch-3.2 uses

zooekeper-3.4.13,

while it would be harmless to add the exclusion even for

zooekeeper-3.4.13.




-









https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff

-









https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff

-









https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff


These are relevant to directory structure used by Bigtop

package.

If the fix does not break the tarball dist,
it would be nice to have these on Hadoop too.



-









https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff


This is for aarch64 and ppe64le lacking required phantomjs.
It is only acceptable for Bigtop not running tests of

YARN-UI2

on

packaging.

Hadoop needs the phantomjs for testing YARN-UI2.


On 2021/08/16 2:59, Brahma Reddy Battula wrote:

Thanks, I'll check them out.

On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka <

aajis...@apache.org>

wrote:



Thanks Brahma for cutting branch-3.2.3.

In Apache Bigtop, there are some patches applied to Hadoop

3.2.2.












https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop


In these patches, how about backporting the following

issues

to

branch-3.2 and branch-3.2.3?
- HADOOP-14922
- HADOOP-15939
- HADOOP-17569

In addition, there are some patches that don't have JIRA

issue

ID.

Maybe we need to create JIRAs and fix those.
-










https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff

-










https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-f

Re: Hadoop-3.2.3 Release Update

[Brahma Reddy Battula]

Sorry, I was looking for some CVE’s fixes.will make it progress on this…


On Tue, 11 Jan 2022 at 5:58 PM, Wei-Chiu Chuang  wrote:

> Is this still making progress?
>
> On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula 
> wrote:
>
> > Hi Akira,
> >
> > Thanks for your email!!
> >
> > I am evaluating the CVE’s which needs to go for this release..
> >
> > Will update soon!!
> >
> >
> > On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka 
> wrote:
> >
> > > Hi Brahma,
> > >
> > > What is the release process going on? Is there any blocker for the RC?
> > >
> > > -Akira
> > >
> > > On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He 
> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > The feature 'BPServiceActor processes commands from NameNode
> > > > asynchronously' has been ready for both branch-3.2 and branch-3.2.3.
> > > While
> > > > cherry-picking there is only minor conflict, So I checked in
> directly.
> > > BTW,
> > > > run some unit tests and build pseudo cluster to verify, it seems to
> > work
> > > > fine.
> > > > FYI.
> > > >
> > > > Regards,
> > > > - He Xiaoqiao
> > > >
> > > > On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula <
> > bra...@apache.org
> > > >
> > > > wrote:
> > > >
> > > >> Please go ahead. Let me know any help required on review.
> > > >>
> > > >> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He 
> > > wrote:
> > > >>
> > > >>> Hi Brahma,
> > > >>>
> > > >>> I plan to involve HDFS-14997 and related JIRAs if possible. I have
> > > >>> resolved the conflict and verified them locally.
> > > >>> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
> > > >>> I would like to hear some more response that if we have enough time
> > to
> > > >>> wait for it to be ready.
> > > >>> Thanks.
> > > >>>
> > > >>> Best Regards,
> > > >>> - He Xiaoqiao
> > > >>>
> > > >>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He 
> > > wrote:
> > > >>>
> > >  Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3.
> FYI.
> > > 
> > >  On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula <
> > > bra...@apache.org>
> > >  wrote:
> > > 
> > > > Hi All,
> > > >
> > > > Waiting for the following jira to commit to hadoop-3.2.3 , mostly
> > > this
> > > > can
> > > > be done by this week,then I will try to create the RC next if
> there
> > > is
> > > > no
> > > > objection.
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15160
> > > >
> > > >
> > > >
> > > > On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula <
> > > > bra...@apache.org>
> > > > wrote:
> > > >
> > > > > @Akira Ajisaka   and @Masatake Iwasaki
> > > > > 
> > > > > Looks all are build related issues when you try with bigtop. We
> > can
> > > > > discuss and prioritize this.. Will connect with you guys.
> > > > >
> > > > > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
> > > > > iwasak...@oss.nttdata.co.jp> wrote:
> > > > >
> > > > >> >> -
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> > > > >> >
> > > > >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > > > >> > we do not see the issue usually since branch-3.2 uses
> > > > zooekeper-3.4.13,
> > > > >> > while it would be harmless to add the exclusion even for
> > > > >> zooekeeper-3.4.13.
> > > > >>
> > > > >> I filed HADOOP-17849 for this.
> > > > >>
> > > > >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
> > > > >> > Thanks for bringing this up, Akira. Let me explain some
> > > > background.
> > > > >> >
> > > > >> >
> > > > >> >> -
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> > > > >> >
> > > > >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > > > >> > we do not see the issue usually since branch-3.2 uses
> > > > zooekeper-3.4.13,
> > > > >> > while it would be harmless to add the exclusion even for
> > > > >> zooekeeper-3.4.13.
> > > > >> >
> > > > >> >
> > > > >> >> -
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> > > > >> >> -
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> > > > >> >> -
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> > > > >> >
> > > > >> > These are relevant to directory structure used by Bigtop
> > > package.
> > > > >> > If the fix does not break the tarball dist,
> > > > >> > it would be nice to have these on Hadoop too.
> > > > >> >
> > > > >> >

Re: Hadoop-3.2.3 Release Update

[Wei-Chiu Chuang]

Is this still making progress?

On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula 
wrote:

> Hi Akira,
>
> Thanks for your email!!
>
> I am evaluating the CVE’s which needs to go for this release..
>
> Will update soon!!
>
>
> On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka  wrote:
>
> > Hi Brahma,
> >
> > What is the release process going on? Is there any blocker for the RC?
> >
> > -Akira
> >
> > On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He  wrote:
> >
> > > Hi Brahma,
> > >
> > > The feature 'BPServiceActor processes commands from NameNode
> > > asynchronously' has been ready for both branch-3.2 and branch-3.2.3.
> > While
> > > cherry-picking there is only minor conflict, So I checked in directly.
> > BTW,
> > > run some unit tests and build pseudo cluster to verify, it seems to
> work
> > > fine.
> > > FYI.
> > >
> > > Regards,
> > > - He Xiaoqiao
> > >
> > > On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula <
> bra...@apache.org
> > >
> > > wrote:
> > >
> > >> Please go ahead. Let me know any help required on review.
> > >>
> > >> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He 
> > wrote:
> > >>
> > >>> Hi Brahma,
> > >>>
> > >>> I plan to involve HDFS-14997 and related JIRAs if possible. I have
> > >>> resolved the conflict and verified them locally.
> > >>> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
> > >>> I would like to hear some more response that if we have enough time
> to
> > >>> wait for it to be ready.
> > >>> Thanks.
> > >>>
> > >>> Best Regards,
> > >>> - He Xiaoqiao
> > >>>
> > >>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He 
> > wrote:
> > >>>
> >  Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI.
> > 
> >  On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula <
> > bra...@apache.org>
> >  wrote:
> > 
> > > Hi All,
> > >
> > > Waiting for the following jira to commit to hadoop-3.2.3 , mostly
> > this
> > > can
> > > be done by this week,then I will try to create the RC next if there
> > is
> > > no
> > > objection.
> > >
> > > https://issues.apache.org/jira/browse/HDFS-15160
> > >
> > >
> > >
> > > On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula <
> > > bra...@apache.org>
> > > wrote:
> > >
> > > > @Akira Ajisaka   and @Masatake Iwasaki
> > > > 
> > > > Looks all are build related issues when you try with bigtop. We
> can
> > > > discuss and prioritize this.. Will connect with you guys.
> > > >
> > > > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
> > > > iwasak...@oss.nttdata.co.jp> wrote:
> > > >
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> > > >> >
> > > >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > > >> > we do not see the issue usually since branch-3.2 uses
> > > zooekeper-3.4.13,
> > > >> > while it would be harmless to add the exclusion even for
> > > >> zooekeeper-3.4.13.
> > > >>
> > > >> I filed HADOOP-17849 for this.
> > > >>
> > > >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
> > > >> > Thanks for bringing this up, Akira. Let me explain some
> > > background.
> > > >> >
> > > >> >
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> > > >> >
> > > >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > > >> > we do not see the issue usually since branch-3.2 uses
> > > zooekeper-3.4.13,
> > > >> > while it would be harmless to add the exclusion even for
> > > >> zooekeeper-3.4.13.
> > > >> >
> > > >> >
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> > > >> >
> > > >> > These are relevant to directory structure used by Bigtop
> > package.
> > > >> > If the fix does not break the tarball dist,
> > > >> > it would be nice to have these on Hadoop too.
> > > >> >
> > > >> >
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
> > > >> >
> > > >> > This is for aarch64 and ppe64le lacking required phantomjs.
> > > >> > It is only acceptable for Bigtop not running tests of YARN-UI2
> > on
> > > >> packaging.
> > > >> > Hadoop needs the phantomjs for testing YARN-UI2.
> > > >> >
> > >

Re: Hadoop-3.2.3 Release Update

[Wei-Chiu Chuang]

Hi to raise the awareness,
it looks like reverting the FoldedTreeSet HDFS-13671
 breaks TestBlockManager
in branch-3.2.  Branch-3.3 is good.

tracking jira: HDFS-16258 

On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula 
wrote:

> Hi Akira,
>
> Thanks for your email!!
>
> I am evaluating the CVE’s which needs to go for this release..
>
> Will update soon!!
>
>
> On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka  wrote:
>
> > Hi Brahma,
> >
> > What is the release process going on? Is there any blocker for the RC?
> >
> > -Akira
> >
> > On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He  wrote:
> >
> > > Hi Brahma,
> > >
> > > The feature 'BPServiceActor processes commands from NameNode
> > > asynchronously' has been ready for both branch-3.2 and branch-3.2.3.
> > While
> > > cherry-picking there is only minor conflict, So I checked in directly.
> > BTW,
> > > run some unit tests and build pseudo cluster to verify, it seems to
> work
> > > fine.
> > > FYI.
> > >
> > > Regards,
> > > - He Xiaoqiao
> > >
> > > On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula <
> bra...@apache.org
> > >
> > > wrote:
> > >
> > >> Please go ahead. Let me know any help required on review.
> > >>
> > >> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He 
> > wrote:
> > >>
> > >>> Hi Brahma,
> > >>>
> > >>> I plan to involve HDFS-14997 and related JIRAs if possible. I have
> > >>> resolved the conflict and verified them locally.
> > >>> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
> > >>> I would like to hear some more response that if we have enough time
> to
> > >>> wait for it to be ready.
> > >>> Thanks.
> > >>>
> > >>> Best Regards,
> > >>> - He Xiaoqiao
> > >>>
> > >>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He 
> > wrote:
> > >>>
> >  Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI.
> > 
> >  On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula <
> > bra...@apache.org>
> >  wrote:
> > 
> > > Hi All,
> > >
> > > Waiting for the following jira to commit to hadoop-3.2.3 , mostly
> > this
> > > can
> > > be done by this week,then I will try to create the RC next if there
> > is
> > > no
> > > objection.
> > >
> > > https://issues.apache.org/jira/browse/HDFS-15160
> > >
> > >
> > >
> > > On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula <
> > > bra...@apache.org>
> > > wrote:
> > >
> > > > @Akira Ajisaka   and @Masatake Iwasaki
> > > > 
> > > > Looks all are build related issues when you try with bigtop. We
> can
> > > > discuss and prioritize this.. Will connect with you guys.
> > > >
> > > > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
> > > > iwasak...@oss.nttdata.co.jp> wrote:
> > > >
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> > > >> >
> > > >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > > >> > we do not see the issue usually since branch-3.2 uses
> > > zooekeper-3.4.13,
> > > >> > while it would be harmless to add the exclusion even for
> > > >> zooekeeper-3.4.13.
> > > >>
> > > >> I filed HADOOP-17849 for this.
> > > >>
> > > >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
> > > >> > Thanks for bringing this up, Akira. Let me explain some
> > > background.
> > > >> >
> > > >> >
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> > > >> >
> > > >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > > >> > we do not see the issue usually since branch-3.2 uses
> > > zooekeper-3.4.13,
> > > >> > while it would be harmless to add the exclusion even for
> > > >> zooekeeper-3.4.13.
> > > >> >
> > > >> >
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> > > >> >
> > > >> > These are relevant to directory structure used by Bigtop
> > package.
> > > >> > If the fix does not break the tarball dist,
> > > >> > it would be nice to have these on Hadoop too.
> > > >> >
> > > >> >
> > > >> >> -
> > > >>
> > >
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
> > > >> >
> > > >> > This is for aarch64

Re: Hadoop-3.2.3 Release Update

[Brahma Reddy Battula]

Hi Akira,

Thanks for your email!!

I am evaluating the CVE’s which needs to go for this release..

Will update soon!!


On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka  wrote:

> Hi Brahma,
>
> What is the release process going on? Is there any blocker for the RC?
>
> -Akira
>
> On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He  wrote:
>
> > Hi Brahma,
> >
> > The feature 'BPServiceActor processes commands from NameNode
> > asynchronously' has been ready for both branch-3.2 and branch-3.2.3.
> While
> > cherry-picking there is only minor conflict, So I checked in directly.
> BTW,
> > run some unit tests and build pseudo cluster to verify, it seems to work
> > fine.
> > FYI.
> >
> > Regards,
> > - He Xiaoqiao
> >
> > On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula  >
> > wrote:
> >
> >> Please go ahead. Let me know any help required on review.
> >>
> >> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He 
> wrote:
> >>
> >>> Hi Brahma,
> >>>
> >>> I plan to involve HDFS-14997 and related JIRAs if possible. I have
> >>> resolved the conflict and verified them locally.
> >>> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
> >>> I would like to hear some more response that if we have enough time to
> >>> wait for it to be ready.
> >>> Thanks.
> >>>
> >>> Best Regards,
> >>> - He Xiaoqiao
> >>>
> >>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He 
> wrote:
> >>>
>  Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI.
> 
>  On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula <
> bra...@apache.org>
>  wrote:
> 
> > Hi All,
> >
> > Waiting for the following jira to commit to hadoop-3.2.3 , mostly
> this
> > can
> > be done by this week,then I will try to create the RC next if there
> is
> > no
> > objection.
> >
> > https://issues.apache.org/jira/browse/HDFS-15160
> >
> >
> >
> > On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula <
> > bra...@apache.org>
> > wrote:
> >
> > > @Akira Ajisaka   and @Masatake Iwasaki
> > > 
> > > Looks all are build related issues when you try with bigtop. We can
> > > discuss and prioritize this.. Will connect with you guys.
> > >
> > > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
> > > iwasak...@oss.nttdata.co.jp> wrote:
> > >
> > >> >> -
> > >>
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> > >> >
> > >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > >> > we do not see the issue usually since branch-3.2 uses
> > zooekeper-3.4.13,
> > >> > while it would be harmless to add the exclusion even for
> > >> zooekeeper-3.4.13.
> > >>
> > >> I filed HADOOP-17849 for this.
> > >>
> > >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
> > >> > Thanks for bringing this up, Akira. Let me explain some
> > background.
> > >> >
> > >> >
> > >> >> -
> > >>
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> > >> >
> > >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > >> > we do not see the issue usually since branch-3.2 uses
> > zooekeper-3.4.13,
> > >> > while it would be harmless to add the exclusion even for
> > >> zooekeeper-3.4.13.
> > >> >
> > >> >
> > >> >> -
> > >>
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> > >> >> -
> > >>
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> > >> >> -
> > >>
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> > >> >
> > >> > These are relevant to directory structure used by Bigtop
> package.
> > >> > If the fix does not break the tarball dist,
> > >> > it would be nice to have these on Hadoop too.
> > >> >
> > >> >
> > >> >> -
> > >>
> >
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
> > >> >
> > >> > This is for aarch64 and ppe64le lacking required phantomjs.
> > >> > It is only acceptable for Bigtop not running tests of YARN-UI2
> on
> > >> packaging.
> > >> > Hadoop needs the phantomjs for testing YARN-UI2.
> > >> >
> > >> >
> > >> > On 2021/08/16 2:59, Brahma Reddy Battula wrote:
> > >> >> Thanks, I'll check them out.
> > >> >>
> > >> >> On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka <
> > aajis...@apache.org>
> > >> wrote:
> > >> >>
> > >> >>> Thanks Brahma for cutting branch-3.2.3.
> > >> >>>
> > >> >>> In Apache Bigtop, there are some patches applied to Hadoop
> > 3.2.2.
> > >> >>>
> 

Re: Hadoop-3.2.3 Release Update

[Akira Ajisaka]

Hi Brahma,

What is the release process going on? Is there any blocker for the RC?

-Akira

On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He  wrote:

> Hi Brahma,
>
> The feature 'BPServiceActor processes commands from NameNode
> asynchronously' has been ready for both branch-3.2 and branch-3.2.3. While
> cherry-picking there is only minor conflict, So I checked in directly. BTW,
> run some unit tests and build pseudo cluster to verify, it seems to work
> fine.
> FYI.
>
> Regards,
> - He Xiaoqiao
>
> On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula 
> wrote:
>
>> Please go ahead. Let me know any help required on review.
>>
>> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He  wrote:
>>
>>> Hi Brahma,
>>>
>>> I plan to involve HDFS-14997 and related JIRAs if possible. I have
>>> resolved the conflict and verified them locally.
>>> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
>>> I would like to hear some more response that if we have enough time to
>>> wait for it to be ready.
>>> Thanks.
>>>
>>> Best Regards,
>>> - He Xiaoqiao
>>>
>>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He  wrote:
>>>
 Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI.

 On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula 
 wrote:

> Hi All,
>
> Waiting for the following jira to commit to hadoop-3.2.3 , mostly this
> can
> be done by this week,then I will try to create the RC next if there is
> no
> objection.
>
> https://issues.apache.org/jira/browse/HDFS-15160
>
>
>
> On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula <
> bra...@apache.org>
> wrote:
>
> > @Akira Ajisaka   and @Masatake Iwasaki
> > 
> > Looks all are build related issues when you try with bigtop. We can
> > discuss and prioritize this.. Will connect with you guys.
> >
> > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
> > iwasak...@oss.nttdata.co.jp> wrote:
> >
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> >> >
> >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> >> > we do not see the issue usually since branch-3.2 uses
> zooekeper-3.4.13,
> >> > while it would be harmless to add the exclusion even for
> >> zooekeeper-3.4.13.
> >>
> >> I filed HADOOP-17849 for this.
> >>
> >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
> >> > Thanks for bringing this up, Akira. Let me explain some
> background.
> >> >
> >> >
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> >> >
> >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> >> > we do not see the issue usually since branch-3.2 uses
> zooekeper-3.4.13,
> >> > while it would be harmless to add the exclusion even for
> >> zooekeeper-3.4.13.
> >> >
> >> >
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> >> >
> >> > These are relevant to directory structure used by Bigtop package.
> >> > If the fix does not break the tarball dist,
> >> > it would be nice to have these on Hadoop too.
> >> >
> >> >
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
> >> >
> >> > This is for aarch64 and ppe64le lacking required phantomjs.
> >> > It is only acceptable for Bigtop not running tests of YARN-UI2 on
> >> packaging.
> >> > Hadoop needs the phantomjs for testing YARN-UI2.
> >> >
> >> >
> >> > On 2021/08/16 2:59, Brahma Reddy Battula wrote:
> >> >> Thanks, I'll check them out.
> >> >>
> >> >> On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka <
> aajis...@apache.org>
> >> wrote:
> >> >>
> >> >>> Thanks Brahma for cutting branch-3.2.3.
> >> >>>
> >> >>> In Apache Bigtop, there are some patches applied to Hadoop
> 3.2.2.
> >> >>>
> >> >>>
> >>
> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop
> >> >>>
> >> >>> In these patches, how about backporting the following issues to
> >> >>> branch-3.2 and branch-3.2.3?
> >> >>> - HADOOP-14922
> >> >>> - HADOOP-15939
> >> >>> - HADOOP-17569
> >> >>>
> >> >>> In addition, there are some patches that don't have JIRA issue
> ID.
> >> >>> Maybe we need to create JIRAs and fix those.
> >> >>> -

Re: Hadoop-3.2.3 Release Update

[Xiaoqiao He]

Hi Brahma,

The feature 'BPServiceActor processes commands from NameNode
asynchronously' has been ready for both branch-3.2 and branch-3.2.3. While
cherry-picking there is only minor conflict, So I checked in directly. BTW,
run some unit tests and build pseudo cluster to verify, it seems to work
fine.
FYI.

Regards,
- He Xiaoqiao

On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula 
wrote:

> Please go ahead. Let me know any help required on review.
>
> On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He  wrote:
>
>> Hi Brahma,
>>
>> I plan to involve HDFS-14997 and related JIRAs if possible. I have
>> resolved the conflict and verified them locally.
>> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
>> I would like to hear some more response that if we have enough time to
>> wait for it to be ready.
>> Thanks.
>>
>> Best Regards,
>> - He Xiaoqiao
>>
>> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He  wrote:
>>
>>> Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI.
>>>
>>> On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula 
>>> wrote:
>>>
 Hi All,

 Waiting for the following jira to commit to hadoop-3.2.3 , mostly this
 can
 be done by this week,then I will try to create the RC next if there is
 no
 objection.

 https://issues.apache.org/jira/browse/HDFS-15160



 On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula >>> >
 wrote:

 > @Akira Ajisaka   and @Masatake Iwasaki
 > 
 > Looks all are build related issues when you try with bigtop. We can
 > discuss and prioritize this.. Will connect with you guys.
 >
 > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
 > iwasak...@oss.nttdata.co.jp> wrote:
 >
 >> >> -
 >>
 https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
 >> >
 >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
 >> > we do not see the issue usually since branch-3.2 uses
 zooekeper-3.4.13,
 >> > while it would be harmless to add the exclusion even for
 >> zooekeeper-3.4.13.
 >>
 >> I filed HADOOP-17849 for this.
 >>
 >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
 >> > Thanks for bringing this up, Akira. Let me explain some background.
 >> >
 >> >
 >> >> -
 >>
 https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
 >> >
 >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
 >> > we do not see the issue usually since branch-3.2 uses
 zooekeper-3.4.13,
 >> > while it would be harmless to add the exclusion even for
 >> zooekeeper-3.4.13.
 >> >
 >> >
 >> >> -
 >>
 https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
 >> >> -
 >>
 https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
 >> >> -
 >>
 https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
 >> >
 >> > These are relevant to directory structure used by Bigtop package.
 >> > If the fix does not break the tarball dist,
 >> > it would be nice to have these on Hadoop too.
 >> >
 >> >
 >> >> -
 >>
 https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
 >> >
 >> > This is for aarch64 and ppe64le lacking required phantomjs.
 >> > It is only acceptable for Bigtop not running tests of YARN-UI2 on
 >> packaging.
 >> > Hadoop needs the phantomjs for testing YARN-UI2.
 >> >
 >> >
 >> > On 2021/08/16 2:59, Brahma Reddy Battula wrote:
 >> >> Thanks, I'll check them out.
 >> >>
 >> >> On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka <
 aajis...@apache.org>
 >> wrote:
 >> >>
 >> >>> Thanks Brahma for cutting branch-3.2.3.
 >> >>>
 >> >>> In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.
 >> >>>
 >> >>>
 >>
 https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop
 >> >>>
 >> >>> In these patches, how about backporting the following issues to
 >> >>> branch-3.2 and branch-3.2.3?
 >> >>> - HADOOP-14922
 >> >>> - HADOOP-15939
 >> >>> - HADOOP-17569
 >> >>>
 >> >>> In addition, there are some patches that don't have JIRA issue
 ID.
 >> >>> Maybe we need to create JIRAs and fix those.
 >> >>> -
 >> >>>
 >>
 https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
 >> >>> -
 >> >>>
 >>
 https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
 >> >>> -
 >> >>>
 

Re: Hadoop-3.2.3 Release Update

[Brahma Reddy Battula]

Please go ahead. Let me know any help required on review.

On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He  wrote:

> Hi Brahma,
>
> I plan to involve HDFS-14997 and related JIRAs if possible. I have
> resolved the conflict and verified them locally.
> It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
> I would like to hear some more response that if we have enough time to
> wait for it to be ready.
> Thanks.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He  wrote:
>
>> Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI.
>>
>> On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula 
>> wrote:
>>
>>> Hi All,
>>>
>>> Waiting for the following jira to commit to hadoop-3.2.3 , mostly this
>>> can
>>> be done by this week,then I will try to create the RC next if there is no
>>> objection.
>>>
>>> https://issues.apache.org/jira/browse/HDFS-15160
>>>
>>>
>>>
>>> On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula 
>>> wrote:
>>>
>>> > @Akira Ajisaka   and @Masatake Iwasaki
>>> > 
>>> > Looks all are build related issues when you try with bigtop. We can
>>> > discuss and prioritize this.. Will connect with you guys.
>>> >
>>> > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
>>> > iwasak...@oss.nttdata.co.jp> wrote:
>>> >
>>> >> >> -
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>>> >> >
>>> >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
>>> >> > we do not see the issue usually since branch-3.2 uses
>>> zooekeper-3.4.13,
>>> >> > while it would be harmless to add the exclusion even for
>>> >> zooekeeper-3.4.13.
>>> >>
>>> >> I filed HADOOP-17849 for this.
>>> >>
>>> >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
>>> >> > Thanks for bringing this up, Akira. Let me explain some background.
>>> >> >
>>> >> >
>>> >> >> -
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>>> >> >
>>> >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
>>> >> > we do not see the issue usually since branch-3.2 uses
>>> zooekeper-3.4.13,
>>> >> > while it would be harmless to add the exclusion even for
>>> >> zooekeeper-3.4.13.
>>> >> >
>>> >> >
>>> >> >> -
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
>>> >> >> -
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
>>> >> >> -
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
>>> >> >
>>> >> > These are relevant to directory structure used by Bigtop package.
>>> >> > If the fix does not break the tarball dist,
>>> >> > it would be nice to have these on Hadoop too.
>>> >> >
>>> >> >
>>> >> >> -
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
>>> >> >
>>> >> > This is for aarch64 and ppe64le lacking required phantomjs.
>>> >> > It is only acceptable for Bigtop not running tests of YARN-UI2 on
>>> >> packaging.
>>> >> > Hadoop needs the phantomjs for testing YARN-UI2.
>>> >> >
>>> >> >
>>> >> > On 2021/08/16 2:59, Brahma Reddy Battula wrote:
>>> >> >> Thanks, I'll check them out.
>>> >> >>
>>> >> >> On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka <
>>> aajis...@apache.org>
>>> >> wrote:
>>> >> >>
>>> >> >>> Thanks Brahma for cutting branch-3.2.3.
>>> >> >>>
>>> >> >>> In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.
>>> >> >>>
>>> >> >>>
>>> >>
>>> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop
>>> >> >>>
>>> >> >>> In these patches, how about backporting the following issues to
>>> >> >>> branch-3.2 and branch-3.2.3?
>>> >> >>> - HADOOP-14922
>>> >> >>> - HADOOP-15939
>>> >> >>> - HADOOP-17569
>>> >> >>>
>>> >> >>> In addition, there are some patches that don't have JIRA issue ID.
>>> >> >>> Maybe we need to create JIRAs and fix those.
>>> >> >>> -
>>> >> >>>
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>>> >> >>> -
>>> >> >>>
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
>>> >> >>> -
>>> >> >>>
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
>>> >> >>> -
>>> >> >>>
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
>>> >> >>> -
>>> >> >>>
>>> >>
>>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
>>> >> >>>
>>> >> >>> Thanks and regards,
>>> >> >>> Akira
>>> >> >>>
>>> >> >>> On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He <
>>>

Re: Hadoop-3.2.3 Release Update

[Xiaoqiao He]

Hi Brahma,

I plan to involve HDFS-14997 and related JIRAs if possible. I have resolved
the conflict and verified them locally.
It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113.
I would like to hear some more response that if we have enough time to wait
for it to be ready.
Thanks.

Best Regards,
- He Xiaoqiao

On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He  wrote:

> Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI.
>
> On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula 
> wrote:
>
>> Hi All,
>>
>> Waiting for the following jira to commit to hadoop-3.2.3 , mostly this can
>> be done by this week,then I will try to create the RC next if there is no
>> objection.
>>
>> https://issues.apache.org/jira/browse/HDFS-15160
>>
>>
>>
>> On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula 
>> wrote:
>>
>> > @Akira Ajisaka   and @Masatake Iwasaki
>> > 
>> > Looks all are build related issues when you try with bigtop. We can
>> > discuss and prioritize this.. Will connect with you guys.
>> >
>> > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
>> > iwasak...@oss.nttdata.co.jp> wrote:
>> >
>> >> >> -
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>> >> >
>> >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
>> >> > we do not see the issue usually since branch-3.2 uses
>> zooekeper-3.4.13,
>> >> > while it would be harmless to add the exclusion even for
>> >> zooekeeper-3.4.13.
>> >>
>> >> I filed HADOOP-17849 for this.
>> >>
>> >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
>> >> > Thanks for bringing this up, Akira. Let me explain some background.
>> >> >
>> >> >
>> >> >> -
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>> >> >
>> >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
>> >> > we do not see the issue usually since branch-3.2 uses
>> zooekeper-3.4.13,
>> >> > while it would be harmless to add the exclusion even for
>> >> zooekeeper-3.4.13.
>> >> >
>> >> >
>> >> >> -
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
>> >> >> -
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
>> >> >> -
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
>> >> >
>> >> > These are relevant to directory structure used by Bigtop package.
>> >> > If the fix does not break the tarball dist,
>> >> > it would be nice to have these on Hadoop too.
>> >> >
>> >> >
>> >> >> -
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
>> >> >
>> >> > This is for aarch64 and ppe64le lacking required phantomjs.
>> >> > It is only acceptable for Bigtop not running tests of YARN-UI2 on
>> >> packaging.
>> >> > Hadoop needs the phantomjs for testing YARN-UI2.
>> >> >
>> >> >
>> >> > On 2021/08/16 2:59, Brahma Reddy Battula wrote:
>> >> >> Thanks, I'll check them out.
>> >> >>
>> >> >> On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka > >
>> >> wrote:
>> >> >>
>> >> >>> Thanks Brahma for cutting branch-3.2.3.
>> >> >>>
>> >> >>> In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.
>> >> >>>
>> >> >>>
>> >>
>> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop
>> >> >>>
>> >> >>> In these patches, how about backporting the following issues to
>> >> >>> branch-3.2 and branch-3.2.3?
>> >> >>> - HADOOP-14922
>> >> >>> - HADOOP-15939
>> >> >>> - HADOOP-17569
>> >> >>>
>> >> >>> In addition, there are some patches that don't have JIRA issue ID.
>> >> >>> Maybe we need to create JIRAs and fix those.
>> >> >>> -
>> >> >>>
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>> >> >>> -
>> >> >>>
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
>> >> >>> -
>> >> >>>
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
>> >> >>> -
>> >> >>>
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
>> >> >>> -
>> >> >>>
>> >>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
>> >> >>>
>> >> >>> Thanks and regards,
>> >> >>> Akira
>> >> >>>
>> >> >>> On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He <
>> hexiaoq...@apache.org>
>> >> >>> wrote:
>> >> 
>> >>  Thanks Brahma for initiating this and making hadoop-3.2.3 release
>> >> happen.
>> >> 
>> >>  I would like to validate the HBase project (both the latest
>> release
>> >> and
>> >>  trunk branch).
>> >>  Chao S

Re: Hadoop-3.2.3 Release Update

[Xiaoqiao He]

Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI.

On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula 
wrote:

> Hi All,
>
> Waiting for the following jira to commit to hadoop-3.2.3 , mostly this can
> be done by this week,then I will try to create the RC next if there is no
> objection.
>
> https://issues.apache.org/jira/browse/HDFS-15160
>
>
>
> On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula 
> wrote:
>
> > @Akira Ajisaka   and @Masatake Iwasaki
> > 
> > Looks all are build related issues when you try with bigtop. We can
> > discuss and prioritize this.. Will connect with you guys.
> >
> > On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
> > iwasak...@oss.nttdata.co.jp> wrote:
> >
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> >> >
> >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> >> > we do not see the issue usually since branch-3.2 uses
> zooekeper-3.4.13,
> >> > while it would be harmless to add the exclusion even for
> >> zooekeeper-3.4.13.
> >>
> >> I filed HADOOP-17849 for this.
> >>
> >> On 2021/08/16 12:02, Masatake Iwasaki wrote:
> >> > Thanks for bringing this up, Akira. Let me explain some background.
> >> >
> >> >
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> >> >
> >> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> >> > we do not see the issue usually since branch-3.2 uses
> zooekeper-3.4.13,
> >> > while it would be harmless to add the exclusion even for
> >> zooekeeper-3.4.13.
> >> >
> >> >
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> >> >
> >> > These are relevant to directory structure used by Bigtop package.
> >> > If the fix does not break the tarball dist,
> >> > it would be nice to have these on Hadoop too.
> >> >
> >> >
> >> >> -
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
> >> >
> >> > This is for aarch64 and ppe64le lacking required phantomjs.
> >> > It is only acceptable for Bigtop not running tests of YARN-UI2 on
> >> packaging.
> >> > Hadoop needs the phantomjs for testing YARN-UI2.
> >> >
> >> >
> >> > On 2021/08/16 2:59, Brahma Reddy Battula wrote:
> >> >> Thanks, I'll check them out.
> >> >>
> >> >> On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka 
> >> wrote:
> >> >>
> >> >>> Thanks Brahma for cutting branch-3.2.3.
> >> >>>
> >> >>> In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.
> >> >>>
> >> >>>
> >>
> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop
> >> >>>
> >> >>> In these patches, how about backporting the following issues to
> >> >>> branch-3.2 and branch-3.2.3?
> >> >>> - HADOOP-14922
> >> >>> - HADOOP-15939
> >> >>> - HADOOP-17569
> >> >>>
> >> >>> In addition, there are some patches that don't have JIRA issue ID.
> >> >>> Maybe we need to create JIRAs and fix those.
> >> >>> -
> >> >>>
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> >> >>> -
> >> >>>
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> >> >>> -
> >> >>>
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> >> >>> -
> >> >>>
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> >> >>> -
> >> >>>
> >>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
> >> >>>
> >> >>> Thanks and regards,
> >> >>> Akira
> >> >>>
> >> >>> On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He  >
> >> >>> wrote:
> >> 
> >>  Thanks Brahma for initiating this and making hadoop-3.2.3 release
> >> happen.
> >> 
> >>  I would like to validate the HBase project (both the latest release
> >> and
> >>  trunk branch).
> >>  Chao Sun will validate the Spark Project (Got in touch with Chao
> >> >>> already).
> >>  once RC is out.
> >> 
> >>  Thanks and Regards,
> >>  - He Xiaoqiao
> >> 
> >> 
> >>  On Tue, Aug 10, 2021 at 5:54 PM Brahma Reddy Battula <
> >> bra...@apache.org>
> >>  wrote:
> >> 
> >> > Hi All,
> >> >
> >> > I cut branch-3.2.3 and it is ready for release. Please commit to
> >> > branch-3.2.3 if any critical/blocker issues need to go.
> >> >
> >> > *This time I am thinking of having downstream 

Re: Hadoop-3.2.3 Release Update

[Brahma Reddy Battula]

Hi All,

Waiting for the following jira to commit to hadoop-3.2.3 , mostly this can
be done by this week,then I will try to create the RC next if there is no
objection.

https://issues.apache.org/jira/browse/HDFS-15160



On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula 
wrote:

> @Akira Ajisaka   and @Masatake Iwasaki
> 
> Looks all are build related issues when you try with bigtop. We can
> discuss and prioritize this.. Will connect with you guys.
>
> On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
> iwasak...@oss.nttdata.co.jp> wrote:
>
>> >> -
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>> >
>> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
>> > we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13,
>> > while it would be harmless to add the exclusion even for
>> zooekeeper-3.4.13.
>>
>> I filed HADOOP-17849 for this.
>>
>> On 2021/08/16 12:02, Masatake Iwasaki wrote:
>> > Thanks for bringing this up, Akira. Let me explain some background.
>> >
>> >
>> >> -
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>> >
>> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
>> > we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13,
>> > while it would be harmless to add the exclusion even for
>> zooekeeper-3.4.13.
>> >
>> >
>> >> -
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
>> >> -
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
>> >> -
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
>> >
>> > These are relevant to directory structure used by Bigtop package.
>> > If the fix does not break the tarball dist,
>> > it would be nice to have these on Hadoop too.
>> >
>> >
>> >> -
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
>> >
>> > This is for aarch64 and ppe64le lacking required phantomjs.
>> > It is only acceptable for Bigtop not running tests of YARN-UI2 on
>> packaging.
>> > Hadoop needs the phantomjs for testing YARN-UI2.
>> >
>> >
>> > On 2021/08/16 2:59, Brahma Reddy Battula wrote:
>> >> Thanks, I'll check them out.
>> >>
>> >> On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka 
>> wrote:
>> >>
>> >>> Thanks Brahma for cutting branch-3.2.3.
>> >>>
>> >>> In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.
>> >>>
>> >>>
>> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop
>> >>>
>> >>> In these patches, how about backporting the following issues to
>> >>> branch-3.2 and branch-3.2.3?
>> >>> - HADOOP-14922
>> >>> - HADOOP-15939
>> >>> - HADOOP-17569
>> >>>
>> >>> In addition, there are some patches that don't have JIRA issue ID.
>> >>> Maybe we need to create JIRAs and fix those.
>> >>> -
>> >>>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
>> >>> -
>> >>>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
>> >>> -
>> >>>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
>> >>> -
>> >>>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
>> >>> -
>> >>>
>> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
>> >>>
>> >>> Thanks and regards,
>> >>> Akira
>> >>>
>> >>> On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He 
>> >>> wrote:
>> 
>>  Thanks Brahma for initiating this and making hadoop-3.2.3 release
>> happen.
>> 
>>  I would like to validate the HBase project (both the latest release
>> and
>>  trunk branch).
>>  Chao Sun will validate the Spark Project (Got in touch with Chao
>> >>> already).
>>  once RC is out.
>> 
>>  Thanks and Regards,
>>  - He Xiaoqiao
>> 
>> 
>>  On Tue, Aug 10, 2021 at 5:54 PM Brahma Reddy Battula <
>> bra...@apache.org>
>>  wrote:
>> 
>> > Hi All,
>> >
>> > I cut branch-3.2.3 and it is ready for release. Please commit to
>> > branch-3.2.3 if any critical/blocker issues need to go.
>> >
>> > *This time I am thinking of having downstream projects and
>> companies'
>> > voices,let's know how this can go. *
>> >
>> > - Planning to check with downstream projects like Spark,HBase
>> and
>> >>> Hive
>> > if they can help on validation(Or running their UT on this
>> branch)
>> > - Collecting info from Companies who are already deployed and
>> using
>> >>> the
>> > branch-3.2
>> >
>> >
>> > so that we can 

Re: Hadoop-3.2.3 Release Update

[Brahma Reddy Battula]

@Akira Ajisaka   and @Masatake Iwasaki

Looks all are build related issues when you try with bigtop. We can discuss
and prioritize this.. Will connect with you guys.

On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki <
iwasak...@oss.nttdata.co.jp> wrote:

> >> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> >
> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13,
> > while it would be harmless to add the exclusion even for
> zooekeeper-3.4.13.
>
> I filed HADOOP-17849 for this.
>
> On 2021/08/16 12:02, Masatake Iwasaki wrote:
> > Thanks for bringing this up, Akira. Let me explain some background.
> >
> >
> >> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> >
> > This is for building hadoop-3.2.2 against zookeeper-3.4.14.
> > we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13,
> > while it would be harmless to add the exclusion even for
> zooekeeper-3.4.13.
> >
> >
> >> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> >> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> >> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> >
> > These are relevant to directory structure used by Bigtop package.
> > If the fix does not break the tarball dist,
> > it would be nice to have these on Hadoop too.
> >
> >
> >> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
> >
> > This is for aarch64 and ppe64le lacking required phantomjs.
> > It is only acceptable for Bigtop not running tests of YARN-UI2 on
> packaging.
> > Hadoop needs the phantomjs for testing YARN-UI2.
> >
> >
> > On 2021/08/16 2:59, Brahma Reddy Battula wrote:
> >> Thanks, I'll check them out.
> >>
> >> On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka 
> wrote:
> >>
> >>> Thanks Brahma for cutting branch-3.2.3.
> >>>
> >>> In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.
> >>>
> >>>
> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop
> >>>
> >>> In these patches, how about backporting the following issues to
> >>> branch-3.2 and branch-3.2.3?
> >>> - HADOOP-14922
> >>> - HADOOP-15939
> >>> - HADOOP-17569
> >>>
> >>> In addition, there are some patches that don't have JIRA issue ID.
> >>> Maybe we need to create JIRAs and fix those.
> >>> -
> >>>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> >>> -
> >>>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> >>> -
> >>>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> >>> -
> >>>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> >>> -
> >>>
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
> >>>
> >>> Thanks and regards,
> >>> Akira
> >>>
> >>> On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He 
> >>> wrote:
> 
>  Thanks Brahma for initiating this and making hadoop-3.2.3 release
> happen.
> 
>  I would like to validate the HBase project (both the latest release
> and
>  trunk branch).
>  Chao Sun will validate the Spark Project (Got in touch with Chao
> >>> already).
>  once RC is out.
> 
>  Thanks and Regards,
>  - He Xiaoqiao
> 
> 
>  On Tue, Aug 10, 2021 at 5:54 PM Brahma Reddy Battula <
> bra...@apache.org>
>  wrote:
> 
> > Hi All,
> >
> > I cut branch-3.2.3 and it is ready for release. Please commit to
> > branch-3.2.3 if any critical/blocker issues need to go.
> >
> > *This time I am thinking of having downstream projects and companies'
> > voices,let's know how this can go. *
> >
> > - Planning to check with downstream projects like Spark,HBase and
> >>> Hive
> > if they can help on validation(Or running their UT on this
> branch)
> > - Collecting info from Companies who are already deployed and
> using
> >>> the
> > branch-3.2
> >
> >
> > so that we can make a more stable release on 3.2 (so that features
> >>> released
> > on this branch impact can be known) ,please let me know anybody from
> >>> these
> > communities who can help on this.
> >
> >
> > Planning to create RC by this month end. Any suggestions are welcome.
> >
> >
> >
> > --Brahma Reddy Battula
> >
> >>>
> >
> > ---

Re: Hadoop-3.2.3 Release Update

[Masatake Iwasaki]

- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff


This is for building hadoop-3.2.2 against zookeeper-3.4.14.
we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13,
while it would be harmless to add the exclusion even for zooekeeper-3.4.13.


I filed HADOOP-17849 for this.

On 2021/08/16 12:02, Masatake Iwasaki wrote:

Thanks for bringing this up, Akira. Let me explain some background.



- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff


This is for building hadoop-3.2.2 against zookeeper-3.4.14.
we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13,
while it would be harmless to add the exclusion even for zooekeeper-3.4.13.



- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff


These are relevant to directory structure used by Bigtop package.
If the fix does not break the tarball dist,
it would be nice to have these on Hadoop too.



- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff


This is for aarch64 and ppe64le lacking required phantomjs.
It is only acceptable for Bigtop not running tests of YARN-UI2 on packaging.
Hadoop needs the phantomjs for testing YARN-UI2.


On 2021/08/16 2:59, Brahma Reddy Battula wrote:

Thanks, I'll check them out.

On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka  wrote:


Thanks Brahma for cutting branch-3.2.3.

In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.

https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop

In these patches, how about backporting the following issues to
branch-3.2 and branch-3.2.3?
- HADOOP-14922
- HADOOP-15939
- HADOOP-17569

In addition, there are some patches that don't have JIRA issue ID.
Maybe we need to create JIRAs and fix those.
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff

Thanks and regards,
Akira

On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He 
wrote:


Thanks Brahma for initiating this and making hadoop-3.2.3 release happen.

I would like to validate the HBase project (both the latest release and
trunk branch).
Chao Sun will validate the Spark Project (Got in touch with Chao

already).

once RC is out.

Thanks and Regards,
- He Xiaoqiao


On Tue, Aug 10, 2021 at 5:54 PM Brahma Reddy Battula 
wrote:


Hi All,

I cut branch-3.2.3 and it is ready for release. Please commit to
branch-3.2.3 if any critical/blocker issues need to go.

*This time I am thinking of having downstream projects and companies'
voices,let's know how this can go. *

    - Planning to check with downstream projects like Spark,HBase and

Hive

    if they can help on validation(Or running their UT on this branch)
    - Collecting info from Companies who are already deployed and using

the

    branch-3.2


so that we can make a more stable release on 3.2 (so that features

released

on this branch impact can be known) ,please let me know anybody from

these

communities who can help on this.


Planning to create RC by this month end. Any suggestions are welcome.



--Brahma Reddy Battula





-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: Hadoop-3.2.3 Release Update

[Masatake Iwasaki]

Thanks for bringing this up, Akira. Let me explain some background.



- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff


This is for building hadoop-3.2.2 against zookeeper-3.4.14.
we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13,
while it would be harmless to add the exclusion even for zooekeeper-3.4.13.



- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff


These are relevant to directory structure used by Bigtop package.
If the fix does not break the tarball dist,
it would be nice to have these on Hadoop too.



- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff


This is for aarch64 and ppe64le lacking required phantomjs.
It is only acceptable for Bigtop not running tests of YARN-UI2 on packaging.
Hadoop needs the phantomjs for testing YARN-UI2.


On 2021/08/16 2:59, Brahma Reddy Battula wrote:

Thanks, I'll check them out.

On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka  wrote:


Thanks Brahma for cutting branch-3.2.3.

In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.

https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop

In these patches, how about backporting the following issues to
branch-3.2 and branch-3.2.3?
- HADOOP-14922
- HADOOP-15939
- HADOOP-17569

In addition, there are some patches that don't have JIRA issue ID.
Maybe we need to create JIRAs and fix those.
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
-
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff

Thanks and regards,
Akira

On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He 
wrote:


Thanks Brahma for initiating this and making hadoop-3.2.3 release happen.

I would like to validate the HBase project (both the latest release and
trunk branch).
Chao Sun will validate the Spark Project (Got in touch with Chao

already).

once RC is out.

Thanks and Regards,
- He Xiaoqiao


On Tue, Aug 10, 2021 at 5:54 PM Brahma Reddy Battula 
wrote:


Hi All,

I cut branch-3.2.3 and it is ready for release. Please commit to
branch-3.2.3 if any critical/blocker issues need to go.

*This time I am thinking of having downstream projects and companies'
voices,let's know how this can go. *

- Planning to check with downstream projects like Spark,HBase and

Hive

if they can help on validation(Or running their UT on this branch)
- Collecting info from Companies who are already deployed and using

the

branch-3.2


so that we can make a more stable release on 3.2 (so that features

released

on this branch impact can be known) ,please let me know anybody from

these

communities who can help on this.


Planning to create RC by this month end. Any suggestions are welcome.



--Brahma Reddy Battula





-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: Hadoop-3.2.3 Release Update

[Brahma Reddy Battula]

Thanks, I'll check them out.

On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka  wrote:

> Thanks Brahma for cutting branch-3.2.3.
>
> In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.
>
> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop
>
> In these patches, how about backporting the following issues to
> branch-3.2 and branch-3.2.3?
> - HADOOP-14922
> - HADOOP-15939
> - HADOOP-17569
>
> In addition, there are some patches that don't have JIRA issue ID.
> Maybe we need to create JIRAs and fix those.
> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
> -
> https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff
>
> Thanks and regards,
> Akira
>
> On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He 
> wrote:
> >
> > Thanks Brahma for initiating this and making hadoop-3.2.3 release happen.
> >
> > I would like to validate the HBase project (both the latest release and
> > trunk branch).
> > Chao Sun will validate the Spark Project (Got in touch with Chao
> already).
> > once RC is out.
> >
> > Thanks and Regards,
> > - He Xiaoqiao
> >
> >
> > On Tue, Aug 10, 2021 at 5:54 PM Brahma Reddy Battula 
> > wrote:
> >
> > > Hi All,
> > >
> > > I cut branch-3.2.3 and it is ready for release. Please commit to
> > > branch-3.2.3 if any critical/blocker issues need to go.
> > >
> > > *This time I am thinking of having downstream projects and companies'
> > > voices,let's know how this can go. *
> > >
> > >- Planning to check with downstream projects like Spark,HBase and
> Hive
> > >if they can help on validation(Or running their UT on this branch)
> > >- Collecting info from Companies who are already deployed and using
> the
> > >branch-3.2
> > >
> > >
> > > so that we can make a more stable release on 3.2 (so that features
> released
> > > on this branch impact can be known) ,please let me know anybody from
> these
> > > communities who can help on this.
> > >
> > >
> > > Planning to create RC by this month end. Any suggestions are welcome.
> > >
> > >
> > >
> > > --Brahma Reddy Battula
> > >
>
-- 



--Brahma Reddy Battula

Re: Hadoop-3.2.3 Release Update

[Akira Ajisaka]

Thanks Brahma for cutting branch-3.2.3.

In Apache Bigtop, there are some patches applied to Hadoop 3.2.2.
https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop

In these patches, how about backporting the following issues to
branch-3.2 and branch-3.2.3?
- HADOOP-14922
- HADOOP-15939
- HADOOP-17569

In addition, there are some patches that don't have JIRA issue ID.
Maybe we need to create JIRAs and fix those.
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff
- 
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff

Thanks and regards,
Akira

On Wed, Aug 11, 2021 at 12:31 PM Xiaoqiao He  wrote:
>
> Thanks Brahma for initiating this and making hadoop-3.2.3 release happen.
>
> I would like to validate the HBase project (both the latest release and
> trunk branch).
> Chao Sun will validate the Spark Project (Got in touch with Chao already).
> once RC is out.
>
> Thanks and Regards,
> - He Xiaoqiao
>
>
> On Tue, Aug 10, 2021 at 5:54 PM Brahma Reddy Battula 
> wrote:
>
> > Hi All,
> >
> > I cut branch-3.2.3 and it is ready for release. Please commit to
> > branch-3.2.3 if any critical/blocker issues need to go.
> >
> > *This time I am thinking of having downstream projects and companies'
> > voices,let's know how this can go. *
> >
> >- Planning to check with downstream projects like Spark,HBase and Hive
> >if they can help on validation(Or running their UT on this branch)
> >- Collecting info from Companies who are already deployed and using the
> >branch-3.2
> >
> >
> > so that we can make a more stable release on 3.2 (so that features released
> > on this branch impact can be known) ,please let me know anybody from these
> > communities who can help on this.
> >
> >
> > Planning to create RC by this month end. Any suggestions are welcome.
> >
> >
> >
> > --Brahma Reddy Battula
> >

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: Hadoop-3.2.3 Release Update

[Xiaoqiao He]

Thanks Brahma for initiating this and making hadoop-3.2.3 release happen.

I would like to validate the HBase project (both the latest release and
trunk branch).
Chao Sun will validate the Spark Project (Got in touch with Chao already).
once RC is out.

Thanks and Regards,
- He Xiaoqiao


On Tue, Aug 10, 2021 at 5:54 PM Brahma Reddy Battula 
wrote:

> Hi All,
>
> I cut branch-3.2.3 and it is ready for release. Please commit to
> branch-3.2.3 if any critical/blocker issues need to go.
>
> *This time I am thinking of having downstream projects and companies'
> voices,let's know how this can go. *
>
>- Planning to check with downstream projects like Spark,HBase and Hive
>if they can help on validation(Or running their UT on this branch)
>- Collecting info from Companies who are already deployed and using the
>branch-3.2
>
>
> so that we can make a more stable release on 3.2 (so that features released
> on this branch impact can be known) ,please let me know anybody from these
> communities who can help on this.
>
>
> Planning to create RC by this month end. Any suggestions are welcome.
>
>
>
> --Brahma Reddy Battula
>

Hadoop-3.2.3 Release Update

[Brahma Reddy Battula]

Hi All,

I cut branch-3.2.3 and it is ready for release. Please commit to
branch-3.2.3 if any critical/blocker issues need to go.

*This time I am thinking of having downstream projects and companies'
voices,let's know how this can go. *

   - Planning to check with downstream projects like Spark,HBase and Hive
   if they can help on validation(Or running their UT on this branch)
   - Collecting info from Companies who are already deployed and using the
   branch-3.2


so that we can make a more stable release on 3.2 (so that features released
on this branch impact can be known) ,please let me know anybody from these
communities who can help on this.


Planning to create RC by this month end. Any suggestions are welcome.



--Brahma Reddy Battula