Re: [VOTE] Release Apache Hadoop 3.0.2 (RC1)

[Lei Xu]

Hi, everybody,

The 3.0.2 vote has passed with 4 binding +1s, 1 non-binding +1 and no -1s.

Thanks everyone for contributing, testing and voting for this release!



On Fri, Apr 20, 2018 at 4:00 PM, Xiao Chen  wrote:
> Thanks Eddy for the effort!
>
> +1 (binding)
>
> Downloaded src tarball and verified checksums
> Built from src
> Started a pseudo distributed hdfs cluster
> Verified basic hdfs operations work
> Sanity checked webui and logs
>
> Best,
> -Xiao
>
>
> -Xiao
>
> On Fri, Apr 20, 2018 at 1:44 AM, 俊平堵  wrote:
>>
>> Thanks Lei for the work!
>>
>> +1 (binding), base on following verification work:
>> - built succeed from source
>> - verified signature
>> - deployed a pseudo cluster and run some simple MR jobs (PI, sleep,
>> terasort, etc.)
>> - checked HDFS/YARN daemons' UI
>> - Tried some rolling upgrade related features: MR over DistributedCache,
>> NM
>> Restart with work preserving, etc.
>>
>> Thanks,
>>
>> Junping
>>
>> 2018-04-17 7:59 GMT+08:00 Lei Xu :
>>
>> > Hi, All
>> >
>> > I've created release candidate RC-1 for Apache Hadoop 3.0.2, to
>> > address missing source jars in the maven repository in RC-0.
>> >
>> > Thanks Ajay Kumar for spotting the error.
>> >
>> > Please note: this is an amendment for Apache Hadoop 3.0.1 release to
>> > fix shaded jars in apache maven repository. The codebase of 3.0.2
>> > release is the same as 3.0.1.  New bug fixes will be included in
>> > Apache Hadoop 3.0.3 instead.
>> >
>> > The release page is:
>> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
>> >
>> > New RC is available at: http://home.apache.org/~lei/hadoop-3.0.2-RC1/
>> >
>> > The git tag is release-3.0.2-RC1, and the latest commit is
>> > 5c141f7c0f24c12cb8704a6ccc1ff8ec991f41ee, which is the same as RC-0.
>> >
>> > The maven artifacts are available at:
>> > https://repository.apache.org/content/repositories/orgapachehadoop-1102/
>> >
>> > Please try the release, especially, *verify the maven artifacts*, and
>> > vote.
>> >
>> > The vote will run 5 days, ending 4/21/2018.
>> >
>> > Here is my +1.
>> >
>> > Best,
>> >
>> > -
>> > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
>> > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>> >
>> >
>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[VOTE] Release Apache Hadoop 3.0.2 (RC1)

[Lei Xu]

Hi, All

I've created release candidate RC-1 for Apache Hadoop 3.0.2, to
address missing source jars in the maven repository in RC-0.

Thanks Ajay Kumar for spotting the error.

Please note: this is an amendment for Apache Hadoop 3.0.1 release to
fix shaded jars in apache maven repository. The codebase of 3.0.2
release is the same as 3.0.1.  New bug fixes will be included in
Apache Hadoop 3.0.3 instead.

The release page is:
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release

New RC is available at: http://home.apache.org/~lei/hadoop-3.0.2-RC1/

The git tag is release-3.0.2-RC1, and the latest commit is
5c141f7c0f24c12cb8704a6ccc1ff8ec991f41ee, which is the same as RC-0.

The maven artifacts are available at:
https://repository.apache.org/content/repositories/orgapachehadoop-1102/

Please try the release, especially, *verify the maven artifacts*, and vote.

The vote will run 5 days, ending 4/21/2018.

Here is my +1.

Best,

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC0)

[Lei Xu]

Ajay, thanks for spotting this.

I am working on fix the deploy.

On Tue, Apr 10, 2018 at 8:32 AM, Ajay Kumar  wrote:
> Thanks Lie for working on this.
>
>- Downloaded src tarball and verified checksums
>- Built from src on mac with java 1.8.0_111
>- Built a pseudo distributed hdfs cluster
>- Run test mr jobs (pi, dfsio,wordcount
>- Verified basic hdfs operations
>- Basic validation for webui
>
> ** I checked maven artifacts and it seems source jars are not there (checked 
> hadoop-hdfs , hadoop-client). Not sure if they are required for release.
>
>
> On 4/9/18, 4:19 PM, "Xiao Chen"  wrote:
>
> Thanks Eddy for the effort!
>
> +1 (binding)
>
>- Downloaded src tarball and verified checksums
>- Built from src
>- Started a pseudo distributed hdfs cluster
>- Verified basic hdfs operations work
>- Sanity checked logs / webui
>
> Best,
> -Xiao
>
>
> On Mon, Apr 9, 2018 at 11:28 AM, Eric Payne 
> 
> wrote:
>
> > Thanks a lot for working to produce this release.
> >
> > +1 (binding)
> > Tested the following:
> > - built from source and installed on 6-node pseudo-cluster
> > - tested Capacity Scheduler FairOrderingPolicy and FifoOrderingPolicy to
> > determine that capacity was assigned as expected in each case
> > - tested user weights with FifoOrderingPolicy to ensure that weights 
> were
>     > assigned to users as expected.
> >
> > Eric Payne
> >
> >
> >
> >
> >
> >
> > On Friday, April 6, 2018, 1:17:10 PM CDT, Lei Xu  
> wrote:
> >
> >
> >
> >
> >
> > Hi, All
> >
> > I've created release candidate RC-0 for Apache Hadoop 3.0.2.
> >
> > Please note: this is an amendment for Apache Hadoop 3.0.1 release to
> > fix shaded jars in apache maven repository. The codebase of 3.0.2
> > release is the same as 3.0.1.  New bug fixes will be included in
> > Apache Hadoop 3.0.3 instead.
> >
> > The release page is:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
> >
> > New RC is available at: http://home.apache.org/~lei/hadoop-3.0.2-RC0/
> >
> > The git tag is release-3.0.2-RC0, and the latest commit is
> > 5c141f7c0f24c12cb8704a6ccc1ff8ec991f41ee
> >
> > The maven artifacts are available at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1096/
> >
> > Please try the release, especially, *verify the maven artifacts*, and 
> vote.
> >
> > The vote will run 5 days, ending 4/11/2018.
> >
> > Thanks for everyone who helped to spot the error and proposed fixes!
> >
> > -
> > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
> >
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> >
>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[VOTE] Release Apache Hadoop 3.0.2 (RC0)

[Lei Xu]

Hi, All

I've created release candidate RC-0 for Apache Hadoop 3.0.2.

Please note: this is an amendment for Apache Hadoop 3.0.1 release to
fix shaded jars in apache maven repository. The codebase of 3.0.2
release is the same as 3.0.1.  New bug fixes will be included in
Apache Hadoop 3.0.3 instead.

The release page is:
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release

New RC is available at: http://home.apache.org/~lei/hadoop-3.0.2-RC0/

The git tag is release-3.0.2-RC0, and the latest commit is
5c141f7c0f24c12cb8704a6ccc1ff8ec991f41ee

The maven artifacts are available at
https://repository.apache.org/content/repositories/orgapachehadoop-1096/

Please try the release, especially, *verify the maven artifacts*, and vote.

The vote will run 5 days, ending 4/11/2018.

Thanks for everyone who helped to spot the error and proposed fixes!

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

[Lei Xu]

Hi, All

Thanks Arpit to find that 3.0.1 artifacts missed shaded jars.  I will
create a new 3.0.2 release with the same codebase as 3.0.1, but deploy
it as shaded jars. A new vote thread will start later this week.

After that, 3.0.3 release will be cut from branch-3.0 and includes the
new bug fixes. Please change the target version of new fixes to 3.0.3.
I will also go over JIRAs to fix targeted versions for committed changes.

Best,

On Mon, Apr 2, 2018 at 6:08 PM, Lei Xu  wrote:
> Hi, Arpit
>
> I followed this instruction
> https://wiki.apache.org/hadoop/HowToRelease
>
> It instructs to use "mvn deploy -Psign -DskipTests -DskipShade".
>
> It seems wrong, given that 3.0.0 is a shaded jar.
> I will do another deploy to fix it.
>
>
> On Mon, Apr 2, 2018 at 5:31 PM, Arpit Agarwal  
> wrote:
>> Hi Lei,
>>
>> It looks like the release artefacts have dummy shaded jars. E.g.
>>
>> Repository Path:  
>> /org/apache/hadoop/hadoop-client-runtime/3.0.1/hadoop-client-runtime-3.0.1.jar
>> Uploaded by:  lei
>> Size: 44.47 KB
>> Uploaded Date:Fri Mar 16 2018 15:50:42 GMT-0700 (PDT)
>> Last Modified:Fri Mar 16 2018 15:50:42 GMT-0700 (PDT)
>>
>> https://repository.apache.org/index.html#view-repositories;releases~browsestorage~/org/apache/hadoop/hadoop-client-runtime/3.0.1/hadoop-client-runtime-3.0.1.jar
>>
>> Am I looking at this wrong or is this supposed to be the shaded jar which is 
>> ~20MB?
>>
>> Thanks,
>> Arpit
>>
>>
>>
>> On 3/23/18, 10:18 AM, "Lei Xu"  wrote:
>>
>> Hi, All
>>
>> Thanks everyone for voting! The vote passes successfully with 6
>> binding +1s, 7 non-binding +1s and no -1s.
>>
>> I will work on the staging and releases.
>>
>> Best,
>>
>>
>> On Fri, Mar 23, 2018 at 5:10 AM, Kuhu Shukla  
>> wrote:
>> > +1 (non-binding)
>> >
>> > Built from source.
>> > Installed on a pseudo distributed cluster.
>> > Ran word count job and basic hdfs commands.
>> >
>> > Thank you for the effort on this release.
>> >
>> > Regards,
>> > Kuhu
>> >
>> > On Thu, Mar 22, 2018 at 5:25 PM, Elek, Marton  wrote:
>> >
>> >>
>> >> +1 (non binding)
>> >>
>> >> I did a full build from source code, created a docker container and 
>> did
>> >> various basic level tests with robotframework based automation and
>> >> docker-compose based pseudo clusters[1].
>> >>
>> >> Including:
>> >>
>> >> * Hdfs federation smoke test
>> >> * Basic ViewFS configuration
>> >> * Yarn example jobs
>> >> * Spark example jobs (with and without yarn)
>> >> * Simple hive table creation
>> >>
>> >> Marton
>> >>
>> >>
>> >> [1]: https://github.com/flokkr/runtime-compose
>> >>
>> >> On 03/18/2018 05:11 AM, Lei Xu wrote:
>> >>
>> >>> Hi, all
>> >>>
>> >>> I've created release candidate RC-1 for Apache Hadoop 3.0.1
>> >>>
>> >>> Apache Hadoop 3.0.1 will be the first bug fix release for Apache
>> >>> Hadoop 3.0 release. It includes 49 bug fixes and security fixes, 
>> which
>> >>> include 12
>> >>> blockers and 17 are critical.
>> >>>
>> >>> Please note:
>> >>> * HDFS-12990. Change default NameNode RPC port back to 8020. It makes
>> >>> incompatible changes to Hadoop 3.0.0.  After 3.0.1 releases, Apache
>> >>> Hadoop 3.0.0 will be deprecated due to this change.
>> >>>
>> >>> The release page is:
>> >>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
>> >>>
>> >>> New RC is available at: http://home.apache.org/~lei/hadoop-3.0.1-RC1/
>> >>>
>> >>> The git tag is release-3.0.1-RC1, and the latest commit is
>> >>> 496dc57cc2e4f4da117f7a8e3840aaeac0c1d2d0
>> >>>
>> >>> The maven artifacts are available at:
>> >>> 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1081/
>> >>>
>> >>> Please try t

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

[Lei Xu]

Hi, Arpit

I followed this instruction
https://wiki.apache.org/hadoop/HowToRelease

It instructs to use "mvn deploy -Psign -DskipTests -DskipShade".

It seems wrong, given that 3.0.0 is a shaded jar.
I will do another deploy to fix it.


On Mon, Apr 2, 2018 at 5:31 PM, Arpit Agarwal  wrote:
> Hi Lei,
>
> It looks like the release artefacts have dummy shaded jars. E.g.
>
> Repository Path:  
> /org/apache/hadoop/hadoop-client-runtime/3.0.1/hadoop-client-runtime-3.0.1.jar
> Uploaded by:  lei
> Size: 44.47 KB
> Uploaded Date:Fri Mar 16 2018 15:50:42 GMT-0700 (PDT)
> Last Modified:Fri Mar 16 2018 15:50:42 GMT-0700 (PDT)
>
> https://repository.apache.org/index.html#view-repositories;releases~browsestorage~/org/apache/hadoop/hadoop-client-runtime/3.0.1/hadoop-client-runtime-3.0.1.jar
>
> Am I looking at this wrong or is this supposed to be the shaded jar which is 
> ~20MB?
>
> Thanks,
> Arpit
>
>
>
> On 3/23/18, 10:18 AM, "Lei Xu"  wrote:
>
> Hi, All
>
> Thanks everyone for voting! The vote passes successfully with 6
> binding +1s, 7 non-binding +1s and no -1s.
>
> I will work on the staging and releases.
>
> Best,
>
>
> On Fri, Mar 23, 2018 at 5:10 AM, Kuhu Shukla  
> wrote:
> > +1 (non-binding)
> >
> > Built from source.
> > Installed on a pseudo distributed cluster.
> > Ran word count job and basic hdfs commands.
> >
> > Thank you for the effort on this release.
> >
> > Regards,
> > Kuhu
> >
> > On Thu, Mar 22, 2018 at 5:25 PM, Elek, Marton  wrote:
> >
> >>
> >> +1 (non binding)
> >>
> >> I did a full build from source code, created a docker container and did
> >> various basic level tests with robotframework based automation and
> >> docker-compose based pseudo clusters[1].
> >>
> >> Including:
> >>
> >> * Hdfs federation smoke test
>     >> * Basic ViewFS configuration
> >> * Yarn example jobs
> >> * Spark example jobs (with and without yarn)
> >> * Simple hive table creation
> >>
> >> Marton
> >>
> >>
> >> [1]: https://github.com/flokkr/runtime-compose
> >>
> >> On 03/18/2018 05:11 AM, Lei Xu wrote:
> >>
> >>> Hi, all
> >>>
> >>> I've created release candidate RC-1 for Apache Hadoop 3.0.1
> >>>
> >>> Apache Hadoop 3.0.1 will be the first bug fix release for Apache
> >>> Hadoop 3.0 release. It includes 49 bug fixes and security fixes, which
> >>> include 12
> >>> blockers and 17 are critical.
> >>>
> >>> Please note:
> >>> * HDFS-12990. Change default NameNode RPC port back to 8020. It makes
> >>> incompatible changes to Hadoop 3.0.0.  After 3.0.1 releases, Apache
> >>> Hadoop 3.0.0 will be deprecated due to this change.
> >>>
> >>> The release page is:
> >>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
> >>>
> >>> New RC is available at: http://home.apache.org/~lei/hadoop-3.0.1-RC1/
> >>>
> >>> The git tag is release-3.0.1-RC1, and the latest commit is
> >>> 496dc57cc2e4f4da117f7a8e3840aaeac0c1d2d0
> >>>
> >>> The maven artifacts are available at:
> >>> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1081/
> >>>
> >>> Please try the release and vote; the vote will run for the usual 5
> >>> days, ending on 3/22/2017 6pm PST time.
> >>>
> >>> Thanks!
> >>>
> >>> -
> >>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> >>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >>>
> >>>
> >> -
> >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >>
> >>
>
>
>
> --
> Lei (Eddy) Xu
> Software Engineer, Cloudera
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

[Lei Xu]

Hi, All

Thanks everyone for voting! The vote passes successfully with 6
binding +1s, 7 non-binding +1s and no -1s.

I will work on the staging and releases.

Best,


On Fri, Mar 23, 2018 at 5:10 AM, Kuhu Shukla  wrote:
> +1 (non-binding)
>
> Built from source.
> Installed on a pseudo distributed cluster.
> Ran word count job and basic hdfs commands.
>
> Thank you for the effort on this release.
>
> Regards,
> Kuhu
>
> On Thu, Mar 22, 2018 at 5:25 PM, Elek, Marton  wrote:
>
>>
>> +1 (non binding)
>>
>> I did a full build from source code, created a docker container and did
>> various basic level tests with robotframework based automation and
>> docker-compose based pseudo clusters[1].
>>
>> Including:
>>
>> * Hdfs federation smoke test
>> * Basic ViewFS configuration
>> * Yarn example jobs
>> * Spark example jobs (with and without yarn)
>> * Simple hive table creation
>>
>> Marton
>>
>>
>> [1]: https://github.com/flokkr/runtime-compose
>>
>> On 03/18/2018 05:11 AM, Lei Xu wrote:
>>
>>> Hi, all
>>>
>>> I've created release candidate RC-1 for Apache Hadoop 3.0.1
>>>
>>> Apache Hadoop 3.0.1 will be the first bug fix release for Apache
>>> Hadoop 3.0 release. It includes 49 bug fixes and security fixes, which
>>> include 12
>>> blockers and 17 are critical.
>>>
>>> Please note:
>>> * HDFS-12990. Change default NameNode RPC port back to 8020. It makes
>>> incompatible changes to Hadoop 3.0.0.  After 3.0.1 releases, Apache
>>> Hadoop 3.0.0 will be deprecated due to this change.
>>>
>>> The release page is:
>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
>>>
>>> New RC is available at: http://home.apache.org/~lei/hadoop-3.0.1-RC1/
>>>
>>> The git tag is release-3.0.1-RC1, and the latest commit is
>>> 496dc57cc2e4f4da117f7a8e3840aaeac0c1d2d0
>>>
>>> The maven artifacts are available at:
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1081/
>>>
>>> Please try the release and vote; the vote will run for the usual 5
>>> days, ending on 3/22/2017 6pm PST time.
>>>
>>> Thanks!
>>>
>>> -
>>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>>>
>>>
>> -
>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>
>>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

[Lei Xu]

Sure, Akira

.mds files are uploaded to http://home.apache.org/~lei/hadoop-3.0.1-RC1/

On Sun, Mar 18, 2018 at 6:04 PM, Akira Ajisaka
 wrote:
> Hi Lei,
>
> Would you provide SHA checksum files instead of MD5?
> http://www.apache.org/dev/release-distribution#sigs-and-sums
>
> -Akira
>
>
> On 2018/03/18 13:11, Lei Xu wrote:
>>
>> Hi, all
>>
>> I've created release candidate RC-1 for Apache Hadoop 3.0.1
>>
>> Apache Hadoop 3.0.1 will be the first bug fix release for Apache
>> Hadoop 3.0 release. It includes 49 bug fixes and security fixes, which
>> include 12
>> blockers and 17 are critical.
>>
>> Please note:
>> * HDFS-12990. Change default NameNode RPC port back to 8020. It makes
>> incompatible changes to Hadoop 3.0.0.  After 3.0.1 releases, Apache
>> Hadoop 3.0.0 will be deprecated due to this change.
>>
>> The release page is:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
>>
>> New RC is available at: http://home.apache.org/~lei/hadoop-3.0.1-RC1/
>>
>> The git tag is release-3.0.1-RC1, and the latest commit is
>> 496dc57cc2e4f4da117f7a8e3840aaeac0c1d2d0
>>
>> The maven artifacts are available at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1081/
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 3/22/2017 6pm PST time.
>>
>> Thanks!
>>
>> -
>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[VOTE] Release Apache Hadoop 3.0.1 (RC1)

[Lei Xu]

Hi, all

I've created release candidate RC-1 for Apache Hadoop 3.0.1

Apache Hadoop 3.0.1 will be the first bug fix release for Apache
Hadoop 3.0 release. It includes 49 bug fixes and security fixes, which
include 12
blockers and 17 are critical.

Please note:
* HDFS-12990. Change default NameNode RPC port back to 8020. It makes
incompatible changes to Hadoop 3.0.0.  After 3.0.1 releases, Apache
Hadoop 3.0.0 will be deprecated due to this change.

The release page is:
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release

New RC is available at: http://home.apache.org/~lei/hadoop-3.0.1-RC1/

The git tag is release-3.0.1-RC1, and the latest commit is
496dc57cc2e4f4da117f7a8e3840aaeac0c1d2d0

The maven artifacts are available at:
https://repository.apache.org/content/repositories/orgapachehadoop-1081/

Please try the release and vote; the vote will run for the usual 5
days, ending on 3/22/2017 6pm PST time.

Thanks!

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC0)

[Lei Xu]

Hi, All

Thank you so much for the votes so far!
I will make a new RC1 in the next few days to include the latest
fixes. Please wait and vote for the new RC1.

Best regards,

On Thu, Feb 22, 2018 at 1:17 PM, Ajay Kumar  wrote:
> +1 non-binding
>
>
>
> Build from source in CentOs7.
> Deployed 4 node secure federated cluster (HDFS HA,Non-HA YARN)
> Run TestDFSIO(write/read), pi and wordcount.
> Verified basic hdfs command
>
>
>
> Thanks,
>
> Ajay
>
>
>
> On 2/20/18, 11:16 PM, "Vinayakumar B"  wrote:
>
>
>
> +1 Binding,
>
>
>
> 1. Checked out code from tag.
>
> 2. Built and deployed a 3-node docker cluster in secure mode (HDFS HA,
>
> Non-HA YARN). Docker script available in [1], if anyone need,
>
> 3. Ran wordcount job successfully.
>
> 4. Verified basic webhdfs commands using curl.
>
> 5. Verified basic HDFS commands
>
>
>
>
>
> [1] https://github.com/vinayakumarb/docker-cluster-deployer
>
>
>
> Thanks,
>
> -Vinay
>
>
>
> On Wed, Feb 21, 2018 at 5:41 AM, Lei Xu  wrote:
>
>
>
> > Dear PMC and committers
>
> >
>
> > Please help to verify and vote this release. I will extend the voting
>
> > period to Friday (Feb 23th) of this week.
>
> >
>
> > Thanks!
>
> >
>
> > On Thu, Feb 15, 2018 at 3:36 PM, Lei Xu  wrote:
>
> > > Hi, all
>
> > >
>
> > > I've created release candidate 0 for Apache Hadoop 3.0.1
>
> > >
>
> > > Apache Hadoop 3.0.1 will be the first bug fix release for Apache
>
> > > Hadoop 3.0 release. It includes 49 bug fixes, which include 10
>
> > > blockers and 8 are critical.
>
> > >
>
> > > Please note:
>
> > > * HDFS-12990. Change default NameNode RPC port back to 8020. It
> makes
>
> > > incompatible changes to Hadoop 3.0.0.  After 3.0.1 releases, Apache
>
> > > Hadoop 3.0.0 will be deprecated due to this change.
>
> > >
>
> > > The release page is:
>
> > >
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
>
> > >
>
> > > New RC is available at:
> http://home.apache.org/~lei/hadoop-3.0.1-RC0/
>
> > >
>
> > > The git tag is release-3.0.1-RC0, and the latest commit is
>
> > > 494d075055b52b0cc922bc25237e231bb3771c90
>
> > >
>
> > > The maven artifacts are available:
>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1078/
>
> > >
>
> > > Please try the release and vote; the vote will run for the usual 5
>
> > > days, ending on 2/20/2017 6pm PST time.
>
> > >
>
> > > Thanks!
>
> > >
>
> > > --
>
>> > Lei (Eddy) Xu
>
> >
>
> >
>
> >
>
> > --
>
> > Lei (Eddy) Xu
>
> > Software Engineer, Cloudera
>
> >
>
> > -
>
> > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
>
> > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
> >
>
> >
>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC0)

[Lei Xu]

Dear PMC and committers

Please help to verify and vote this release. I will extend the voting
period to Friday (Feb 23th) of this week.

Thanks!

On Thu, Feb 15, 2018 at 3:36 PM, Lei Xu  wrote:
> Hi, all
>
> I've created release candidate 0 for Apache Hadoop 3.0.1
>
> Apache Hadoop 3.0.1 will be the first bug fix release for Apache
> Hadoop 3.0 release. It includes 49 bug fixes, which include 10
> blockers and 8 are critical.
>
> Please note:
> * HDFS-12990. Change default NameNode RPC port back to 8020. It makes
> incompatible changes to Hadoop 3.0.0.  After 3.0.1 releases, Apache
> Hadoop 3.0.0 will be deprecated due to this change.
>
> The release page is:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release
>
> New RC is available at: http://home.apache.org/~lei/hadoop-3.0.1-RC0/
>
> The git tag is release-3.0.1-RC0, and the latest commit is
> 494d075055b52b0cc922bc25237e231bb3771c90
>
> The maven artifacts are available:
> https://repository.apache.org/content/repositories/orgapachehadoop-1078/
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 2/20/2017 6pm PST time.
>
> Thanks!
>
> --
> Lei (Eddy) Xu



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[VOTE] Release Apache Hadoop 3.0.1 (RC0)

[Lei Xu]

Hi, all

I've created release candidate 0 for Apache Hadoop 3.0.1

Apache Hadoop 3.0.1 will be the first bug fix release for Apache
Hadoop 3.0 release. It includes 49 bug fixes, which include 10
blockers and 8 are critical.

Please note:
* HDFS-12990. Change default NameNode RPC port back to 8020. It makes
incompatible changes to Hadoop 3.0.0.  After 3.0.1 releases, Apache
Hadoop 3.0.0 will be deprecated due to this change.

The release page is:
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.0+Release

New RC is available at: http://home.apache.org/~lei/hadoop-3.0.1-RC0/

The git tag is release-3.0.1-RC0, and the latest commit is
494d075055b52b0cc922bc25237e231bb3771c90

The maven artifacts are available:
https://repository.apache.org/content/repositories/orgapachehadoop-1078/

Please try the release and vote; the vote will run for the usual 5
days, ending on 2/20/2017 6pm PST time.

Thanks!

-- 
Lei (Eddy) Xu

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: Apache Hadoop 3.0.1 Release plan

[Lei Xu]

Hi, Brahma

Thanks for reminder. YARN-5742 does not look like a blocker to me. I
will create a RC right after HADOOP-14060.

On Thu, Feb 8, 2018 at 7:35 AM, Kihwal Lee  wrote:
> HADOOP-14060 is a blocker.  Daryn will add more detail to the jira or to
> this thread.
>
> On Thu, Feb 8, 2018 at 7:01 AM, Brahma Reddy Battula 
> wrote:
>>
>> Hi Eddy,
>>
>> HDFS-12990 got committed to 3.0.1,can we have RC for 3.0.1 (only
>> YARN-5742
>> blocker is open )  ?
>>
>>
>> On Sat, Feb 3, 2018 at 12:40 AM, Chris Douglas 
>> wrote:
>>
>> > On Fri, Feb 2, 2018 at 10:22 AM, Arpit Agarwal
>> > 
>> > wrote:
>> > > Do you plan to roll an RC with an uncommitted fix? That isn't the
>> > > right
>> > approach.
>> >
>> > The fix will be committed to the release branch. We'll vote on the
>> > release, and if it receives a majority of +1 votes then it becomes
>> > 3.0.1. That's how the PMC decides how to move forward. In this case,
>> > that will also resolve whether or not it can be committed to trunk.
>> >
>> > If this logic is unpersuasive, then we can require a 2/3 majority to
>> > replace the codebase. Either way, the PMC will vote to define the
>> > consensus view when it is not emergent.
>> >
>> > > This issue has good visibility and enough discussion.
>> >
>> > Yes, it has. We always prefer consensus to voting, but when discussion
>> > reveals that complete consensus is impossible, we still need a way
>> > forward. This is rare, and usually reserved for significant changes
>> > (like merging YARN). Frankly, it's embarrassing to resort to it here,
>> > but here we are.
>> >
>> > > If there is a binding veto in effect then the change must be
>> > > abandoned.
>> > Else you should be able to proceed with committing. However, 3.0.0 must
>> > be
>> > called out as an abandoned release if we commit it.
>> >
>> > This is not accurate. A binding veto from any committer halts
>> > progress, but the PMC sets the direction of the project. That includes
>> > making decisions that are not universally accepted. -C
>> >
>> > > On 2/1/18, 3:01 PM, "Lei Xu"  wrote:
>> > >
>> > > Sounds good to me, ATM.
>> > >
>> > > On Thu, Feb 1, 2018 at 2:34 PM, Aaron T. Myers 
>> > wrote:
>> > > > Hey Anu,
>> > > >
>> > > > My feeling on HDFS-12990 is that we've discussed it quite a bit
>> > already and
>> > > > it doesn't seem at this point like either side is going to
>> > > budge.
>> > I'm
>> > > > certainly happy to have a phone call about it, but I don't
>> > > expect
>> > that we'd
>> > > > make much progress.
>> > > >
>> > > > My suggestion is that we simply include the patch posted to
>> > HDFS-12990 in
>> > > > the 3.0.1 RC and call this issue out clearly in the subsequent
>> > VOTE thread
>> > > > for the 3.0.1 release. Eddy, are you up for that?
>> > > >
>> > > > Best,
>> > > > Aaron
>> > > >
>> > > > On Thu, Feb 1, 2018 at 1:13 PM, Lei Xu  wrote:
>> > > >>
>> > > >> +Xiao
>> > >     >>
>> > > >> My understanding is that we will have this for 3.0.1.   Xiao,
>> > could
>> > > >> you give your inputs here?
>> > > >>
>> > > >> On Thu, Feb 1, 2018 at 11:55 AM, Anu Engineer <
>> > aengin...@hortonworks.com>
>> > > >> wrote:
>> > > >> > Hi Eddy,
>> > > >> >
>> > > >> > Thanks for driving this release. Just a quick question, do we
>> > have time
>> > > >> > to close this issue?
>> > > >> > https://issues.apache.org/jira/browse/HDFS-12990
>> > > >> >
>> > > >> > or are we abandoning it? I believe that this is the last
>> > > window
>> > for us
>> > > >> > to fix this issue.
>> > > >> >
>> > > >> > Should we have a call and get this resolved one way or
>> > > another?
>> > > >&

Re: Apache Hadoop 3.0.1 Release plan

[Lei Xu]

Sounds good to me, ATM.

On Thu, Feb 1, 2018 at 2:34 PM, Aaron T. Myers  wrote:
> Hey Anu,
>
> My feeling on HDFS-12990 is that we've discussed it quite a bit already and
> it doesn't seem at this point like either side is going to budge. I'm
> certainly happy to have a phone call about it, but I don't expect that we'd
> make much progress.
>
> My suggestion is that we simply include the patch posted to HDFS-12990 in
> the 3.0.1 RC and call this issue out clearly in the subsequent VOTE thread
> for the 3.0.1 release. Eddy, are you up for that?
>
> Best,
> Aaron
>
> On Thu, Feb 1, 2018 at 1:13 PM, Lei Xu  wrote:
>>
>> +Xiao
>>
>> My understanding is that we will have this for 3.0.1.   Xiao, could
>> you give your inputs here?
>>
>> On Thu, Feb 1, 2018 at 11:55 AM, Anu Engineer 
>> wrote:
>> > Hi Eddy,
>> >
>> > Thanks for driving this release. Just a quick question, do we have time
>> > to close this issue?
>> > https://issues.apache.org/jira/browse/HDFS-12990
>> >
>> > or are we abandoning it? I believe that this is the last window for us
>> > to fix this issue.
>> >
>> > Should we have a call and get this resolved one way or another?
>> >
>> > Thanks
>> > Anu
>> >
>> > On 2/1/18, 10:51 AM, "Lei Xu"  wrote:
>> >
>> > Hi, All
>> >
>> > I just cut branch-3.0.1 from branch-3.0.  Please make sure all
>> > patches
>> > targeted to 3.0.1 being checked in both branch-3.0 and branch-3.0.1.
>> >
>> > Thanks!
>> > Eddy
>> >
>> > On Tue, Jan 9, 2018 at 11:17 AM, Lei Xu  wrote:
>> > > Hi, All
>> > >
>> > > We have released Apache Hadoop 3.0.0 in December [1]. To further
>> > > improve the quality of release, we plan to cut branch-3.0.1 branch
>> > > tomorrow for the preparation of Apache Hadoop 3.0.1 release. The
>> > focus
>> > > of 3.0.1 will be fixing blockers (3), critical bugs (1) and bug
>> > fixes
>> > > [2].  No new features and improvement should be included.
>> > >
>> > > We plan to cut branch-3.0.1 tomorrow (Jan 10th) and vote for RC on
>> > Feb
>> > > 1st, targeting for Feb 9th release.
>> > >
>> > > Please feel free to share your insights.
>> > >
>> > > [1]
>> > https://www.mail-archive.com/general@hadoop.apache.org/msg07757.html
>> > > [2] https://issues.apache.org/jira/issues/?filter=12342842
>> > >
>> > > Best,
>> > > --
>> > > Lei (Eddy) Xu
>> > > Software Engineer, Cloudera
>> >
>> >
>> >
>> > --
>> > Lei (Eddy) Xu
>> > Software Engineer, Cloudera
>> >
>> >
>> > -
>> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>> >
>> >
>> >
>>
>>
>>
>> --
>> Lei (Eddy) Xu
>> Software Engineer, Cloudera
>>
>> -
>> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: Apache Hadoop 3.0.1 Release plan

[Lei Xu]

+Xiao

My understanding is that we will have this for 3.0.1.   Xiao, could
you give your inputs here?

On Thu, Feb 1, 2018 at 11:55 AM, Anu Engineer  wrote:
> Hi Eddy,
>
> Thanks for driving this release. Just a quick question, do we have time to 
> close this issue?
> https://issues.apache.org/jira/browse/HDFS-12990
>
> or are we abandoning it? I believe that this is the last window for us to fix 
> this issue.
>
> Should we have a call and get this resolved one way or another?
>
> Thanks
> Anu
>
> On 2/1/18, 10:51 AM, "Lei Xu"  wrote:
>
> Hi, All
>
> I just cut branch-3.0.1 from branch-3.0.  Please make sure all patches
> targeted to 3.0.1 being checked in both branch-3.0 and branch-3.0.1.
>
> Thanks!
> Eddy
>
> On Tue, Jan 9, 2018 at 11:17 AM, Lei Xu  wrote:
> > Hi, All
> >
> > We have released Apache Hadoop 3.0.0 in December [1]. To further
> > improve the quality of release, we plan to cut branch-3.0.1 branch
> > tomorrow for the preparation of Apache Hadoop 3.0.1 release. The focus
> > of 3.0.1 will be fixing blockers (3), critical bugs (1) and bug fixes
> > [2].  No new features and improvement should be included.
> >
> > We plan to cut branch-3.0.1 tomorrow (Jan 10th) and vote for RC on Feb
> > 1st, targeting for Feb 9th release.
> >
> > Please feel free to share your insights.
> >
> > [1] https://www.mail-archive.com/general@hadoop.apache.org/msg07757.html
> > [2] https://issues.apache.org/jira/issues/?filter=12342842
> >
> > Best,
> > --
> > Lei (Eddy) Xu
> > Software Engineer, Cloudera
>
>
>
> --
> Lei (Eddy) Xu
> Software Engineer, Cloudera
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: Apache Hadoop 3.0.1 Release plan

[Lei Xu]

Hi, All

I just cut branch-3.0.1 from branch-3.0.  Please make sure all patches
targeted to 3.0.1 being checked in both branch-3.0 and branch-3.0.1.

Thanks!
Eddy

On Tue, Jan 9, 2018 at 11:17 AM, Lei Xu  wrote:
> Hi, All
>
> We have released Apache Hadoop 3.0.0 in December [1]. To further
> improve the quality of release, we plan to cut branch-3.0.1 branch
> tomorrow for the preparation of Apache Hadoop 3.0.1 release. The focus
> of 3.0.1 will be fixing blockers (3), critical bugs (1) and bug fixes
> [2].  No new features and improvement should be included.
>
> We plan to cut branch-3.0.1 tomorrow (Jan 10th) and vote for RC on Feb
> 1st, targeting for Feb 9th release.
>
> Please feel free to share your insights.
>
> [1] https://www.mail-archive.com/general@hadoop.apache.org/msg07757.html
> [2] https://issues.apache.org/jira/issues/?filter=12342842
>
> Best,
> --
> Lei (Eddy) Xu
> Software Engineer, Cloudera



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: Apache Hadoop 3.0.1 Release plan

[Lei Xu]

Hi, Andrew and Jason

Thanks for the feedback!

I will follow the pattern and cut the branch on Feb 1st and start the
RC vote around Feb 5th then.

Best

On Tue, Jan 9, 2018 at 11:35 AM, Jason Lowe  wrote:
> Is it necessary to cut the branch so far ahead of the release?  branch-3.0
> is already a maintenance line for 3.0.x releases.  Is there a known
> feature/improvement planned to go into branch-3.0 that is not desirable for
> the 3.0.1 release?
>
> I have found in the past that branching so early leads to many useful fixes
> being unnecessarily postponed to future releases because committers forget
> to pick to the new, relatively long-lived patch branch.  This becomes
> especially true if blockers end up dragging out the ultimate release date,
> which has historically been quite common.  My preference would be to cut
> this branch as close to the RC as possible.
>
> Jason
>
>
> On Tue, Jan 9, 2018 at 1:17 PM, Lei Xu  wrote:
>>
>> Hi, All
>>
>> We have released Apache Hadoop 3.0.0 in December [1]. To further
>> improve the quality of release, we plan to cut branch-3.0.1 branch
>> tomorrow for the preparation of Apache Hadoop 3.0.1 release. The focus
>> of 3.0.1 will be fixing blockers (3), critical bugs (1) and bug fixes
>> [2].  No new features and improvement should be included.
>>
>> We plan to cut branch-3.0.1 tomorrow (Jan 10th) and vote for RC on Feb
>> 1st, targeting for Feb 9th release.
>>
>> Please feel free to share your insights.
>>
>> [1] https://www.mail-archive.com/general@hadoop.apache.org/msg07757.html
>> [2] https://issues.apache.org/jira/issues/?filter=12342842
>>
>> Best,
>> --
>> Lei (Eddy) Xu
>> Software Engineer, Cloudera
>>
>> -
>> To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
>>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: Apache Hadoop 3.0.1 Release plan

[Lei Xu]

Hi, Andrew and Jason

Thanks for the feedback!

I will follow the pattern and cut the branch on Feb 1st and start the
RC vote around Feb 5th then.

Best

On Tue, Jan 9, 2018 at 11:35 AM, Jason Lowe  wrote:
> Is it necessary to cut the branch so far ahead of the release?  branch-3.0
> is already a maintenance line for 3.0.x releases.  Is there a known
> feature/improvement planned to go into branch-3.0 that is not desirable for
> the 3.0.1 release?
>
> I have found in the past that branching so early leads to many useful fixes
> being unnecessarily postponed to future releases because committers forget
> to pick to the new, relatively long-lived patch branch.  This becomes
> especially true if blockers end up dragging out the ultimate release date,
> which has historically been quite common.  My preference would be to cut
> this branch as close to the RC as possible.
>
> Jason
>
>
> On Tue, Jan 9, 2018 at 1:17 PM, Lei Xu  wrote:
>>
>> Hi, All
>>
>> We have released Apache Hadoop 3.0.0 in December [1]. To further
>> improve the quality of release, we plan to cut branch-3.0.1 branch
>> tomorrow for the preparation of Apache Hadoop 3.0.1 release. The focus
>> of 3.0.1 will be fixing blockers (3), critical bugs (1) and bug fixes
>> [2].  No new features and improvement should be included.
>>
>> We plan to cut branch-3.0.1 tomorrow (Jan 10th) and vote for RC on Feb
>> 1st, targeting for Feb 9th release.
>>
>> Please feel free to share your insights.
>>
>> [1] https://www.mail-archive.com/general@hadoop.apache.org/msg07757.html
>> [2] https://issues.apache.org/jira/issues/?filter=12342842
>>
>> Best,
>> --
>> Lei (Eddy) Xu
>> Software Engineer, Cloudera
>>
>> -
>> To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
>>
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Apache Hadoop 3.0.1 Release plan

[Lei Xu]

Hi, All

We have released Apache Hadoop 3.0.0 in December [1]. To further
improve the quality of release, we plan to cut branch-3.0.1 branch
tomorrow for the preparation of Apache Hadoop 3.0.1 release. The focus
of 3.0.1 will be fixing blockers (3), critical bugs (1) and bug fixes
[2].  No new features and improvement should be included.

We plan to cut branch-3.0.1 tomorrow (Jan 10th) and vote for RC on Feb
1st, targeting for Feb 9th release.

Please feel free to share your insights.

[1] https://www.mail-archive.com/general@hadoop.apache.org/msg07757.html
[2] https://issues.apache.org/jira/issues/?filter=12342842

Best,
-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

[Lei Xu]

+1 (binding)

* Verified src tarball and bin tarball, verified md5 of each.
* Build source with -Pdist,native
* Started a pseudo cluster
* Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
dfs put/get/cat on "/" with XOR-2-1 policy.

Thanks Andrew for this great effort!

Best,


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang  wrote:
> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang 
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar 
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen"  wrote:
>>>
>>> +1 (binding)
>>>
>>> - downloaded src tarball, verified md5
>>> - built from source with jdk1.8.0_112
>>> - started a pseudo cluster with hdfs and kms
>>> - sanity checked encryption related operations working
>>> - sanity checked webui and logs.
>>>
>>> -Xiao
>>>
>>> On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers 
>>> wrote:
>>>
>>> > +1 (binding)
>>> >
>>> > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>> > - verified the checksum
>>> > - brought up a secure pseudo distributed cluster
>>> > - did some basic file system operations (mkdir, list, put, cat) and
>>> > confirmed that everything was working
>>> > - confirmed that the web UI worked
>>> >
>>> > Best,
>>> > Aaron
>>> >
>>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.w...@cloudera.com>
>>> > wrote:
>>> >
>>> > > Hi all,
>>> > >
>>> > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>> > > who contributed to this release, especially those who jumped on
>>> the
>>> > issues
>>> > > found in RC0.
>>> > >
>>> > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>> > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>> > >
>>> > > You can find the artifacts here:
>>> > >
>>> > > http://home.apache.org/~wang/3.0.0-RC1/
>>> > >
>>> > > I've done the traditional testing of building from the source
>>> tarball and
>>> > > running a Pi job on a single node cluster. I also verified that
>>> the
>>> > shaded
>>> > > jars are not empty.
>>> > >
>>> > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>> > change)
>>> > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>> > > Available here:
>>> > >
>>> > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>> > >
>>> > > This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>> > > Pacific. My +1 to start.
>>> > >
>>> > > Best,
>>> > > Andrew
>>> > >
>>> >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>>
>>
>>
>>
>>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

[Lei Xu]

+1 binding

Run the following steps:

* Check md5 of sources and package.
* Run a YARN + HDFS pseudo cluster.
* Run terasuite on YARN.
* Run HDFS CLIs (ls , rm , etc)


On Mon, Nov 20, 2017 at 12:58 PM, Vinod Kumar Vavilapalli
 wrote:
> Quick question.
>
> I used to be able (in 2.x line) to create dist tarballs (mvn clean install 
> -Pdist -Dtar -DskipTests -Dmaven.javadoc.skip=true) from the source being 
> voted on (hadoop-3.0.0-src.tar.gz).
>
> The idea is to install HDFS, YARN, MR separately in separate root-directories 
> from the generated individual dist tarballs.
>
> But now I see that HDFS and common dist tarballs are empty
> -rw-r--r--  1 vinodkv  staff 45 Nov 20 12:39 
> ./hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0.tar.gz -
> -rw-r--r--  1 vinodkv  staff 45 Nov 20 12:40 
> ./hadoop-hdfs-project/hadoop-hdfs/target/hadoop-hdfs-3.0.0.tar.gz
>
> But YARN and MR are fine
> -rw-r--r--  1 vinodkv  staff   64474187 Nov 20 12:41 
> ./hadoop-yarn-project/target/hadoop-yarn-project-3.0.0.tar.gz
> -rw-r--r--  1 vinodkv  staff   21674457 Nov 20 12:41 
> ./hadoop-mapreduce-project/target/hadoop-mapreduce-3.0.0.tar.gz
>
> Is it just me? Or is this broken?
>
> Thanks
> +Vinod
>
>> On Nov 14, 2017, at 1:34 PM, Andrew Wang  wrote:
>>
>> Hi folks,
>>
>> Thanks as always to the many, many contributors who helped with this
>> release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are
>> available here:
>>
>> http://people.apache.org/~wang/3.0.0-RC0/
>>
>> This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific.
>>
>> 3.0.0 GA contains 291 fixed JIRA issues since 3.0.0-beta1. Notable
>> additions include the merge of YARN resource types, API-based configuration
>> of the CapacityScheduler, and HDFS router-based federation.
>>
>> I've done my traditional testing with a pseudo cluster and a Pi job. My +1
>> to start.
>>
>> Best,
>> Andrew
>
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

[Lei Xu]

Hey,  Weiwei and Jitendra

Thanks a lot for this large effort to bring us ozone.

* As the current state of Ozone implementation, what are the major
benefits of using today’s Ozone over HDFS?  Giving that its missing
features like HDFS-12680 and HDFS-12697, being disabled by default,
and the closing of Hadoop 3.0 release, should we wait for a late merge
when Ozone is more mature ? Or more generally, why should this merge
to a release branch happen now, when Ozone is not yet usable by users?
Staying on a feature branch seems like it's still the right place to
me.
* For the existing HDFS user, could you address the semantic gaps
between Ozone / Ozone File System and HDFS. It would be great to
illustrate what is the expected use cases for Ozone giving its
different architecture and design decisions?  Like no append, no
atomic rename and etc.
* A follow question, was it able to run any of today’s Hadoop
applications (MR, Spark, Impala, Presto and etc) on Ozone directly, or
against OZoneFileSystem? I think a performance / scalability gain or
extended functionality should be the prerequisites for the merge.
Additionally, I believe such tests will reveal the potential caveats
if any.
* Ozone’s architecture shows great potential to address NN
scalability.  However it looks like a XXL effort to me, considering
the fact that 1) the community had multiple unfinished attempts to
simply separate namespace and block management within the same NN
process, and 2) many existing features like snapshot, append, erasure
coding, and etc, are not straightforward to be implemented in today’s
ozone design. Could you share your opinions on this matter?
* How stable is the ozone client? Should we mark them as unstable for
now? Also giving the significant difference between OzoneClient and
HdfsClient, should move it to a separated package or even a project? I
second Konstantin’s option to separate ozone from HDFS.
* Please add sections to the end-user and system admin oriented
documents for deploying and operating SCM, KSM, and also the chunk
servers on DataNodes. Additionally, the introduction in
“OZoneGettingStarted.md” is still building ozone from feature branch
HDFS-7240.

Best regards,

On Mon, Oct 23, 2017 at 11:10 AM, Jitendra Pandey
 wrote:
> I have filed https://issues.apache.org/jira/browse/HDFS-12697 to ensure ozone 
> stays disabled in a secure environment.
> Since ozone is disabled by default and will not come with security on, it 
> will not expose any new attack surface in a Hadoop deployment.
> Ozone security effort will need a detailed design and discussion on a 
> community jira. Hopefully, that effort will start soon after the merge.
>
> Thanks
> jitendra
>
> On 10/20/17, 2:40 PM, "larry mccay"  wrote:
>
> All -
>
> I broke this list of questions out into a separate DISCUSS thread where we
> can iterate over how a security audit process at merge time might look and
> whether it is even something that we want to take on.
>
> I will try and continue discussion on that thread and drive that to some
> conclusion before bringing it into any particular merge discussion.
>
> thanks,
>
> --larry
>
> On Fri, Oct 20, 2017 at 12:37 PM, larry mccay  wrote:
>
> > I previously sent this same email from my work email and it doesn't seem
> > to have gone through - resending from apache account (apologizing up 
> from
> > for the length)
> >
> > For such sizable merges in Hadoop, I would like to start doing security
> > audits in order to have an initial idea of the attack surface, the
> > protections available for known threats, what sort of configuration is
> > being used to launch processes, etc.
> >
> > I dug into the architecture documents while in the middle of this list -
> > nice docs!
> > I do intend to try and make a generic check list like this for such
> > security audits in the future so a lot of this is from that but I tried 
> to
> > also direct specific questions from those docs as well.
> >
> > 1. UIs
> > I see there are at least two UIs - Storage Container Manager and Key 
> Space
> > Manager. There are a number of typical vulnerabilities that we find in 
> UIs
> >
> > 1.1. What sort of validation is being done on any accepted user input?
> > (pointers to code would be appreciated)
> > 1.2. What explicit protections have been built in for (pointers to code
> > would be appreciated):
> >   1.2.1. cross site scripting
> >   1.2.2. cross site request forgery
> >   1.2.3. click jacking (X-Frame-Options)
> > 1.3. What sort of authentication is required for access to the UIs?
> > 1.4. What authorization is available for determining who can access what
> > capabilities of the UIs for either viewing, modifying data or affecting
> > object stores and related processes?
> > 1.5. Are the UIs built with proxying in mind by leveraging X-Forwarded
> > headers?
>

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha4-RC0

[Lei Xu]

+1 (binding)

Ran the following tests:
* Deploy a pesudo cluster using tar ball, run pi.
* Verified MD5 of tar balls for both src and dist.
* Build src tarball with -Pdist,tar

Thanks Andrew for the efforts!

On Thu, Jul 6, 2017 at 3:44 PM, Andrew Wang  wrote:
> Thanks all for the votes so far!
>
> I think we're still at a single binding +1 from myself, so I'll leave this
> vote open until we reach the minimum threshold of 3. I'm still hoping to
> can push the release out before the weekend.
>
> On Thu, Jul 6, 2017 at 2:58 PM, Vijaya Krishna Kalluru Subbarao <
> vij...@cloudera.com> wrote:
>
>> Ran Smokes and BVTs covering basic sanity testing(10+ tests ran) for all
>> these components:
>>
>>- Mapreduce(compression, archives, pipes, JHS),
>>- Avro(AvroMapreduce, HadoopAvro, HiveAvro, SqoopAvro),
>>- HBase(Balancer, compression, ImportExport, Snapshots, Schema
>>change),
>>- Oozie(Hive, Pig, Spark),
>>- Pig(PigAvro, PigParquet, PigCompression),
>>- Search(SolrCtlBasic, SolrRequestForwading, SolrSSLConfiguration).
>>
>> +1 non-binding.
>>
>> Regards,
>> Vijay
>>
>> On Thu, Jul 6, 2017 at 2:39 PM, Eric Badger > > wrote:
>>
>>> - Verified all checksums signatures
>>> - Built from src on macOS 10.12.5 with Java 1.8.0u65
>>> - Deployed single node pseudo cluster
>>> - Successfully ran sleep and pi jobs
>>> - Navigated the various UIs
>>>
>>> +1 (non-binding)
>>>
>>> Thanks,
>>>
>>> Eric
>>>
>>> On Thursday, July 6, 2017 3:31 PM, Aaron Fabbri 
>>> wrote:
>>>
>>>
>>>
>>> Thanks for the hard work on this!  +1 (non-binding)
>>>
>>> - Built from source tarball on OS X w/ Java 1.8.0_45.
>>> - Deployed mini/pseudo cluster.
>>> - Ran grep and wordcount examples.
>>> - Poked around ResourceManager and JobHistory UIs.
>>> - Ran all s3a integration tests in US West 2.
>>>
>>>
>>>
>>> On Thu, Jul 6, 2017 at 10:20 AM, Xiao Chen  wrote:
>>>
>>> > Thanks Andrew!
>>> > +1 (non-binding)
>>> >
>>> >- Verified md5's, checked tarball sizes are reasonable
>>> >- Built source tarball and deployed a pseudo-distributed cluster with
>>> >hdfs/kms
>>> >- Tested basic hdfs/kms operations
>>> >- Sanity checked webuis/logs
>>> >
>>> >
>>> > -Xiao
>>> >
>>> > On Wed, Jul 5, 2017 at 10:33 PM, John Zhuge 
>>> wrote:
>>> >
>>> > > +1 (non-binding)
>>> > >
>>> > >
>>> > >- Verified checksums and signatures of the tarballs
>>> > >- Built source with native, Java 1.8.0_131 on Mac OS X 10.12.5
>>> > >- Cloud connectors:
>>> > >   - A few S3A integration tests
>>> > >   - A few ADL live unit tests
>>> > >- Deployed both binary and built source to a pseudo cluster, passed
>>> > the
>>> > >following sanity tests in insecure, SSL, and SSL+Kerberos mode:
>>> > >   - HDFS basic and ACL
>>> > >   - DistCp basic
>>> > >   - WordCount (skipped in Kerberos mode)
>>> > >   - KMS and HttpFS basic
>>> > >
>>> > > Thanks Andrew for the great effort!
>>> > >
>>> > > On Wed, Jul 5, 2017 at 1:33 PM, Eric Payne >> > > invalid>
>>> > > wrote:
>>> > >
>>> > > > Thanks Andrew.
>>> > > > I downloaded the source, built it, and installed it onto a pseudo
>>> > > > distributed 4-node cluster.
>>> > > >
>>> > > > I ran mapred and streaming test cases, including sleep and
>>> wordcount.
>>> > > > +1 (non-binding)
>>> > > > -Eric
>>> > > >
>>> > > >   From: Andrew Wang 
>>> > > >  To: "common-...@hadoop.apache.org" ;
>>> "
>>> > > > hdfs-...@hadoop.apache.org" ; "
>>> > > > mapreduce-dev@hadoop.apache.org" ;
>>> "
>>> > > > yarn-...@hadoop.apache.org" 
>>> > > >  Sent: Thursday, June 29, 2017 9:41 PM
>>> > > >  Subject: [VOTE] Release Apache Hadoop 3.0.0-alpha4-RC0
>>> > > >
>>> > > > Hi all,
>>> > > >
>>> > > > As always, thanks to the many, many contributors who helped with
>>> this
>>> > > > release! I've prepared an RC0 for 3.0.0-alpha4:
>>> > > >
>>> > > > http://home.apache.org/~wang/3.0.0-alpha4-RC0/
>>> > > >
>>> > > > The standard 5-day vote would run until midnight on Tuesday, July
>>> 4th.
>>> > > > Given that July 4th is a holiday in the US, I expect this vote might
>>> > have
>>> > > > to be extended, but I'd like to close the vote relatively soon
>>> after.
>>> > > >
>>> > > > I've done my traditional testing of a pseudo-distributed cluster
>>> with a
>>> > > > single task pi job, which was successful.
>>> > > >
>>> > > > Normally my testing would end there, but I'm slightly more confident
>>> > this
>>> > > > time. At Cloudera, we've successfully packaged and deployed a
>>> snapshot
>>> > > from
>>> > > > a few days ago, and run basic smoke tests. Some bugs found from this
>>> > > > include HDFS-11956, which fixes backwards compat with Hadoop 2
>>> clients,
>>> > > and
>>> > > > the revert of HDFS-11696, which broke NN QJM HA setup.
>>> > > >
>>> > > > Vijay is working on a test run with a fuller test suite (the
>>> results of
>>> > > > which we can hopefully post soon).
>>> > > >
>>> > > > My +1 to start,
>>> > > >
>>> > > > Best,
>>> > > > Andrew
>>> > > >
>>> > > 

Re: [VOTE] Release Apache Hadoop 2.6.5 (RC0)

[Lei Xu]

+1

The steps I've done:

* Downloaded release tar and source tar, verified MD5.
* Run a HDFS cluster, and copy files between local filesystem and HDFS.


On Tue, Sep 27, 2016 at 1:28 PM, Sangjin Lee  wrote:
> Hi folks,
>
> I have created a release candidate RC0 for the Apache Hadoop 2.6.5 release
> (the next maintenance release in the 2.6.x release line). Below are the
> details of this release candidate:
>
> The RC is available for validation at:
> http://home.apache.org/~sjlee/hadoop-2.6.5-RC0/.
>
> The RC tag in git is release-2.6.5-RC0 and its git commit is
> 6939fc935fba5651fdb33386d88aeb8e875cf27a.
>
> The maven artifacts are staged via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1048/.
>
> You can find my public key at
> http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS.
>
> Please try the release and vote. The vote will run for the usual 5 days.
> Huge thanks to Chris Trezzo for spearheading the release management and
> doing all the work!
>
> Thanks,
> Sangjin



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.7.1 RC0

[Lei Xu]

+1 binding

Downloaded src and bin distribution, verified md5, sha1 and sha256
checksums of both tar files.
Built src using mvn package.
Ran a pseudo HDFS cluster
Ran dfs -put some files, and checked files on NN's web interface.



On Mon, Jun 29, 2015 at 11:54 AM, Wangda Tan  wrote:
> +1 (non-binding)
>
> Compiled and deployed a single node cluster, tried to change node labels
> and run distributed_shell with node label specified.
>
> On Mon, Jun 29, 2015 at 10:30 AM, Ted Yu  wrote:
>
>> +1 (non-binding)
>>
>> Compiled hbase branch-1 with Java 1.8.0_45
>> Ran unit test suite which passed.
>>
>> On Mon, Jun 29, 2015 at 7:22 AM, Steve Loughran 
>> wrote:
>>
>> >
>> > +1 binding from me.
>> >
>> > Tests:
>> >
>> > Rebuild slider with Hadoop.version=2.7.1; ran all the tests including
>> > against a secure cluster.
>> > Repeated for windows running Java 8.
>> >
>> > All tests passed
>> >
>> >
>> > > On 29 Jun 2015, at 09:45, Vinod Kumar Vavilapalli 
>> > wrote:
>> > >
>> > > Hi all,
>> > >
>> > > I've created a release candidate RC0 for Apache Hadoop 2.7.1.
>> > >
>> > > As discussed before, this is the next stable release to follow up
>> 2.6.0,
>> > > and the first stable one in the 2.7.x line.
>> > >
>> > > The RC is available for validation at:
>> > > *http://people.apache.org/~vinodkv/hadoop-2.7.1-RC0/
>> > > *
>> > >
>> > > The RC tag in git is: release-2.7.1-RC0
>> > >
>> > > The maven artifacts are available via repository.apache.org at
>> > > *
>> > https://repository.apache.org/content/repositories/orgapachehadoop-1019/
>> > > <
>> > https://repository.apache.org/content/repositories/orgapachehadoop-1019/
>> >*
>> > >
>> > > Please try the release and vote; the vote will run for the usual 5
>> days.
>> > >
>> > > Thanks,
>> > > Vinod
>> > >
>> > > PS: It took 2 months instead of the planned [1] 2 weeks in getting this
>> > > release out: post-mortem in a separate thread.
>> > >
>> > > [1]: A 2.7.1 release to follow up 2.7.0
>> > > http://markmail.org/thread/zwzze6cqqgwq4rmw
>> >
>> >
>>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera