Re: Apache Hadoop 3.1.1 release plan

[Weiwei Yang]

Hi Wangda

I would propose to have https://issues.apache.org/jira/browse/YARN-8015 
included in 3.1.1.

Once this is done, we get both intra and inter placement constraint covered so 
users could start to explore this feature. Otherwise the functionality is 
pretty limited. It has been Patch Available for a while, I just promoted it 
targeting to 3.1.1. Hope that makes sense.

Thanks!

--
Weiwei

On 11 May 2018, 9:02 AM +0800, Wangda Tan , wrote:
Hi all,

As we previously proposed RC time (May 1st), we want to release 3.1.1
sooner if possible. As of now, 3.1.1 has 187 fixes [1] on top of 3.1.0, and
there're 10 open blockers/criticals which target to 3.1.1 [2]. I just
posted comments to these open criticals/blockers ticket owners asking about
statuses.

If everybody agrees, I propose start code freeze of branch-3.1 from Sat PDT
time this week, only blockers/criticals can be committed to branch-3.1. To
avoid the burden of committers, I want to delay cutting branch-3.1.1 as
late as possible. If you have any major/minors (For severe issues please
update priorities) tickets want to go to 3.1.1, please reply this email
thread and we can look at them and make a call together.

Please feel free to share your comments and suggestions.

Thanks,
Wangda

[1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop Map/Reduce")
AND status = Resolved AND fixVersion = 3.1.1
[2] project in (YARN, HADOOP, MAPREDUCE, "Hadoop Development Tools") AND
priority in (Blocker, Critical) AND resolution = Unresolved AND "Target
Version/s" = 3.1.1 ORDER BY priority DESC


On Thu, May 10, 2018 at 5:48 PM, Wangda Tan  wrote:

Thanks Brahma/Sunil,

For YARN-8265, it is a too big change for 3.1.1, I just removed 3.1.1 from
target version.
For YARN-8236, it is a severe issue and I think it is close to finish.



On Thu, May 10, 2018 at 3:08 AM, Sunil G  wrote:


Thanks Brahma.
Yes, Billie is reviewing YARN-8265 and I am helping in YARN-8236.

- Sunil


On Thu, May 10, 2018 at 2:25 PM Brahma Reddy Battula <
brahmareddy.batt...@huawei.com> wrote:

Thanks Wangda Tan for driving the 3.1.1 release.Yes,This can be better
addition to 3.1 line release for improving quality.

Looks only following two are pending which are in review state. Hope you
are monitoring these two.

https://issues.apache.org/jira/browse/YARN-8265
https://issues.apache.org/jira/browse/YARN-8236



Note : https://issues.apache.org/jira/browse/YARN-8247==> committed
branch-3.1


-Original Message-
From: Wangda Tan [mailto:wheele...@gmail.com]
Sent: 19 April 2018 17:49
To: Hadoop Common ;
mapreduce-...@hadoop.apache.org; Hdfs-dev ;
yarn-...@hadoop.apache.org
Subject: Apache Hadoop 3.1.1 release plan

Hi, All

We have released Apache Hadoop 3.1.0 on Apr 06. To further improve the
quality of the release, we plan to release 3.1.1 at May 06. The focus of
3.1.1 will be fixing blockers / critical bugs and other enhancements. So
far there are 100 JIRAs [1] have fix version marked to 3.1.1.

We plan to cut branch-3.1.1 on May 01 and vote for RC on the same day.

Please feel free to share your insights.

Thanks,
Wangda Tan

[1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop
Map/Reduce") AND fixVersion = 3.1.1

Re: Apache Hadoop 3.1.1 release plan

[Wangda Tan]

Hi all,

As we previously proposed RC time (May 1st), we want to release 3.1.1
sooner if possible. As of now, 3.1.1 has 187 fixes [1] on top of 3.1.0, and
there're 10 open blockers/criticals which target to 3.1.1 [2]. I just
posted comments to these open criticals/blockers ticket owners asking about
statuses.

If everybody agrees, I propose start code freeze of branch-3.1 from Sat PDT
time this week, only blockers/criticals can be committed to branch-3.1. To
avoid the burden of committers, I want to delay cutting branch-3.1.1 as
late as possible. If you have any major/minors (For severe issues please
update priorities) tickets want to go to 3.1.1, please reply this email
thread and we can look at them and make a call together.

Please feel free to share your comments and suggestions.

Thanks,
Wangda

[1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop Map/Reduce")
AND status = Resolved AND  fixVersion = 3.1.1
[2] project in (YARN, HADOOP, MAPREDUCE, "Hadoop Development Tools") AND
priority in (Blocker, Critical) AND resolution = Unresolved AND "Target
Version/s" = 3.1.1 ORDER BY priority DESC


On Thu, May 10, 2018 at 5:48 PM, Wangda Tan  wrote:

> Thanks Brahma/Sunil,
>
> For YARN-8265, it is a too big change for 3.1.1, I just removed 3.1.1 from
> target version.
> For YARN-8236, it is a severe issue and I think it is close to finish.
>
>
>
> On Thu, May 10, 2018 at 3:08 AM, Sunil G  wrote:
>
>>
>> Thanks Brahma.
>> Yes, Billie is reviewing YARN-8265 and I am helping in YARN-8236.
>>
>> - Sunil
>>
>>
>> On Thu, May 10, 2018 at 2:25 PM Brahma Reddy Battula <
>> brahmareddy.batt...@huawei.com> wrote:
>>
>>> Thanks Wangda Tan for driving the 3.1.1 release.Yes,This can be better
>>> addition to 3.1 line release for improving quality.
>>>
>>> Looks only following two are pending which are in review state. Hope you
>>> are monitoring these two.
>>>
>>> https://issues.apache.org/jira/browse/YARN-8265
>>> https://issues.apache.org/jira/browse/YARN-8236
>>>
>>>
>>>
>>> Note : https://issues.apache.org/jira/browse/YARN-8247==> committed
>>> branch-3.1
>>>
>>>
>>> -Original Message-
>>> From: Wangda Tan [mailto:wheele...@gmail.com]
>>> Sent: 19 April 2018 17:49
>>> To: Hadoop Common ;
>>> mapreduce-...@hadoop.apache.org; Hdfs-dev ;
>>> yarn-...@hadoop.apache.org
>>> Subject: Apache Hadoop 3.1.1 release plan
>>>
>>> Hi, All
>>>
>>> We have released Apache Hadoop 3.1.0 on Apr 06. To further improve the
>>> quality of the release, we plan to release 3.1.1 at May 06. The focus of
>>> 3.1.1 will be fixing blockers / critical bugs and other enhancements. So
>>> far there are 100 JIRAs [1] have fix version marked to 3.1.1.
>>>
>>> We plan to cut branch-3.1.1 on May 01 and vote for RC on the same day.
>>>
>>> Please feel free to share your insights.
>>>
>>> Thanks,
>>> Wangda Tan
>>>
>>> [1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop
>>> Map/Reduce") AND fixVersion = 3.1.1
>>>
>>
>

Re: Apache Hadoop 3.1.1 release plan

[Wangda Tan]

Thanks Brahma/Sunil,

For YARN-8265, it is a too big change for 3.1.1, I just removed 3.1.1 from
target version.
For YARN-8236, it is a severe issue and I think it is close to finish.



On Thu, May 10, 2018 at 3:08 AM, Sunil G  wrote:

>
> Thanks Brahma.
> Yes, Billie is reviewing YARN-8265 and I am helping in YARN-8236.
>
> - Sunil
>
>
> On Thu, May 10, 2018 at 2:25 PM Brahma Reddy Battula <
> brahmareddy.batt...@huawei.com> wrote:
>
>> Thanks Wangda Tan for driving the 3.1.1 release.Yes,This can be better
>> addition to 3.1 line release for improving quality.
>>
>> Looks only following two are pending which are in review state. Hope you
>> are monitoring these two.
>>
>> https://issues.apache.org/jira/browse/YARN-8265
>> https://issues.apache.org/jira/browse/YARN-8236
>>
>>
>>
>> Note : https://issues.apache.org/jira/browse/YARN-8247==> committed
>> branch-3.1
>>
>>
>> -Original Message-
>> From: Wangda Tan [mailto:wheele...@gmail.com]
>> Sent: 19 April 2018 17:49
>> To: Hadoop Common ;
>> mapreduce-...@hadoop.apache.org; Hdfs-dev ;
>> yarn-...@hadoop.apache.org
>> Subject: Apache Hadoop 3.1.1 release plan
>>
>> Hi, All
>>
>> We have released Apache Hadoop 3.1.0 on Apr 06. To further improve the
>> quality of the release, we plan to release 3.1.1 at May 06. The focus of
>> 3.1.1 will be fixing blockers / critical bugs and other enhancements. So
>> far there are 100 JIRAs [1] have fix version marked to 3.1.1.
>>
>> We plan to cut branch-3.1.1 on May 01 and vote for RC on the same day.
>>
>> Please feel free to share your insights.
>>
>> Thanks,
>> Wangda Tan
>>
>> [1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop
>> Map/Reduce") AND fixVersion = 3.1.1
>>
>

[jira] [Created] (HDDS-43) Rename hdsl to hdds in hadoop-ozone/acceptance-test/README.md

[Sandeep Nemuri (JIRA)]

Sandeep Nemuri created HDDS-43:
--

 Summary: Rename hdsl to hdds in 
hadoop-ozone/acceptance-test/README.md
 Key: HDDS-43
 URL: https://issues.apache.org/jira/browse/HDDS-43
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Sandeep Nemuri
Assignee: Sandeep Nemuri


Rename hdsl to hdds in hadoop-ozone/acceptance-test/README.md



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13544) Improve logging in JournalNode for federated cluster

[Hanisha Koneru (JIRA)]

Hanisha Koneru created HDFS-13544:
-

 Summary: Improve logging in JournalNode for federated cluster
 Key: HDFS-13544
 URL: https://issues.apache.org/jira/browse/HDFS-13544
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: federation, hdfs
Reporter: Hanisha Koneru
Assignee: Hanisha Koneru


In a federated cluster,when two namespaces utilize the same JournalSet, it is 
difficult to decode some of the log statements as to which Namespace it is 
logging for. 
For example, the following two log statements do not tell us which Namespace 
the edit log belongs to.
{code:java}
INFO  server.Journal (Journal.java:prepareRecovery(773)) - Prepared recovery 
for segment 1: segmentState { startTxId: 1 endTxId: 10 isInProgress: true } 
lastWriterEpoch: 1 lastCommittedTxId: 10

INFO  server.Journal (Journal.java:acceptRecovery(826)) - Synchronizing log 
startTxId: 1 endTxId: 11 isInProgress: true: old segment startTxId: 1 endTxId: 
10 isInProgress: true is not the right length{code}

We should add the NameserviceID or the JournalID to appropriate JournalNode 
logs to help with debugging.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDDS-42) Inconsistent module names and descriptions

[Tsz Wo Nicholas Sze (JIRA)]

Tsz Wo Nicholas Sze created HDDS-42:
---

 Summary: Inconsistent module names and descriptions
 Key: HDDS-42
 URL: https://issues.apache.org/jira/browse/HDDS-42
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze


The hdds/ozone module names and descriptions are inconsistent:
- Missing "Hadoop" in some cases.
- Inconsistent use of acronyms.
- Inconsistent capitalization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/

[May 9, 2018 1:58:42 AM] (fabbri) HADOOP-15420 s3guard ITestS3GuardToolLocal 
failures in diff tests (Gabor
[May 9, 2018 4:51:41 AM] (msingh) HDDS-33. Ozone : Fix the test logic in
[May 9, 2018 6:30:43 AM] (rohithsharmaks) YARN-8239. [UI2] Clicking on Node 
Manager UI under AM container info /
[May 9, 2018 6:34:31 AM] (rohithsharmaks) YARN-8264. [UI2] GPU Info tab 
disappears if we click any sub link under
[May 9, 2018 6:47:04 AM] (rohithsharmaks) YARN-8260. [UI2] Per-application 
tracking URL is no longer available in
[May 9, 2018 1:36:07 PM] (msingh) HDDS-20. Ozone: Add support for rename key 
within a bucket for rpc
[May 9, 2018 2:50:21 PM] (msingh) HDDS-28. Ozone:Duplicate declaration in
[May 9, 2018 5:32:51 PM] (eyang) YARN-8261.  Fixed a bug in creation of 
localized container directory.   
[May 9, 2018 9:15:51 PM] (mackrorysd) HADOOP-15356. Make HTTP timeout 
configurable in ADLS connector.
[May 9, 2018 11:52:09 PM] (inigoiri) HDFS-13537. TestHdfsHelper does not 
generate jceks path properly for




-1 overall


The following subsystems voted -1:
asflicense findbugs unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

FindBugs :

   module:hadoop-hdds/common 
   Found reliance on default encoding in 
org.apache.hadoop.utils.MetadataKeyFilters$KeyPrefixFilter.filterKey(byte[], 
byte[], byte[]):in 
org.apache.hadoop.utils.MetadataKeyFilters$KeyPrefixFilter.filterKey(byte[], 
byte[], byte[]): String.getBytes() At MetadataKeyFilters.java:[line 97] 

Failed junit tests :

   hadoop.util.TestReadWriteDiskValidator 
   hadoop.util.TestBasicDiskValidator 
   hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem 
   hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy 
   hadoop.hdfs.TestLeaseRecovery2 
   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy 
   
hadoop.yarn.server.nodemanager.containermanager.scheduler.TestContainerSchedulerQueuing
 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/diff-compile-javac-root.txt
  [328K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/whitespace-eol.txt
  [9.4M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/whitespace-tabs.txt
  [1.1M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/xml.txt
  [4.0K]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [268K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-hdds_common-warnings.html
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [64K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [48K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [28K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [24K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/777/artifact/out/branch-findbugs-hadoop-ozone_tools.txt
  [4.0K]
   

[jira] [Created] (HDDS-40) Separating packaging of Ozone/HDDS from the main Hadoop

[Elek, Marton (JIRA)]

Elek, Marton created HDDS-40:


 Summary: Separating packaging of Ozone/HDDS from the main Hadoop
 Key: HDDS-40
 URL: https://issues.apache.org/jira/browse/HDDS-40
 Project: Hadoop Distributed Data Store
  Issue Type: Improvement
Reporter: Elek, Marton
Assignee: Elek, Marton


According to the community vote, Ozone/Hdds release cycle should be independent 
from the Hadoop release cycle.

To make it possible we need a separated ozone package.

*The current state:*

We have just one output tar/directory under hadoop-dist (hadoop-3.2.0). It 
includes all the hdfs/yarn/mapreduce/hdds binaries and libraries. (Jar files 
are put in separated directory).

The hdds components and hdfs compobebts all could be started from the bin. 

*Proposed version*

Create a sepearated hadoop-dist/ozone-2.1.0 which contains only the hdfs AND 
hdds components. Both the hdfs namenode and hdds datanode/scm/ksm could be 
started from the ozone-2.1.0 package. 

Hdds packages would be removed from the original hadoop-3.2.0 directory.

This is a relatively small change. On further JIRAs we need to :

 * Create a shaded datanode plugin which could be used with any existing hadoop 
cluster
 * Use standalone ObjectStore/Ozone server instead of the Namenode+Datanod 
plugin.
 * Add test cases for both the ozone-only and the mixed clusters (ozone + hdfs)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.4 (RC0)

[Zsolt Venczel]

Thanks Junping for working on this!

+1 (non-binding)
  - checked out git tag release-2.8.4-RC0
  - successfully run "mvn clean package -Pdist,native -Dtar -DskipTests"
(Ubuntu 16.04.4 LTS)
  - started hadoop clulster with 1 master and 2 slaves
  - run pi estimator, teragen, terasort, teravalidate
  - verified Web UI (file browser)

Thanks,
Zsolt



On Thu, May 10, 2018 at 8:50 AM, Takanobu Asanuma 
wrote:

> Thanks for working on this, Junping!
>
> +1 (non-binding)
>- verified checksums
>- succeeded "mvn clean package -Pdist,native -Dtar -DskipTests" (CentOS
> 7)
>- started hadoop cluster with 1 master and 5 slaves
>- run TeraGen/TeraSort
>- verified Web UI (NN, RM, JobHistory, Timeline)
>- verified some Archival Storage operations
>
> Thanks,
> -Takanobu
>
> > -Original Message-
> > From: Gabor Bota [mailto:gabor.b...@cloudera.com]
> > Sent: Wednesday, May 09, 2018 5:20 PM
> > To: ajay.ku...@hortonworks.com
> > Cc: junping...@apache.org; Hadoop Common ;
> > Hdfs-dev ; mapreduce-...@hadoop.apache.org;
> > yarn-...@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.4 (RC0)
> >
> >   Thanks for the work Junping!
> >
> >   +1 (non-binding)
> >
> >-   checked out git tag release-2.8.4-RC0
> >-   hadoop-aws unit tests ran successfully
> >-   built from source on Mac OS X 10.13.4, java 8.0.171-oracle
> >-   deployed on a 3 node cluster (HDFS HA, Non-HA YARN)
> >-   verified pi job (yarn), teragen, terasort and teravalidate
> >
> >
> >   Regards,
> >   Gabor Bota
> >
> > On Wed, May 9, 2018 at 12:14 AM Ajay Kumar 
> > wrote:
> >
> > > Thanks for work on this, Junping!!
> > >
> > > +1 (non-binding)
> > >   - verified binary checksum
> > > - built from source and setup 4 node cluster
> > > - run basic hdfs command
> > > - run wordcount, pi & TestDFSIO (read/write)
> > > - basic check for NN UI
> > >
> > > Best,
> > > Ajay
> > >
> > > On 5/8/18, 10:41 AM, "俊平堵"  wrote:
> > >
> > > Hi all,
> > >  I've created the first release candidate (RC0) for Apache
> Hadoop
> > > 2.8.4. This is our next maint release to follow up 2.8.3. It
> includes
> > > 77
> > > important fixes and improvements.
> > >
> > > The RC artifacts are available at:
> > > http://home.apache.org/~junping_du/hadoop-2.8.4-RC0
> > >
> > > The RC tag in git is: release-2.8.4-RC0
> > >
> > > The maven artifacts are available via repository.apache.org<
> > > http://repository.apache.org> at:
> > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-111
> > 8
> > >
> > > Please try the release and vote; the vote will run for the
> usual
> > 5
> > > working days, ending on 5/14/2018 PST time.
> > >
> > > Thanks,
> > >
> > > Junping
> > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> > >
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>

Re: Apache Hadoop 3.1.1 release plan

[Sunil G]

Thanks Brahma.
Yes, Billie is reviewing YARN-8265 and I am helping in YARN-8236.

- Sunil


On Thu, May 10, 2018 at 2:25 PM Brahma Reddy Battula <
brahmareddy.batt...@huawei.com> wrote:

> Thanks Wangda Tan for driving the 3.1.1 release.Yes,This can be better
> addition to 3.1 line release for improving quality.
>
> Looks only following two are pending which are in review state. Hope you
> are monitoring these two.
>
> https://issues.apache.org/jira/browse/YARN-8265
> https://issues.apache.org/jira/browse/YARN-8236
>
>
>
> Note : https://issues.apache.org/jira/browse/YARN-8247==> committed
> branch-3.1
>
>
> -Original Message-
> From: Wangda Tan [mailto:wheele...@gmail.com]
> Sent: 19 April 2018 17:49
> To: Hadoop Common ;
> mapreduce-...@hadoop.apache.org; Hdfs-dev ;
> yarn-...@hadoop.apache.org
> Subject: Apache Hadoop 3.1.1 release plan
>
> Hi, All
>
> We have released Apache Hadoop 3.1.0 on Apr 06. To further improve the
> quality of the release, we plan to release 3.1.1 at May 06. The focus of
> 3.1.1 will be fixing blockers / critical bugs and other enhancements. So
> far there are 100 JIRAs [1] have fix version marked to 3.1.1.
>
> We plan to cut branch-3.1.1 on May 01 and vote for RC on the same day.
>
> Please feel free to share your insights.
>
> Thanks,
> Wangda Tan
>
> [1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop Map/Reduce")
> AND fixVersion = 3.1.1
>

RE: Apache Hadoop 3.1.1 release plan

[Brahma Reddy Battula]

Thanks Wangda Tan for driving the 3.1.1 release.Yes,This can be better addition 
to 3.1 line release for improving quality.

Looks only following two are pending which are in review state. Hope you are 
monitoring these two.

https://issues.apache.org/jira/browse/YARN-8265
https://issues.apache.org/jira/browse/YARN-8236



Note : https://issues.apache.org/jira/browse/YARN-8247==> committed branch-3.1


-Original Message-
From: Wangda Tan [mailto:wheele...@gmail.com] 
Sent: 19 April 2018 17:49
To: Hadoop Common ; 
mapreduce-...@hadoop.apache.org; Hdfs-dev ; 
yarn-...@hadoop.apache.org
Subject: Apache Hadoop 3.1.1 release plan

Hi, All

We have released Apache Hadoop 3.1.0 on Apr 06. To further improve the quality 
of the release, we plan to release 3.1.1 at May 06. The focus of 3.1.1 will be 
fixing blockers / critical bugs and other enhancements. So far there are 100 
JIRAs [1] have fix version marked to 3.1.1.

We plan to cut branch-3.1.1 on May 01 and vote for RC on the same day.

Please feel free to share your insights.

Thanks,
Wangda Tan

[1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop Map/Reduce") AND 
fixVersion = 3.1.1

[jira] [Resolved] (HDDS-36) Ozone: Enable checkstyle for HDDS

[Lokesh Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HDDS-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lokesh Jain resolved HDDS-36.
-
Resolution: Not A Problem

> Ozone: Enable checkstyle for HDDS
> -
>
> Key: HDDS-36
> URL: https://issues.apache.org/jira/browse/HDDS-36
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Reporter: Lokesh Jain
>Assignee: Lokesh Jain
>Priority: Major
> Fix For: 0.2.1
>
>
> Before merge we had disabled the checkstyle for hadoop-hdds and hadoop-ozone. 
> Since we have our own subprojects now we can enable them with hdds profile.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

RE: [VOTE] Release Apache Hadoop 2.8.4 (RC0)

[Takanobu Asanuma]

Thanks for working on this, Junping!

+1 (non-binding)
   - verified checksums
   - succeeded "mvn clean package -Pdist,native -Dtar -DskipTests" (CentOS 7)
   - started hadoop cluster with 1 master and 5 slaves
   - run TeraGen/TeraSort
   - verified Web UI (NN, RM, JobHistory, Timeline)
   - verified some Archival Storage operations

Thanks,
-Takanobu

> -Original Message-
> From: Gabor Bota [mailto:gabor.b...@cloudera.com]
> Sent: Wednesday, May 09, 2018 5:20 PM
> To: ajay.ku...@hortonworks.com
> Cc: junping...@apache.org; Hadoop Common ;
> Hdfs-dev ; mapreduce-...@hadoop.apache.org;
> yarn-...@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.4 (RC0)
> 
>   Thanks for the work Junping!
> 
>   +1 (non-binding)
> 
>-   checked out git tag release-2.8.4-RC0
>-   hadoop-aws unit tests ran successfully
>-   built from source on Mac OS X 10.13.4, java 8.0.171-oracle
>-   deployed on a 3 node cluster (HDFS HA, Non-HA YARN)
>-   verified pi job (yarn), teragen, terasort and teravalidate
> 
> 
>   Regards,
>   Gabor Bota
> 
> On Wed, May 9, 2018 at 12:14 AM Ajay Kumar 
> wrote:
> 
> > Thanks for work on this, Junping!!
> >
> > +1 (non-binding)
> >   - verified binary checksum
> > - built from source and setup 4 node cluster
> > - run basic hdfs command
> > - run wordcount, pi & TestDFSIO (read/write)
> > - basic check for NN UI
> >
> > Best,
> > Ajay
> >
> > On 5/8/18, 10:41 AM, "俊平堵"  wrote:
> >
> > Hi all,
> >  I've created the first release candidate (RC0) for Apache Hadoop
> > 2.8.4. This is our next maint release to follow up 2.8.3. It includes
> > 77
> > important fixes and improvements.
> >
> > The RC artifacts are available at:
> > http://home.apache.org/~junping_du/hadoop-2.8.4-RC0
> >
> > The RC tag in git is: release-2.8.4-RC0
> >
> > The maven artifacts are available via repository.apache.org<
> > http://repository.apache.org> at:
> >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-111
> 8
> >
> > Please try the release and vote; the vote will run for the usual
> 5
> > working days, ending on 5/14/2018 PST time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
> > -
> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-13543) when datanode have some unmounted disks, disk balancer should skip these disks not throw IllegalArgumentException

[luoge123 (JIRA)]

luoge123 created HDFS-13543:
---

 Summary: when datanode have some unmounted disks, disk balancer 
should skip these disks not throw IllegalArgumentException
 Key: HDFS-13543
 URL: https://issues.apache.org/jira/browse/HDFS-13543
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: diskbalancer
Affects Versions: 3.0.0
Reporter: luoge123


when datanode has an unmounted disk, disk balancer get disk capacity from 
report is zero, this will case getVolumeInfoFromStorageReports throw 
IllegalArgumentException
{code:java}
java.lang.IllegalArgumentException
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:72)
at 
org.apache.hadoop.hdfs.server.diskbalancer.datamodel.DiskBalancerVolume.setUsed(DiskBalancerVolume.java:268)
at 
org.apache.hadoop.hdfs.server.diskbalancer.connectors.DBNameNodeConnector.getVolumeInfoFromStorageReports(DBNameNodeConnector.java:148)
at 
org.apache.hadoop.hdfs.server.diskbalancer.connectors.DBNameNodeConnector.getNodes(DBNameNodeConnector.java:90)
at 
org.apache.hadoop.hdfs.server.diskbalancer.datamodel.DiskBalancerCluster.readClusterInfo(DiskBalancerCluster.java:133)
at 
org.apache.hadoop.hdfs.server.diskbalancer.command.Command.readClusterInfo(Command.java:123)
at 
org.apache.hadoop.hdfs.server.diskbalancer.command.ReportCommand.execute(ReportCommand.java:74)
at 
org.apache.hadoop.hdfs.tools.DiskBalancerCLI.dispatch(DiskBalancerCLI.java:468)
at org.apache.hadoop.hdfs.tools.DiskBalancerCLI.run(DiskBalancerCLI.java:183)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.hdfs.tools.DiskBalancerCLI.main(DiskBalancerCLI.java:164)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org