date:20170724

Re: Pre-Commit build is failing

2017-07-24 Thread Konstantin Shvachko

Or should we backport the entire HADOOP-11917
 ?

Thanks,
--Konst

On Mon, Jul 24, 2017 at 6:56 PM, Konstantin Shvachko 
wrote:

> Allen,
>
> Should we add "patchprocess/" to .gitignore, is that the problem for 2.7?
>
> Thanks,
> --Konstantin
>
> On Fri, Jul 21, 2017 at 6:24 PM, Konstantin Shvachko  > wrote:
>
>> What stuff? Is there a jira?
>> It did work like a week ago. Is it a new Yetus requirement.
>> Anyways I can commit a change to fix the build on our side.
>> Just need to know what is missing.
>>
>> Thanks,
>> --Konst
>>
>> On Fri, Jul 21, 2017 at 5:50 PM, Allen Wittenauer <
>> a...@effectivemachines.com> wrote:
>>
>>>
>>> > On Jul 21, 2017, at 5:46 PM, Konstantin Shvachko 
>>> wrote:
>>> >
>>> > + d...@yetus.apache.org
>>> >
>>> > Guys, could you please take a look. Seems like Yetus problem with
>>> > pre-commit build for branch-2.7.
>>>
>>>
>>> branch-2.7 is missing stuff in .gitignore.
>>
>>
>>
>

Re: zstd compression

2017-07-24 Thread Andrew Wang

I think it'd still be worth asking FB to relicense zstandard. Being able to
bundle it in the release would make it easier to use, since I doubt there
are zstandard packages in the default OS repos.

Sean, have you already filed an issue with zstandard?

On Mon, Jul 17, 2017 at 1:30 PM, Jason Lowe 
wrote:

> I think we are OK to leave support for the zstd codec in the Hadoop code
> base.  I asked Chris Mattman for clarification, noting that the support for
> the zstd codec requires the user to install the zstd headers and libraries
> and then configure it to be included in the native Hadoop build.  The
> Hadoop releases are not shipping any zstd code (e.g.: headers or libraries)
> nor does it require zstd as a mandatory dependency.  Here's what he said:
>
>
> On Monday, July 17, 2017 11:07 AM, Chris Mattmann 
> wrote:
>
> > Hi Jason,
> >
> > This sounds like an optional dependency on a Cat-X software. This isn’t
> the only type of compression
> > that is allowed within Hadoop, correct? If it is truly optional and you
> have gone to that level of detail
> > below to make the user opt in, and if we are not shipping zstd with our
> products (source code releases),
> > then this is an acceptable usage.
> >
> > Cheers,
> > Chris
>
>
> So I think we are in the clear with respect to zstd usage as long as we
> keep it as an optional codec where the user needs to get the headers and
> libraries for zstd and configure it into the native Hadoop build.
>
> Jason
>
> On Monday, July 17, 2017 9:44 AM, Sean Busbey  wrote:
>
>
>
> I know that the HBase community is also looking at what to do about
>
> our inclusion of zstd. We've had it in releases since late 2016. My
>
> plan was to request that they relicense it.
>
>
> Perhaps the Hadoop PMC could join HBase in the request?
>
>
> On Sun, Jul 16, 2017 at 8:11 PM, Allen Wittenauer
>
>  wrote:
>
> >
>
> > It looks like HADOOP-13578 added Facebook's zstd compression
> codec.  Unfortunately, that codec is using the same 3-clause BSD (LICENSE
> file) + patent grant license (PATENTS file) that React is using and RocksDB
> was using.
>
> >
>
> > Should that code get reverted?
>
> >
>
> >
>
> >
>
> > -
>
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
> >
>
>
>
>
> --
>
> busbey
>
>
> -
>
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

Re: Pre-Commit build is failing

2017-07-24 Thread Konstantin Shvachko

Allen,

Should we add "patchprocess/" to .gitignore, is that the problem for 2.7?

Thanks,
--Konstantin

On Fri, Jul 21, 2017 at 6:24 PM, Konstantin Shvachko 
wrote:

> What stuff? Is there a jira?
> It did work like a week ago. Is it a new Yetus requirement.
> Anyways I can commit a change to fix the build on our side.
> Just need to know what is missing.
>
> Thanks,
> --Konst
>
> On Fri, Jul 21, 2017 at 5:50 PM, Allen Wittenauer <
> a...@effectivemachines.com> wrote:
>
>>
>> > On Jul 21, 2017, at 5:46 PM, Konstantin Shvachko 
>> wrote:
>> >
>> > + d...@yetus.apache.org
>> >
>> > Guys, could you please take a look. Seems like Yetus problem with
>> > pre-commit build for branch-2.7.
>>
>>
>> branch-2.7 is missing stuff in .gitignore.
>
>
>

[jira] [Created] (HADOOP-14684) get rid of "skipCorrupt" flag

2017-07-24 Thread Sergey Shelukhin (JIRA)

Sergey Shelukhin created HADOOP-14684:
-

 Summary: get rid of "skipCorrupt" flag
 Key: HADOOP-14684
 URL: https://issues.apache.org/jira/browse/HADOOP-14684
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Sergey Shelukhin


The error that caused the issue was a long time ago and it's probably ok to get 
rid of this flag.
Perhaps we should provide a small tool to overwrite these files without the 
corrupt values.
cc [~prasanth_j]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14683) FileStatus.compareTo binary compat issue between 2.7 and 2.8

2017-07-24 Thread Sergey Shelukhin (JIRA)

Sergey Shelukhin created HADOOP-14683:
-

 Summary: FileStatus.compareTo binary compat issue between 2.7 and 
2.8
 Key: HADOOP-14683
 URL: https://issues.apache.org/jira/browse/HADOOP-14683
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Sergey Shelukhin


See HIVE-17133. Looks like the signature change is causing issues; according to 
[~jnp] this is a public API.
Is it possible to add the old overload back in a point release on 2.8? That way 
we can avoid creating yet another shim for this in Hive.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-14682) cmake Makefiles in hadoop-common don't properly respect -Dopenssl.prefix

2017-07-24 Thread Ravi Prakash (JIRA)

Ravi Prakash created HADOOP-14682:
-

 Summary: cmake Makefiles in hadoop-common don't properly respect 
-Dopenssl.prefix
 Key: HADOOP-14682
 URL: https://issues.apache.org/jira/browse/HADOOP-14682
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Ravi Prakash


Allen reported that while running tests, cmake didn't properly respect 
-Dopenssl.prefix that would allow us to build and run the tests with different 
versions of OpenSSL.
https://issues.apache.org/jira/browse/HADOOP-14597?focusedCommentId=16092114=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16092114

I too encountered some funny stuff while trying to build with a non-default 
OpenSSL library. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re:

2017-07-24 Thread Junping Du

I have done the change.

All committers,

  2.8.2 release is supposed to be a stable/production release for 
branch-2.8. For commits to go for 2.8.2 release (only important and low risk 
bug fixes), please commit to trunk, branch-2, branch-2.8 and branch-2.8.2. For 
unimportant or high risk bug fixes/improvements, please commit to branch-2.8 
(trunk/branch-2) only and mark JIRA fixed as 2.8.3. Thanks for your cooperation!

Thanks,

Junping

From: Junping Du
Sent: Monday, July 24, 2017 10:36 AM
To: Brahma Reddy Battula; Vinod Kumar Vavilapalli
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe; 
humbed...@apache.org
Subject: Re: Apache Hadoop 2.8.2 Release Plan

Nice catch, Brahma. Actually, this is not supposed to be happen as we all 
should know the patch should firstly get landed on branch-2.8 (as well as 
trunk, branch-2) before landed on branch-2.8.2. Anyway, we will always check 
JIRAs claim to fixed in a specific release version with commits actually 
landing on the releasing branch before kicking off RC. So, I am not too worry 
about this mistaking behaviors as it happens in every releases.

If no other concerns, I will do the branch update in next 30 minutes.

Thanks,

Junping

From: Brahma Reddy Battula 
Sent: Sunday, July 23, 2017 1:50 AM
To: Junping Du; Vinod Kumar Vavilapalli
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe; 
humbed...@apache.org
Subject: Re: Apache Hadoop 2.8.2 Release Plan

Just executed the "git log branch-2.8 ^branch-2.8.2" found two commits are 
missed (HDFS-8312 and HADOOP-13867 ) in branch-2.8.I just pushed this two 
commits.Hope we'll not miss any commits which present in only in branch-2.8.2.

From: Junping Du 
Sent: Saturday, July 22, 2017 5:57 AM
To: Vinod Kumar Vavilapalli
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe; 
humbed...@apache.org
Subject: Re: Apache Hadoop 2.8.2 Release Plan

Already get back from Daniel who is from ASF INFRA team, I plan to do following 
operations on next Monday morning:
1. Drop current branch-2.8.2 and recut branch-2.8.2 from branch-2.8
2. Drop abandoned branch-2.8.1 and rename branch-2.8.1-private to branch-2.8.1 
where we just released 2.8.1 from.
I will also adjust fix version on all affected JIRA accordingly.

If you have any concerns on above operations, please raise it before the end of 
this Sunday (7/23).

Thanks,

Junping

From: Junping Du 
Sent: Friday, July 21, 2017 2:29 PM
To: Vinod Kumar Vavilapalli
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe
Subject: Re: Apache Hadoop 2.8.2 Release Plan

Make sense, just raise: https://issues.apache.org/jira/browse/INFRA-14669

Thanks,

Junping

From: Vinod Kumar Vavilapalli 
Sent: Friday, July 21, 2017 12:31 PM
To: Junping Du
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe
Subject: Re: Apache Hadoop 2.8.2 Release Plan

Junping,

If we are looking at a month, I’d not rebranch branch-2.8.2 right now given how 
these things go. We can just continue to commit on branch-2.8 for now.

I also think we should just follow up with ASF INFRA and clean up the branches
 - Delete branch-2.8.2 so that we can recreate it afresh a little later.
 - branch-2.8.1 is also stale and it should be deleted. branch-2.8.1-private 
should be renamed to branch-2.8.1

Thanks
+Vinod

> On Jul 21, 2017, at 11:23 AM, Junping Du  wrote:
>
> Thanks for suggestions, Jason and Kihwal!
> +1 on releasing 2.8.2 on latest branch-2.8 too. Practically, if branch-2.8.2 
> cannot be abandoned/replaced (suspect all branches are read-only now), I will 
> manually merge all commits that not landed on 2.8.2 yet.
>
> Thanks,
>
> Junping
> 
> From: Jason Lowe 
> Sent: Friday, July 21, 2017 8:17 AM
> To: Kihwal Lee; Junping Du; common-dev@hadoop.apache.org; 
> hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; 
> yarn-...@hadoop.apache.org
> Subject: Re: Apache Hadoop 2.8.2 Release Plan
>
> +1 to base the 2.8.2 release off of the more recent activity on branch-2.8.  
> Because branch-2.8.2 was cut so long ago it is missing a lot of fixes that 
> are in branch-2.8.  There also are a lot of JIRAs that claim they are fixed 
> in 2.8.2 but are not in branch-2.8.2.

[DISCUSS] Looking to a 2.9.0 release

2017-07-24 Thread Subru Krishnan

Folks,

With the release for 2.8, we would like to look ahead to 2.9 release as
there are many features/improvements in branch-2 (about 1062 commits), that
are in need of a release vechile.

Here's our first cut of the proposal from the YARN side:

   1. Scheduler improvements (decoupling allocation from node heartbeat,
   allocation ID, concurrency fixes, LightResource etc).
   2. Timeline Service v2
   3. Opportunistic containers
   4. Federation

We would like to hear a formal list from HDFS & Hadoop (& MapReduce if any)
and will update the Roadmap wiki accordingly.

Considering our familiarity with the above mentioned YARN features, we
would like to volunteer as the co-RMs for 2.9.0.

We want to keep the timeline at 8-12 weeks to keep the release pragmatic.

Feedback?

-Subru/Arun

Re: Apache Hadoop 2.8.2 Release Plan

2017-07-24 Thread Junping Du

Nice catch, Brahma. Actually, this is not supposed to be happen as we all 
should know the patch should firstly get landed on branch-2.8 (as well as 
trunk, branch-2) before landed on branch-2.8.2. Anyway, we will always check 
JIRAs claim to fixed in a specific release version with commits actually 
landing on the releasing branch before kicking off RC. So, I am not too worry 
about this mistaking behaviors as it happens in every releases.

If no other concerns, I will do the branch update in next 30 minutes.


Thanks,


Junping



From: Brahma Reddy Battula 
Sent: Sunday, July 23, 2017 1:50 AM
To: Junping Du; Vinod Kumar Vavilapalli
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe; 
humbed...@apache.org
Subject: Re: Apache Hadoop 2.8.2 Release Plan



Just executed the "git log branch-2.8 ^branch-2.8.2" found two commits are 
missed (HDFS-8312 and HADOOP-13867 ) in branch-2.8.I just pushed this two 
commits.Hope we'll not miss any commits which present in only in branch-2.8.2.


From: Junping Du 
Sent: Saturday, July 22, 2017 5:57 AM
To: Vinod Kumar Vavilapalli
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe; 
humbed...@apache.org
Subject: Re: Apache Hadoop 2.8.2 Release Plan

Already get back from Daniel who is from ASF INFRA team, I plan to do following 
operations on next Monday morning:
1. Drop current branch-2.8.2 and recut branch-2.8.2 from branch-2.8
2. Drop abandoned branch-2.8.1 and rename branch-2.8.1-private to branch-2.8.1 
where we just released 2.8.1 from.
I will also adjust fix version on all affected JIRA accordingly.

If you have any concerns on above operations, please raise it before the end of 
this Sunday (7/23).


Thanks,

Junping


From: Junping Du 
Sent: Friday, July 21, 2017 2:29 PM
To: Vinod Kumar Vavilapalli
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe
Subject: Re: Apache Hadoop 2.8.2 Release Plan

Make sense, just raise: https://issues.apache.org/jira/browse/INFRA-14669

Thanks,

Junping

From: Vinod Kumar Vavilapalli 
Sent: Friday, July 21, 2017 12:31 PM
To: Junping Du
Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org; Jason Lowe
Subject: Re: Apache Hadoop 2.8.2 Release Plan

Junping,

If we are looking at a month, I’d not rebranch branch-2.8.2 right now given how 
these things go. We can just continue to commit on branch-2.8 for now.

I also think we should just follow up with ASF INFRA and clean up the branches
 - Delete branch-2.8.2 so that we can recreate it afresh a little later.
 - branch-2.8.1 is also stale and it should be deleted. branch-2.8.1-private 
should be renamed to branch-2.8.1

Thanks
+Vinod

> On Jul 21, 2017, at 11:23 AM, Junping Du  wrote:
>
> Thanks for suggestions, Jason and Kihwal!
> +1 on releasing 2.8.2 on latest branch-2.8 too. Practically, if branch-2.8.2 
> cannot be abandoned/replaced (suspect all branches are read-only now), I will 
> manually merge all commits that not landed on 2.8.2 yet.
>
> Thanks,
>
> Junping
> 
> From: Jason Lowe 
> Sent: Friday, July 21, 2017 8:17 AM
> To: Kihwal Lee; Junping Du; common-dev@hadoop.apache.org; 
> hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; 
> yarn-...@hadoop.apache.org
> Subject: Re: Apache Hadoop 2.8.2 Release Plan
>
> +1 to base the 2.8.2 release off of the more recent activity on branch-2.8.  
> Because branch-2.8.2 was cut so long ago it is missing a lot of fixes that 
> are in branch-2.8.  There also are a lot of JIRAs that claim they are fixed 
> in 2.8.2 but are not in branch-2.8.2.  Having the 2.8.2 release be based on 
> recent activity in branch-2.8 would solve both of these issues, and we'd only 
> need to move the handful of JIRAs that have marked themselves correctly as 
> fixed in 2.8.3 to be fixed in 2.8.2.
>
> Jason
>
>
>On Friday, July 21, 2017 10:01 AM, Kihwal Lee 
>  wrote:
>
>
> Thanks for driving the next 2.8 release, Junping. While I was committing a 
> blocker for 2.7.4, I noticed some of the jiras are back-ported to 2.7, but 
> missing in branch-2.8.2.  Perhaps it is safer and easier to simply rebranch 
> 2.8.2.
> Thanks,Kihwal
>
> On Thursday, July 20, 2017, 3:32:16 PM CDT, Junping Du  
> wrote:
>
> Hi all,
>Per Vinod's previous email, we just announce Apache Hadoop 2.8.1 get 
> released today which is

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-07-24 Thread Apache Jenkins Server

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/474/

[Jul 24, 2017 6:09:03 AM] (sunilg) YARN-6102. RMActiveService context to be 
updated with new RMContext on




-1 overall


The following subsystems voted -1:
findbugs unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

FindBugs :

   module:hadoop-hdfs-project/hadoop-hdfs-client 
   Possible exposure of partially initialized object in 
org.apache.hadoop.hdfs.DFSClient.initThreadsNumForStripedReads(int) At 
DFSClient.java:object in 
org.apache.hadoop.hdfs.DFSClient.initThreadsNumForStripedReads(int) At 
DFSClient.java:[line 2888] 
   org.apache.hadoop.hdfs.server.protocol.SlowDiskReports.equals(Object) 
makes inefficient use of keySet iterator instead of entrySet iterator At 
SlowDiskReports.java:keySet iterator instead of entrySet iterator At 
SlowDiskReports.java:[line 105] 

FindBugs :

   module:hadoop-hdfs-project/hadoop-hdfs 
   Possible null pointer dereference in 
org.apache.hadoop.hdfs.qjournal.server.JournalNode.getJournalsStatus() due to 
return value of called method Dereferenced at 
JournalNode.java:org.apache.hadoop.hdfs.qjournal.server.JournalNode.getJournalsStatus()
 due to return value of called method Dereferenced at JournalNode.java:[line 
302] 
   
org.apache.hadoop.hdfs.server.common.HdfsServerConstants$StartupOption.setClusterId(String)
 unconditionally sets the field clusterId At HdfsServerConstants.java:clusterId 
At HdfsServerConstants.java:[line 193] 
   
org.apache.hadoop.hdfs.server.common.HdfsServerConstants$StartupOption.setForce(int)
 unconditionally sets the field force At HdfsServerConstants.java:force At 
HdfsServerConstants.java:[line 217] 
   
org.apache.hadoop.hdfs.server.common.HdfsServerConstants$StartupOption.setForceFormat(boolean)
 unconditionally sets the field isForceFormat At 
HdfsServerConstants.java:isForceFormat At HdfsServerConstants.java:[line 229] 
   
org.apache.hadoop.hdfs.server.common.HdfsServerConstants$StartupOption.setInteractiveFormat(boolean)
 unconditionally sets the field isInteractiveFormat At 
HdfsServerConstants.java:isInteractiveFormat At HdfsServerConstants.java:[line 
237] 
   Possible null pointer dereference in 
org.apache.hadoop.hdfs.server.datanode.DataStorage.linkBlocksHelper(File, File, 
int, HardLink, boolean, File, List) due to return value of called method 
Dereferenced at 
DataStorage.java:org.apache.hadoop.hdfs.server.datanode.DataStorage.linkBlocksHelper(File,
 File, int, HardLink, boolean, File, List) due to return value of called method 
Dereferenced at DataStorage.java:[line 1339] 
   Possible null pointer dereference in 
org.apache.hadoop.hdfs.server.namenode.NNStorageRetentionManager.purgeOldLegacyOIVImages(String,
 long) due to return value of called method Dereferenced at 
NNStorageRetentionManager.java:org.apache.hadoop.hdfs.server.namenode.NNStorageRetentionManager.purgeOldLegacyOIVImages(String,
 long) due to return value of called method Dereferenced at 
NNStorageRetentionManager.java:[line 258] 
   Possible null pointer dereference in 
org.apache.hadoop.hdfs.server.namenode.NNUpgradeUtil$1.visitFile(Path, 
BasicFileAttributes) due to return value of called method Dereferenced at 
NNUpgradeUtil.java:org.apache.hadoop.hdfs.server.namenode.NNUpgradeUtil$1.visitFile(Path,
 BasicFileAttributes) due to return value of called method Dereferenced at 
NNUpgradeUtil.java:[line 133] 
   Useless condition:argv.length >= 1 at this point At DFSAdmin.java:[line 
2096] 
   Useless condition:numBlocks == -1 at this point At 
ImageLoaderCurrent.java:[line 727] 

FindBugs :

   
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
 
   Useless object stored in variable removedNullContainers of method 
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
 At NodeStatusUpdaterImpl.java:removedNullContainers of method 
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
 At NodeStatusUpdaterImpl.java:[line 642] 
   
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
 makes inefficient use of keySet iterator instead of entrySet iterator At 
NodeStatusUpdaterImpl.java:keySet iterator instead of entrySet iterator At 
NodeStatusUpdaterImpl.java:[line 719] 
   Hard coded reference to an absolute pathname in 
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
 At DockerLinuxContainerRuntime.java:absolute pathname in

[jira] [Created] (HADOOP-14681) Remove MockitoMaker class

2017-07-24 Thread Andras Bokor (JIRA)

Andras Bokor created HADOOP-14681:
-

 Summary: Remove MockitoMaker class
 Key: HADOOP-14681
 URL: https://issues.apache.org/jira/browse/HADOOP-14681
 Project: Hadoop Common
  Issue Type: Bug
  Components: test
Reporter: Andras Bokor
Assignee: Andras Bokor


I would remove MockitoMaker class and use the standard way to mock objects.
For developers it's harder to read and misleading since it's using the 
deprecated syntax.
In addition, it is only used at only some places so we using Mockito on a 
not-unified way. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Maven inconsistently overriding Zookeeper version

2017-07-24 Thread Akira Ajisaka


Filed YARN-6859 and attached the patch there.

-Akira

On 2017/07/19 14:06, Akira Ajisaka wrote:

Hi Sean,

Probably test scope is missing in the zookeeper dependency of 
hadoop-yarn-server-resource-manager test-jar.
Attaching a patch to fix this.

Here is a log after applying the patch:
https://gist.github.com/aajisaka/057cb3d6d26c05a541f5b5de06f70ded

Regards,
Akira

On 2017/07/19 2:26, Sean Mackrory wrote:

There's some Maven magic going on here that I don't understand:
https://gist.github.com/mackrorysd/61c689f04c3595bcda9c256ec6b2da75

On line 2 of the gist, you can see me checking which ZooKeeper artifacts
get picked up when running dependency:tree with the ZooKeeper version
overridden with -Dzookeeper.version. It's all 3.5.3-beta, the version I'm
trying to override it to.

On line 84 of the gist, you can see me doing a clean build of Hadoop with
the same ZooKeeper version, but at the end it appears that
hadoop-yarn-server-resourcemanager is sometimes depending on 3.4.9 (the
version originally in the POM) and other times 3.5.3-beta.

I can't seem to work around that, or even explain it. Anybody have ideas?





-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Pre-Commit build is failing

Re: zstd compression

Re: Pre-Commit build is failing

[jira] [Created] (HADOOP-14684) get rid of "skipCorrupt" flag

[jira] [Created] (HADOOP-14683) FileStatus.compareTo binary compat issue between 2.7 and 2.8

[jira] [Created] (HADOOP-14682) cmake Makefiles in hadoop-common don't properly respect -Dopenssl.prefix

Re:

[DISCUSS] Looking to a 2.9.0 release

Re: Apache Hadoop 2.8.2 Release Plan

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[jira] [Created] (HADOOP-14681) Remove MockitoMaker class

Re: Maven inconsistently overriding Zookeeper version

12 matches

Site Navigation

Mail list logo

Footer information