[jira] [Created] (YARN-3903) Disable preemption at Queue level for Fair Scheduler

2015-07-08 Thread He Tianyi (JIRA) 
He Tianyi created YARN-3903:
---

 Summary: Disable preemption at Queue level for Fair Scheduler
 Key: YARN-3903
 URL: https://issues.apache.org/jira/browse/YARN-3903
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: fairscheduler
Affects Versions: 2.3.0
 Environment: 3.16.0-0.bpo.4-amd64 #1 SMP Debian 3.16.7-ckt2-1~bpo70+1 
(2014-12-08) x86_64
Reporter: He Tianyi
Priority: Trivial


YARN-2056 supports disabling preemption at queue level for CapacityScheduler.
As for fair scheduler, we recently encountered the same need.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (YARN-3902) Fair scheduler preempts ApplicationMaster

2015-07-08 Thread He Tianyi (JIRA) 
He Tianyi created YARN-3902:
---

 Summary: Fair scheduler preempts ApplicationMaster
 Key: YARN-3902
 URL: https://issues.apache.org/jira/browse/YARN-3902
 Project: Hadoop YARN
  Issue Type: Bug
  Components: fairscheduler
Affects Versions: 2.3.0
 Environment: 3.16.0-0.bpo.4-amd64 #1 SMP Debian 3.16.7-ckt2-1~bpo70+1 
(2014-12-08) x86_64
Reporter: He Tianyi


YARN-2022 have fixed the similar issue related to CapacityScheduler.
However, FairScheduler still suffer, preempting AM while other normal 
containers running out there.

I think we should take the same approach, avoid AM being preempted unless there 
is no container running other than AM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Failed: YARN-3800 PreCommit Build #8466

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3800
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8466/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 329 lines...]
|  +1  | whitespace  |  0m 4s | The patch has no lines that end in 
|  | || whitespace.
|  +1  |install  |  1m 22s| mvn install still works. 
|  +1  |eclipse:eclipse  |  0m 33s| The patch built with 
|  | || eclipse:eclipse.
|  +1  |   findbugs  |  1m 24s| The patch does not introduce any 
|  | || new Findbugs (version 3.0.0)
|  | || warnings.
|  -1  | yarn tests  |  51m 5s| Tests failed in 
|  | || hadoop-yarn-server-resourcemanager.
|  | |  89m 37s   | 


 Reason | Tests
 Failed unit tests  |  
hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744375/YARN-3800.005.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2e3d83f |
| checkstyle |  
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build@2/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
 |
| hadoop-yarn-server-resourcemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build@2/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8466/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
c721de5e7d581501f4ccb3c90c3bd5ff4e09f788 logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8465
Archived 30 artifacts
Archive block size is 32768
Received 1 blocks and 13114660 bytes
Compression is 0.2%
Took 2.8 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService.testGetAllocationFileFromClasspath

Error Message:
null

Stack Trace:
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService.testGetAllocationFileFromClasspath(TestAllocationFileLoaderService.java:54)

Success: YARN-2194 PreCommit Build #8465

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-2194
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8465/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 255 lines...]
|  | || messages.
|  +1  |  release audit  |  0m 19s| The applied patch does not increase 
|  | || the total number of release audit
|  | || warnings.
|  +1  | checkstyle  |  0m 38s| There were no new checkstyle 
|  | || issues.
|  +1  | whitespace  |  0m 0s | The patch has no lines that end in 
|  | || whitespace.
|  +1  |install  |  1m 20s| mvn install still works. 
|  +1  |eclipse:eclipse  |  0m 32s| The patch built with 
|  | || eclipse:eclipse.
|  +1  |   findbugs  |  1m 13s| The patch does not introduce any 
|  | || new Findbugs (version 3.0.0)
|  | || warnings.
|  +1  | yarn tests  |  6m 15s| Tests passed in 
|  | || hadoop-yarn-server-nodemanager.
|  | |  24m 42s   | 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744373/YARN-2194-7.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 2e3d83f |
| hadoop-yarn-server-nodemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8465/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
e770858c2d0084c453d09a70c2bb5f50fdca5e93 logged out




  Finished build.




Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 27 artifacts
Archive block size is 32768
Received 30 blocks and 5728103 bytes
Compression is 14.6%
Took 4.4 sec
Description set: YARN-2194
Recording test results
Email was triggered for: Success
Sending email for trigger: Success



###
## FAILED TESTS (if any) 
##
All tests passed

Failed: YARN-3836 PreCommit Build #8464

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3836
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8464/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 320 lines...]
|  | || the total number of release audit
|  | || warnings.
|  +1  | checkstyle  |  0m 51s| There were no new checkstyle 
|  | || issues.
|  +1  | whitespace  |  0m 0s | The patch has no lines that end in 
|  | || whitespace.
|  +1  |install  |  1m 49s| mvn install still works. 
|  +1  |eclipse:eclipse  |  0m 45s| The patch built with 
|  | || eclipse:eclipse.
|  +1  |   findbugs  |  3m 24s| The patch does not introduce any 
|  | || new Findbugs (version 3.0.0)
|  | || warnings.
|  +1  | yarn tests  |  0m 24s| Tests passed in hadoop-yarn-api. 
|  +1  | yarn tests  |  2m 6s | Tests passed in hadoop-yarn-common. 
|  | |  46m 32s   | 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744367/YARN-3836-YARN-2928.001.patch
 |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / 4c5f88f |
| hadoop-yarn-api test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-api.txt
 |
| hadoop-yarn-common test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-common.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8464/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
6c057365bd5034cb2a9b5940d40c027ffe7458d1 logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 39 artifacts
Archive block size is 32768
Received 33 blocks and 9411192 bytes
Compression is 10.3%
Took 2.1 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
All tests passed

Failed: YARN-3116 PreCommit Build #8461

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3116
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8461/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 394 lines...]
|  | || warnings.
|  +1  | yarn tests  |  0m 26s| Tests passed in hadoop-yarn-api. 
|  +1  | yarn tests  |  1m 58s| Tests passed in hadoop-yarn-common. 
|  +1  | yarn tests  |  6m 17s| Tests passed in 
|  | || hadoop-yarn-server-nodemanager.
|  -1  | yarn tests  |  51m 6s| Tests failed in 
|  | || hadoop-yarn-server-resourcemanager.
|  | |  107m 48s  | 


   Reason | Tests
 Timed out tests  |  
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAppRunnability 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744340/YARN-3116.v7.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2e3d83f |
| checkstyle |  
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/diffcheckstylehadoop-yarn-common.txt
 |
| hadoop-yarn-api test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-api.txt
 |
| hadoop-yarn-common test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-common.txt
 |
| hadoop-yarn-server-nodemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
 |
| hadoop-yarn-server-resourcemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8461/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
4fbf8c807455673a58ab4fdb0d8c65fc27513777 logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 70 artifacts
Archive block size is 32768
Received 62 blocks and 30769116 bytes
Compression is 6.2%
Took 6.4 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
All tests passed

Failed: YARN-3852 PreCommit Build #8463

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3852
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8463/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 242 lines...]
|  -1  | whitespace  |  0m 4s | The patch has 1 line(s) that end in 
|  | || whitespace. Use git apply
|  | || --whitespace=fix.
|  +1  |install  |  1m 19s| mvn install still works. 
|  +1  |eclipse:eclipse  |  0m 33s| The patch built with 
|  | || eclipse:eclipse.
|  -1  | yarn tests  |  5m 59s| Tests failed in 
|  | || hadoop-yarn-server-nodemanager.
|  | |  21m 31s   | 


 Reason | Tests
 Failed unit tests  |  hadoop.yarn.server.nodemanager.TestDeletionService 
|  
hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor
 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744358/YARN-3852.patch |
| Optional Tests | javac unit |
| git revision | trunk / 2e3d83f |
| whitespace | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/whitespace.txt
 |
| hadoop-yarn-server-nodemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8463/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
73301fd0ff85fe80a503349cde031158fd8de9d2 logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 14 artifacts
Archive block size is 32768
Received 31 blocks and 2962830 bytes
Compression is 25.5%
Took 0.8 sec
[description-setter] Could not determine description.
Recording test results
Publish JUnit test result report is waiting for a checkpoint on 
PreCommit-YARN-Build #8462
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
2 tests failed.
FAILED:  
org.apache.hadoop.yarn.server.nodemanager.TestDeletionService.testAbsDelete

Error Message:
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/org.apache.hadoop.yarn.server.nodemanager.TestDeletionService/5490551821339358793

Stack Trace:
java.nio.file.NoSuchFileException: 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/org.apache.hadoop.yarn.server.nodemanager.TestDeletionService/5490551821339358793
at 
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at 
sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at 
sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at 
sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:97)
at java.nio.file.Files.readAttributes(Files.java:1684)
at 
org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.(RawLocalFileSystem.java:653)
at 
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:634)
at 
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:850)
at 
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalF

Failed: YARN-3878 PreCommit Build #8462

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3878
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8462/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 288 lines...]
|  +1  |  release audit  |  0m 22s| The applied patch does not increase 
|  | || the total number of release audit
|  | || warnings.
|  +1  | checkstyle  |  0m 28s| There were no new checkstyle 
|  | || issues.
|  +1  | whitespace  |  0m 0s | The patch has no lines that end in 
|  | || whitespace.
|  +1  |install  |  1m 23s| mvn install still works. 
|  +1  |eclipse:eclipse  |  0m 33s| The patch built with 
|  | || eclipse:eclipse.
|  +1  |   findbugs  |  1m 35s| The patch does not introduce any 
|  | || new Findbugs (version 3.0.0)
|  | || warnings.
|  +1  | yarn tests  |  1m 58s| Tests passed in hadoop-yarn-common. 
|  | |  39m 37s   | 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744343/YARN-3878.08.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2e3d83f |
| hadoop-yarn-common test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build@2/patchprocess/testrun_hadoop-yarn-common.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8462/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
0d68d7f0a15b6c479b0c7fcd652e1ebc9bb2f96c logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 27 artifacts
Archive block size is 32768
Received 1 blocks and 9680713 bytes
Compression is 0.3%
Took 2.1 sec
[description-setter] Could not determine description.
Recording test results
Publish JUnit test result report is waiting for a checkpoint on 
PreCommit-YARN-Build #8461
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
All tests passed

Re: [RESULT][VOTE] Release Apache Hadoop 2.7.1 RC0

2015-07-08 Thread Vinod Kumar Vavilapalli 
The website and mirrors updates came back. Sending out an announcement now.

Thanks
+Vinod

> On Jul 6, 2015, at 4:31 PM, Vinod Kumar Vavilapalli  
> wrote:
> 
> Here’s my +1 to end the vote.
> 
> With 6 binding +1s, 12 non-binding +1s, this release-vote passes.
> 
> Thanks everyone who voted!
> 
> I’ll follow up on the post-voting release process.
> 
> Thanks
> +Vinod
> 
> 
>> On Jun 29, 2015, at 1:45 AM, Vinod Kumar Vavilapalli  
>> wrote:
>> 
>> Hi all,
>> 
>> I've created a release candidate RC0 for Apache Hadoop 2.7.1.
>> 
>> As discussed before, this is the next stable release to follow up 2.6.0,
>> and the first stable one in the 2.7.x line.
>> 
>> The RC is available for validation at:
>> *http://people.apache.org/~vinodkv/hadoop-2.7.1-RC0/
>> *
>> 
>> The RC tag in git is: release-2.7.1-RC0
>> 
>> The maven artifacts are available via repository.apache.org at
>> *https://repository.apache.org/content/repositories/orgapachehadoop-1019/
>> *
>> 
>> Please try the release and vote; the vote will run for the usual 5 days.
>> 
>> Thanks,
>> Vinod
>> 
>> PS: It took 2 months instead of the planned [1] 2 weeks in getting this
>> release out: post-mortem in a separate thread.
>> 
>> [1]: A 2.7.1 release to follow up 2.7.0
>> http://markmail.org/thread/zwzze6cqqgwq4rmw
>

Failed: YARN-3047 PreCommit Build #8460

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3047
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8460/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 398 lines...]
|   0  | shellcheck  |  1m 23s| Shellcheck was not available. 
|  +1  | whitespace  |  0m 0s | The patch has no lines that end in 
|  | || whitespace.
|  +1  |install  |  1m 41s| mvn install still works. 
|  +1  |eclipse:eclipse  |  0m 39s| The patch built with 
|  | || eclipse:eclipse.
|  +1  |   findbugs  |  3m 56s| The patch does not introduce any 
|  | || new Findbugs (version 3.0.0)
|  | || warnings.
|  +1  | yarn tests  |  0m 21s| Tests passed in hadoop-yarn-api. 
|  +1  | yarn tests  |  1m 58s| Tests passed in hadoop-yarn-common. 
|  +1  | yarn tests  |  1m 23s| Tests passed in 
|  | || hadoop-yarn-server-timelineservice.
|  | |  46m 31s   | 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744326/YARN-3047-YARN-2928.13.patch
 |
| Optional Tests | shellcheck javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / 499ce52 |
| hadoop-yarn-api test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-api.txt
 |
| hadoop-yarn-common test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-common.txt
 |
| hadoop-yarn-server-timelineservice test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-timelineservice.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8460/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
e488caebf7e48c9b716194371a70a6dad624bf00 logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 53 artifacts
Archive block size is 32768
Received 155 blocks and 14935103 bytes
Compression is 25.4%
Took 3.1 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
All tests passed

Failed: YARN-1012 PreCommit Build #8459

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-1012
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8459/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 355 lines...]
|  +1  | yarn tests  |  7m 1s | Tests passed in hadoop-yarn-client. 
|  +1  | yarn tests  |  0m 25s| Tests passed in 
|  | || hadoop-yarn-server-common.
|  -1  | yarn tests  |  6m 4s | Tests failed in 
|  | || hadoop-yarn-server-nodemanager.
|  | |  55m 45s   | 


 Reason | Tests
  FindBugs  |  module:hadoop-yarn-server-common 
 Failed unit tests  |  
hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor
 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744329/YARN-1012-11.patch |
| Optional Tests | javac unit findbugs checkstyle javadoc |
| git revision | trunk / 2e3d83f |
| Pre-patch Findbugs warnings | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/trunkFindbugsWarningshadoop-yarn-server-common.html
 |
| Findbugs warnings | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html
 |
| hadoop-yarn-client test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-client.txt
 |
| hadoop-yarn-server-common test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-common.txt
 |
| hadoop-yarn-server-nodemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8459/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
e1f406150896c9b8f2352eba93bedfbdfb66c61b logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 57 artifacts
Archive block size is 32768
Received 38 blocks and 13465041 bytes
Compression is 8.5%
Took 3 sec
[description-setter] Could not determine description.
Recording test results
Publish JUnit test result report is waiting for a checkpoint on 
PreCommit-YARN-Build #8458
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor.testSquashCGroupOperationsWithValidOperations

Error Message:
expected:<...n/container_01/tasks[,net_cls/hadoop_yarn/container_01/tasks,]blkio/hadoop_yarn/co...>
 but 
was:<...n/container_01/tasks[%net_cls/hadoop_yarn/container_01/tasks%]blkio/hadoop_yarn/co...>

Stack Trace:
org.junit.ComparisonFailure: 
expected:<...n/container_01/tasks[,net_cls/hadoop_yarn/container_01/tasks,]blkio/hadoop_yarn/co...>
 but 
was:<...n/container_01/tasks[%net_cls/hadoop_yarn/container_01/tasks%]blkio/hadoop_yarn/co...>
at org.junit.Assert.assertEquals(Assert.java:115)
at org.junit.Assert.assertEquals(Assert.java:144)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor.testSquashCGroupOperationsWithValidOperations(TestPrivilegedOperationExecutor.java:227)

Failed: YARN-3900 PreCommit Build #8458

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3900
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8458/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 341 lines...]
|  | || warnings.
|  +1  | yarn tests  |  1m 58s| Tests passed in hadoop-yarn-common. 
|  +1  | yarn tests  |  3m 10s| Tests passed in 
|  | || 
hadoop-yarn-server-applicationhistory
|  | || service.
|  -1  | yarn tests  |  50m 56s   | Tests failed in 
|  | || hadoop-yarn-server-resourcemanager.
|  | |  100m 29s  | 


 Reason | Tests
 Failed unit tests  |  
hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
 
|  
hadoop.yarn.server.resourcemanager.reservation.TestFairReservationSystem 
|  
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744322/YARN-3900.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2e3d83f |
| hadoop-yarn-common test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build@2/patchprocess/testrun_hadoop-yarn-common.txt
 |
| hadoop-yarn-server-applicationhistoryservice test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build@2/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
 |
| hadoop-yarn-server-resourcemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build@2/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8458/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
200622f3f37c1c967cdfaed670c5b1e1cb1e446f logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 57 artifacts
Archive block size is 32768
Received 10 blocks and 14790711 bytes
Compression is 2.2%
Took 3.1 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
4 tests failed.
REGRESSION:  
org.apache.hadoop.yarn.server.resourcemanager.reservation.TestFairReservationSystem.testFairReservationSystemInitialize

Error Message:
org/apache/hadoop/util/StringInterner

Stack Trace:
java.lang.NoClassDefFoundError: org/apache/hadoop/util/StringInterner
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at 
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2681)
at 
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2558)
at 
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2469)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1294)
at 
org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider.getRecordFactory(RecordFactoryProvider.java:49)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairSchedulerTestBase.(FairSchedulerTestBase.java:56)
at 
org.apache.hadoop.yarn.server.resourcemanager.res

[jira] [Created] (YARN-3901) Populate flow run data in the flow_run table

2015-07-08 Thread Vrushali C (JIRA) 
Vrushali C created YARN-3901:


 Summary: Populate flow run data in the flow_run table
 Key: YARN-3901
 URL: https://issues.apache.org/jira/browse/YARN-3901
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Vrushali C
Assignee: Vrushali C



As per the schema proposed in YARN-3815 in 
https://issues.apache.org/jira/secure/attachment/12743391/hbase-schema-proposal-for-aggregation.pdf

filing jira to track creation and population of data in the flow run table. 

Some points that are being  considered:
- Stores per flow run information aggregated across applications, flow version
RM’s collector writes to on app creation and app completion
- Per App collector writes to it for metric updates at a slower frequency than 
the metric updates to application table
primary key: cluster ! user ! flow ! flow run id
- Only the latest version of flow-level aggregated metrics will be kept, even 
if the entity and application level keep a timeseries.
- The running_apps column will be incremented on app creation, and decremented 
on app completion.
- For min_start_time the RM writer will simply write a value with the tag for 
the applicationId. A coprocessor will return the min value of all written 
values. - 
- Upon flush and compactions, the min value between all the cells of this 
column will be written to the cell without any tag (empty tag) and all the 
other cells will be discarded.
- Ditto for the max_end_time, but then the max will be kept.
- Tags are represented as #type:value. The type can be not set (0), or can 
indicate running (1) or complete (2). In those cases (for metrics) only 
complete app metrics are collapsed on compaction.
- The m! values are aggregated (summed) upon read. Only when applications are 
completed (indicated by tag type 2) can the values be collapsed.
- The application ids that have completed and been aggregated into the flow 
numbers are retained in a separate column for historical tracking: we don’t 
want to re-aggregate for those upon replay






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Failed: YARN-3047 PreCommit Build #8457

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3047
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8457/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 419 lines...]
|  +1  |   findbugs  |  2m 12s| The patch does not introduce any 
|  | || new Findbugs (version ) warnings.
|  -1  | yarn tests  |  0m 18s| Tests failed in hadoop-yarn-api. 
|  -1  | yarn tests  |  0m 19s| Tests failed in hadoop-yarn-common. 
|  -1  | yarn tests  |  0m 12s| Tests failed in 
|  | || hadoop-yarn-server-timelineservice.
|  | |  42m 50s   | 


Reason | Tests
 Failed build  |  hadoop-yarn-api 
   |  hadoop-yarn-common 
   |  hadoop-yarn-server-timelineservice 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744326/YARN-3047-YARN-2928.13.patch
 |
| Optional Tests | shellcheck javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / 499ce52 |
| checkstyle |  
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/diffcheckstylehadoop-yarn-api.txt
 |
| hadoop-yarn-api test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-api.txt
 |
| hadoop-yarn-common test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-common.txt
 |
| hadoop-yarn-server-timelineservice test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-timelineservice.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8457/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
bc909d6cd0f5153fc78d26b05794d686d24ac793 logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 44 artifacts
Archive block size is 32768
Received 156 blocks and 23075740 bytes
Compression is 18.1%
Took 4.7 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Created] (YARN-3900) Protobuf layout of yarn_security_token causes errors in other protos that include it

2015-07-08 Thread Anubhav Dhoot (JIRA) 
Anubhav Dhoot created YARN-3900:
---

 Summary: Protobuf layout  of yarn_security_token causes errors in 
other protos that include it
 Key: YARN-3900
 URL: https://issues.apache.org/jira/browse/YARN-3900
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Anubhav Dhoot
Assignee: Anubhav Dhoot


Because of the subdirectory server used in 
{{hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/proto/server/yarn_security_token.proto}}
 there are errors in other protos that include them.
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Failed: YARN-313 PreCommit Build #8456

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-313
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8456/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 404 lines...]
|  | || new Findbugs (version 3.0.0)
|  | || warnings.
|  +1  | yarn tests  |  0m 24s| Tests passed in hadoop-yarn-api. 
|  -1  | yarn tests  |  6m 51s| Tests failed in hadoop-yarn-client. 
|  +1  | yarn tests  |  1m 58s| Tests passed in hadoop-yarn-common. 
|  +1  | yarn tests  |  51m 24s   | Tests passed in 
|  | || hadoop-yarn-server-resourcemanager.
|  | |  109m 21s  | 


 Reason | Tests
 Failed unit tests  |  hadoop.yarn.client.cli.TestRMAdminCLI 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744257/YARN-313-v6.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 4119ad3 |
| checkstyle |  
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/diffcheckstylehadoop-yarn-api.txt
 |
| hadoop-yarn-api test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-api.txt
 |
| hadoop-yarn-client test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-client.txt
 |
| hadoop-yarn-common test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-common.txt
 |
| hadoop-yarn-server-resourcemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8456/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
139da787c88473af96472a4586439ee302eded8a logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 72 artifacts
Archive block size is 32768
Received 65 blocks and 29675607 bytes
Compression is 6.7%
Took 6.3 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  
org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testRefreshNodesWithGracefulTimeout

Error Message:

Wanted but not invoked:
resourceManagerAdministrationProtocol.refreshNodes(
decommissionType: GRACEFUL
);
-> at 
org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testRefreshNodesWithGracefulTimeout(TestRMAdminCLI.java:198)
Actually, there were zero interactions with this mock.


Stack Trace:
org.mockito.exceptions.verification.WantedButNotInvoked: 
Wanted but not invoked:
resourceManagerAdministrationProtocol.refreshNodes(
decommissionType: GRACEFUL
);
-> at 
org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testRefreshNodesWithGracefulTimeout(TestRMAdminCLI.java:198)
Actually, there were zero interactions with this mock.

at 
org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testRefreshNodesWithGracefulTimeout(TestRMAdminCLI.java:198)

[jira] [Resolved] (YARN-3899) Add equals and hashCode to TimelineEntity

2015-07-08 Thread Li Lu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Li Lu resolved YARN-3899.
-
Resolution: Duplicate

Duplicate to YARN-3836. 

> Add equals and hashCode to TimelineEntity
> -
>
> Key: YARN-3899
> URL: https://issues.apache.org/jira/browse/YARN-3899
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Li Lu
>Assignee: Li Lu
>
> We need to add equals and hashCode methods for timeline entity so that we can 
> easily tell if two timeline entities are equal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (YARN-3899) Add equals and hashCode to TimelineEntity

2015-07-08 Thread Li Lu (JIRA) 
Li Lu created YARN-3899:
---

 Summary: Add equals and hashCode to TimelineEntity
 Key: YARN-3899
 URL: https://issues.apache.org/jira/browse/YARN-3899
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Li Lu
Assignee: Li Lu


We need to add equals and hashCode methods for timeline entity so that we can 
easily tell if two timeline entities are equal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Failed: YARN-3896 PreCommit Build #8455

2015-07-08 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/YARN-3896
Build: https://builds.apache.org/job/PreCommit-YARN-Build/8455/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 291 lines...]
|  +1  |  release audit  |  0m 22s| The applied patch does not increase 
|  | || the total number of release audit
|  | || warnings.
|  +1  | checkstyle  |  0m 23s| There were no new checkstyle 
|  | || issues.
|  +1  | whitespace  |  0m 0s | The patch has no lines that end in 
|  | || whitespace.
|  +1  |install  |  1m 33s| mvn install still works. 
|  +1  |eclipse:eclipse  |  0m 35s| The patch built with 
|  | || eclipse:eclipse.
|  +1  |   findbugs  |  1m 24s| The patch does not introduce any 
|  | || new Findbugs (version 3.0.0)
|  | || warnings.
|  +1  | yarn tests  |  50m 59s   | Tests passed in 
|  | || hadoop-yarn-server-resourcemanager.
|  | |  87m 46s   | 


|| Subsystem || Report/Notes ||

| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12744222/YARN-3896.01.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / bd4e109 |
| hadoop-yarn-server-resourcemanager test log | 
/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/8455/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |




  Adding comment to JIRA




Comment added.
6e84257a2c38478725ad8eadec30da817ec9b90b logged out




  Finished build.




Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-YARN-Build #8445
Archived 27 artifacts
Archive block size is 32768
Received 51 blocks and 7795540 bytes
Compression is 17.7%
Took 9.6 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Created] (YARN-3898) YARN web console only proxies GET to application master but doesn't provide any feedback for other HTTP methods

2015-07-08 Thread Kam Kasravi (JIRA) 
Kam Kasravi created YARN-3898:
-

 Summary: YARN web console only proxies GET to application master 
but doesn't provide any feedback for other HTTP methods
 Key: YARN-3898
 URL: https://issues.apache.org/jira/browse/YARN-3898
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn
Affects Versions: 2.7.1
Reporter: Kam Kasravi
Priority: Minor


YARN web console should provide some feedback when filtering (and preventing) 
DELETE, POST, PUT, etc



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: IMPORTANT: automatic changelog creation

2015-07-08 Thread Allen Wittenauer 

I think it defeats the purpose ofu

On Jul 8, 2015, at 12:13 AM, Tsuyoshi Ozawa  wrote:

> +1, thanks Allen and Andrew for taking lots effort!
> 
>> Is there any possibility that, we can restrict someone from editing the
>> issue in jira once its marked as "closed" after release?
> 
> Vinay's comment looks considerable for us to me. What do you think?


If you lock closed liras, then how does re-open work when something 
gets reverted?

I’m also not sure what purpose it ultimately serves.  Is it the fear 
that the release data that gets shipped with the src tar ball will be wrong? Is 
it that the data might change?  That happens now and is pretty much unfixable 
no matter what one does. [1]  Besides, it’s going to be *extremely* valuable 
for RMs to be able to edit the JIRA data when cutting a release, especially 
given how many people are refusing to write release notes for incompatible 
changes. Being easily audit-able _and easily fixable!_ is a huge win.

It really is a feature that this data can be changed.

Let’s say there is a change. Next release, we can re-roll the old 
change and release notes data for all releases and it will be correct on the 
website, etc, from that point on.

FWIW, this is one of the reasons why we really should be publishing 
trunk’s doc-set to the website.  It’s generally more accurate than the last 
release. Never mind everyone seeming to think that “current” = trunk and 
getting confused when they write a doc patch that doesn’t apply to trunk.

[1] As part of the JIRA cleanup last year,  I added hundreds of JIRAs to 
2.0.0-alpha and 0.23.x last year that were incorrectly marked for 0.24 and 
3.0.0.  I doubt anyone but me (and the future 3.0.0 RM?) really cares.

Re: IMPORTANT: automatic changelog creation

2015-07-08 Thread Sean Busbey 
On Jul 8, 2015 2:13 AM, "Tsuyoshi Ozawa"  wrote:
>
> +1, thanks Allen and Andrew for taking lots effort!
>
> > Is there any possibility that, we can restrict someone from editing the
> > issue in jira once its marked as "closed" after release?
>
> Vinay's comment looks considerable for us to me. What do you think?
>

Mistakes happen, even during the release process.

Presuming the set of folks who can edit closed tickets is already
restricted to contributors, why not assume any edits are the community
making things more accurate?

-- 
Sean

Build failed in Jenkins: Hadoop-Yarn-trunk #980

2015-07-08 Thread Apache Jenkins Server 
See 

Changes:

[vinayakumarb] HADOOP-12117. Potential NPE from Configuration#loadProperty with 
allowNullValueProperties set. (Contributed by zhihai xu)

[vvasudev] YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. 
Contributed by Wei Yan.

[vinayakumarb] HADOOP-12186. ActiveStandbyElector shouldn't call 
monitorLockNodeAsync multiple times (Contributed by zhihai xu)

[cdouglas] HADOOP-5732. Add SFTPFileSystem. Contributed by Ramtin Boustani and 
Inigo Goiri

[jing9] Revert "HDFS-8652. Track BlockInfo instead of Block in 
CorruptReplicasMap. Contributed by Jing Zhao."

[cdouglas] MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in 
MapReduce Tutorial. Contributed by Tsuyoshi Ozawa

[wang] HADOOP-12193. Rename Touchz.java to Touch.java.

[wang] HADOOP-12195. Add annotation to package-info.java file to workaround 
MCOMPILER-205.

[xyao] HDFS-8711. setSpaceQuota command should print the available storage type 
when input storage type is wrong. Contributed by Brahma Reddy Battula.

[wheat9] HDFS-8620. Clean up the checkstyle warinings about ClientProtocol. 
Contributed by Takanobu Asanuma.

[jianhe] YARN-3892. Fixed NPE on RMStateStore#serviceStop when 
CapacityScheduler#serviceInit fails. Contributed by Bibin A Chundatt

[aajisaka] YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy 
Battula.

--
[...truncated 7252 lines...]
 [exec] /usr/bin/cmake -E cmake_link_script 
CMakeFiles/container-executor.dir/link.txt --verbose=1
 [exec] /usr/bin/cc   -g -O2 -Wall -pthread  -D_GNU_SOURCE 
CMakeFiles/container-executor.dir/main/native/container-executor/impl/main.c.o  
-o target/usr/local/bin/container-executor -rdynamic libcontainer.a 
 [exec] make[2]: Leaving directory 
`
 [exec] /usr/bin/cmake -E cmake_progress_report 

  3
 [exec] [ 75%] Built target container-executor
 [exec] make -f CMakeFiles/test-container-executor.dir/build.make 
CMakeFiles/test-container-executor.dir/depend
 [exec] make[2]: Entering directory 
`
 [exec] cd 

 && /usr/bin/cmake -E cmake_depends "Unix Makefiles" 

 

 

 

 

 --color=
 [exec] Dependee 
"
 is newer than depender 
"
 [exec] Dependee 
"
 is newer than depender 
"
 [exec] Scanning dependencies of target test-container-executor
 [exec] make[2]: Leaving directory 
`
 [exec] make -f CMakeFiles/test-container-executor.dir/build.make 
CMakeFiles/test-container-executor.dir/build
 [exec] make[2]: Entering directory 
`

Hadoop-Yarn-trunk - Build # 980 - Still Failing

2015-07-08 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Yarn-trunk/980/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 7445 lines...]
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop YARN  SUCCESS [  3.218 s]
[INFO] Apache Hadoop YARN API  SUCCESS [03:30 min]
[INFO] Apache Hadoop YARN Common . SUCCESS [02:59 min]
[INFO] Apache Hadoop YARN Server . SUCCESS [  0.096 s]
[INFO] Apache Hadoop YARN Server Common .. SUCCESS [ 45.310 s]
[INFO] Apache Hadoop YARN NodeManager  FAILURE [05:44 min]
[INFO] Apache Hadoop YARN Web Proxy .. SKIPPED
[INFO] Apache Hadoop YARN ApplicationHistoryService .. SKIPPED
[INFO] Apache Hadoop YARN ResourceManager  SKIPPED
[INFO] Apache Hadoop YARN Server Tests ... SKIPPED
[INFO] Apache Hadoop YARN Client . SKIPPED
[INFO] Apache Hadoop YARN SharedCacheManager . SKIPPED
[INFO] Apache Hadoop YARN Applications ... SKIPPED
[INFO] Apache Hadoop YARN DistributedShell ... SKIPPED
[INFO] Apache Hadoop YARN Unmanaged Am Launcher .. SKIPPED
[INFO] Apache Hadoop YARN Site ... SKIPPED
[INFO] Apache Hadoop YARN Registry ... SKIPPED
[INFO] Apache Hadoop YARN Project  SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 13:04 min
[INFO] Finished at: 2015-07-08T10:45:27+00:00
[INFO] Final Memory: 77M/914M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-yarn-server-nodemanager: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Yarn-trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-yarn-server-nodemanager
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Updating HADOOP-12117
Updating YARN-3892
Updating YARN-2194
Updating HADOOP-5732
Updating HDFS-8620
Updating MCOMPILER-205
Updating MAPREDUCE-6038
Updating HDFS-8652
Updating YARN-3690
Updating HADOOP-12186
Updating HADOOP-12195
Updating HADOOP-12193
Updating HDFS-8711
Sending e-mails to: yarn-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
1 tests failed.
REGRESSION:  
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor.testSquashCGroupOperationsWithValidOperations

Error Message:
expected:<...n/container_01/tasks[,net_cls/hadoop_yarn/container_01/tasks,]blkio/hadoop_yarn/co...>
 but 
was:<...n/container_01/tasks[%net_cls/hadoop_yarn/container_01/tasks%]blkio/hadoop_yarn/co...>

Stack Trace:
org.junit.ComparisonFailure: 
expected:<...n/container_01/tasks[,net_cls/hadoop_yarn/container_01/tasks,]blkio/hadoop_yarn/co...>
 but 
was:<...n/container_01/tasks[%net_cls/hadoop_yarn/container_01/tasks%]blkio/hadoop_yarn/co...>
at org.junit.Assert.assertEquals(Assert.java:115)
at org.junit.Assert.assertEquals(Assert.java:144)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor.testSquashCGroupOperationsWithValidOperations(TestPrivilegedOperationExecutor.java:227)

Build failed in Jenkins: Hadoop-Yarn-trunk-Java8 #250

2015-07-08 Thread Apache Jenkins Server 
See 

Changes:

[vinayakumarb] HADOOP-12117. Potential NPE from Configuration#loadProperty with 
allowNullValueProperties set. (Contributed by zhihai xu)

[vvasudev] YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. 
Contributed by Wei Yan.

[vinayakumarb] HADOOP-12186. ActiveStandbyElector shouldn't call 
monitorLockNodeAsync multiple times (Contributed by zhihai xu)

[cdouglas] HADOOP-5732. Add SFTPFileSystem. Contributed by Ramtin Boustani and 
Inigo Goiri

[jing9] Revert "HDFS-8652. Track BlockInfo instead of Block in 
CorruptReplicasMap. Contributed by Jing Zhao."

[cdouglas] MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in 
MapReduce Tutorial. Contributed by Tsuyoshi Ozawa

[wang] HADOOP-12193. Rename Touchz.java to Touch.java.

[wang] HADOOP-12195. Add annotation to package-info.java file to workaround 
MCOMPILER-205.

[xyao] HDFS-8711. setSpaceQuota command should print the available storage type 
when input storage type is wrong. Contributed by Brahma Reddy Battula.

[wheat9] HDFS-8620. Clean up the checkstyle warinings about ClientProtocol. 
Contributed by Takanobu Asanuma.

[jianhe] YARN-3892. Fixed NPE on RMStateStore#serviceStop when 
CapacityScheduler#serviceInit fails. Contributed by Bibin A Chundatt

[aajisaka] YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy 
Battula.

--
[...truncated 4212 lines...]
Tests run: 8, Failures: 0, Errors: 0, Skipped: 8, Time elapsed: 0.807 sec - in 
org.apache.hadoop.yarn.server.nodemanager.TestLinuxContainerExecutor
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.yarn.server.nodemanager.webapp.TestContainerLogsPage
Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 1.118 sec - in 
org.apache.hadoop.yarn.server.nodemanager.webapp.TestContainerLogsPage
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServices
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.696 sec - in 
org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServices
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running 
org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesContainers
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.112 sec - in 
org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesContainers
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServer
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.156 sec - in 
org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServer
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.463 sec - in 
org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running 
org.apache.hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.915 sec - in 
org.apache.hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.yarn.server.nodemanager.TestNodeManager
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.111 sec - in 
org.apache.hadoop.yarn.server.nodemanager.TestNodeManager
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running org.apache.hadoop.yarn.server.nodemanager.TestNodeHealthService
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.984 sec - in 
org.apache.hadoop.yarn.server.nodemanager.TestNodeHealthService
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running 
org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.097 sec - in 
org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=768m; 
support was removed in 8.0
Running 
org.apache.hadoop.yarn.server.nodemanager.util.TestNodeManagerHardwareUtils
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.682 sec - in 
org.apache.hadoop.yarn.server.nodemanager.util.TestNodeManagerHardwareUt

Hadoop-Yarn-trunk-Java8 - Build # 250 - Still Failing

2015-07-08 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/250/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 4405 lines...]
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop YARN  SUCCESS [  3.774 s]
[INFO] Apache Hadoop YARN API  SUCCESS [01:20 min]
[INFO] Apache Hadoop YARN Common . SUCCESS [02:34 min]
[INFO] Apache Hadoop YARN Server . SUCCESS [  0.056 s]
[INFO] Apache Hadoop YARN Server Common .. SUCCESS [ 39.721 s]
[INFO] Apache Hadoop YARN NodeManager  FAILURE [05:57 min]
[INFO] Apache Hadoop YARN Web Proxy .. SKIPPED
[INFO] Apache Hadoop YARN ApplicationHistoryService .. SKIPPED
[INFO] Apache Hadoop YARN ResourceManager  SKIPPED
[INFO] Apache Hadoop YARN Server Tests ... SKIPPED
[INFO] Apache Hadoop YARN Client . SKIPPED
[INFO] Apache Hadoop YARN SharedCacheManager . SKIPPED
[INFO] Apache Hadoop YARN Applications ... SKIPPED
[INFO] Apache Hadoop YARN DistributedShell ... SKIPPED
[INFO] Apache Hadoop YARN Unmanaged Am Launcher .. SKIPPED
[INFO] Apache Hadoop YARN Site ... SKIPPED
[INFO] Apache Hadoop YARN Registry ... SKIPPED
[INFO] Apache Hadoop YARN Project  SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 10:37 min
[INFO] Finished at: 2015-07-08T10:44:11+00:00
[INFO] Final Memory: 81M/1093M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on 
project hadoop-yarn-server-nodemanager: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Yarn-trunk-Java8/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-yarn-server-nodemanager
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Updating HADOOP-12117
Updating YARN-3892
Updating YARN-2194
Updating HADOOP-5732
Updating HDFS-8620
Updating MCOMPILER-205
Updating MAPREDUCE-6038
Updating HDFS-8652
Updating YARN-3690
Updating HADOOP-12186
Updating HADOOP-12195
Updating HADOOP-12193
Updating HDFS-8711
Sending e-mails to: yarn-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
1 tests failed.
REGRESSION:  
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor.testSquashCGroupOperationsWithValidOperations

Error Message:
expected:<...n/container_01/tasks[,net_cls/hadoop_yarn/container_01/tasks,]blkio/hadoop_yarn/co...>
 but 
was:<...n/container_01/tasks[%net_cls/hadoop_yarn/container_01/tasks%]blkio/hadoop_yarn/co...>

Stack Trace:
org.junit.ComparisonFailure: 
expected:<...n/container_01/tasks[,net_cls/hadoop_yarn/container_01/tasks,]blkio/hadoop_yarn/co...>
 but 
was:<...n/container_01/tasks[%net_cls/hadoop_yarn/container_01/tasks%]blkio/hadoop_yarn/co...>
at org.junit.Assert.assertEquals(Assert.java:115)
at org.junit.Assert.assertEquals(Assert.java:144)
at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.TestPrivilegedOperationExecutor.testSquashCGroupOperationsWithValidOperations(TestPrivilegedOperationExecutor.java:227)

[jira] [Created] (YARN-3897) "Too many links" in NM log dir

2015-07-08 Thread Hong Zhiguo (JIRA) 
Hong Zhiguo created YARN-3897:
-

 Summary: "Too many links" in NM log dir
 Key: YARN-3897
 URL: https://issues.apache.org/jira/browse/YARN-3897
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Hong Zhiguo
Assignee: Hong Zhiguo
Priority: Minor


Users need to left container logs more than one day. On some nodes of our busy 
cluster, the number of subdirs of {yarn.nodemanager.log-dirs} may reach 32000, 
which is the defaul limit of ext3 file system. As a result, we got errors when 
initiating containers:
"Failed to create directory 
{yarn.nodemanager.log-dirs}/logs/application_1435111082717_1341740 - Too many 
links"

log aggregation is not an option for us because of the heavy pressure on 
namenode. With a cluster of 5K nodes and 20k log files per node, it's not 
acceptable to aggregate some many files to hdfs.

Since ext3 is still widely used, we'd better do something to avoid such error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (YARN-3896) RMNode transitioned from RUNNING to REBOOTED because its response id had not been reset

2015-07-08 Thread Jun Gong (JIRA) 
Jun Gong created YARN-3896:
--

 Summary: RMNode transitioned from RUNNING to REBOOTED because its 
response id had not been reset
 Key: YARN-3896
 URL: https://issues.apache.org/jira/browse/YARN-3896
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Reporter: Jun Gong
Assignee: Jun Gong


{noformat}
2015-07-03 16:49:39,075 INFO org.apache.hadoop.yarn.util.RackResolver: Resolved 
10.208.132.153 to /default-rack
2015-07-03 16:49:39,075 INFO 
org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: Reconnect 
from the node at: 10.208.132.153
2015-07-03 16:49:39,075 INFO 
org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: 
NodeManager from node 10.208.132.153(cmPort: 8041 httpPort: 8080) registered 
with capability: , assigned nodeId 
10.208.132.153:8041
2015-07-03 16:49:39,104 INFO 
org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: Too far 
behind rm response id:2506413 nm response id:0
2015-07-03 16:49:39,137 INFO 
org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Deactivating 
Node 10.208.132.153:8041 as it is now REBOOTED
2015-07-03 16:49:39,137 INFO 
org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: 
10.208.132.153:8041 Node Transitioned from RUNNING to REBOOTED
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (YARN-3895) Support ACLs in TimelineReader

2015-07-08 Thread Varun Saxena (JIRA) 
Varun Saxena created YARN-3895:
--

 Summary: Support ACLs in TimelineReader
 Key: YARN-3895
 URL: https://issues.apache.org/jira/browse/YARN-3895
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: timelineserver
Affects Versions: YARN-2928
Reporter: Varun Saxena
Assignee: Varun Saxena






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: IMPORTANT: automatic changelog creation

2015-07-08 Thread Tsuyoshi Ozawa 
+1, thanks Allen and Andrew for taking lots effort!

> Is there any possibility that, we can restrict someone from editing the
> issue in jira once its marked as "closed" after release?

Vinay's comment looks considerable for us to me. What do you think?

- Tsuyoshi

On Wed, Jul 8, 2015 at 3:57 PM, Akira AJISAKA
 wrote:
> +1, thanks Allen and Andrew.
>
> Regards,
> Akira
>
>
> On 7/3/15 22:31, Devaraj K wrote:
>>
>> +1
>>
>> Thanks Allen and Andrew for your efforts on this.
>>
>> Thanks
>> Devaraj
>>
>> On Fri, Jul 3, 2015 at 11:29 AM, Varun Vasudev 
>> wrote:
>>
>>> +1
>>>
>>> Many thanks to Allen and Andrew for driving this.
>>>
>>> -Varun
>>>
>>>
>>>
>>> On 7/3/15, 10:25 AM, "Vinayakumar B"  wrote:
>>>
 +1 for the auto generation.

 bq. Besides, after a release R1 is out, someone may (accidentally or
 intentionally) modify the JIRA summary.
 Is there any possibility that, we can restrict someone from editing the
 issue in jira once its marked as "closed" after release?

 Regards,
 Vinay

 On Fri, Jul 3, 2015 at 8:32 AM, Karthik Kambatla 
>>>
>>> wrote:


> Huge +1
>
> On Thursday, July 2, 2015, Chris Nauroth 
>>>
>>> wrote:
>
>
>> +1
>>
>> Thank you to Allen for the script, and thank you to Andrew for
>> volunteering to drive the conversion.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 7/2/15, 2:01 PM, "Andrew Wang" 
> >
>>
>> wrote:
>>
>>> Hi all,
>>>
>>> I want to revive the discussion on this thread, since the overhead of
>>> CHANGES.txt came up again in the context of backporting fixes for
>>> maintenance releases.
>>>
>>> Allen's automatic generation script (HADOOP-11731) went into trunk
>>>
>>> but
>
> not
>>>
>>> branch-2, so we're still maintaining CHANGES.txt everywhere. What do
>>> people
>>> think about backporting this to branch-2 and then removing
>>>
>>> CHANGES.txt
>>>
>>> from
>>> trunk/branch-2 (HADOOP-11792)? Based on discussion on this thread
>>>
>>> and in
>>>
>>> HADOOP-11731, we seem to agree that CHANGES.txt is an unreliable
>>>
>>> source
>
> of
>>>
>>> information, and JIRA is at least as reliable and probably much more
>>>
>>> so.
>>>
>>> Thus I don't see any downsides to backporting it.
>>>
>>> Would like to hear everyone's thoughts on this, I'm willing to drive
>>>
>>> the
>>>
>>> effort.
>>>
>>> Thanks,
>>> Andrew
>>>
>>> On Thu, Apr 2, 2015 at 2:00 PM, Tsz Wo Sze
>>>
>>> 
>>>
>>> wrote:
>>>
 Generating change log from JIRA is a good idea.  It bases on an
 assumption
 that each JIRA has an accurate summary (a.k.a. JIRA title) to
>>>
>>> reflect

 the
 committed change. Unfortunately, the assumption is invalid for many
 cases
 since we never enforce that the JIRA summary must be the same as
>>>
>>> the

 change
 log.  We may compare the current CHANGES.txt with the generated
>>>
>>> change

 log.  I beg the diff is long.
 Besides, after a release R1 is out, someone may (accidentally or
 intentionally) modify the JIRA summary.  Then, the entry for the
>>>
>>> same

 item
 in a later release R2 could be different from the one in R1.
 I agree that manually editing CHANGES.txt is not a perfect
>>>
>>> solution.

 However, it works well in the past for many releases.  I suggest we
>
> keep

 the current dev workflow.  Try using the new script provided by
 HADOOP-11731 to generate the next release.  If everything works
>>>
>>> well,
>
> we

 shell remove CHANGES.txt and revise the dev workflow.  What do you
 think?
 Regards,Tsz-Wo


   On Thursday, April 2, 2015 12:57 PM, Allen Wittenauer <
 a...@altiscale.com > wrote:





 On Apr 2, 2015, at 12:40 PM, Vinod Kumar Vavilapalli <
 vino...@hortonworks.com > wrote:

>
> We'd then doing two commits for every patch. Let's simply not
>>>
>>> remove

 CHANGES.txt from trunk, keep the existing dev workflow, but doc the
 release
 process to remove CHANGES.txt in trunk at the time of a release
>>>
>>> going

 out
 of trunk.



 Might as well copy branch-2¹s changes.txt into trunk then. (or
>>>
>>> 2.7¹s.

 Last I looked, people updated branch-2 and not 2.7¹s or vice versa
>>>
>>> for

 some
 patches that went into both branches.)  So that folks who are
 committing to
 both branches and want to cherry pick all changes can.

 I mean, trunk¹s is very very very wrong. R

[jira] [Created] (YARN-3894) RM startup should fail for wrong CS xml NodeLabel capacity configuration

2015-07-08 Thread Bibin A Chundatt (JIRA) 
Bibin A Chundatt created YARN-3894:
--

 Summary: RM startup should fail for wrong CS xml NodeLabel 
capacity configuration 
 Key: YARN-3894
 URL: https://issues.apache.org/jira/browse/YARN-3894
 Project: Hadoop YARN
  Issue Type: Bug
  Components: capacityscheduler
Reporter: Bibin A Chundatt
Assignee: Bibin A Chundatt
Priority: Critical


Currently in capacity Scheduler when capacity configuration is wrong
RM shutdown is the current behaviour, but not incase of NodeLabels capacity 
mismatch


In {{CapacityScheduler#initializeQueues}}

{code}
  private void initializeQueues(CapacitySchedulerConfiguration conf)
throws IOException {   
root = 
parseQueue(this, conf, null, CapacitySchedulerConfiguration.ROOT, 
queues, queues, noop);
labelManager.reinitializeQueueLabels(getQueueToLabels());
root = 
parseQueue(this, conf, null, CapacitySchedulerConfiguration.ROOT, 
queues, queues, noop);
LOG.info("Initialized root queue " + root);
initializeQueueMappings();
setQueueAcls(authorizer, queues);
  }
{code}

{{labelManager}} is initialized from queues and calculation for Label level 
capacity mismatch happens in {{parseQueue}} . So during initialization 
{{parseQueue}} the labels will be empty . 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)