Re: JIRA dependencies and Jenkins
On 07/22/15 03:08 PM, Chris Nauroth wrote: Jenkins will not recognize inter-dependencies of multiple jiras and try to pull in dependent patches. You'd need to wait for the dependent patch to get reviewed and committed first, and then trigger the Jenkins run for the second patch. OK. Sometimes people like to get a preview Jenkins run to see an early health check instead of waiting for the commit. You can do that by uploading a consolidated patch file to the second jira that includes all of the changes and applies cleanly to trunk. After the first patch gets committed, you would once again upload a new patch file on the second jira containing just the second set of changes. (The final Jenkins run should be done on what is actually getting committed within the scope of that second jira.) Looks like I did a preview Jenkins run by mistake. OK, thanks for the info. Will wait for the integration of the first patch. Martin Thanks for the patches, Martin! --Chris Nauroth On 7/22/15, 6:29 AM, Martin Walsh martin.wa...@oracle.com wrote: I am working on the following two bugs: https://issues.apache.org/jira/browse/HADOOP-12184 https://issues.apache.org/jira/browse/HADOOP-7824 HADOOP-12184 is in the Patch Available state and is marked as a requirement for HADOOP-7824. When I submit the patch for HADOOP-7824 it will touch the same section of code that HADOOP-12184 has done. Therefore my patch will only apply cleanly if HADOOP-12184 patch is applied first. Is Jenkins smart enough to pull in required patches as detailed in the JIRA and apply them first, or does the dependency need to be committed back to the repository? Thanks, Martin
[jira] [Resolved] (HADOOP-7824) Native IO uses wrong constants almost everywhere
[ https://issues.apache.org/jira/browse/HADOOP-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HADOOP-7824. -- Resolution: Fixed Fix Version/s: 2.8.0 committed to 2.8, thanks! Native IO uses wrong constants almost everywhere Key: HADOOP-7824 URL: https://issues.apache.org/jira/browse/HADOOP-7824 Project: Hadoop Common Issue Type: Sub-task Components: native Affects Versions: 0.20.204.0, 0.20.205.0, 1.0.3, 0.23.0, 2.0.0-alpha, 3.0.0 Environment: Mac OS X, Linux, Solaris, Windows, ... Reporter: Dmytro Shteflyuk Assignee: Martin Walsh Labels: hadoop Fix For: 2.8.0 Attachments: HADOOP-7824.001.patch, HADOOP-7824.002.patch, HADOOP-7824.patch, HADOOP-7824.patch, hadoop-7824.txt Constants like O_CREAT, O_EXCL, etc. have different values on OS X and many other operating systems. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HADOOP-7824) Native IO uses wrong constants almost everywhere
[ https://issues.apache.org/jira/browse/HADOOP-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe reopened HADOOP-7824: -- Native IO uses wrong constants almost everywhere Key: HADOOP-7824 URL: https://issues.apache.org/jira/browse/HADOOP-7824 Project: Hadoop Common Issue Type: Sub-task Components: native Affects Versions: 0.20.204.0, 0.20.205.0, 1.0.3, 0.23.0, 2.0.0-alpha, 3.0.0 Environment: Mac OS X, Linux, Solaris, Windows, ... Reporter: Dmytro Shteflyuk Assignee: Martin Walsh Labels: hadoop Fix For: 2.8.0 Attachments: HADOOP-7824.001.patch, HADOOP-7824.002.patch, HADOOP-7824.patch, HADOOP-7824.patch, hadoop-7824.txt Constants like O_CREAT, O_EXCL, etc. have different values on OS X and many other operating systems. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12257) add gradle build tool support
Allen Wittenauer created HADOOP-12257: - Summary: add gradle build tool support Key: HADOOP-12257 URL: https://issues.apache.org/jira/browse/HADOOP-12257 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer We should add support for gradle. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12255) Add support for rubocop
Allen Wittenauer created HADOOP-12255: - Summary: Add support for rubocop Key: HADOOP-12255 URL: https://issues.apache.org/jira/browse/HADOOP-12255 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer We should support rubocop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12256) we should add support for ruby-lint
Allen Wittenauer created HADOOP-12256: - Summary: we should add support for ruby-lint Key: HADOOP-12256 URL: https://issues.apache.org/jira/browse/HADOOP-12256 Project: Hadoop Common Issue Type: Sub-task Affects Versions: HADOOP-12111 Reporter: Allen Wittenauer -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12254) test-patch.sh should run findbugs if ony findbugs-exclude.xml has changed
Varun Saxena created HADOOP-12254: - Summary: test-patch.sh should run findbugs if ony findbugs-exclude.xml has changed Key: HADOOP-12254 URL: https://issues.apache.org/jira/browse/HADOOP-12254 Project: Hadoop Common Issue Type: Bug Environment: We can run findbugs for all the submodules. Reporter: Varun Saxena -- This message was sent by Atlassian JIRA (v6.3.4#6332)
JIRA dependencies and Jenkins
I am working on the following two bugs: https://issues.apache.org/jira/browse/HADOOP-12184 https://issues.apache.org/jira/browse/HADOOP-7824 HADOOP-12184 is in the Patch Available state and is marked as a requirement for HADOOP-7824. When I submit the patch for HADOOP-7824 it will touch the same section of code that HADOOP-12184 has done. Therefore my patch will only apply cleanly if HADOOP-12184 patch is applied first. Is Jenkins smart enough to pull in required patches as detailed in the JIRA and apply them first, or does the dependency need to be committed back to the repository? Thanks, Martin
Re: Planning Hadoop 2.6.1 release
Hi Vinod, We've gone through the various lists of upstream jiras and wanted to provide our take on what we'd like to see in 2.6.1 release. In addition, we have ~58 jiras (some a group of work such as DN maintenance state) that we already have in production in a pre-2.6 release. I've gone through and selected those that seem relevant for 2.6.1. Jiras already marked with 2.6.1-candidate (we agree should be part of 2.6.1): HADOOP-11802 HDFS-7489 HDFS-7533 HDFS-7587 HDFS-7596 HDFS-7707 HDFS-7742 HDFS-8046 HDFS-8072 HDFS-8127 MAPREDUCE-6303 MAPREDUCE-6361 YARN-2414 YARN-2905 YARN-3369 YARN-3485 YARN-3585 YARN-3641 HDFS-7443 HDFS-7575 Jiras not yet marked with 2.6.1-candidate that we'd like to see in 2.6.1: HDFS-7213 HDFS-7446 HDFS-7704 HDFS-7788 HDFS-7884 HDFS-7894 HDFS-7916 HDFS-7929 HDFS-7930 HDFS-7980 HDFS-8245 HDFS-8270 HDFS-8404 HDFS-8480 HDFS-8486 MAPREDUCE-6238 YARN-2856 MAPREDUCE-6300 YARN-2952 YARN-2997 YARN-3094 YARN-3222 YARN-3238 YARN-3464 YARN-3526 YARN-3850 Jiras that we're already running in production pre-2.6 that we'd like in 2.6.1-candidate: HADOOP-11812 (committed in 2.8) MAPREDUCE-5649 (committed in 2.8) YARN-3231 (committed in 2.7) MAPREDUCE-6166 (committed in 2.7) HADOOP-11295 (committed in 2.7) HDFS-7314 (committed in 2.8) HDFS-7182 (committed in 2.7) MAPREDUCE-5465 (committed in 2.8) Lower priority jira that we're already running in production in pre-2.6 that are more a nice to have: HDFS-7281 (committed in 3.0) YARN-3176 (not yet committed in OSS) Note that these lists are a culmination of lots of work by many people in our team, so credit goes to all of them. Any possible typo or mistake in copying jira ids are entirely to be blamed on me. Thanks, Joep On Tue, Jul 21, 2015 at 2:15 AM, Akira AJISAKA ajisa...@oss.nttdata.co.jp wrote: Thanks Vinod for updating the candidate list. I'd like to include the followings 12 JIRAs: * YARN-3641 * YARN-3585 * YARN-2910 * HDFS-8431 * HDFS-7830 * HDFS-7763 * HDFS-7742 * HDFS-7235 * HDFS-7225 * MAPREDUCE-6324 * HADOOP-11934 * HADOOP-11491 Thanks, Akira On 7/18/15 11:13, Vinod Kumar Vavilapalli wrote: - I also have a bunch of patches that I’d like to include, will update them right away. I’ve just finished this. The latest 2.6.1-candidate list is up at 64 JIRAs. Others, please look at the list and post anything else you’d like to get included for 2.6.1. Thanks +Vinod On Jul 15, 2015, at 6:24 PM, Vinod Kumar Vavilapalli vino...@hortonworks.commailto:vino...@hortonworks.com wrote: Alright, I’d like to make progress while the issue is hot. I created a label to discuss on the candidate list of patches: https://issues.apache.org/jira/issues/?jql=labels%20%3D%202.6.1-candidate https://issues.apache.org/jira/issues/?jql=labels%20=%202.6.1-candidate Next steps, I’ll do the following - Review 2.7 and 2.8 blocker/critical tickets and see what makes sense for 2.6.1 and add as candidates - I haven’t reviewed the current list yet, the seed list is from this email thread. Will review them. - I also have a bunch of patches that I’d like to include, will update them right away. Others, please look at the current list and let me know what else you’d like to include. I’d like to keep this ‘candidate-collection’ cycle’ for a max of a week and then start the release process. @Akira, let’s sync up offline on how to take this forward in terms of the release process. Thanks +Vinod
[jira] [Resolved] (HADOOP-12106) JceAesCtrCryptoCodec.java may have an issue with Cypher.update(ByteBuffer, ByteBuffer)
[ https://issues.apache.org/jira/browse/HADOOP-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Reix resolved HADOOP-12106. Resolution: Invalid Release Note: The issue was in the IBM JVM which did not integrate a fix that was integrated into OpenJDK. JceAesCtrCryptoCodec.java may have an issue with Cypher.update(ByteBuffer, ByteBuffer) -- Key: HADOOP-12106 URL: https://issues.apache.org/jira/browse/HADOOP-12106 Project: Hadoop Common Issue Type: Bug Components: security Affects Versions: 2.6.0, 2.7.0 Environment: Hadoop 2.60 and 2.7+ - AIX/PowerPC/IBMJVM - Ubuntu/i386/IBMJVM Reporter: Tony Reix Attachments: mvn.Test.TestCryptoStreamsForLocalFS.res20.AIX.Errors, mvn.Test.TestCryptoStreamsForLocalFS.res20.Ubuntu-i386.IBMJVM.Errors, mvn.Test.TestCryptoStreamsForLocalFS.res22.OpenJDK.Errors On AIX (IBM JVM available only), many sub-tests of : org.apache.hadoop.crypto.TestCryptoStreamsForLocalFS fail: Tests run: 13, Failures: 5, Errors: 1, Skipped: - testCryptoIV - testSeek - testSkip - testAvailable - testPositionedRead When testing SAME exact code on Ubuntu/i386 : - with OpenJDK, all tests are OK - with IBM JVM, tests randomly fail. The issue may be in the IBM JVM, or in some Hadoop code that not perfectly handles differences due to different IBM JVM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12258) Need translate java.nio.file.NoSuchFileException to FileNotFoundException in DeprecatedRawLocalFileStatus constructor
zhihai xu created HADOOP-12258:
--
Summary: Need translate java.nio.file.NoSuchFileException to
FileNotFoundException in DeprecatedRawLocalFileStatus constructor
Key: HADOOP-12258
URL: https://issues.apache.org/jira/browse/HADOOP-12258
Project: Hadoop Common
Issue Type: Bug
Components: fs
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Blocker
need translate java.nio.file.NoSuchFileException to FileNotFoundException in
DeprecatedRawLocalFileStatus constructor.
HADOOP-12045 adds nio to support access time, but nio will create
java.nio.file.NoSuchFileException instead of FileNotFoundException.
many hadoop codes depend on FileNotFoundException to decide whether a file
exists. for example {{FileContext.util().exists()}}.
{code}
public boolean exists(final Path f) throws AccessControlException,
UnsupportedFileSystemException, IOException {
try {
FileStatus fs = FileContext.this.getFileStatus(f);
assert fs != null;
return true;
} catch (FileNotFoundException e) {
return false;
}
}
{code}
same for {{FileSystem#exists}}
{code}
public boolean exists(Path f) throws IOException {
try {
return getFileStatus(f) != null;
} catch (FileNotFoundException e) {
return false;
}
}
{code}
NoSuchFileException will break these functions.
Several test failures are caused by this issue:
https://builds.apache.org/job/PreCommit-YARN-Build/8630/testReport/org.apache.hadoop.yarn.server.nodemanager/TestDeletionService/testRelativeDelete/
https://builds.apache.org/job/PreCommit-YARN-Build/8632/testReport/org.apache.hadoop.yarn.server.nodemanager/TestDeletionService/testAbsDelete/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-12259) Utility to Dynamic port allocation
Brahma Reddy Battula created HADOOP-12259: - Summary: Utility to Dynamic port allocation Key: HADOOP-12259 URL: https://issues.apache.org/jira/browse/HADOOP-12259 Project: Hadoop Common Issue Type: Bug Components: test, util Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula As per discussion in YARN-3528 and [~rkanter] comment [here | https://issues.apache.org/jira/browse/YARN-3528?focusedCommentId=14637700page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14637700 ] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: JIRA dependencies and Jenkins
Jenkins will not recognize inter-dependencies of multiple jiras and try to pull in dependent patches. You'd need to wait for the dependent patch to get reviewed and committed first, and then trigger the Jenkins run for the second patch. Sometimes people like to get a preview Jenkins run to see an early health check instead of waiting for the commit. You can do that by uploading a consolidated patch file to the second jira that includes all of the changes and applies cleanly to trunk. After the first patch gets committed, you would once again upload a new patch file on the second jira containing just the second set of changes. (The final Jenkins run should be done on what is actually getting committed within the scope of that second jira.) Thanks for the patches, Martin! --Chris Nauroth On 7/22/15, 6:29 AM, Martin Walsh martin.wa...@oracle.com wrote: I am working on the following two bugs: https://issues.apache.org/jira/browse/HADOOP-12184 https://issues.apache.org/jira/browse/HADOOP-7824 HADOOP-12184 is in the Patch Available state and is marked as a requirement for HADOOP-7824. When I submit the patch for HADOOP-7824 it will touch the same section of code that HADOOP-12184 has done. Therefore my patch will only apply cleanly if HADOOP-12184 patch is applied first. Is Jenkins smart enough to pull in required patches as detailed in the JIRA and apply them first, or does the dependency need to be committed back to the repository? Thanks, Martin
Restructuring and Expansion of the Hadoop wiki
Hi All, As we had seen so many repetitive queries/questions from the user mailing list, its better to stream-line such queries, so that ever user first look into this guide before asking any query. I propose to restructure the Hadoop-wiki with more comprehensive way with common issues raised on the user@ list. Traditionally, the place to handle this kind of queries is in the Apache Hadoop wiki(Mostly FAQ/Troubleshooting), with pages like below. https://wiki.apache.org/hadoop/ConnectionRefused I think, we can give wiki permissions to interested contributors. Thanks Regards Brahma Reddy Battula
