[jira] [Commented] (HADOOP-11104) org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical parameter checking
[ https://issues.apache.org/jira/browse/HADOOP-11104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903249#comment-14903249 ] Ray Chiang commented on HADOOP-11104: - RE: Failed unit test TestZKFailoverController passes in my tree. > org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical parameter > checking > - > > Key: HADOOP-11104 > URL: https://issues.apache.org/jira/browse/HADOOP-11104 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Chiang >Assignee: Ray Chiang >Priority: Minor > Labels: BB2015-05-TBR, newbie, supportability > Attachments: HADOOP-11104.001.patch, HADOOP-11104.002.patch, > HADOOP-11104.003.patch > > > Passing a negative value to the interval field of > MetricsRegistry#newQuantiles should throw a MetricsException with a clear > error message. The current stack trace looks something like: > java.lang.IllegalArgumentException: null > at > java.util.concurrent.ScheduledThreadPoolExecutor.scheduleAtFixedRate(ScheduledThreadPoolExecutor.java:420) > at > org.apache.hadoop.metrics2.lib.MutableQuantiles.(MutableQuantiles.java:107) > at > org.apache.hadoop.metrics2.lib.MetricsRegistry.newQuantiles(MetricsRegistry.java:200) > Along similar lines, should the other methods like > MetricsRegistry#newCounter() also have parameter checking for negative > int/long values? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11890) Uber-JIRA: Hadoop should support IPv6
[ https://issues.apache.org/jira/browse/HADOOP-11890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903225#comment-14903225 ] Colin Patrick McCabe commented on HADOOP-11890: --- Good work, guys. We usually make the specific things we're doing on a branch into subtasks... do you mind if I move HADOOP-11582, HADOOP-11990, HDFS-8078, etc. as subtasks rather than separate JIRAs? > Uber-JIRA: Hadoop should support IPv6 > - > > Key: HADOOP-11890 > URL: https://issues.apache.org/jira/browse/HADOOP-11890 > Project: Hadoop Common > Issue Type: Improvement > Components: net >Reporter: Nate Edel >Assignee: Nate Edel > Labels: ipv6 > > Hadoop currently treats IPv6 as unsupported. Track related smaller issues to > support IPv6. > (Current case here is mainly HBase on HDFS, so any suggestions about other > test cases/workload are really appreciated.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11104) org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical parameter checking
[
https://issues.apache.org/jira/browse/HADOOP-11104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903226#comment-14903226
]
Hadoop QA commented on HADOOP-11104:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 24m 30s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 10m 48s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 14m 48s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 30s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 38s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 2m 15s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 42s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 43s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 28m 22s | Tests failed in
hadoop-common. |
| | | 86m 19s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.ha.TestZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12761524/HADOOP-11104.003.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / cc2b473 |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7686/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7686/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7686/console |
This message was automatically generated.
> org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical parameter
> checking
> -
>
> Key: HADOOP-11104
> URL: https://issues.apache.org/jira/browse/HADOOP-11104
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Minor
> Labels: BB2015-05-TBR, newbie, supportability
> Attachments: HADOOP-11104.001.patch, HADOOP-11104.002.patch,
> HADOOP-11104.003.patch
>
>
> Passing a negative value to the interval field of
> MetricsRegistry#newQuantiles should throw a MetricsException with a clear
> error message. The current stack trace looks something like:
> java.lang.IllegalArgumentException: null
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.scheduleAtFixedRate(ScheduledThreadPoolExecutor.java:420)
> at
> org.apache.hadoop.metrics2.lib.MutableQuantiles.(MutableQuantiles.java:107)
> at
> org.apache.hadoop.metrics2.lib.MetricsRegistry.newQuantiles(MetricsRegistry.java:200)
> Along similar lines, should the other methods like
> MetricsRegistry#newCounter() also have parameter checking for negative
> int/long values?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12421) Add jitter to RetryInvocationHandler
[ https://issues.apache.org/jira/browse/HADOOP-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HADOOP-12421: --- Attachment: HADOOP-12421-v2.patch > Add jitter to RetryInvocationHandler > > > Key: HADOOP-12421 > URL: https://issues.apache.org/jira/browse/HADOOP-12421 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HADOOP-12421-v1.patch, HADOOP-12421-v2.patch > > > Calls to NN can become synchronized across a cluster during NN failover. This > leads to a spike in requests until things recover. Making an already tricky > time worse. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12421) Add jitter to RetryInvocationHandler
[ https://issues.apache.org/jira/browse/HADOOP-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902032#comment-14902032 ] Elliott Clark commented on HADOOP-12421: Whoops yeah retry time can be 0. > Add jitter to RetryInvocationHandler > > > Key: HADOOP-12421 > URL: https://issues.apache.org/jira/browse/HADOOP-12421 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HADOOP-12421-v1.patch > > > Calls to NN can become synchronized across a cluster during NN failover. This > leads to a spike in requests until things recover. Making an already tricky > time worse. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12428) Fix inconsistency between log-level guards and statements
[
https://issues.apache.org/jira/browse/HADOOP-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902009#comment-14902009
]
Hudson commented on HADOOP-12428:
-
SUCCESS: Integrated in Hadoop-Yarn-trunk #1161 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/1161/])
HADOOP-12428. Fix inconsistency between log-level guards and statements.
Contributed by Jagadesh Kiran N and Jackie Chang. (ozawa: rev
dfd807afab0fae3839c9cc5d552aa030f956)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
*
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/SerialJobFactory.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerRequestor.java
* hadoop-yarn-project/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/NMTokenSelector.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java
> Fix inconsistency between log-level guards and statements
> -
>
> Key: HADOOP-12428
> URL: https://issues.apache.org/jira/browse/HADOOP-12428
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jackie Chang
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Labels: BB2015-05-TBR
> Fix For: 2.8.0
>
> Attachments: HADOOP-9995-00.patch, HADOOP-9995.patch,
> MAPREDUCE-6468-01.patch, MAPREDUCE-6468-02.patch, MAPREDUCE-6468-03.patch,
> MAPREDUCE-6468-04.patch
>
>
> Developers use logs to do in-house debugging. These log statements are later
> demoted to less severe levels and usually are guarded by their matching
> severity levels. However, we do see inconsistencies in trunk. A log statement
> like
> {code}
>if (LOG.isDebugEnabled()) {
> LOG.info("Assigned container (" + allocated + ") "
> {code}
> doesn't make much sense because the log message is actually only printed out
> in DEBUG-level. We do see previous issues tried to correct this
> inconsistency. I am proposing a comprehensive correction over trunk.
> Doug Cutting pointed it out in HADOOP-312:
> https://issues.apache.org/jira/browse/HADOOP-312?focusedCommentId=12429498=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12429498
> HDFS-1611 also corrected this inconsistency.
> This could have been avoided by switching from log4j to slf4j's {} format
> like CASSANDRA-625 (2010/3) and ZOOKEEPER-850 (2012/1), which gives cleaner
> code and slightly higher performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12428) Fix inconsistency between log-level guards and statements
[
https://issues.apache.org/jira/browse/HADOOP-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902014#comment-14902014
]
Hudson commented on HADOOP-12428:
-
SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #429 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/429/])
HADOOP-12428. Fix inconsistency between log-level guards and statements.
Contributed by Jagadesh Kiran N and Jackie Chang. (ozawa: rev
dfd807afab0fae3839c9cc5d552aa030f956)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/SerialJobFactory.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerRequestor.java
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java
*
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/NMTokenSelector.java
> Fix inconsistency between log-level guards and statements
> -
>
> Key: HADOOP-12428
> URL: https://issues.apache.org/jira/browse/HADOOP-12428
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jackie Chang
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Labels: BB2015-05-TBR
> Fix For: 2.8.0
>
> Attachments: HADOOP-9995-00.patch, HADOOP-9995.patch,
> MAPREDUCE-6468-01.patch, MAPREDUCE-6468-02.patch, MAPREDUCE-6468-03.patch,
> MAPREDUCE-6468-04.patch
>
>
> Developers use logs to do in-house debugging. These log statements are later
> demoted to less severe levels and usually are guarded by their matching
> severity levels. However, we do see inconsistencies in trunk. A log statement
> like
> {code}
>if (LOG.isDebugEnabled()) {
> LOG.info("Assigned container (" + allocated + ") "
> {code}
> doesn't make much sense because the log message is actually only printed out
> in DEBUG-level. We do see previous issues tried to correct this
> inconsistency. I am proposing a comprehensive correction over trunk.
> Doug Cutting pointed it out in HADOOP-312:
> https://issues.apache.org/jira/browse/HADOOP-312?focusedCommentId=12429498=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12429498
> HDFS-1611 also corrected this inconsistency.
> This could have been avoided by switching from log4j to slf4j's {} format
> like CASSANDRA-625 (2010/3) and ZOOKEEPER-850 (2012/1), which gives cleaner
> code and slightly higher performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11104) org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical parameter checking
[ https://issues.apache.org/jira/browse/HADOOP-11104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated HADOOP-11104: Status: Patch Available (was: Open) Forgot to submit patch. > org.apache.hadoop.metrics2.lib.MetricsRegistry needs numerical parameter > checking > - > > Key: HADOOP-11104 > URL: https://issues.apache.org/jira/browse/HADOOP-11104 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ray Chiang >Assignee: Ray Chiang >Priority: Minor > Labels: BB2015-05-TBR, newbie, supportability > Attachments: HADOOP-11104.001.patch, HADOOP-11104.002.patch, > HADOOP-11104.003.patch > > > Passing a negative value to the interval field of > MetricsRegistry#newQuantiles should throw a MetricsException with a clear > error message. The current stack trace looks something like: > java.lang.IllegalArgumentException: null > at > java.util.concurrent.ScheduledThreadPoolExecutor.scheduleAtFixedRate(ScheduledThreadPoolExecutor.java:420) > at > org.apache.hadoop.metrics2.lib.MutableQuantiles.(MutableQuantiles.java:107) > at > org.apache.hadoop.metrics2.lib.MetricsRegistry.newQuantiles(MetricsRegistry.java:200) > Along similar lines, should the other methods like > MetricsRegistry#newCounter() also have parameter checking for negative > int/long values? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-4258) the test patch script should check for filenames that differ only in case
[ https://issues.apache.org/jira/browse/HADOOP-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902886#comment-14902886 ] Kengo Seki commented on HADOOP-4258: Sorry [~jagadesh.kiran], I'm on vacation until Sep. 27th. I'll review your patch next Monday. > the test patch script should check for filenames that differ only in case > - > > Key: HADOOP-4258 > URL: https://issues.apache.org/jira/browse/HADOOP-4258 > Project: Hadoop Common > Issue Type: Sub-task > Components: build, test >Reporter: Owen O'Malley >Assignee: Jagadesh Kiran N > Labels: test-patch > Attachments: HADOOP-4258.001.patch, > HADOOP-4258.HADOOP-12111.00.patch, HADOOP-4258.HADOOP-12111.01.patch, > HADOOP-4258.HADOOP-12111.02.patch > > > It would be nice if the test patch script warned about filenames that differ > only in case. We recently had a patch committed that had a pair of colliding > filenames and subversion broke badly on my Mac. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12429) NullPointerException when underreplicated blocks are there
Syed Akram created HADOOP-12429: --- Summary: NullPointerException when underreplicated blocks are there Key: HADOOP-12429 URL: https://issues.apache.org/jira/browse/HADOOP-12429 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.1 Reporter: Syed Akram Assignee: Syed Akram 2015-09-22 09:48:47,830 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: dn1:50010:DataXceiver error processing WRITE_BLOCK operation src: /dn1:42973 dst: /dn2:50010 java.lang.NullPointerException at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.(BlockReceiver.java:186) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:677) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:137) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:74) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:251) at java.lang.Thread.run(Thread.java:744) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12361) reorganize repo layout for break from Hadoop code base
[ https://issues.apache.org/jira/browse/HADOOP-12361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12361: - Issue Type: Task (was: Sub-task) Parent: (was: HADOOP-12111) > reorganize repo layout for break from Hadoop code base > -- > > Key: HADOOP-12361 > URL: https://issues.apache.org/jira/browse/HADOOP-12361 > Project: Hadoop Common > Issue Type: Task > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Sean Busbey >Assignee: Sean Busbey > Attachments: HADOOP-12361.HADOOP-12111.1.sh > > > Reorganize current top level repo to include our starting modules (and only > those modules), so that it's easier to start bringing in new contributors: > * shelldoc > * releasedocmaker > * interface audience > * test-patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12424) Add a function to build unique cache key for Token.
[ https://issues.apache.org/jira/browse/HADOOP-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903145#comment-14903145 ] Xiaobing Zhou commented on HADOOP-12424: [~owen.omalley] can you also review it? Thanks. This is actual separation of patch of HDFS-8855 that fails the build since it doesn't compile hadoop common changes. > Add a function to build unique cache key for Token. > --- > > Key: HADOOP-12424 > URL: https://issues.apache.org/jira/browse/HADOOP-12424 > Project: Hadoop Common > Issue Type: Bug >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou >Priority: Critical > Attachments: HADOOP-12424.001.patch > > > HDFS-8855 needs facility function from Token to build unique cache key. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (HADOOP-12430) HDFS client gets errors trying to to connect to IPv6 DataNode
[ https://issues.apache.org/jira/browse/HADOOP-12430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe moved HDFS-8078 to HADOOP-12430: - Affects Version/s: (was: 2.6.0) 2.6.0 Component/s: (was: hdfs-client) Key: HADOOP-12430 (was: HDFS-8078) Project: Hadoop Common (was: Hadoop HDFS) > HDFS client gets errors trying to to connect to IPv6 DataNode > - > > Key: HADOOP-12430 > URL: https://issues.apache.org/jira/browse/HADOOP-12430 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: Nate Edel >Assignee: Nate Edel > Labels: BB2015-05-TBR, ipv6 > Attachments: HDFS-8078.10.patch, HDFS-8078.11.patch, > HDFS-8078.12.patch, HDFS-8078.13.patch, HDFS-8078.14.patch, > HDFS-8078.15.patch, HDFS-8078.9.patch, dummy.patch > > > 1st exception, on put: > 15/03/23 18:43:18 WARN hdfs.DFSClient: DataStreamer Exception > java.lang.IllegalArgumentException: Does not contain a valid host:port > authority: 2401:db00:1010:70ba:face:0:8:0:50010 > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212) > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153) > at > org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1607) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1408) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1361) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:588) > Appears to actually stem from code in DataNodeID which assumes it's safe to > append together (ipaddr + ":" + port) -- which is OK for IPv4 and not OK for > IPv6. NetUtils.createSocketAddr( ) assembles a Java URI object, which > requires the format proto://[2401:db00:1010:70ba:face:0:8:0]:50010 > Currently using InetAddress.getByName() to validate IPv6 (guava > InetAddresses.forString has been flaky) but could also use our own parsing. > (From logging this, it seems like a low-enough frequency call that the extra > object creation shouldn't be problematic, and for me the slight risk of > passing in bad input that is not actually an IPv4 or IPv6 address and thus > calling an external DNS lookup is outweighed by getting the address > normalized and avoiding rewriting parsing.) > Alternatively, sun.net.util.IPAddressUtil.isIPv6LiteralAddress() > --- > 2nd exception (on datanode) > 15/04/13 13:18:07 ERROR datanode.DataNode: > dev1903.prn1.facebook.com:50010:DataXceiver error processing unknown > operation src: /2401:db00:20:7013:face:0:7:0:54152 dst: > /2401:db00:11:d010:face:0:2f:0:50010 > java.io.EOFException > at java.io.DataInputStream.readShort(DataInputStream.java:315) > at > org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:58) > at > org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:226) > at java.lang.Thread.run(Thread.java:745) > Which also comes as client error "-get: 2401 is not an IP string literal." > This one has existing parsing logic which needs to shift to the last colon > rather than the first. Should also be a tiny bit faster by using lastIndexOf > rather than split. Could alternatively use the techniques above. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11582) org.apache.hadoop.net.TestDNS failing with NumberFormatException -IPv6 related?
[
https://issues.apache.org/jira/browse/HADOOP-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-11582:
--
Issue Type: Sub-task (was: Bug)
Parent: HADOOP-11890
> org.apache.hadoop.net.TestDNS failing with NumberFormatException -IPv6
> related?
> ---
>
> Key: HADOOP-11582
> URL: https://issues.apache.org/jira/browse/HADOOP-11582
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: net
>Affects Versions: 3.0.0
> Environment: OSX yosemite
>Reporter: Steve Loughran
>Assignee: Varun Saxena
> Labels: ipv6
>
> {{org.apache.hadoop.net.TestDNS}} failing {{java.lang.NumberFormatException:
> For input string: ":3246:9aff:fe80:438f"}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11072) better Logging in DNS.java
[
https://issues.apache.org/jira/browse/HADOOP-11072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-11072:
--
Issue Type: Sub-task (was: Bug)
Parent: HADOOP-11890
> better Logging in DNS.java
> ---
>
> Key: HADOOP-11072
> URL: https://issues.apache.org/jira/browse/HADOOP-11072
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: net
>Affects Versions: 2.3.0, 2.4.0
>Reporter: jay vyas
>Priority: Minor
> Labels: logging
>
> The DNS.java class should be more informative, and possibly fail early when
> reverse DNS is broken. Right now it is vulnerable to a cryptic ArrayIndex
> exception.
> {noformat}
> parts = hostIp.getHostAddress()
> String reverseIp = parts[3] + parts[2] ...
> {noformat}
> During this patch I think we can also improve a couple other minor logging
> statements in the net package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-3619) DNS.getHosts triggers an ArrayIndexOutOfBoundsException in reverseDNS if one of the interfaces is IPv6
[ https://issues.apache.org/jira/browse/HADOOP-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HADOOP-3619: - Issue Type: Sub-task (was: Bug) Parent: HADOOP-11890 > DNS.getHosts triggers an ArrayIndexOutOfBoundsException in reverseDNS if one > of the interfaces is IPv6 > -- > > Key: HADOOP-3619 > URL: https://issues.apache.org/jira/browse/HADOOP-3619 > Project: Hadoop Common > Issue Type: Sub-task > Components: net >Reporter: Steve Loughran >Assignee: Dr. Martin Menzel > Labels: BB2015-05-TBR, ipv6, patch > Attachments: HADOOP-3619-v2.patch > > > reverseDNS tries to split a host address string by ".", and so fails if ":" > is the separator, as it is in IPv6. When it tries to access the parts of the > address, a stack trace is seen. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12430) HDFS client gets errors trying to to connect to IPv6 DataNode
[ https://issues.apache.org/jira/browse/HADOOP-12430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HADOOP-12430: -- Issue Type: Sub-task (was: Bug) Parent: HADOOP-11890 > HDFS client gets errors trying to to connect to IPv6 DataNode > - > > Key: HADOOP-12430 > URL: https://issues.apache.org/jira/browse/HADOOP-12430 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.6.0 >Reporter: Nate Edel >Assignee: Nate Edel > Labels: BB2015-05-TBR, ipv6 > Attachments: HDFS-8078.10.patch, HDFS-8078.11.patch, > HDFS-8078.12.patch, HDFS-8078.13.patch, HDFS-8078.14.patch, > HDFS-8078.15.patch, HDFS-8078.9.patch, dummy.patch > > > 1st exception, on put: > 15/03/23 18:43:18 WARN hdfs.DFSClient: DataStreamer Exception > java.lang.IllegalArgumentException: Does not contain a valid host:port > authority: 2401:db00:1010:70ba:face:0:8:0:50010 > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212) > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164) > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153) > at > org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1607) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1408) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1361) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:588) > Appears to actually stem from code in DataNodeID which assumes it's safe to > append together (ipaddr + ":" + port) -- which is OK for IPv4 and not OK for > IPv6. NetUtils.createSocketAddr( ) assembles a Java URI object, which > requires the format proto://[2401:db00:1010:70ba:face:0:8:0]:50010 > Currently using InetAddress.getByName() to validate IPv6 (guava > InetAddresses.forString has been flaky) but could also use our own parsing. > (From logging this, it seems like a low-enough frequency call that the extra > object creation shouldn't be problematic, and for me the slight risk of > passing in bad input that is not actually an IPv4 or IPv6 address and thus > calling an external DNS lookup is outweighed by getting the address > normalized and avoiding rewriting parsing.) > Alternatively, sun.net.util.IPAddressUtil.isIPv6LiteralAddress() > --- > 2nd exception (on datanode) > 15/04/13 13:18:07 ERROR datanode.DataNode: > dev1903.prn1.facebook.com:50010:DataXceiver error processing unknown > operation src: /2401:db00:20:7013:face:0:7:0:54152 dst: > /2401:db00:11:d010:face:0:2f:0:50010 > java.io.EOFException > at java.io.DataInputStream.readShort(DataInputStream.java:315) > at > org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:58) > at > org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:226) > at java.lang.Thread.run(Thread.java:745) > Which also comes as client error "-get: 2401 is not an IP string literal." > This one has existing parsing logic which needs to shift to the last colon > rather than the first. Should also be a tiny bit faster by using lastIndexOf > rather than split. Could alternatively use the techniques above. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12122) Hadoop should avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/HADOOP-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HADOOP-12122: -- Issue Type: Sub-task (was: Bug) Parent: HADOOP-11890 > Hadoop should avoid unsafe split and append on fields that might be IPv6 > literals > - > > Key: HADOOP-12122 > URL: https://issues.apache.org/jira/browse/HADOOP-12122 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Nate Edel >Assignee: Nate Edel > Attachments: lets_blow_up_a_lot_of_tests.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-3619) DNS.getHosts triggers an ArrayIndexOutOfBoundsException in reverseDNS if one of the interfaces is IPv6
[
https://issues.apache.org/jira/browse/HADOOP-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903837#comment-14903837
]
Hadoop QA commented on HADOOP-3619:
---
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 20m 41s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 9m 21s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 6s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 12s | The applied patch generated
27 new checkstyle issues (total was 28, now 55). |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 3 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 53s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 23m 3s | Tests failed in
hadoop-common. |
| | | 68m 49s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.fs.TestLocalFsFCStatistics |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12655523/HADOOP-3619-v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / cc2b473 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7688/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7688/artifact/patchprocess/whitespace.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7688/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7688/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7688/console |
This message was automatically generated.
> DNS.getHosts triggers an ArrayIndexOutOfBoundsException in reverseDNS if one
> of the interfaces is IPv6
> --
>
> Key: HADOOP-3619
> URL: https://issues.apache.org/jira/browse/HADOOP-3619
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: net
>Reporter: Steve Loughran
>Assignee: Dr. Martin Menzel
> Labels: BB2015-05-TBR, ipv6, patch
> Attachments: HADOOP-3619-v2.patch
>
>
> reverseDNS tries to split a host address string by ".", and so fails if ":"
> is the separator, as it is in IPv6. When it tries to access the parts of the
> address, a stack trace is seen.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12433) Migrate open jiras to Yetus tracker
[ https://issues.apache.org/jira/browse/HADOOP-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903835#comment-14903835 ] Allen Wittenauer commented on HADOOP-12433: --- Argh sorry. I just committed 7617e5cb775e8af7441faaecdabe03e385a4156f I'll stop doing that. :) > Migrate open jiras to Yetus tracker > --- > > Key: HADOOP-12433 > URL: https://issues.apache.org/jira/browse/HADOOP-12433 > Project: Hadoop Common > Issue Type: Sub-task > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > > move anything that isn't in the HADOOP-12111 branch already over to the yetus > tracker. preferably with nothing landing in the branch after commit > 0a6e5d109b7f28ed3e2ae780898e79ad16faa954 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12434) tombstone yetus branch
Sean Busbey created HADOOP-12434: Summary: tombstone yetus branch Key: HADOOP-12434 URL: https://issues.apache.org/jira/browse/HADOOP-12434 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Sean Busbey Priority: Blocker Tasks to finalize code movement to Apache Yetus 1. tag the HADOOP-12111 branch * preferably at commit 0a6e5d109b7f28ed3e2ae780898e79ad16faa954 * probably something like HADOOP-12111-moved-to-yetus 2. delete the HADOOP-12111 branch 3. iff any patches land after 0a6e5d109b7f28ed3e2ae780898e79ad16faa954, cherry pick over to Yetus / master -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HADOOP-12433) Migrate open jiras to Yetus tracker
[ https://issues.apache.org/jira/browse/HADOOP-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-12433 started by Sean Busbey. > Migrate open jiras to Yetus tracker > --- > > Key: HADOOP-12433 > URL: https://issues.apache.org/jira/browse/HADOOP-12433 > Project: Hadoop Common > Issue Type: Sub-task > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > > move anything that isn't in the HADOOP-12111 branch already over to the yetus > tracker. preferably with nothing landing in the branch after commit > 0a6e5d109b7f28ed3e2ae780898e79ad16faa954 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12427) Upgrade Mockito version to 1.10.19
[
https://issues.apache.org/jira/browse/HADOOP-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903738#comment-14903738
]
Hadoop QA commented on HADOOP-12427:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 19m 13s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 9m 27s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 44s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 29s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 51s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 38s | The patch built with
eclipse:eclipse. |
| | | 43m 25s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12761772/HADOOP-12427.v0.patch |
| Optional Tests | javadoc javac unit |
| git revision | trunk / cc2b473 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7687/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7687/console |
This message was automatically generated.
> Upgrade Mockito version to 1.10.19
> --
>
> Key: HADOOP-12427
> URL: https://issues.apache.org/jira/browse/HADOOP-12427
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Reporter: Giovanni Matteo Fumarola
>Assignee: Giovanni Matteo Fumarola
>Priority: Minor
> Attachments: HADOOP-12427.v0.patch
>
>
> The current version is 1.8.5 - inserted in 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11072) better Logging in DNS.java
[
https://issues.apache.org/jira/browse/HADOOP-11072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-11072:
--
Issue Type: Bug (was: Sub-task)
Parent: (was: HADOOP-3619)
> better Logging in DNS.java
> ---
>
> Key: HADOOP-11072
> URL: https://issues.apache.org/jira/browse/HADOOP-11072
> Project: Hadoop Common
> Issue Type: Bug
> Components: net
>Affects Versions: 2.3.0, 2.4.0
>Reporter: jay vyas
>Priority: Minor
> Labels: logging
>
> The DNS.java class should be more informative, and possibly fail early when
> reverse DNS is broken. Right now it is vulnerable to a cryptic ArrayIndex
> exception.
> {noformat}
> parts = hostIp.getHostAddress()
> String reverseIp = parts[3] + parts[2] ...
> {noformat}
> During this patch I think we can also improve a couple other minor logging
> statements in the net package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12431) NameNode should bind on both IPv6 and IPv4 if running on dual-stack machine and IPv6 enabled
[ https://issues.apache.org/jira/browse/HADOOP-12431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HADOOP-12431: -- Issue Type: Sub-task (was: Improvement) Parent: HADOOP-11890 > NameNode should bind on both IPv6 and IPv4 if running on dual-stack machine > and IPv6 enabled > > > Key: HADOOP-12431 > URL: https://issues.apache.org/jira/browse/HADOOP-12431 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 2.7.1 >Reporter: Nate Edel >Assignee: Nate Edel > Labels: ipv6 > > NameNode works properly on IPv4 or IPv6 single stack (assuming in the latter > case that scripts have been changed to disable preferIPv4Stack, and dependent > on the client/data node fix in HDFS-8078). On dual-stack machines, NameNode > listens only on IPv4 (even ignoring preferIPv6Addresses being set.) > Our initial use case for IPv6 is IPv6-only clusters, but ideally we'd support > binding to both the IPv4 and IPv6 machine addresses so that we can support > heterogenous clusters (some dual-stack and some IPv6-only machines.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (HADOOP-12431) NameNode should bind on both IPv6 and IPv4 if running on dual-stack machine and IPv6 enabled
[ https://issues.apache.org/jira/browse/HADOOP-12431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe moved HDFS-8271 to HADOOP-12431: - Affects Version/s: (was: 2.7.1) 2.7.1 Component/s: (was: namenode) Key: HADOOP-12431 (was: HDFS-8271) Project: Hadoop Common (was: Hadoop HDFS) > NameNode should bind on both IPv6 and IPv4 if running on dual-stack machine > and IPv6 enabled > > > Key: HADOOP-12431 > URL: https://issues.apache.org/jira/browse/HADOOP-12431 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.7.1 >Reporter: Nate Edel >Assignee: Nate Edel > Labels: ipv6 > > NameNode works properly on IPv4 or IPv6 single stack (assuming in the latter > case that scripts have been changed to disable preferIPv4Stack, and dependent > on the client/data node fix in HDFS-8078). On dual-stack machines, NameNode > listens only on IPv4 (even ignoring preferIPv6Addresses being set.) > Our initial use case for IPv6 is IPv6-only clusters, but ideally we'd support > binding to both the IPv4 and IPv6 machine addresses so that we can support > heterogenous clusters (some dual-stack and some IPv6-only machines.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12432) Support for include/exclude lists on IPv6 setup
[ https://issues.apache.org/jira/browse/HADOOP-12432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HADOOP-12432: -- Issue Type: Sub-task (was: Improvement) Parent: HADOOP-11890 > Support for include/exclude lists on IPv6 setup > --- > > Key: HADOOP-12432 > URL: https://issues.apache.org/jira/browse/HADOOP-12432 > Project: Hadoop Common > Issue Type: Sub-task > Environment: This affects only IPv6 cluster setup >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: HDFS-9026-1.patch, HDFS-9026-2.patch, > HDFS-9026-HADOOP-11890.002.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > This is a tracking item for having e2e IPv6 support in HDFS. > Nate did great ground work in HDFS-8078 but for having whole feature working > e2e this one of the items missing. > Basically today NN won't be able to parse IPv6 addresses if they are present > in include or exclude list. > Patch has a dependency (and has been tested on IPv6 only cluster) on top of > HDFS-8078.14.patch > This should be committed to HADOOP-11890 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11630) Allow hadoop.sh to bind to ipv6 conditionally
[ https://issues.apache.org/jira/browse/HADOOP-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HADOOP-11630: -- Fix Version/s: HADOOP-11890 > Allow hadoop.sh to bind to ipv6 conditionally > - > > Key: HADOOP-11630 > URL: https://issues.apache.org/jira/browse/HADOOP-11630 > Project: Hadoop Common > Issue Type: Sub-task > Components: scripts >Affects Versions: 2.6.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Labels: ipv6 > Fix For: HADOOP-11890 > > Attachments: HDFS-7834-branch-2-0.patch, HDFS-7834-trunk-0.patch > > > Currently the bash scripts unconditionally add -Djava.net.preferIPv4Stack=true > While this was needed a while ago. IPV6 on java works much better now and > there should be a way to allow it to bind dual stack if needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11630) Allow hadoop.sh to bind to ipv6 conditionally
[ https://issues.apache.org/jira/browse/HADOOP-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HADOOP-11630: -- Issue Type: Sub-task (was: Improvement) Parent: HADOOP-11890 > Allow hadoop.sh to bind to ipv6 conditionally > - > > Key: HADOOP-11630 > URL: https://issues.apache.org/jira/browse/HADOOP-11630 > Project: Hadoop Common > Issue Type: Sub-task > Components: scripts >Affects Versions: 2.6.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Labels: ipv6 > Attachments: HDFS-7834-branch-2-0.patch, HDFS-7834-trunk-0.patch > > > Currently the bash scripts unconditionally add -Djava.net.preferIPv4Stack=true > While this was needed a while ago. IPV6 on java works much better now and > there should be a way to allow it to bind dual stack if needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (HADOOP-12432) Support for include/exclude lists on IPv6 setup
[ https://issues.apache.org/jira/browse/HADOOP-12432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe moved HDFS-9026 to HADOOP-12432: - Component/s: (was: namenode) Key: HADOOP-12432 (was: HDFS-9026) Project: Hadoop Common (was: Hadoop HDFS) > Support for include/exclude lists on IPv6 setup > --- > > Key: HADOOP-12432 > URL: https://issues.apache.org/jira/browse/HADOOP-12432 > Project: Hadoop Common > Issue Type: Improvement > Environment: This affects only IPv6 cluster setup >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: HDFS-9026-1.patch, HDFS-9026-2.patch, > HDFS-9026-HADOOP-11890.002.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > This is a tracking item for having e2e IPv6 support in HDFS. > Nate did great ground work in HDFS-8078 but for having whole feature working > e2e this one of the items missing. > Basically today NN won't be able to parse IPv6 addresses if they are present > in include or exclude list. > Patch has a dependency (and has been tested on IPv6 only cluster) on top of > HDFS-8078.14.patch > This should be committed to HADOOP-11890 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11890) Uber-JIRA: Hadoop should support IPv6
[
https://issues.apache.org/jira/browse/HADOOP-11890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903780#comment-14903780
]
Colin Patrick McCabe commented on HADOOP-11890:
---
Great. I moved the "related" jiras to be subtasks.
Can you post a little bit more about what admins will have to do to use ipv6 on
their cluster when this branch is done? Maybe some kind of design doc?
Reading through HADOOP-12122, it sounds like the goal is that if an admin sets
{{HADOOP_ALLOW_IPV6}} and then starts using ipv6 addresses in configuration XML
files, everything will Just Work? Does setting {{HADOOP_ALLOW_IPV6}} also
imply that we will prefer ipv6 addresses internally when resolving hostnames to
IP addresses in the software? Or will the software continue to resolve
hostnames to a mix of ipv4 and ipv6 addresses?
If an admin configures ipv6 addresses in configuration XML files but does not
set {{HADOOP_ALLOW_IPV6}}, what kind of error message do we display?
Are we going to introduce some utility functions to parse ipv6 addresses and
ports?
One thing to keep in mind for this work is that for some people, DNS resolution
is really slow. I have had patches -1ed in the past for introducing extra DNS
resolution steps. We need to be careful not to introduce unnecessary DNS
resolution calls (and no, we can't assume everyone caches DNS lookups forever,
even though many people do).
> Uber-JIRA: Hadoop should support IPv6
> -
>
> Key: HADOOP-11890
> URL: https://issues.apache.org/jira/browse/HADOOP-11890
> Project: Hadoop Common
> Issue Type: Improvement
> Components: net
>Reporter: Nate Edel
>Assignee: Nate Edel
> Labels: ipv6
>
> Hadoop currently treats IPv6 as unsupported. Track related smaller issues to
> support IPv6.
> (Current case here is mainly HBase on HDFS, so any suggestions about other
> test cases/workload are really appreciated.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-4258) the test patch script should check for filenames that differ only in case
[ https://issues.apache.org/jira/browse/HADOOP-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903830#comment-14903830 ] Allen Wittenauer commented on HADOOP-4258: -- This doesn't work on OS X. uniq -d and uniq -c cannot be used simultaneously. > the test patch script should check for filenames that differ only in case > - > > Key: HADOOP-4258 > URL: https://issues.apache.org/jira/browse/HADOOP-4258 > Project: Hadoop Common > Issue Type: Sub-task > Components: build, test >Reporter: Owen O'Malley >Assignee: Jagadesh Kiran N > Labels: test-patch > Attachments: HADOOP-4258.001.patch, > HADOOP-4258.HADOOP-12111.00.patch, HADOOP-4258.HADOOP-12111.01.patch, > HADOOP-4258.HADOOP-12111.02.patch > > > It would be nice if the test patch script warned about filenames that differ > only in case. We recently had a patch committed that had a pair of colliding > filenames and subversion broke badly on my Mac. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12433) Migrate open jiras to Yetus tracker
Sean Busbey created HADOOP-12433: Summary: Migrate open jiras to Yetus tracker Key: HADOOP-12433 URL: https://issues.apache.org/jira/browse/HADOOP-12433 Project: Hadoop Common Issue Type: Sub-task Components: yetus Affects Versions: HADOOP-12111 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker move anything that isn't in the HADOOP-12111 branch already over to the yetus tracker. preferably with nothing landing in the branch after commit 0a6e5d109b7f28ed3e2ae780898e79ad16faa954 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12399) Wrong help messages in some test-patch plugins
[
https://issues.apache.org/jira/browse/HADOOP-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903833#comment-14903833
]
Allen Wittenauer commented on HADOOP-12399:
---
+1 committed.
> Wrong help messages in some test-patch plugins
> --
>
> Key: HADOOP-12399
> URL: https://issues.apache.org/jira/browse/HADOOP-12399
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: yetus
>Affects Versions: HADOOP-12111
>Reporter: Kengo Seki
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Labels: newbie
> Fix For: HADOOP-12111
>
> Attachments: HADOOP-12399.HADOOP-12111.00.patch,
> HADOOP-12399.HADOOP-12111.01.patch, HADOOP-12399.HADOOP-12111.02.patch
>
>
> dev-support/personality/bigtop.sh:
> {code}
> 32 function bigtop_usage
> 33 {
> 34 echo "Bigtop specific:"
> 35 echo "--bigtop-puppetsetup=[false|true] execute the bigtop dev setup
> (needs sudo to root)"
> 36 }
> {code}
> s/bigtop-puppetsetup/bigtop-puppet/.
> dev-support/test-patch.d/gradle.sh:
> {code}
> 21 function gradle_usage
> 22 {
> 23 echo "gradle specific:"
> 24 echo "--gradle-cmd=The 'gradle' command to use (default
> 'gradle')"
> 25 echo "--gradlew-cmd=The 'gradle' command to use (default
> 'basedir/gradlew')"
> 26 }
> {code}
> s/'gradle' command/'gradlew' command/ for the latter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12399) Wrong help messages in some test-patch plugins
[
https://issues.apache.org/jira/browse/HADOOP-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-12399:
--
Resolution: Fixed
Fix Version/s: HADOOP-12111
Status: Resolved (was: Patch Available)
> Wrong help messages in some test-patch plugins
> --
>
> Key: HADOOP-12399
> URL: https://issues.apache.org/jira/browse/HADOOP-12399
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: yetus
>Affects Versions: HADOOP-12111
>Reporter: Kengo Seki
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Labels: newbie
> Fix For: HADOOP-12111
>
> Attachments: HADOOP-12399.HADOOP-12111.00.patch,
> HADOOP-12399.HADOOP-12111.01.patch, HADOOP-12399.HADOOP-12111.02.patch
>
>
> dev-support/personality/bigtop.sh:
> {code}
> 32 function bigtop_usage
> 33 {
> 34 echo "Bigtop specific:"
> 35 echo "--bigtop-puppetsetup=[false|true] execute the bigtop dev setup
> (needs sudo to root)"
> 36 }
> {code}
> s/bigtop-puppetsetup/bigtop-puppet/.
> dev-support/test-patch.d/gradle.sh:
> {code}
> 21 function gradle_usage
> 22 {
> 23 echo "gradle specific:"
> 24 echo "--gradle-cmd=The 'gradle' command to use (default
> 'gradle')"
> 25 echo "--gradlew-cmd=The 'gradle' command to use (default
> 'basedir/gradlew')"
> 26 }
> {code}
> s/'gradle' command/'gradlew' command/ for the latter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11990) DNS#reverseDns fails with a NumberFormatException when using an IPv6 DNS server
[
https://issues.apache.org/jira/browse/HADOOP-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-11990:
--
Issue Type: Sub-task (was: Bug)
Parent: HADOOP-11890
> DNS#reverseDns fails with a NumberFormatException when using an IPv6 DNS
> server
> ---
>
> Key: HADOOP-11990
> URL: https://issues.apache.org/jira/browse/HADOOP-11990
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: net
>Affects Versions: 2.5.1
> Environment: java version "1.7.0_45"
> Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
> Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
>Reporter: Benoit Sigoure
> Labels: ipv6
>
> With this resolv.conf:
> {code}
> nameserver 192.168.1.1
> nameserver 2604:5500:3::3
> nameserver 2604:5500:3:3::3
> {code}
> Starting HBase yields the following:
> {code}
> Caused by: java.lang.NumberFormatException: For input string: "5500:3::3"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:492)
> at java.lang.Integer.parseInt(Integer.java:527)
> at com.sun.jndi.dns.DnsClient.(DnsClient.java:122)
> at com.sun.jndi.dns.Resolver.(Resolver.java:61)
> at com.sun.jndi.dns.DnsContext.getResolver(DnsContext.java:570)
> at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:430)
> at
> com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:231)
> at
> com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:139)
> at
> com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:103)
> at
> javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:142)
> at org.apache.hadoop.net.DNS.reverseDns(DNS.java:84)
> at org.apache.hadoop.net.DNS.getHosts(DNS.java:241)
> at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:344)
> at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:362)
> at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:341)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getHostname(RSRpcServices.java:825)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:782)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:195)
> at
> org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:477)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:492)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:333)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:276)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
> ... 7 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11890) Uber-JIRA: Hadoop should support IPv6
[ https://issues.apache.org/jira/browse/HADOOP-11890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903423#comment-14903423 ] Nate Edel commented on HADOOP-11890: Yes, please do. > Uber-JIRA: Hadoop should support IPv6 > - > > Key: HADOOP-11890 > URL: https://issues.apache.org/jira/browse/HADOOP-11890 > Project: Hadoop Common > Issue Type: Improvement > Components: net >Reporter: Nate Edel >Assignee: Nate Edel > Labels: ipv6 > > Hadoop currently treats IPv6 as unsupported. Track related smaller issues to > support IPv6. > (Current case here is mainly HBase on HDFS, so any suggestions about other > test cases/workload are really appreciated.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12123) Add example test-patch plugin for commit message format
[ https://issues.apache.org/jira/browse/HADOOP-12123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12123: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > Add example test-patch plugin for commit message format > --- > > Key: HADOOP-12123 > URL: https://issues.apache.org/jira/browse/HADOOP-12123 > Project: Hadoop Common > Issue Type: New Feature > Components: Test Patch >Affects Versions: 0.1.0 >Reporter: Sean Busbey > > add an example test plugin to ensures when we get a diff that includes a > commit message that that commit message starts with a jira ID. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12128) test-patch docker mode doesn't support non-project personalities
[ https://issues.apache.org/jira/browse/HADOOP-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12128: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch docker mode doesn't support non-project personalities > > > Key: HADOOP-12128 > URL: https://issues.apache.org/jira/browse/HADOOP-12128 > Project: Hadoop Common > Issue Type: Improvement > Components: Test Patch >Reporter: Allen Wittenauer > > If you provide a personality file that doesn't match the project name, docker > mode won't use it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11977) test-patch.sh smarter java handling
[ https://issues.apache.org/jira/browse/HADOOP-11977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-11977: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch.sh smarter java handling > --- > > Key: HADOOP-11977 > URL: https://issues.apache.org/jira/browse/HADOOP-11977 > Project: Hadoop Common > Issue Type: Improvement > Components: Test Patch >Reporter: Allen Wittenauer > Labels: yetus > > I suspect an optimization could be made that when a patch to java code > doesn't touch either a method definition or a comment, that javadoc might not > need to fire off. Additionally, if a patch only touches a comment, then it > should only do javadoc tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12139) patch to a unit test should only trigger that unit test
[ https://issues.apache.org/jira/browse/HADOOP-12139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12139: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > patch to a unit test should only trigger that unit test > --- > > Key: HADOOP-12139 > URL: https://issues.apache.org/jira/browse/HADOOP-12139 > Project: Hadoop Common > Issue Type: Improvement > Components: yetus >Reporter: Allen Wittenauer > > A great optimization would be to determine if only a unit test was patched > and run just that unit test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11909) test-patch javac & javadoc output clarity
[ https://issues.apache.org/jira/browse/HADOOP-11909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-11909: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch javac & javadoc output clarity > - > > Key: HADOOP-11909 > URL: https://issues.apache.org/jira/browse/HADOOP-11909 > Project: Hadoop Common > Issue Type: Improvement > Components: Test Patch >Reporter: Allen Wittenauer > > It should be possible to use similar code as is in the checkstyle plugin to > just print the errors/warnings of relevant lines rather than the full diff. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12227) need an API for MODULE_STATUS_TIMER
[ https://issues.apache.org/jira/browse/HADOOP-12227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12227: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > need an API for MODULE_STATUS_TIMER > --- > > Key: HADOOP-12227 > URL: https://issues.apache.org/jira/browse/HADOOP-12227 > Project: Hadoop Common > Issue Type: Improvement > Components: yetus >Reporter: Allen Wittenauer > > As shown by checkstyle and findbugs and probably others, it'd be good to have > an API for plugins to use to modify MODULE_STATUS_TIMER similar to what we > have for the local timer (offset_clock). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12262) deal with "Pending Closed" in releasedocmaker
[ https://issues.apache.org/jira/browse/HADOOP-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12262: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > deal with "Pending Closed" in releasedocmaker > - > > Key: HADOOP-12262 > URL: https://issues.apache.org/jira/browse/HADOOP-12262 > Project: Hadoop Common > Issue Type: Improvement > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Allen Wittenauer > > YARN-3641 was set into "Pending Closed" state. 2.7.1 was then marked as > Closed, preventing all future updates to that JIRA. There are two things we > could do here from a releasedocmaker perspective: > a) provide a hook into lint mode that warns about Pending Closed > b) provide a flag that treats Pending Closed the same as Fixed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12276) plugin validator
[ https://issues.apache.org/jira/browse/HADOOP-12276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12276: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > plugin validator > > > Key: HADOOP-12276 > URL: https://issues.apache.org/jira/browse/HADOOP-12276 > Project: Hadoop Common > Issue Type: New Feature > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Allen Wittenauer > > Build a simple plugin validation tool that reports what capabilities a plugin > has. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12293) committer mode for smart-apply-patch
[ https://issues.apache.org/jira/browse/HADOOP-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12293: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > committer mode for smart-apply-patch > > > Key: HADOOP-12293 > URL: https://issues.apache.org/jira/browse/HADOOP-12293 > Project: Hadoop Common > Issue Type: New Feature > Components: Test Patch >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-12293.HADOOP-12111.00.patch > > > It'd be nice if smart-apply-patch had a committer mode that: > a) always stripped whitespace > b) always added new files -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12305) add support for javascript linter
[ https://issues.apache.org/jira/browse/HADOOP-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12305: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > add support for javascript linter > - > > Key: HADOOP-12305 > URL: https://issues.apache.org/jira/browse/HADOOP-12305 > Project: Hadoop Common > Issue Type: New Feature > Components: Test Patch >Reporter: Kengo Seki > > Some projects have its own javascript code other than third-party's, so it > may be useful. > JSLint or JSHint or ESLint? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12306) make test-patch rubocop plugin configurable
[ https://issues.apache.org/jira/browse/HADOOP-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12306: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > make test-patch rubocop plugin configurable > --- > > Key: HADOOP-12306 > URL: https://issues.apache.org/jira/browse/HADOOP-12306 > Project: Hadoop Common > Issue Type: Improvement > Components: Test Patch >Reporter: Kengo Seki > > As with HADOOP-12286, it must be useful if users can change the rules about > indentation, naming convention, max line length, enabling/disabling some > checks, etc. as suitable for their project's personality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12399) Wrong help messages in some test-patch plugins
[
https://issues.apache.org/jira/browse/HADOOP-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903886#comment-14903886
]
Allen Wittenauer commented on HADOOP-12399:
---
This has been cherry-picked over to the yetus repo.
> Wrong help messages in some test-patch plugins
> --
>
> Key: HADOOP-12399
> URL: https://issues.apache.org/jira/browse/HADOOP-12399
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: yetus
>Affects Versions: HADOOP-12111
>Reporter: Kengo Seki
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Labels: newbie
> Fix For: HADOOP-12111
>
> Attachments: HADOOP-12399.HADOOP-12111.00.patch,
> HADOOP-12399.HADOOP-12111.01.patch, HADOOP-12399.HADOOP-12111.02.patch
>
>
> dev-support/personality/bigtop.sh:
> {code}
> 32 function bigtop_usage
> 33 {
> 34 echo "Bigtop specific:"
> 35 echo "--bigtop-puppetsetup=[false|true] execute the bigtop dev setup
> (needs sudo to root)"
> 36 }
> {code}
> s/bigtop-puppetsetup/bigtop-puppet/.
> dev-support/test-patch.d/gradle.sh:
> {code}
> 21 function gradle_usage
> 22 {
> 23 echo "gradle specific:"
> 24 echo "--gradle-cmd=The 'gradle' command to use (default
> 'gradle')"
> 25 echo "--gradlew-cmd=The 'gradle' command to use (default
> 'basedir/gradlew')"
> 26 }
> {code}
> s/'gradle' command/'gradlew' command/ for the latter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12120) detect and flag flakey tests
[ https://issues.apache.org/jira/browse/HADOOP-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12120: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > detect and flag flakey tests > > > Key: HADOOP-12120 > URL: https://issues.apache.org/jira/browse/HADOOP-12120 > Project: Hadoop Common > Issue Type: New Feature > Components: Test Patch >Reporter: Sean Busbey > > we run patch-test very often and in HBase there are some tests that are > flakey but don't appear to fail in pre-commit. we should be able to detect > these and get them followed up on. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12132) test-patch needs Git 1.7.3+
[ https://issues.apache.org/jira/browse/HADOOP-12132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12132: - Issue Type: Bug (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch needs Git 1.7.3+ > --- > > Key: HADOOP-12132 > URL: https://issues.apache.org/jira/browse/HADOOP-12132 > Project: Hadoop Common > Issue Type: Bug > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Kengo Seki >Priority: Minor > > test-patch can run `git clean -xdf -e ...`, but -e is supported only 1.7.3+. > Version checking and documentation are needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12137) build environment and unit tests
[ https://issues.apache.org/jira/browse/HADOOP-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12137: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > build environment and unit tests > > > Key: HADOOP-12137 > URL: https://issues.apache.org/jira/browse/HADOOP-12137 > Project: Hadoop Common > Issue Type: Improvement > Components: Test Patch >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer >Priority: Critical > Attachments: HADOOP-12137.HADOOP-12111.00.patch, > HADOOP-12137.HADOOP-12111.01.patch > > > We need to have some way to build (esp the documentation!) and unit tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-11903) test-patch plugin for checking file name conventions
[ https://issues.apache.org/jira/browse/HADOOP-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-11903: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch plugin for checking file name conventions > > > Key: HADOOP-11903 > URL: https://issues.apache.org/jira/browse/HADOOP-11903 > Project: Hadoop Common > Issue Type: New Feature > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Allen Wittenauer >Assignee: Kengo Seki > Attachments: HADOOP-11903.HADOOP-12111.00.patch, > HADOOP-11903.HADOOP-12111.01.patch, HADOOP-11903.HADOOP-12111.02.patch > > > Providing yetus with the functionality to detect newly added files which > doesn't follow the naming conventions. As for java, it also can be used for > checking classnames. In the past, we've named things like > DefaultResourceCalculator, DefaultContainerExecutor, and DefaultCodec that do > nothing but cause problems down the road since they are effectively version > and functionality locked forever. If these examples had been named what they > truly were (e.g., MemoryResourceCalculator, SimpleContainerExecutor, and > GZipCodec), the defaults could then be changed in the future in a compatible > way. One way to enforce this is to prevent the creation of new classes called > Default-anything. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-9112) test-patch should -1 for @Tests without a timeout
[ https://issues.apache.org/jira/browse/HADOOP-9112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-9112: Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch should -1 for @Tests without a timeout > - > > Key: HADOOP-9112 > URL: https://issues.apache.org/jira/browse/HADOOP-9112 > Project: Hadoop Common > Issue Type: New Feature > Components: build >Reporter: Todd Lipcon > Attachments: HADOOP-9112-1.patch, HADOOP-9112-2.patch, > HADOOP-9112-3.patch, HADOOP-9112-4.patch, HADOOP-9112-5.patch, > HADOOP-9112-6.patch, HADOOP-9112-7.patch > > > With our current test running infrastructure, if a test with no timeout set > runs too long, it triggers a surefire-wide timeout, which for some reason > doesn't show up as a failed test in the test-patch output. Given that, we > should require that all tests have a timeout set, and have test-patch enforce > this with a simple check -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12432) Support for include/exclude lists on IPv6 setup
[
https://issues.apache.org/jira/browse/HADOOP-12432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903943#comment-14903943
]
Hadoop QA commented on HADOOP-12432:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 11s | Findbugs (version ) appears to
be broken on HADOOP-11890. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 1s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 15s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 25s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 33s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 29s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 23s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 158m 46s | Tests failed in hadoop-hdfs. |
| | | 202m 21s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestDFSInputStream |
| | hadoop.hdfs.server.blockmanagement.TestPendingDataNodeMessages |
| | hadoop.hdfs.TestSeekBug |
| | hadoop.hdfs.TestFileAppendRestart |
| | hadoop.TestRefreshCallQueue |
| | hadoop.hdfs.TestGetBlocks |
| | hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling |
| | hadoop.hdfs.qjournal.client.TestQJMWithFaults |
| | hadoop.hdfs.TestHDFSServerPorts |
| | hadoop.hdfs.TestRollingUpgradeRollback |
| | hadoop.hdfs.TestDatanodeDeath |
| | hadoop.hdfs.qjournal.client.TestQuorumJournalManagerUnit |
| | hadoop.hdfs.TestParallelShortCircuitLegacyRead |
| | hadoop.hdfs.TestFileAppend4 |
| | hadoop.hdfs.TestDFSClientRetries |
| | hadoop.hdfs.TestDataTransferKeepalive |
| | hadoop.hdfs.TestReservedRawPaths |
| | hadoop.hdfs.TestExternalBlockReader |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory |
| | hadoop.tracing.TestTracingShortCircuitLocalRead |
| | hadoop.hdfs.TestFileCreationClient |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.TestLocalDFS |
| | hadoop.hdfs.qjournal.server.TestJournal |
| | hadoop.hdfs.TestListFilesInFileContext |
| | hadoop.hdfs.TestReadWhileWriting |
| | hadoop.hdfs.TestDatanodeReport |
| | hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks |
| | hadoop.hdfs.TestParallelRead |
| | hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer |
| | hadoop.hdfs.TestHDFSTrash |
| | hadoop.hdfs.TestDFSRollback |
| | hadoop.hdfs.security.token.block.TestBlockToken |
| | hadoop.hdfs.TestFileLengthOnClusterRestart |
| | hadoop.hdfs.TestFileCreation |
| | hadoop.hdfs.TestSetrepIncreasing |
| | hadoop.hdfs.TestDatanodeConfig |
| | hadoop.hdfs.crypto.TestHdfsCryptoStreams |
| | hadoop.hdfs.qjournal.client.TestEpochsAreUnique |
| | hadoop.hdfs.security.TestDelegationToken |
| | hadoop.tracing.TestTracing |
| | hadoop.hdfs.TestClientReportBadBlock |
| | hadoop.hdfs.TestDataTransferProtocol |
| | hadoop.hdfs.TestWriteRead |
| | hadoop.hdfs.TestClose |
| | hadoop.hdfs.TestFileAppend |
| | hadoop.hdfs.TestBalancerBandwidth |
| | hadoop.hdfs.TestMiniDFSCluster |
| | hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages |
| | hadoop.hdfs.TestDatanodeRegistration |
| | hadoop.hdfs.TestDFSShellGenericOptions |
| | hadoop.hdfs.TestRenameWhileOpen |
| | hadoop.hdfs.qjournal.client.TestQuorumJournalManager |
| | hadoop.hdfs.TestAbandonBlock |
| | hadoop.hdfs.TestRestartDFS |
| | hadoop.hdfs.TestSmallBlock |
| | hadoop.hdfs.TestDatanodeLayoutUpgrade |
| | hadoop.hdfs.TestSafeMode |
| | hadoop.hdfs.server.blockmanagement.TestReplicationPolicy |
| | hadoop.hdfs.TestDFSInotifyEventInputStream |
| | hadoop.hdfs.TestEncryptionZones |
| | hadoop.hdfs.TestDFSRemove |
| | hadoop.hdfs.TestHFlush |
| | hadoop.hdfs.TestMultiThreadedHflush |
| | hadoop.net.TestNetworkTopology |
| | hadoop.hdfs.TestClientBlockVerification |
| | hadoop.hdfs.TestAppendDifferentChecksum |
| | hadoop.hdfs.TestRead |
| | hadoop.hdfs.TestDFSFinalize |
| |
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement |
| | hadoop.hdfs.TestCrcCorruption
[jira] [Updated] (HADOOP-12141) test-patch should check for API breakage without version # change
[ https://issues.apache.org/jira/browse/HADOOP-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12141: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch should check for API breakage without version # change > - > > Key: HADOOP-12141 > URL: https://issues.apache.org/jira/browse/HADOOP-12141 > Project: Hadoop Common > Issue Type: New Feature > Components: Test Patch >Reporter: Sean Busbey > > HBase has some automated tooling for checking for API breakage with the Java > API Compliance Checker tool. It'd be nice ot have it as a pre-commit plugin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12223) Better support for Easter
[ https://issues.apache.org/jira/browse/HADOOP-12223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12223: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > Better support for Easter > - > > Key: HADOOP-12223 > URL: https://issues.apache.org/jira/browse/HADOOP-12223 > Project: Hadoop Common > Issue Type: New Feature > Components: Test Patch >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12278) releasedocmaker should use the Important flag in jira
[ https://issues.apache.org/jira/browse/HADOOP-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12278: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > releasedocmaker should use the Important flag in jira > - > > Key: HADOOP-12278 > URL: https://issues.apache.org/jira/browse/HADOOP-12278 > Project: Hadoop Common > Issue Type: Improvement > Components: Release Doc Maker >Reporter: Allen Wittenauer > > Apache JIRA has an Important flag in certain projects. We should pick it up > and treat it similarly to incompatible changes (special section in changes, > always in release notes, etc) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12307) make test-patch ruby-lint plugin configurable
[ https://issues.apache.org/jira/browse/HADOOP-12307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12307: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > make test-patch ruby-lint plugin configurable > - > > Key: HADOOP-12307 > URL: https://issues.apache.org/jira/browse/HADOOP-12307 > Project: Hadoop Common > Issue Type: Improvement > Components: Test Patch >Reporter: Kengo Seki > > As with HADOOP-12286, it must be useful if users can change the rules about > indentation, naming convention, max line length, enabling/disabling some > checks, etc. as suitable for their project's personality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12308) make test-patch perlcritic plugin configurable
[ https://issues.apache.org/jira/browse/HADOOP-12308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12308: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > make test-patch perlcritic plugin configurable > -- > > Key: HADOOP-12308 > URL: https://issues.apache.org/jira/browse/HADOOP-12308 > Project: Hadoop Common > Issue Type: Improvement > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Kengo Seki > > As with HADOOP-12286, it must be useful if users can change the rules about > indentation, naming convention, max line length, enabling/disabling some > checks, etc. as suitable for their project's personality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12341) patch file confuses test-patch (date format problems)
[ https://issues.apache.org/jira/browse/HADOOP-12341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12341: - Issue Type: Bug (was: Sub-task) Parent: (was: HADOOP-12111) > patch file confuses test-patch (date format problems) > - > > Key: HADOOP-12341 > URL: https://issues.apache.org/jira/browse/HADOOP-12341 > Project: Hadoop Common > Issue Type: Bug > Components: Test Patch >Reporter: Allen Wittenauer > Attachments: HADOOP-12326.002.patch, > HADOOP-12375.HADOOP-12111.02.patch > > > This was attached to HADOOP-12326 . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12372) Add support for rspec with junit
[
https://issues.apache.org/jira/browse/HADOOP-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-12372:
-
Issue Type: New Feature (was: Sub-task)
Parent: (was: HADOOP-12111)
> Add support for rspec with junit
>
>
> Key: HADOOP-12372
> URL: https://issues.apache.org/jira/browse/HADOOP-12372
> Project: Hadoop Common
> Issue Type: New Feature
> Components: yetus
>Reporter: Travis Thompson
>
> rspec already supports junit format with a plugin, just run like this:
> {noformat}
> rspec -f JUnit -o results.xml
> {noformat}
> This will allow any generic rspec to work as well as chefspec (chef unit
> testing). This will also soon allow for kitchen test (chef functional
> testing) to work as well (it's also using rspec, but currently isn't
> configurable, see: https://github.com/test-kitchen/busser-serverspec/issues/9)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12433) Migrate open jiras to Yetus tracker
[ https://issues.apache.org/jira/browse/HADOOP-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903884#comment-14903884 ] Allen Wittenauer commented on HADOOP-12433: --- I've cherry-picked that change over. > Migrate open jiras to Yetus tracker > --- > > Key: HADOOP-12433 > URL: https://issues.apache.org/jira/browse/HADOOP-12433 > Project: Hadoop Common > Issue Type: Sub-task > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > > move anything that isn't in the HADOOP-12111 branch already over to the yetus > tracker. preferably with nothing landing in the branch after commit > 0a6e5d109b7f28ed3e2ae780898e79ad16faa954 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12221) document a coding style guide
[ https://issues.apache.org/jira/browse/HADOOP-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12221: - Issue Type: Task (was: Sub-task) Parent: (was: HADOOP-12111) > document a coding style guide > - > > Key: HADOOP-12221 > URL: https://issues.apache.org/jira/browse/HADOOP-12221 > Project: Hadoop Common > Issue Type: Task > Components: website and documentation >Reporter: Allen Wittenauer > > We should document a style guide, keeping in mind things like > http://wiki.bash-hackers.org/scripting/obsolete . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12285) Reorganize test-patch footer table
[
https://issues.apache.org/jira/browse/HADOOP-12285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-12285:
-
Issue Type: Improvement (was: Sub-task)
Parent: (was: HADOOP-12111)
> Reorganize test-patch footer table
> --
>
> Key: HADOOP-12285
> URL: https://issues.apache.org/jira/browse/HADOOP-12285
> Project: Hadoop Common
> Issue Type: Improvement
> Components: yetus
>Reporter: Kengo Seki
>
> In response to:
> https://issues.apache.org/jira/browse/HADOOP-12266?focusedCommentId=14647672=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14647672
> {quote}
> * when further information is needed, we often publish that in the footer as
> well, and having all of these versions published in the footer makes those
> hard to find
> * are we better off actually adding a fourth table just for versions and
> other operating information? or just exercising better control over the
> footer table to force versions lower than logs? in code, versions could be in
> a separate table but in display one table
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12299) should test-patch enforce some releasedocmaker constraints?
[ https://issues.apache.org/jira/browse/HADOOP-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12299: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > should test-patch enforce some releasedocmaker constraints? > --- > > Key: HADOOP-12299 > URL: https://issues.apache.org/jira/browse/HADOOP-12299 > Project: Hadoop Common > Issue Type: New Feature > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Allen Wittenauer > > In other words, if JIRA is in use, should test-patch flag incompatible > patches without release notes? Is there some other way we could be testing > for releasedoc problems? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12354) github_find_jira_title in test-patch github plugin returns 0 even if jira_determine_issue failed
[
https://issues.apache.org/jira/browse/HADOOP-12354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-12354:
-
Issue Type: Bug (was: Sub-task)
Parent: (was: HADOOP-12111)
> github_find_jira_title in test-patch github plugin returns 0 even if
> jira_determine_issue failed
>
>
> Key: HADOOP-12354
> URL: https://issues.apache.org/jira/browse/HADOOP-12354
> Project: Hadoop Common
> Issue Type: Bug
> Components: yetus
>Affects Versions: HADOOP-12111
>Reporter: Kengo Seki
>
> So the following check for $\{GITHUB_ISSUE} seems to be skipped and
> github_determine_issue seems to succeed almost always.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12131) The different JDKs which versions are the same aren't distiguished on multi-JDK mode
[
https://issues.apache.org/jira/browse/HADOOP-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-12131:
-
Issue Type: Bug (was: Sub-task)
Parent: (was: HADOOP-12111)
> The different JDKs which versions are the same aren't distiguished on
> multi-JDK mode
>
>
> Key: HADOOP-12131
> URL: https://issues.apache.org/jira/browse/HADOOP-12131
> Project: Hadoop Common
> Issue Type: Bug
> Components: yetus
>Affects Versions: HADOOP-12111
>Reporter: Kengo Seki
>Assignee: Kengo Seki
>
> If one installs the same version of Oracle JDK and OpenJDK and enables the
> multi-JDK option, output files are overwritten by the last one because the
> same value is assigned to $jdk and $fn. Example:
> {code}
>
>
> Pre-patch trunk javac compilation
>
>
> mvn --batch-mode clean compile -DskipTests -Pnative -Drequire.libwebhdfs
> -Drequire.snappy -Drequire.openssl -Drequire.fuse -Drequire.test.libhadoop
> -Ptest-patch >
> /tmp/test-patch-hadoop/25579/branch-javac-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
> 2>&1
> Elapsed: 1m 0s
> mvn --batch-mode clean compile -DskipTests -Pnative -Drequire.libwebhdfs
> -Drequire.snappy -Drequire.openssl -Drequire.fuse -Drequire.test.libhadoop
> -Ptest-patch >
> /tmp/test-patch-hadoop/25579/branch-javac-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
> 2>&1
> Elapsed: 0m 54s
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12044) test-patch should handle module movement
[ https://issues.apache.org/jira/browse/HADOOP-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12044: - Issue Type: Bug (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch should handle module movement > > > Key: HADOOP-12044 > URL: https://issues.apache.org/jira/browse/HADOOP-12044 > Project: Hadoop Common > Issue Type: Bug > Components: test, yetus >Affects Versions: HADOOP-12111 >Reporter: Sean Busbey > > test-patch reported a variety of errors against HADOOP-11804 caused by the > fact that a module moved -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12174) compute a code quality index
[ https://issues.apache.org/jira/browse/HADOOP-12174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12174: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > compute a code quality index > > > Key: HADOOP-12174 > URL: https://issues.apache.org/jira/browse/HADOOP-12174 > Project: Hadoop Common > Issue Type: New Feature > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Allen Wittenauer >Priority: Minor > > Just an idea: > Compute a code quality index based upon checkstyle, findbugs, and shellcheck > but with all exclusions and disables turned off. Generate a number pre-patch > and post-patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12166) catching findbugs failures
[ https://issues.apache.org/jira/browse/HADOOP-12166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12166: - Issue Type: Bug (was: Sub-task) Parent: (was: HADOOP-12111) > catching findbugs failures > -- > > Key: HADOOP-12166 > URL: https://issues.apache.org/jira/browse/HADOOP-12166 > Project: Hadoop Common > Issue Type: Bug > Components: yetus >Reporter: Allen Wittenauer > > When findbugs crashes or can't execute something, it would be good if we > could get more information rather than it being a mystery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12233) if CHANGED_FILES is corrupt, find_changed_modules never returns
[ https://issues.apache.org/jira/browse/HADOOP-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12233: - Issue Type: Bug (was: Sub-task) Parent: (was: HADOOP-12111) > if CHANGED_FILES is corrupt, find_changed_modules never returns > --- > > Key: HADOOP-12233 > URL: https://issues.apache.org/jira/browse/HADOOP-12233 > Project: Hadoop Common > Issue Type: Bug > Components: yetus >Affects Versions: HADOOP-12111 >Reporter: Allen Wittenauer >Assignee: Kengo Seki > Fix For: HADOOP-12111 > > Attachments: HADOOP-12233.HADOOP-12111.00.patch, > HADOOP-12233.HADOOP-12111.01.patch, HADOOP-12233.HADOOP-12111.02.patch > > > In building some unit tests, did a negative tests and hit this condition. We > should put a limit on how many times we loop in the find_x_dirs code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12330) Decouple smart-apply-patch from apache
[ https://issues.apache.org/jira/browse/HADOOP-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12330: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > Decouple smart-apply-patch from apache > -- > > Key: HADOOP-12330 > URL: https://issues.apache.org/jira/browse/HADOOP-12330 > Project: Hadoop Common > Issue Type: Improvement > Components: Test Patch >Reporter: Allen Wittenauer > > Now that test-patch has been decoupled from Apache JIRA, the same work should > be done to smart-apply-patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12365) plugin for password, key, etc detection
[ https://issues.apache.org/jira/browse/HADOOP-12365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12365: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > plugin for password, key, etc detection > --- > > Key: HADOOP-12365 > URL: https://issues.apache.org/jira/browse/HADOOP-12365 > Project: Hadoop Common > Issue Type: New Feature > Components: Test Patch >Reporter: Allen Wittenauer > > It'd be nice to have a plugin that checks for these security vulnerabilities, > esp for non-Apache source. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12427) Upgrade Mockito version to 1.10.19
[ https://issues.apache.org/jira/browse/HADOOP-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated HADOOP-12427: -- Description: The current version is 1.8.5 - inserted in 2011. was: The current version is 1.8.5 - inserted in 2011. The new version works better with PowerMockito - Mock for Static methods. > Upgrade Mockito version to 1.10.19 > -- > > Key: HADOOP-12427 > URL: https://issues.apache.org/jira/browse/HADOOP-12427 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Giovanni Matteo Fumarola >Assignee: Giovanni Matteo Fumarola >Priority: Minor > > The current version is 1.8.5 - inserted in 2011. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12427) Upgrade Mockito version to 1.10.19
[ https://issues.apache.org/jira/browse/HADOOP-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated HADOOP-12427: -- Attachment: HADOOP-12427.v0.patch > Upgrade Mockito version to 1.10.19 > -- > > Key: HADOOP-12427 > URL: https://issues.apache.org/jira/browse/HADOOP-12427 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Giovanni Matteo Fumarola >Assignee: Giovanni Matteo Fumarola >Priority: Minor > Attachments: HADOOP-12427.v0.patch > > > The current version is 1.8.5 - inserted in 2011. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12427) Upgrade Mockito version to 1.10.19
[ https://issues.apache.org/jira/browse/HADOOP-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated HADOOP-12427: -- Status: Patch Available (was: Open) No test needed > Upgrade Mockito version to 1.10.19 > -- > > Key: HADOOP-12427 > URL: https://issues.apache.org/jira/browse/HADOOP-12427 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Giovanni Matteo Fumarola >Assignee: Giovanni Matteo Fumarola >Priority: Minor > Attachments: HADOOP-12427.v0.patch > > > The current version is 1.8.5 - inserted in 2011. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12427) Upgrade Mockito version to 1.10.19
[ https://issues.apache.org/jira/browse/HADOOP-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola updated HADOOP-12427: -- Issue Type: Improvement (was: Wish) > Upgrade Mockito version to 1.10.19 > -- > > Key: HADOOP-12427 > URL: https://issues.apache.org/jira/browse/HADOOP-12427 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: Giovanni Matteo Fumarola >Assignee: Giovanni Matteo Fumarola >Priority: Minor > > The current version is 1.8.5 - inserted in 2011. > The new version works better with PowerMockito - Mock for Static methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12377) releasedocmaker should support JIRA authentication
[ https://issues.apache.org/jira/browse/HADOOP-12377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12377: - Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > releasedocmaker should support JIRA authentication > -- > > Key: HADOOP-12377 > URL: https://issues.apache.org/jira/browse/HADOOP-12377 > Project: Hadoop Common > Issue Type: New Feature > Components: Release Doc Maker >Reporter: Allen Wittenauer > > This is needed for non-wide-open JIRA installations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12373) Add support for foodcritic
[
https://issues.apache.org/jira/browse/HADOOP-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HADOOP-12373:
-
Issue Type: New Feature (was: Sub-task)
Parent: (was: HADOOP-12111)
> Add support for foodcritic
> --
>
> Key: HADOOP-12373
> URL: https://issues.apache.org/jira/browse/HADOOP-12373
> Project: Hadoop Common
> Issue Type: New Feature
> Components: Test Patch
>Reporter: Travis Thompson
>
> Foodcritic is the chef "best practices" linter
> (http://acrmp.github.io/foodcritic/)
> Successful output:
> {noformat}
> ± master ✗ → foodcritic .
> ± master ✗ → echo $?
> 0
> {noformat}
> Unsuccessful output:
> {noformat}
> ± master ✗ → foodcritic .
> FC048: Prefer Mixlib::ShellOut: ./recipes/server.rb:482
> ± master ✗ → echo $?
> 0
> {noformat}
> Here's what caused the failure:
> {noformat}
> 482 `echo "this is bad practice"`
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-11897) test-patch.sh plugins should abbreviate the path
[ https://issues.apache.org/jira/browse/HADOOP-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-11897: - Issue Type: Improvement (was: Sub-task) Parent: (was: HADOOP-12111) > test-patch.sh plugins should abbreviate the path > > > Key: HADOOP-11897 > URL: https://issues.apache.org/jira/browse/HADOOP-11897 > Project: Hadoop Common > Issue Type: Improvement > Components: test, yetus >Affects Versions: HADOOP-12111 >Reporter: Allen Wittenauer >Priority: Trivial > Labels: newbie > > The current output of checkstyle, shellcheck, and whitespace results in very > long file name paths. It might be useful to abbreviate them in some way, > maybe removing the entire path and leaving just the filename or last > directory+filename. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-4258) the test patch script should check for filenames that differ only in case
[ https://issues.apache.org/jira/browse/HADOOP-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-4258: Issue Type: New Feature (was: Sub-task) Parent: (was: HADOOP-12111) > the test patch script should check for filenames that differ only in case > - > > Key: HADOOP-4258 > URL: https://issues.apache.org/jira/browse/HADOOP-4258 > Project: Hadoop Common > Issue Type: New Feature > Components: Test Patch >Reporter: Owen O'Malley >Assignee: Jagadesh Kiran N > Labels: test-patch > Attachments: HADOOP-4258.001.patch, > HADOOP-4258.HADOOP-12111.00.patch, HADOOP-4258.HADOOP-12111.01.patch, > HADOOP-4258.HADOOP-12111.02.patch > > > It would be nice if the test patch script warned about filenames that differ > only in case. We recently had a patch committed that had a pair of colliding > filenames and subversion broke badly on my Mac. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12421) Add jitter to RetryInvocationHandler
[
https://issues.apache.org/jira/browse/HADOOP-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902183#comment-14902183
]
Hadoop QA commented on HADOOP-12421:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 27m 3s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 10m 49s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 29s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 31s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 2m 13s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 50s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 9s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 23m 58s | Tests failed in
hadoop-common. |
| | | 80m 44s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.ipc.TestDecayRpcScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12761582/HADOOP-12421-v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / dfd807a |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7685/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7685/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7685/console |
This message was automatically generated.
> Add jitter to RetryInvocationHandler
>
>
> Key: HADOOP-12421
> URL: https://issues.apache.org/jira/browse/HADOOP-12421
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HADOOP-12421-v1.patch, HADOOP-12421-v2.patch
>
>
> Calls to NN can become synchronized across a cluster during NN failover. This
> leads to a spike in requests until things recover. Making an already tricky
> time worse.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12428) Fix inconsistency between log-level guards and statements
[
https://issues.apache.org/jira/browse/HADOOP-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902242#comment-14902242
]
Hudson commented on HADOOP-12428:
-
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #402 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/402/])
HADOOP-12428. Fix inconsistency between log-level guards and statements.
Contributed by Jagadesh Kiran N and Jackie Chang. (ozawa: rev
dfd807afab0fae3839c9cc5d552aa030f956)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
*
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/NMTokenSelector.java
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
*
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/SerialJobFactory.java
* hadoop-yarn-project/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerRequestor.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
> Fix inconsistency between log-level guards and statements
> -
>
> Key: HADOOP-12428
> URL: https://issues.apache.org/jira/browse/HADOOP-12428
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jackie Chang
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Labels: BB2015-05-TBR
> Fix For: 2.8.0
>
> Attachments: HADOOP-9995-00.patch, HADOOP-9995.patch,
> MAPREDUCE-6468-01.patch, MAPREDUCE-6468-02.patch, MAPREDUCE-6468-03.patch,
> MAPREDUCE-6468-04.patch
>
>
> Developers use logs to do in-house debugging. These log statements are later
> demoted to less severe levels and usually are guarded by their matching
> severity levels. However, we do see inconsistencies in trunk. A log statement
> like
> {code}
>if (LOG.isDebugEnabled()) {
> LOG.info("Assigned container (" + allocated + ") "
> {code}
> doesn't make much sense because the log message is actually only printed out
> in DEBUG-level. We do see previous issues tried to correct this
> inconsistency. I am proposing a comprehensive correction over trunk.
> Doug Cutting pointed it out in HADOOP-312:
> https://issues.apache.org/jira/browse/HADOOP-312?focusedCommentId=12429498=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12429498
> HDFS-1611 also corrected this inconsistency.
> This could have been avoided by switching from log4j to slf4j's {} format
> like CASSANDRA-625 (2010/3) and ZOOKEEPER-850 (2012/1), which gives cleaner
> code and slightly higher performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12428) Fix inconsistency between log-level guards and statements
[
https://issues.apache.org/jira/browse/HADOOP-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902209#comment-14902209
]
Hudson commented on HADOOP-12428:
-
FAILURE: Integrated in Hadoop-Hdfs-trunk #2340 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2340/])
HADOOP-12428. Fix inconsistency between log-level guards and statements.
Contributed by Jagadesh Kiran N and Jackie Chang. (ozawa: rev
dfd807afab0fae3839c9cc5d552aa030f956)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
*
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/SerialJobFactory.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerRequestor.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/NMTokenSelector.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java
*
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* hadoop-common-project/hadoop-common/CHANGES.txt
> Fix inconsistency between log-level guards and statements
> -
>
> Key: HADOOP-12428
> URL: https://issues.apache.org/jira/browse/HADOOP-12428
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jackie Chang
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Labels: BB2015-05-TBR
> Fix For: 2.8.0
>
> Attachments: HADOOP-9995-00.patch, HADOOP-9995.patch,
> MAPREDUCE-6468-01.patch, MAPREDUCE-6468-02.patch, MAPREDUCE-6468-03.patch,
> MAPREDUCE-6468-04.patch
>
>
> Developers use logs to do in-house debugging. These log statements are later
> demoted to less severe levels and usually are guarded by their matching
> severity levels. However, we do see inconsistencies in trunk. A log statement
> like
> {code}
>if (LOG.isDebugEnabled()) {
> LOG.info("Assigned container (" + allocated + ") "
> {code}
> doesn't make much sense because the log message is actually only printed out
> in DEBUG-level. We do see previous issues tried to correct this
> inconsistency. I am proposing a comprehensive correction over trunk.
> Doug Cutting pointed it out in HADOOP-312:
> https://issues.apache.org/jira/browse/HADOOP-312?focusedCommentId=12429498=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12429498
> HDFS-1611 also corrected this inconsistency.
> This could have been avoided by switching from log4j to slf4j's {} format
> like CASSANDRA-625 (2010/3) and ZOOKEEPER-850 (2012/1), which gives cleaner
> code and slightly higher performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12428) Fix inconsistency between log-level guards and statements
[
https://issues.apache.org/jira/browse/HADOOP-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902086#comment-14902086
]
Hudson commented on HADOOP-12428:
-
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2367 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2367/])
HADOOP-12428. Fix inconsistency between log-level guards and statements.
Contributed by Jagadesh Kiran N and Jackie Chang. (ozawa: rev
dfd807afab0fae3839c9cc5d552aa030f956)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
* hadoop-common-project/hadoop-common/CHANGES.txt
* hadoop-yarn-project/CHANGES.txt
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenSelector.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
*
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/SerialJobFactory.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/NMTokenSelector.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerRequestor.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
*
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
> Fix inconsistency between log-level guards and statements
> -
>
> Key: HADOOP-12428
> URL: https://issues.apache.org/jira/browse/HADOOP-12428
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jackie Chang
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Labels: BB2015-05-TBR
> Fix For: 2.8.0
>
> Attachments: HADOOP-9995-00.patch, HADOOP-9995.patch,
> MAPREDUCE-6468-01.patch, MAPREDUCE-6468-02.patch, MAPREDUCE-6468-03.patch,
> MAPREDUCE-6468-04.patch
>
>
> Developers use logs to do in-house debugging. These log statements are later
> demoted to less severe levels and usually are guarded by their matching
> severity levels. However, we do see inconsistencies in trunk. A log statement
> like
> {code}
>if (LOG.isDebugEnabled()) {
> LOG.info("Assigned container (" + allocated + ") "
> {code}
> doesn't make much sense because the log message is actually only printed out
> in DEBUG-level. We do see previous issues tried to correct this
> inconsistency. I am proposing a comprehensive correction over trunk.
> Doug Cutting pointed it out in HADOOP-312:
> https://issues.apache.org/jira/browse/HADOOP-312?focusedCommentId=12429498=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12429498
> HDFS-1611 also corrected this inconsistency.
> This could have been avoided by switching from log4j to slf4j's {} format
> like CASSANDRA-625 (2010/3) and ZOOKEEPER-850 (2012/1), which gives cleaner
> code and slightly higher performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
