[jira] [Commented] (HADOOP-15483) Upgrade jquery to version 3.3.1

2018-06-02 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-15483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499038#comment-16499038
 ] 

genericqa commented on HADOOP-15483:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m  
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
 8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m 
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
32m 10s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . 
hadoop-ozone hadoop-ozone/acceptance-test {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
39s{color} | {color:red} hadoop-ozone/ozone-manager in trunk has 1 extant 
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 
11s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 30m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 27m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
 8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m 
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 5861 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
10m  1s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . 
hadoop-ozone hadoop-ozone/acceptance-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  6m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}146m 45s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
38s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}353m 24s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
|   | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
|   | hadoop.hdfs.server.namenode.TestReencryptionWithKMS |
|   | hadoop.hdfs.client.impl.TestBlockReaderLocal |
|   | hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun |
|   | hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema 
|
|   | 
hadoop.yarn

[jira] [Created] (HADOOP-15509) Release Hadoop 2.7.7

2018-06-02 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15509:
---

 Summary: Release Hadoop 2.7.7
 Key: HADOOP-15509
 URL: https://issues.apache.org/jira/browse/HADOOP-15509
 Project: Hadoop Common
  Issue Type: Task
  Components: build
Affects Versions: 2.7.6
Reporter: Steve Loughran
Assignee: Steve Loughran


Time to get a new Hadoop 2.7.x out the door.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Work started] (HADOOP-15509) Release Hadoop 2.7.7

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HADOOP-15509 started by Steve Loughran.
---
> Release Hadoop 2.7.7
> 
>
> Key: HADOOP-15509
> URL: https://issues.apache.org/jira/browse/HADOOP-15509
> Project: Hadoop Common
>  Issue Type: Task
>  Components: build
>Affects Versions: 2.7.6
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>
> Time to get a new Hadoop 2.7.x out the door.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14307) GraphiteSink can specify which tags to export into the metrics prefix

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-14307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14307:

Target Version/s: 2.7.8  (was: 2.7.7)

> GraphiteSink can specify which tags to export into the metrics prefix
> -
>
> Key: HADOOP-14307
> URL: https://issues.apache.org/jira/browse/HADOOP-14307
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: metrics
>Affects Versions: 2.7.0, 2.7.1, 2.7.2, 2.7.3
>Reporter: Damien Claveau
>Assignee: Damien Claveau
>Priority: Minor
> Fix For: 2.7.0, 2.7.1, 2.7.2, 2.7.3
>
>
> This Jira is a feature proposal to add the ability in GraphiteSink to specify 
> which Tag (Name/Value) pairs are to be flattened in the metric prefix.
> The motivation for this is that currently, all the tags are included in the 
> prefix like this : 
> graphite_prefix.Context=$context.Process=$process.Tag1=$value1.Tag2=$value2..Tag9=$value9.metric
> This requires a bunch of rewriting rules (and complex regexp) in the metric 
> aggregation servers (carbon-relay, carbon-aggregator, ...) .
> The feature would be exactly the same as the solution implemented in the 
> GangliaSink : https://issues.apache.org/jira/browse/HADOOP-7507
> See also the commit : 
> https://github.com/apache/hadoop/commit/2ca9c8d926a8eeb871b2868e6eb4dfb97d7dc63d



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14544) DistCp documentation for command line options is misaligned.

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-14544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14544:

Target Version/s: 2.7.8  (was: 2.7.7)

> DistCp documentation for command line options is misaligned.
> 
>
> Key: HADOOP-14544
> URL: https://issues.apache.org/jira/browse/HADOOP-14544
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 2.7.3
>Reporter: Chris Nauroth
>Assignee: Nanda kumar
>Priority: Minor
> Attachments: DistCp 2.7.3 Documentation.png
>
>
> In the DistCp documentation, the Command Line Options section appears to be 
> misaligned/incorrect in some of the Notes for release 2.7.3.  This is the 
> current stable version, so it's likely that users will drive into this 
> version of the document.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14989) metrics2 JMX cache refresh result in inconsistent Mutable(Stat|Rate) values

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14989:

Target Version/s: 2.7.8  (was: 2.7.7)

> metrics2 JMX cache refresh result in inconsistent Mutable(Stat|Rate) values
> ---
>
> Key: HADOOP-14989
> URL: https://issues.apache.org/jira/browse/HADOOP-14989
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: metrics
>Affects Versions: 2.6.5
>Reporter: Erik Krogen
>Priority: Critical
> Attachments: HADOOP-14989.test.patch
>
>
> While doing some digging in the metrics2 system recently, we noticed that the 
> way {{MutableStat}} values are collected (and thus {{MutableRate}}, since it 
> is based off of {{MutableStat}}) mean that every time the value is 
> snapshotted, all previous information is lost. So every time a JMX cache 
> refresh occurs, it resets the {{MutableStat}}, meaning that all configured 
> metrics sinks do not consider the previous statistics in their emitted 
> values. The same behavior is true if you configured multiple sink periods.
> {{MutableStat}}, to compute its average value, maintains a total value since 
> last snapshot, as well as operation count since last snapshot. Upon 
> snapshotting, the average is calculated as (total / opCount) and placed into 
> a gauge metric, and total / operation count are cleared. So the average value 
> represents the average since the last snapshot. If we have only a single sink 
> period ever snapshotting, this would result in the expected behavior that the 
> value is the average over the reporting period. However, if multiple sink 
> periods are configured, or if the JMX cache is refreshed, this is another 
> snapshot operation. So, for example, if you have a FileSink configured at a 
> 60 second interval and your JMX cache refreshes itself 1 second before the 
> FileSink period fires, the values emitted to your FileSink only represent 
> averages _over the last one second_.
> A few ways to solve this issue:
> * Make {{MutableRate}} manage its own average refresh, similar to 
> {{MutableQuantiles}}, which has a refresh thread and saves a snapshot of the 
> last quantile values that it will serve up until the next refresh. Given how 
> many {{MutableRate}} metrics there are, a thread per metric is not really 
> feasible, but could be done on e.g. a per-source basis. This has some 
> downsides: if multiple sinks are configured with different periods, what is 
> the right refresh period for the {{MutableRate}}? 
> * Make {{MutableRate}} emit two counters, one for total and one for operation 
> count, rather than an average gauge and an operation count counter. The 
> average could then be calculated downstream from this information. This is 
> cumbersome for operators and not backwards compatible. To improve on both of 
> those downsides, we could have it keep the current behavior but 
> _additionally_ emit the total as a counter. The snapshotted average is 
> probably sufficient in the common case (we've been using it for years), and 
> when more guaranteed accuracy is required, the average could be derived from 
> the total and operation count.
> The two above suggestions will fix this for both JMX and multiple sink 
> periods, but may be overkill. Multiple sink periods are probably not 
> necessary though we should at least document the behavior.
> Open to suggestions & input here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-12630) Misuse of sun.misc.Unsafe by org.apache.hadoop.io.FastByteComparisons causes misaligned memory access coredumps

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-12630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12630:

Target Version/s: 2.7.8  (was: 2.7.7)

> Misuse of sun.misc.Unsafe by org.apache.hadoop.io.FastByteComparisons causes 
> misaligned memory access coredumps
> ---
>
> Key: HADOOP-12630
> URL: https://issues.apache.org/jira/browse/HADOOP-12630
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: native
>Affects Versions: 2.6.0, 2.7.0, 3.0.0-alpha1
> Environment: Solaris SPARC
>Reporter: Alan Burlison
>Assignee: Alan Burlison
>Priority: Major
>
> Misuse of sun.misc.unsafe by {{org.apache.hadoop.io.FastByteComparisons}} 
> causes misaligned memory accesses and results in coredumps. Stack traces 
> below:
> {code}
> hadoop-tools/hadoop-gridmix/core
>  --- called from signal handler with signal 10 (SIGBUS) ---
>  7717fa40 Unsafe_GetLong (18c000, 7e2fd6d8, 0, 19, 
> 775d4be0, 10018c000) + 158
>  70810dcc * sun/misc/Unsafe.getLong(Ljava/lang/Object;J)J+-30004
>  70810d70 * sun/misc/Unsafe.getLong(Ljava/lang/Object;J)J+0
>  70806d58 * 
> org/apache/hadoop/io/FastByteComparisons$LexicographicalComparerHolder$UnsafeComparer.compareTo([BII[BII)I+91
>  (line 405)
>  70806cb4 * 
> org/apache/hadoop/io/FastByteComparisons$LexicographicalComparerHolder$UnsafeComparer.compareTo(Ljava/lang/Object;IILjava/lang/Object;II)I+16
>  (line 264)
>  7080783c * 
> org/apache/hadoop/io/FastByteComparisons.compareTo([BII[BII)I+11 (line 92)
>  70806cb4 * 
> org/apache/hadoop/io/WritableComparator.compareBytes([BII[BII)I+8 (line 376)
>  70806cb4 * 
> org/apache/hadoop/mapred/gridmix/GridmixRecord$Comparator.compare([BII[BII)I+61
>  (line 522)
>  70806cb4 * 
> org/apache/hadoop/mapred/gridmix/TestGridmixRecord.binSortTest(Lorg/apache/hadoop/mapred/gridmix/GridmixRecord;Lorg/apache/hadoop/mapred/gridmix/GridmixRecord;IILorg/apache/hadoop/io/WritableComparator;)V+280
>  (line 268)
>  70806f44 * 
> org/apache/hadoop/mapred/gridmix/TestGridmixRecord.testBaseRecord()V+57 (line 
> 482)
> {code}
> This also causes {{hadoop-mapreduce-project/hadoop-mapreduce-examples/core}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-12861) RPC client fails too quickly when server connection limit is reached

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-12861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12861:

Target Version/s: 2.7.8  (was: 2.7.7)

> RPC client fails too quickly when server connection limit is reached
> 
>
> Key: HADOOP-12861
> URL: https://issues.apache.org/jira/browse/HADOOP-12861
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: ipc
>Affects Versions: 2.7.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Major
>
> The NN's rpc server immediately closes new client connections when a 
> connection limit is reached. The client rapidly retries a small number of 
> times with no delay which causes clients to fail quickly. If the connection 
> is refused or timedout, the connection retry policy tries with backoff. 
> Clients should treat a reset connection as a connection failure so the 
> connection retry policy is used.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14388) Don't set the key password if there is a problem reading SSL configuration

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-14388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14388:

Target Version/s: 2.7.8  (was: 2.7.7)

> Don't set the key password if there is a problem reading SSL configuration
> --
>
> Key: HADOOP-14388
> URL: https://issues.apache.org/jira/browse/HADOOP-14388
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 2.8.0, 2.7.3
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
>Priority: Minor
> Attachments: HADOOP-14388.patch
>
>
> When launching MiniDFSCluster in a test, with 
> "dfs.data.transfer.protection=integrity" and without specifying a 
> ssl-server.xml, the code hangs on "builder.build()". 
> This is because in HttpServer2, it is setting a null value on the 
> SslSocketConnector:
> c.setKeyPassword(keyPassword);
> Instead, this call should be inside the "if (keystore != null) {" block. Once 
> this is done the code exits as expected with an error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-15253) Should update maxQueueSize when refresh call queue

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-15253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-15253:

Target Version/s: 2.7.8  (was: 2.7.7)

> Should update maxQueueSize when refresh call queue
> --
>
> Key: HADOOP-15253
> URL: https://issues.apache.org/jira/browse/HADOOP-15253
> Project: Hadoop Common
>  Issue Type: Improvement
>Affects Versions: 2.8.2
>Reporter: Tao Jie
>Assignee: Tao Jie
>Priority: Minor
> Attachments: HADOOP-15253.001.patch, HADOOP-15253.002.patch, 
> HADOOP-15253.003.patch, HADOOP-15253.004.patch
>
>
> When calling {{dfsadmin -refreshCallQueue}} to update CallQueue instance, 
> {{maxQueueSize}} should also be updated.
> In case of changing CallQueue instance to FairCallQueue, the length of each 
> queue in FairCallQueue would be 1/priorityLevels of original length of 
> DefaultCallQueue. So it would be helpful for us to set the length of 
> callqueue to a proper value.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14064) Native maven build fails for non-default zlib location

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14064:

Target Version/s: 2.7.8  (was: 2.7.7)

> Native maven build fails for non-default zlib location
> --
>
> Key: HADOOP-14064
> URL: https://issues.apache.org/jira/browse/HADOOP-14064
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build, native
>Affects Versions: 2.7.3
>Reporter: David James
>Priority: Major
>  Labels: build-problem, native
> Fix For: 2.7.3
>
>
> This bug prevents me from building a native Hadoop for my system. 
> The current build fails for me if zlib is not located the usual location(s).
> I looked into ways to pass the necessary information to the Maven build. I 
> found nothing that worked short of modifying the pom.xml of hadoop-common.
> Luckily, this is easy for the project maintainers to fix -- see patch below 
> -- since the build already includes the FindZLIB module for cmake 
> (https://cmake.org/cmake/help/v3.0/module/FindZLIB.html). All that is missing 
> is to allow maven command line parameters pass through to FindZLIB.
> I suggest the following patch (works for me) that accepts two command line 
> arguments, `zlib.lib` and `zlib.include` and passes them to cmake as 
> ZLIB_LIBRARY and ZLIB_INCLUDE_DIR, respectively.
> It would be used like this:
> {code:none}
> mvn package -Pdist,native -DskipTests -Dtar \
> -Dzlib.lib=$HOME/lib/libz.so \
> -Dzlib.include=$HOME/include
> {code}
> Here is my suggested patch:
> {code:none}
> diff -r hadoop-2.7.3-src/hadoop-common-project/hadoop-common/pom.xml 
> hadoop-2.7.3-src-patched/hadoop-common-project/hadoop-common/pom.xml
> 521a522,523
> > 
> > 
> 605c607
> <   
> ---
> >   
> 643a646,647
> > 
> > 
> {code}
> Suggestions and improvements are welcome.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14078) TestRaceWhenRelogin fails occasionally

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14078:

Target Version/s: 2.7.8  (was: 2.7.7)

> TestRaceWhenRelogin fails occasionally
> --
>
> Key: HADOOP-14078
> URL: https://issues.apache.org/jira/browse/HADOOP-14078
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.9.0, 2.7.4, 2.6.6, 2.8.1, 3.0.0-alpha4
> Environment: Precommit jenkins
>Reporter: Wei-Chiu Chuang
>Priority: Major
>
> HADOOP-13433 added this test class and it failed in a few precommit jobs like 
> this one: 
> https://builds.apache.org/job/PreCommit-HADOOP-Build/11616/testReport/org.apache.hadoop.security/TestRaceWhenRelogin/test/
> There were a lot of errors in the test, starting with this one
> {noformat}
> 2017-02-13 12:26:01,838 ERROR impl.DefaultKdcHandler 
> (DefaultKdcHandler.java:handleMessage(71)) - Error occured while processing 
> request:
> org.apache.kerby.kerberos.kerb.KrbException: Integrity check on decrypted 
> field failed
>   at 
> org.apache.kerby.kerberos.kerb.crypto.enc.KeKiEnc.decryptWith(KeKiEnc.java:127)
>   at 
> org.apache.kerby.kerberos.kerb.crypto.enc.AbstractEncTypeHandler.decrypt(AbstractEncTypeHandler.java:150)
>   at 
> org.apache.kerby.kerberos.kerb.crypto.enc.AbstractEncTypeHandler.decrypt(AbstractEncTypeHandler.java:138)
>   at 
> org.apache.kerby.kerberos.kerb.crypto.EncryptionHandler.decrypt(EncryptionHandler.java:228)
>   at 
> org.apache.kerby.kerberos.kerb.common.EncryptionUtil.unseal(EncryptionUtil.java:136)
>   at 
> org.apache.kerby.kerberos.kerb.server.request.TgsRequest.verifyAuthenticator(TgsRequest.java:138)
>   at 
> org.apache.kerby.kerberos.kerb.server.preauth.builtin.TgtPreauth.verify(TgtPreauth.java:41)
>   at 
> org.apache.kerby.kerberos.kerb.server.preauth.PreauthHandle.verify(PreauthHandle.java:46)
>   at 
> org.apache.kerby.kerberos.kerb.server.preauth.PreauthHandler.verify(PreauthHandler.java:101)
>   at 
> org.apache.kerby.kerberos.kerb.server.request.KdcRequest.preauth(KdcRequest.java:562)
>   at 
> org.apache.kerby.kerberos.kerb.server.request.KdcRequest.process(KdcRequest.java:181)
>   at 
> org.apache.kerby.kerberos.kerb.server.KdcHandler.handleMessage(KdcHandler.java:115)
>   at 
> org.apache.kerby.kerberos.kerb.server.impl.DefaultKdcHandler.handleMessage(DefaultKdcHandler.java:67)
>   at 
> org.apache.kerby.kerberos.kerb.server.impl.DefaultKdcHandler.run(DefaultKdcHandler.java:52)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-11957) if an IOException error is thrown in DomainSocket.close we go into infinite loop.

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-11957:

Target Version/s: 2.7.8  (was: 2.7.7)

> if an IOException error is thrown in DomainSocket.close we go into infinite 
> loop.
> -
>
> Key: HADOOP-11957
> URL: https://issues.apache.org/jira/browse/HADOOP-11957
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: net
>Affects Versions: 2.7.1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
>Priority: Major
> Attachments: HADOOP-11957.001.patch
>
>
> if an IOException error is thrown in DomainSocket.close we go into infinite 
> loop.
> Issue : If the shutdown0(fd) call throws an IOException we break out of the 
> if shutdown call but will continue to loop in the while loop infinitely since 
> we have no way of decrementing the counter. Please scroll down and see the 
> comment marked with Bug Bug to see where the issue is.
> {code:title=DomainSocket.java}
>   @Override
>   public void close() throws IOException {
> // Set the closed bit on this DomainSocket
> int count = 0;
> try {
>   count = refCount.setClosed();
> } catch (ClosedChannelException e) {
>   // Someone else already closed the DomainSocket.
>   return;
> }
> // Wait for all references to go away
> boolean didShutdown = false;
> boolean interrupted = false;
> while (count > 0) {
>   if (!didShutdown) {
> try {
>   // Calling shutdown on the socket will interrupt blocking system
>   // calls like accept, write, and read that are going on in a
>   // different thread.
>   shutdown0(fd);
> } catch (IOException e) {
>   LOG.error("shutdown error: ", e);
> }
> didShutdown = true; 
> // *BUG BUG* <-- Here the code will never exit the loop
> // if the count is greater then 0. we need to break out
> // of the while loop in case of IOException Error
>   }
>   try {
> Thread.sleep(10);
>   } catch (InterruptedException e) {
> interrupted = true;
>   }
>   count = refCount.getReferenceCount();
> }
> // At this point, nobody has a reference to the file descriptor, 
> // and nobody will be able to get one in the future either.
> // We now call close(2) on the file descriptor.
> // After this point, the file descriptor number will be reused by 
> // something else.  Although this DomainSocket object continues to hold 
> // the old file descriptor number (it's a final field), we never use it 
> // again because this DomainSocket is closed.
> close0(fd);
> if (interrupted) {
>   Thread.currentThread().interrupt();
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-12208) Wrong token in the authentication header when there is re-authentication request

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12208:

Target Version/s: 2.7.8  (was: 2.7.7)

> Wrong token in the authentication header when there is re-authentication 
> request
> 
>
> Key: HADOOP-12208
> URL: https://issues.apache.org/jira/browse/HADOOP-12208
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/swift
>Affects Versions: 2.6.0
>Reporter: Gil Vernik
>Assignee: Gil Vernik
>Priority: Major
> Attachments: token-header-fix-0001.patch
>
>
> When authentication token expires, Swift returns 401. In this case the exec 
> method from SwiftRestClient catches this exception and performs another 
> authentication request to renew the token. If authentication successful, exec 
> method retry original request. However, the bug is that retry still uses old 
> token in a header and doesn't update it with a new one.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-12125) Retrying UnknownHostException on a proxy does not actually retry hostname resolution

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-12125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-12125:

Target Version/s: 2.7.8  (was: 2.7.7)

> Retrying UnknownHostException on a proxy does not actually retry hostname 
> resolution
> 
>
> Key: HADOOP-12125
> URL: https://issues.apache.org/jira/browse/HADOOP-12125
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: ipc
>Reporter: Jason Lowe
>Assignee: Rushabh S Shah
>Priority: Major
>
> When RetryInvocationHandler attempts to retry an UnknownHostException the 
> hostname fails to be resolved again.  The InetSocketAddress in the 
> ConnectionId has cached the fact that the hostname is unresolvable, and when 
> the proxy tries to setup a new Connection object with that ConnectionId it 
> checks if the (cached) resolution result is unresolved and immediately throws.
> The end result is we sleep and retry for no benefit.  The hostname resolution 
> is never attempted again.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-15477) Make unjar in RunJar overrideable

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-15477:

Target Version/s: 2.7.8  (was: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 2.7.7, 
2.8.5)

> Make unjar in RunJar overrideable
> -
>
> Key: HADOOP-15477
> URL: https://issues.apache.org/jira/browse/HADOOP-15477
> Project: Hadoop Common
>  Issue Type: Improvement
>Affects Versions: 2.8.3, 2.9.1, 2.7.6, 3.0.2
>Reporter: Johan Gustavsson
>Assignee: Johan Gustavsson
>Priority: Trivial
> Fix For: 3.2.0
>
> Attachments: HADOOP-15477.001.patch, HADOOP-15477.002.patch, 
> HADOOP-15477.003.patch, HADOOP-15477.004.patch
>
>
> Currently Hadoop's RunJar will unjar the jar provided and look for any jars 
> inside and add them to the classpath. Since most deployments doesn't use jar 
> in jar, but rather uberjars this could be rather time consuming at times and 
> can cause issues related to over consumption of inodes, for something that is 
> in many cases is not used.
> For that purpose there should be an env variable to disable this behavior.
>  
> Edit: As requested by [~ajisakaa] in person here is a more detailed 
> description of the issues we are trying to solve with this.
> A good chunk of our workloads are packaged in an uberjar, and are launched as 
> a separate process using the {{hadoop jar}} cli. This is has generally been 
> working out pretty well historically, with sub second launch times and good 
> client isolation. Since bumping the host OS to a version patched with 
> Meltdown/Specter patches we do see from time to time load becoming very high 
> even with only a few client processes running and a single unjar process 
> taking up to 10min. 
> While another simple approach would be to abandon using the {{hadoop jar}} 
> cli this would most likely take a lot more work than simply disabling unjar 
> for the time being.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14075) chown doesn't work with usernames containing '\' character

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-14075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14075:

Target Version/s: 2.7.8  (was: 2.7.7)

> chown doesn't work with usernames containing '\' character
> --
>
> Key: HADOOP-14075
> URL: https://issues.apache.org/jira/browse/HADOOP-14075
> Project: Hadoop Common
>  Issue Type: Sub-task
>Affects Versions: 2.6.0
>Reporter: Attila Bukor
>Assignee: Attila Bukor
>Priority: Major
> Attachments: HADOOP-14075.001.patch, HADOOP-14075.002.patch
>
>
> Usernames containing backslash (e.g. down-level logon names) seem to work 
> fine with Hadoop, except for chown.
> {code}
> $ HADOOP_USER_NAME="FOOBAR\\testuser" hdfs dfs -mkdir /test/testfile1
> $ hdfs dfs -ls /test
> Found 1 items
> drwxrwxr-x   - FOOBAR\testuser supergroup  0 2017-02-10 12:49 
> /test/testfile1
> $ HADOOP_USER_NAME="testuser" hdfs dfs -mkdir /test/testfile2
> $ HADOOP_USER_NAME="hdfs" hdfs dfs -chown "FOOBAR\\testuser" /test/testfile2
> -chown: 'FOOBAR\testuser' does not match expected pattern for [owner][:group].
> Usage: hadoop fs [generic options] -chown [-R] [OWNER][:[GROUP]] PATH...
> $
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-14313) Replace/improve Hadoop's byte[] comparator

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-14313:

Target Version/s: 2.7.8  (was: 2.7.7)

> Replace/improve Hadoop's byte[] comparator
> --
>
> Key: HADOOP-14313
> URL: https://issues.apache.org/jira/browse/HADOOP-14313
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Affects Versions: 2.7.4
>Reporter: Vikas Vishwakarma
>Priority: Major
> Attachments: HADOOP-14313.001.patch, 
> HADOOP-14313.branch-2.7.001.patch, HADOOP-14313.branch-2.7.002.patch
>
>
> Hi,
> Recently we were looking at the Lexicographic byte array comparison in HBase. 
> We did microbenchmark for the byte array comparator of HADOOP ( 
> https://github.com/hanborq/hadoop/blob/master/src/core/org/apache/hadoop/io/FastByteComparisons.java#L161
>  ) , HBase Vs the latest byte array comparator from guava  ( 
> https://github.com/google/guava/blob/master/guava/src/com/google/common/primitives/UnsignedBytes.java#L362
>  ) and observed that the guava main branch version is much faster. 
> Specifically we see very good improvement when the byteArraySize%8 != 0 and 
> also for large byte arrays. I will update the benchmark results using JMH for 
> Hadoop vs Guava. For the jira on HBase, please refer HBASE-17877. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-15129) Datanode caches namenode DNS lookup failure and cannot startup

2018-06-02 Thread Steve Loughran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran updated HADOOP-15129:

Target Version/s: 2.7.8  (was: 2.7.7)

> Datanode caches namenode DNS lookup failure and cannot startup
> --
>
> Key: HADOOP-15129
> URL: https://issues.apache.org/jira/browse/HADOOP-15129
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: ipc
>Affects Versions: 2.8.2
> Environment: Google Compute Engine.
> I'm using Java 8, Debian 8, Hadoop 2.8.2.
>Reporter: Karthik Palaniappan
>Assignee: Karthik Palaniappan
>Priority: Minor
> Attachments: HADOOP-15129.001.patch, HADOOP-15129.002.patch
>
>
> On startup, the Datanode creates an InetSocketAddress to register with each 
> namenode. Though there are retries on connection failure throughout the 
> stack, the same InetSocketAddress is reused.
> InetSocketAddress is an interesting class, because it resolves DNS names to 
> IP addresses on construction, and it is never refreshed. Hadoop re-creates an 
> InetSocketAddress in some cases just in case the remote IP has changed for a 
> particular DNS name: https://issues.apache.org/jira/browse/HADOOP-7472.
> Anyway, on startup, you cna see the Datanode log: "Namenode...remains 
> unresolved" -- referring to the fact that DNS lookup failed.
> {code:java}
> 2017-11-02 16:01:55,115 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
> Refresh request received for nameservices: null
> 2017-11-02 16:01:55,153 WARN org.apache.hadoop.hdfs.DFSUtilClient: Namenode 
> for null remains unresolved for ID null. Check your hdfs-site.xml file to 
> ensure namenodes are configured properly.
> 2017-11-02 16:01:55,156 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
> Starting BPOfferServices for nameservices: 
> 2017-11-02 16:01:55,169 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
> Block pool  (Datanode Uuid unassigned) service to 
> cluster-32f5-m:8020 starting to offer service
> {code}
> The Datanode then proceeds to use this unresolved address, as it may work if 
> the DN is configured to use a proxy. Since I'm not using a proxy, it forever 
> prints out this message:
> {code:java}
> 2017-12-15 00:13:40,712 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: 
> Problem connecting to server: cluster-32f5-m:8020
> 2017-12-15 00:13:45,712 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: 
> Problem connecting to server: cluster-32f5-m:8020
> 2017-12-15 00:13:50,712 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: 
> Problem connecting to server: cluster-32f5-m:8020
> 2017-12-15 00:13:55,713 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: 
> Problem connecting to server: cluster-32f5-m:8020
> 2017-12-15 00:14:00,713 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: 
> Problem connecting to server: cluster-32f5-m:8020
> {code}
> Unfortunately, the log doesn't contain the exception that triggered it, but 
> the culprit is actually in IPC Client: 
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java#L444.
> This line was introduced in https://issues.apache.org/jira/browse/HADOOP-487 
> to give a clear error message when somebody mispells an address.
> However, the fix in HADOOP-7472 doesn't apply here, because that code happens 
> in Client#getConnection after the Connection is constructed.
> My proposed fix (will attach a patch) is to move this exception out of the 
> constructor and into a place that will trigger HADOOP-7472's logic to 
> re-resolve addresses. If the DNS failure was temporary, this will allow the 
> connection to succeed. If not, the connection will fail after ipc client 
> retries (default 10 seconds worth of retries).
> I want to fix this in ipc client rather than just in Datanode startup, as 
> this fixes temporary DNS issues for all of Hadoop.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-15217) org.apache.hadoop.fs.FsUrlConnection does not handle paths with spaces

2018-06-02 Thread Zsolt Venczel (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HADOOP-15217:
---
Attachment: HADOOP-15217.02.patch

> org.apache.hadoop.fs.FsUrlConnection does not handle paths with spaces
> --
>
> Key: HADOOP-15217
> URL: https://issues.apache.org/jira/browse/HADOOP-15217
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs
>Affects Versions: 3.0.0
>Reporter: Joseph Fourny
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HADOOP-15217.01.patch, HADOOP-15217.02.patch, 
> TestCase.java
>
>
> When _FsUrlStreamHandlerFactory_ is registered with _java.net.URL_ (ex: when 
> Spark is initialized), it breaks URLs with spaces (even though they are 
> properly URI-encoded). I traced the problem down to 
> _FSUrlConnection.connect()_ method. It naively gets the path from the URL, 
> which contains encoded spaces, and pases it to 
> _org.apache.hadoop.fs.Path(String)_ constructor. This is not correct, because 
> the docs clearly say that the string must NOT be encoded. Doing so causes 
> double encoding within the Path class (ie: %20 becomes %2520). 
> See attached JUnit test. 
> This test case mimics an issue I ran into when trying to use Commons 
> Configuration 1.9 AFTER initializing Spark. Commons Configuration uses URL 
> class to load configuration files, but Spark installs 
> _FsUrlStreamHandlerFactory_, which hits this issue. For now, we are using an 
> AspectJ aspect to "patch" the bytecode at load time to work-around the issue. 
> The real fix is quite simple. All you need to do is replace this line in 
> _org.apache.hadoop.fs.FsUrlConnection.connect()_:
>         is = fs.open(new Path(url.getPath()));
> with this line:
>      is = fs.open(new Path(url.*toUri()*.getPath()));
> URI.getPath() will correctly decode the path, which is what is expected by 
> _org.apache.hadoop.fs.Path(String)_ constructor.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-15217) org.apache.hadoop.fs.FsUrlConnection does not handle paths with spaces

2018-06-02 Thread Zsolt Venczel (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499199#comment-16499199
 ] 

Zsolt Venczel commented on HADOOP-15217:


Thank you very much [~xiaochen] and [~gabor.bota] for the review!

In my latest patch I updated the Path object construct as suggested.
I added a test timeout rule for the test suite and removed the old construct.

Let me share my thoughts with regards failing a unit test with an error instead 
of an assert: in my understanding unit tests should check for the outcome of a 
logic by asserting rules that are as explicit as possible. On the other hand 
when an error occurs it skips the grip of the asserts and has a more vague 
meaning: something went wrong somewhere but not explicitly in the logic.
Based on the above I feel the catch and fail with an assert approach a bit more 
explicit in stabilizing the code. What do your think?

Cheers and best regards,
Zsolt
 

> org.apache.hadoop.fs.FsUrlConnection does not handle paths with spaces
> --
>
> Key: HADOOP-15217
> URL: https://issues.apache.org/jira/browse/HADOOP-15217
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs
>Affects Versions: 3.0.0
>Reporter: Joseph Fourny
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HADOOP-15217.01.patch, HADOOP-15217.02.patch, 
> TestCase.java
>
>
> When _FsUrlStreamHandlerFactory_ is registered with _java.net.URL_ (ex: when 
> Spark is initialized), it breaks URLs with spaces (even though they are 
> properly URI-encoded). I traced the problem down to 
> _FSUrlConnection.connect()_ method. It naively gets the path from the URL, 
> which contains encoded spaces, and pases it to 
> _org.apache.hadoop.fs.Path(String)_ constructor. This is not correct, because 
> the docs clearly say that the string must NOT be encoded. Doing so causes 
> double encoding within the Path class (ie: %20 becomes %2520). 
> See attached JUnit test. 
> This test case mimics an issue I ran into when trying to use Commons 
> Configuration 1.9 AFTER initializing Spark. Commons Configuration uses URL 
> class to load configuration files, but Spark installs 
> _FsUrlStreamHandlerFactory_, which hits this issue. For now, we are using an 
> AspectJ aspect to "patch" the bytecode at load time to work-around the issue. 
> The real fix is quite simple. All you need to do is replace this line in 
> _org.apache.hadoop.fs.FsUrlConnection.connect()_:
>         is = fs.open(new Path(url.getPath()));
> with this line:
>      is = fs.open(new Path(url.*toUri()*.getPath()));
> URI.getPath() will correctly decode the path, which is what is expected by 
> _org.apache.hadoop.fs.Path(String)_ constructor.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Created] (HADOOP-15510) Randomize baseDir for MiniDFSCluster in TestDFSStripedInputStream and TestDFSStripedInputStreamWithRandomECPolicy

2018-06-02 Thread Anbang Hu (JIRA)
Anbang Hu created HADOOP-15510:
--

 Summary: Randomize baseDir for MiniDFSCluster in 
TestDFSStripedInputStream and TestDFSStripedInputStreamWithRandomECPolicy
 Key: HADOOP-15510
 URL: https://issues.apache.org/jira/browse/HADOOP-15510
 Project: Hadoop Common
  Issue Type: Test
Reporter: Anbang Hu
Assignee: Anbang Hu






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-15217) org.apache.hadoop.fs.FsUrlConnection does not handle paths with spaces

2018-06-02 Thread genericqa (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499225#comment-16499225
 ] 

genericqa commented on HADOOP-15217:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m 
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m  
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m 50s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
30s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 26m 
56s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
3m  8s{color} | {color:orange} root: The patch generated 1 new + 1 unchanged - 
0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
10m  5s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  8m 
43s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m 
36s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
36s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}136m 16s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HADOOP-15217 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12926250/HADOOP-15217.02.patch 
|
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  |
| uname | Linux cc0f24f43476 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 8261f9e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/14717/artifact/ou