[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939451#comment-14939451 ] zhihai xu commented on HADOOP-8437: --- The test failures are not related to the patch. All these tests passed in my local build. +1 LGTM for the latest patch, I will commit it tomorrow if no one objects. > getLocalPathForWrite is not throwing any expection for invalid paths > > > Key: HADOOP-8437 > URL: https://issues.apache.org/jira/browse/HADOOP-8437 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.0.0-alpha, 2.6.0, 3.0.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: HADOOP-8437-003.patch, HADOOP-8437-004.patch, > HADOOP-8437-005.patch, HADOOP-8437.patch, HADOOP-8437_1.patch, > HADOOP-8437_2.patch > > > call dirAllocator.getLocalPathForWrite ( "/InvalidPath", conf ); > Here it will not thrown any exception but earlier version it used throw. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14935019#comment-14935019
]
Hadoop QA commented on HADOOP-8437:
---
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 17m 30s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 58s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 8s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 10s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 27s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 51s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 6m 58s | Tests failed in
hadoop-common. |
| | | 48m 1s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.security.ssl.TestReloadingX509TrustManager |
| Timed out tests | org.apache.hadoop.ha.TestZKFailoverController |
| | org.apache.hadoop.ha.TestZKFailoverControllerStress |
| | org.apache.hadoop.metrics2.lib.TestMutableMetrics |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12764221/HADOOP-8437-005.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / d6fa34e |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7733/console |
This message was automatically generated.
> getLocalPathForWrite is not throwing any expection for invalid paths
>
>
> Key: HADOOP-8437
> URL: https://issues.apache.org/jira/browse/HADOOP-8437
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.0.0-alpha, 2.6.0, 3.0.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-8437-003.patch, HADOOP-8437-004.patch,
> HADOOP-8437-005.patch, HADOOP-8437.patch, HADOOP-8437_1.patch,
> HADOOP-8437_2.patch
>
>
> call dirAllocator.getLocalPathForWrite ( "/InvalidPath", conf );
> Here it will not thrown any exception but earlier version it used throw.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934854#comment-14934854 ] Brahma Reddy Battula commented on HADOOP-8437: -- [~zxu] thanks for taking a look into this issue.. Updated patch based on your comments..Kindly Review.. > getLocalPathForWrite is not throwing any expection for invalid paths > > > Key: HADOOP-8437 > URL: https://issues.apache.org/jira/browse/HADOOP-8437 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.0.0-alpha, 2.6.0, 3.0.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Labels: BB2015-05-TBR > Attachments: HADOOP-8437-003.patch, HADOOP-8437-004.patch, > HADOOP-8437.patch, HADOOP-8437_1.patch, HADOOP-8437_2.patch > > > call dirAllocator.getLocalPathForWrite ( "/InvalidPath", conf ); > Here it will not thrown any exception but earlier version it used throw. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934924#comment-14934924
]
Hadoop QA commented on HADOOP-8437:
---
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 17m 17s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 53s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 9s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 5s | The applied patch generated 1
new checkstyle issues (total was 23, now 24). |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 3 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 27s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 53s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests | 7m 47s | Tests passed in
hadoop-common. |
| | | 48m 31s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12764209/HADOOP-8437-004.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / d6fa34e |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/trunkFindbugsWarningshadoop-common.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/whitespace.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7731/console |
This message was automatically generated.
> getLocalPathForWrite is not throwing any expection for invalid paths
>
>
> Key: HADOOP-8437
> URL: https://issues.apache.org/jira/browse/HADOOP-8437
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.0.0-alpha, 2.6.0, 3.0.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Attachments: HADOOP-8437-003.patch, HADOOP-8437-004.patch,
> HADOOP-8437.patch, HADOOP-8437_1.patch, HADOOP-8437_2.patch
>
>
> call dirAllocator.getLocalPathForWrite ( "/InvalidPath", conf );
> Here it will not thrown any exception but earlier version it used throw.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14908372#comment-14908372
]
zhihai xu commented on HADOOP-8437:
---
thanks [~brahmareddy] for the patch! If the size of {{dirs}} is zero, it will
cause the following exception.
{code}
java.lang.IllegalArgumentException: n must be positive
at java.util.Random.nextInt(Random.java:250)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:243)
{code}
same exception found at HADOOP-6766 and MAPREDUCE-635.
Can we avoid IllegalArgumentException by checking {{dirs.size()}} non-zero
before calling {{dirIndexRandomizer.nextInt}} with the following simpler change?
{code}
if (dirs.size() > 0) {
// randomize the first disk picked in the round-robin selection
dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size());
}
{code}
then it will trigger {{DiskErrorException}} later due to zero length
{{localDirs}} if calling from {{getLocalPathForWrite}} or
{{getLocalPathToRead}} .
> getLocalPathForWrite is not throwing any expection for invalid paths
>
>
> Key: HADOOP-8437
> URL: https://issues.apache.org/jira/browse/HADOOP-8437
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.0.0-alpha, 2.6.0, 3.0.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Labels: BB2015-05-TBR
> Attachments: HADOOP-8437-003.patch, HADOOP-8437.patch,
> HADOOP-8437_1.patch, HADOOP-8437_2.patch
>
>
> call dirAllocator.getLocalPathForWrite ( "/InvalidPath", conf );
> Here it will not thrown any exception but earlier version it used throw.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14908468#comment-14908468
]
zhihai xu commented on HADOOP-8437:
---
By the way, a much easier way for a test case may be just to change the
configuration to a string with only white space.
For example, run the following code:
{code}
conf1.set(CONTEXT, " ");
dirAllocator.getLocalPathForWrite("/test", conf1);
{code}
You can get the following exception:
{code}
java.lang.IllegalArgumentException: n must be positive
at java.util.Random.nextInt(Random.java:300)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:308)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:347)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:115)
at
org.apache.hadoop.fs.TestLocalDirAllocator.testShouldNotthrowNPE(TestLocalDirAllocator.java:320)
{code}
> getLocalPathForWrite is not throwing any expection for invalid paths
>
>
> Key: HADOOP-8437
> URL: https://issues.apache.org/jira/browse/HADOOP-8437
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 2.0.0-alpha, 2.6.0, 3.0.0
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Labels: BB2015-05-TBR
> Attachments: HADOOP-8437-003.patch, HADOOP-8437.patch,
> HADOOP-8437_1.patch, HADOOP-8437_2.patch
>
>
> call dirAllocator.getLocalPathForWrite ( "/InvalidPath", conf );
> Here it will not thrown any exception but earlier version it used throw.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541586#comment-14541586
]
Tsuyoshi Ozawa commented on HADOOP-8437:
[~brahmareddy] thank you for taking this issue. As [~qwertymaniac] mentioned,
we should fix the test case to make it FS independent. Could you try it?
{quote}
I'm fine with the 256 length test, with a comment added along that this depends
on the FS in use to fail and may pass on some FS.
{quote}
getLocalPathForWrite is not throwing any expection for invalid paths
Key: HADOOP-8437
URL: https://issues.apache.org/jira/browse/HADOOP-8437
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha, 3.0.0, 2.6.0
Reporter: Brahma Reddy Battula
Assignee: Brahma Reddy Battula
Labels: BB2015-05-TBR
Attachments: HADOOP-8437-003.patch, HADOOP-8437.patch,
HADOOP-8437_1.patch, HADOOP-8437_2.patch
call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf );
Here it will not thrown any exception but earlier version it used throw.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386111#comment-14386111 ] Brahma Reddy Battula commented on HADOOP-8437: -- [~qwertymaniac] Kindly review the attached patch getLocalPathForWrite is not throwing any expection for invalid paths Key: HADOOP-8437 URL: https://issues.apache.org/jira/browse/HADOOP-8437 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.0-alpha, 3.0.0, 2.6.0 Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Attachments: HADOOP-8437-003.patch, HADOOP-8437.patch, HADOOP-8437_1.patch, HADOOP-8437_2.patch call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf ); Here it will not thrown any exception but earlier version it used throw. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386142#comment-14386142
]
Hadoop QA commented on HADOOP-8437:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708073/HADOOP-8437-003.patch
against trunk revision 232eca9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6022//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/6022//console
This message is automatically generated.
getLocalPathForWrite is not throwing any expection for invalid paths
Key: HADOOP-8437
URL: https://issues.apache.org/jira/browse/HADOOP-8437
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha, 3.0.0, 2.6.0
Reporter: Brahma Reddy Battula
Assignee: Brahma Reddy Battula
Attachments: HADOOP-8437-003.patch, HADOOP-8437.patch,
HADOOP-8437_1.patch, HADOOP-8437_2.patch
call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf );
Here it will not thrown any exception but earlier version it used throw.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761596#comment-13761596 ] Brahma Reddy Battula commented on HADOOP-8437: -- [~qwertymaniac] Hope 256 length test is okie..can you check first patch.. getLocalPathForWrite is not throwing any expection for invalid paths Key: HADOOP-8437 URL: https://issues.apache.org/jira/browse/HADOOP-8437 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.0-alpha Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Attachments: HADOOP-8437_1.patch, HADOOP-8437.patch call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf ); Here it will not thrown any exception but earlier version it used throw. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470076#comment-13470076
]
Brahma Reddy Battula commented on HADOOP-8437:
--
Hi Harsh,
do u have any suggestions on this apart from permissions..? or following will
be okie.
{code}
for (int i = 0; i 256; i++) {
+ invalidPath.append(A);
+}
{code}
I tried with special chars which are not supported by OS..
getLocalPathForWrite is not throwing any expection for invalid paths
Key: HADOOP-8437
URL: https://issues.apache.org/jira/browse/HADOOP-8437
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Brahma Reddy Battula
Assignee: Brahma Reddy Battula
Attachments: HADOOP-8437_1.patch, HADOOP-8437.patch
call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf );
Here it will not thrown any exception but earlier version it used throw.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470755#comment-13470755 ] Harsh J commented on HADOOP-8437: - Would permissions not work? I had tried the special char approach on your patch earlier but Linux supports almost everything we could use as a reasonable test. If permission tweaks are not gonna work, I'm fine with the 256 length test, with a comment added along that this depends on the FS in use to fail and may pass on some FS. getLocalPathForWrite is not throwing any expection for invalid paths Key: HADOOP-8437 URL: https://issues.apache.org/jira/browse/HADOOP-8437 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.0-alpha Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Attachments: HADOOP-8437_1.patch, HADOOP-8437.patch call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf ); Here it will not thrown any exception but earlier version it used throw. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13462631#comment-13462631 ] Brahma Reddy Battula commented on HADOOP-8437: -- Hi Harsh, Thanks a lot for reviewing the patch..I updated patch.. getLocalPathForWrite is not throwing any expection for invalid paths Key: HADOOP-8437 URL: https://issues.apache.org/jira/browse/HADOOP-8437 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.0-alpha Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Attachments: HADOOP-8437_1.patch, HADOOP-8437.patch call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf ); Here it will not thrown any exception but earlier version it used throw. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13462640#comment-13462640 ] Hadoop QA commented on HADOOP-8437: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12546490/HADOOP-8437_1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.TestLocalDirAllocator +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1515//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1515//console This message is automatically generated. getLocalPathForWrite is not throwing any expection for invalid paths Key: HADOOP-8437 URL: https://issues.apache.org/jira/browse/HADOOP-8437 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.0-alpha Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Attachments: HADOOP-8437_1.patch, HADOOP-8437.patch call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf ); Here it will not thrown any exception but earlier version it used throw. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13462669#comment-13462669 ] Harsh J commented on HADOOP-8437: - Doesn't seem like the logic is working (and hence the test failure), can you recheck the patch please Brahma? I'm not sure what we're missing :) getLocalPathForWrite is not throwing any expection for invalid paths Key: HADOOP-8437 URL: https://issues.apache.org/jira/browse/HADOOP-8437 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.0-alpha Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Attachments: HADOOP-8437_1.patch, HADOOP-8437.patch call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf ); Here it will not thrown any exception but earlier version it used throw. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13462682#comment-13462682
]
Harsh J commented on HADOOP-8437:
-
Ok so its cause of my earlier comment:
{code}
+for (int i = 0; i 256; i++) {
+ invalidPath.append(A);
+}
{code}
This creates a condition where the filename request is max allowed filename
length on some FS-es. Since thats not a nice way to do it (it may begin
'passing' on some FSes), lets instead play with permissions or something?
getLocalPathForWrite is not throwing any expection for invalid paths
Key: HADOOP-8437
URL: https://issues.apache.org/jira/browse/HADOOP-8437
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Brahma Reddy Battula
Assignee: Brahma Reddy Battula
Attachments: HADOOP-8437_1.patch, HADOOP-8437.patch
call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf );
Here it will not thrown any exception but earlier version it used throw.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461455#comment-13461455
]
Harsh J commented on HADOOP-8437:
-
Hi Brahma,
Couple of comments:
{code}
+if (0 == dirs.size()) {
{code}
I feel this check can happen slightly more earlier as the hunk itself seems to
suggest that dirs is being utilized before we come to this new check.
{code}
+ throw new IOException(No dirs to select.Total dirs size is 0);
{code}
Please space the two sentences (general rule: Always a space after a
punctuation mark, never before.)
{code}
+for (int i = 0; i 256; i++) {
+ invalidPath.append(A);
+}
{code}
Seems excessive to do this. Just declare a constant string? :-)
{code}
fail(not throwing the exception);
{code}
Lets describe failure strings better since it is a specific test we're carrying
out/expecting a failure from: An exception should have been thrown when an
invalid directory is used.
Thanks!
getLocalPathForWrite is not throwing any expection for invalid paths
Key: HADOOP-8437
URL: https://issues.apache.org/jira/browse/HADOOP-8437
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Brahma Reddy Battula
Assignee: Brahma Reddy Battula
Attachments: HADOOP-8437.patch
call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf );
Here it will not thrown any exception but earlier version it used throw.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284878#comment-13284878
]
Brahma Reddy Battula commented on HADOOP-8437:
--
Following testcases failing because someother(timedout) reason not because of
this fix..
{noformat}
-1 core tests. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.ha.TestZKFailoverController
org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
{noformat}
getLocalPathForWrite is not throwing any expection for invalid paths
Key: HADOOP-8437
URL: https://issues.apache.org/jira/browse/HADOOP-8437
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.1-alpha
Reporter: Brahma Reddy Battula
Assignee: Brahma Reddy Battula
Fix For: 2.0.1-alpha, 3.0.0
Attachments: HADOOP-8437.patch
call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf );
Here it will not thrown any exception but earlier version it used throw.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[ https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284480#comment-13284480 ] Hadoop QA commented on HADOOP-8437: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12529963/HADOOP-8437.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1044//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1044//console This message is automatically generated. getLocalPathForWrite is not throwing any expection for invalid paths Key: HADOOP-8437 URL: https://issues.apache.org/jira/browse/HADOOP-8437 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.1-alpha Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula Fix For: 2.0.1-alpha, 3.0.0 Attachments: HADOOP-8437.patch call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf ); Here it will not thrown any exception but earlier version it used throw. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8437) getLocalPathForWrite is not throwing any expection for invalid paths
[
https://issues.apache.org/jira/browse/HADOOP-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13283223#comment-13283223
]
Brahma Reddy Battula commented on HADOOP-8437:
--
In earlier version
{code}
localDirs = dirs.toArray(new String[dirs.size()]);
dirDF = dfList.toArray(new DF[dirs.size()]);
savedLocalDirs = newLocalDirs;
if (0 == dirs.size()) {
throw new IOException(No dirs to select.Total dirs size is 0);
}
// randomize the first disk picked in the round-robin selection
dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size());
dirNumLastAccessedforKnownSize = dirNumLastAccessed;
{code}
Here it is throwing IOException saying that No dirs to select.Total dirs size
is 0
But in brnach2 and trunk...code is like follwoing..
{code}
localDirs = dirs.toArray(new String[dirs.size()]);
dirDF = dfList.toArray(new DF[dirs.size()]);
savedLocalDirs = newLocalDirs;
// randomize the first disk picked in the round-robin selection
dirNumLastAccessed = dirIndexRandomizer.nextInt(dirs.size());
{code}
Here follwoing check is removed..
{code}
if (0 == dirs.size()) {
throw new IOException(No dirs to select.Total dirs size is 0);
}
{code}
getLocalPathForWrite is not throwing any expection for invalid paths
Key: HADOOP-8437
URL: https://issues.apache.org/jira/browse/HADOOP-8437
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.1-alpha
Reporter: Brahma Reddy Battula
Assignee: Brahma Reddy Battula
call dirAllocator.getLocalPathForWrite ( /InvalidPath, conf );
Here it will not thrown any exception but earlier version it used throw.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
