[jira] [Commented] (MAPREDUCE-7132) JobSplitWriter prints unnecessary warnings if EC is used

2018-10-16 Thread Xiao Chen (JIRA) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652103#comment-16652103
 ] 

Xiao Chen commented on MAPREDUCE-7132:
--

checkstyle is pre-existing. +1 on patch 8. Committing

> JobSplitWriter prints unnecessary warnings if EC is used
> 
>
> Key: MAPREDUCE-7132
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7132
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: client, mrv2
>Affects Versions: 3.1.1
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MAPREDUCE-7132-001.patch, MAPREDUCE-7132-002.patch, 
> MAPREDUCE-7132-003.patch, MAPREDUCE-7132-004.patch, MAPREDUCE-7132-005.patch, 
> MAPREDUCE-7132-006.patch, MAPREDUCE-7132-007.patch, MAPREDUCE-7132-008.patch
>
>
> Currently, {{JobSplitWriter}} compares the number of hosts for a certain 
> block against a static value that comes from 
> {{mapreduce.job.max.split.locations}}. The default value of this property is 
> 10.
> However, an EC schema like RS-10-4 requires at least 14 hosts. In this case, 
> 14 block locations will be returned and {{JobSplitWriter}} prints a warning, 
> which can confuse users.
> A possible solution could check whether EC is enabled for a block and 
> increase this value dynamically if needed. A simpler one is to simply 
> increase the default value to a sensible number like 15.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

[jira] [Commented] (MAPREDUCE-7132) JobSplitWriter prints unnecessary warnings if EC is used

2018-10-16 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651773#comment-16651773
 ] 

Hadoop QA commented on MAPREDUCE-7132:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m 42s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
2s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 59s{color} | {color:orange} 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: 
The patch generated 1 new + 57 unchanged - 1 fixed = 58 total (was 58) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
19m 35s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  7m 
57s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 85m 49s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 |
| JIRA Issue | MAPREDUCE-7132 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12944122/MAPREDUCE-7132-008.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  xml  |
| uname | Linux d2ca4d978fbd 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0bf8a11 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7509/artifact/out/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-B

[jira] [Commented] (MAPREDUCE-7132) JobSplitWriter prints unnecessary warnings if EC is used

2018-10-15 Thread Peter Bacsko (JIRA) 


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650647#comment-16650647
 ] 

Peter Bacsko commented on MAPREDUCE-7132:
-

[~xiaochen] yes, that's right. I'll change that tomorrow.

> JobSplitWriter prints unnecessary warnings if EC is used
> 
>
> Key: MAPREDUCE-7132
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7132
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: client, mrv2
>Affects Versions: 3.1.1
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MAPREDUCE-7132-001.patch, MAPREDUCE-7132-002.patch, 
> MAPREDUCE-7132-003.patch, MAPREDUCE-7132-004.patch, MAPREDUCE-7132-005.patch, 
> MAPREDUCE-7132-006.patch, MAPREDUCE-7132-007.patch
>
>
> Currently, {{JobSplitWriter}} compares the number of hosts for a certain 
> block against a static value that comes from 
> {{mapreduce.job.max.split.locations}}. The default value of this property is 
> 10.
> However, an EC shema like RS-10-4 requires at least 14 hosts. In this case, 
> 14 block locations will be returned and {{JobSplitWriter}} prints a warning, 
> which can confuse users.
> A possible solution could check whether EC is enabled for a block and 
> increase this value dynamically if needed. A simpler one is to simply 
> increase the default value to a sensible number like 15.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org