[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268589#comment-14268589 ] Hadoop QA commented on MAPREDUCE-6205: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12690653/MAPREDUCE-6205.003.patch against trunk revision e13a484. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1219 javac compiler warnings (more than the trunk's current 1218 warnings). {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:red}-1 findbugs{color}. The patch appears to introduce 13 new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The test build failed in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5099//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5099//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Javac warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5099//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5099//console This message is automatically generated. > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.003.patch, MAPREDUCE-6205.patch, > MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268556#comment-14268556 ] Gera Shegalov commented on MAPREDUCE-6205: -- I agree. I think the proposed deprecation chain handles this correctly but we should definitely capture these requirements in unit tests. And I hope you can review it [~jlowe]. > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.003.patch, MAPREDUCE-6205.patch, > MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268495#comment-14268495 ] Jason Lowe commented on MAPREDUCE-6205: --- I think we need to be _very_ careful with a patch like this, or it could easily create more problems than it solves. Something like this can readily break backwards compatibility. For example, if a user does "-Dmapreduce.map.java.opts=X -Dmapred.child.java.opts=Y", then maps should get X and reduces should get Y. We need to make sure that this code doesn't end up setting maps and reduces to Y because of the order we processed those settings and how they're aliased. mapred.child.java.opts is indeed deprecated, but it is not strictly an alias for the map/reduce-specific settings. > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.003.patch, MAPREDUCE-6205.patch, > MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14267431#comment-14267431 ] Masatake Iwasaki commented on MAPREDUCE-6205: - bq. Uncomment both mapreduce.map|reduce.java.opts and set their value as '-Xmx200m'. [~sam liu], [the comment in MAPREDUCE-5130|https://issues.apache.org/jira/browse/MAPREDUCE-5130?focusedCommentId=13623376&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13623376] says it breaks compatibility. > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14267371#comment-14267371 ] sam liu commented on MAPREDUCE-6205: Gera, The root cause of above failure is that after deserializing both the deprecated and the non-deprecated versions of a config are set. So, before deserializing, mapreduce.reduce.java.opts is equals to $ {mapred.child.java.opts}; after deserializing, mapreduce.reduce.java.opts is equals to '-Xmx200m'. To resolve this issue, I have a proposal that in mapred-default.xml: - Comment out the property 'mapred.child.java.opts' - Uncomment both mapreduce.map|reduce.java.opts and set their value as '-Xmx200m'. I think this makes sense, because 'mapred.child.java.opts' is deprecated, but 'mapreduce.map|reduce.java.opts' are recommended by hadoop. With such modifications, test TestWritableJobConf could pass. Also I ran some other regarding tests, and I found only test TestJobConf should be updated till now. What's your comments on the proposal? Thanks! > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265742#comment-14265742 ] sam liu commented on MAPREDUCE-6205: Gera, Thanks for your reminder! After implementing suggestion 3(update mapred-default.xml), the unit test failed on 'assertEquals(conf, deser)', but with different issue: - the expected is: 'mapreduce.reduce.java.opts=${mapred.child.java.opts}, mapreduce.map.java.opts=${mapred.child.java.opts}' - but is: 'mapreduce.reduce.java.opts=-Xmx200m, mapreduce.map.java.opts=-Xmx200m' However, I could get following output by 'conf.get(key)' or 'deser.get(key)' during running this test: 1. The properties of 'conf': mapred.child.java.opts=-Xmx200m mapred.map.child.java.opts=-Xmx200m mapred.reduce.child.java.opts=-Xmx200m mapreduce.map.java.opts=-Xmx200m mapreduce.reduce.java.opts=-Xmx200m 2. The properties of 'deser': mapred.child.java.opts=-Xmx200m mapred.map.child.java.opts=-Xmx200m mapred.reduce.child.java.opts=-Xmx200m mapreduce.map.java.opts=-Xmx200m mapreduce.reduce.java.opts=-Xmx200m Any comments? > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265505#comment-14265505 ] Gera Shegalov commented on MAPREDUCE-6205: -- [~sam liu], the test failure you see should go away if you implement suggestion 3 from my previous comment. > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262063#comment-14262063 ] sam liu commented on MAPREDUCE-6205: Gera, Yes, there is an existing deprecation deltas "mapred.map|reduce.child.java.opts" -> "mapreduce.map|reduce.java.opts". However, after I add the deprecation delta as " mapred.child.java.opts" -> "mapred.map|reduce.child.java.opts", the test TestWritableJobConf#testEmptyConfiguration() failed. In this test, it compares two configurations(conf and deser): -conf: JobConf conf = new JobConf(); -deser: Configuration deser = serDeser(conf); [A] Before I added the new deprecation delta, I got following output during running this test and the test passed. [A.1] The properties of 'conf': mapred.child.java.opts=-Xmx200m 2014-12-31 01:13:12,092 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1013)) - mapred.map.child.java.opts is deprecated. Instead, use mapreduce.map.java.opts mapred.map.child.java.opts=null 2014-12-31 01:13:12,107 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1013)) - mapred.reduce.child.java.opts is deprecated. Instead, use mapreduce.reduce.java.opts mapred.reduce.child.java.opts=null mapreduce.map.java.opts=null mapreduce.reduce.java.opts=null [A.2] The properties of 'deser': mapred.child.java.opts=-Xmx200m mapred.map.child.java.opts=null mapred.reduce.child.java.opts=null mapreduce.map.java.opts=null mapreduce.reduce.java.opts=null [B] After I added the new deprecation delta, I got following output during running this test and the test failed. [B.1] The properties of 'conf': 2014-12-31 01:06:22,313 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1013)) - mapred.child.java.opts is deprecated. Instead, use mapred.map.child.java.opts, mapred.reduce.child.java.opts mapred.child.java.opts=-Xmx200m mapred.map.child.java.opts=null mapred.reduce.child.java.opts=null mapreduce.map.java.opts=null mapreduce.reduce.java.opts=null [B.2] The properties of 'deser': mapred.child.java.opts=-Xmx200m mapred.map.child.java.opts=-Xmx200m mapred.reduce.child.java.opts=-Xmx200m mapreduce.map.java.opts=-Xmx200m mapreduce.reduce.java.opts=-Xmx200m Furthermore, if I did not add the deprecation delta as "mapred.child.java.opts" -> "mapred.map|reduce.child.java.opts", but add the deprecation delta as "mapred.child.java.opts" -> "mapreduce.map|reduce.java.opts", the test TestWritableJobConf#testEmptyConfiguration() could pass. What's your opinion? Thanks! > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260081#comment-14260081 ] Gera Shegalov commented on MAPREDUCE-6205: -- [~sam liu], thanks for the patch. # Please follow the guideline for including a version in patch names under https://wiki.apache.org/hadoop/HowToContribute#Naming_your_patch. # The deprecation delta should be " mapred.child.java.opts" -> { "mapred.map.child.java.opts", "mapred.reduce.child.java.opts" }. Otherwise you introduce non-determinism due to existing deprecation deltas "mapred.map|reduce.child.java.opts" -> "mapreduce.map|reduce.java.opts" # mapred-default.xml should be updated. Currently mapreduce.map|reduce.java.opts is commented out. Instead both should have thw substitution value: {{$\{mapred.child.java.opts\}}} > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259141#comment-14259141 ] sam liu commented on MAPREDUCE-6205: Yes, the MapReduceChildJVM#getChildJavaOpts makes the old property as fallback. However, this solution has two disadvantages: 1. During map/reduce task execution, the program might get incorrect value of new property MRJobConfig.MAP_JAVA_OPTS and MRJobConfig.REDUCE_JAVA_OPTS, if only the value of "mapred.child.java.opts" is defined before. 2. Inconsistent handling style. For most of other deprecated properties, Hadoop will automatically update the value of their new version with the value of old version. However, hadoop does not use same style to handle the deprecation of property "mapred.child.java.opts" > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259140#comment-14259140 ] sam liu commented on MAPREDUCE-6205: Yes, the MapReduceChildJVM#getChildJavaOpts makes the old property as fallback. However, this solution has two disadvantages: 1. During map/reduce task execution, the program might get incorrect value of new property MRJobConfig.MAP_JAVA_OPTS and MRJobConfig.REDUCE_JAVA_OPTS, if only the value of "mapred.child.java.opts" is defined before. 2. Inconsistent handling style. For most of other deprecated properties, Hadoop will automatically update the value of their new version with the value of old version. However, hadoop does not use same style to handle the deprecation of property "mapred.child.java.opts" > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258815#comment-14258815 ] Masatake Iwasaki commented on MAPREDUCE-6205: - The value of "mapred.child.java.opts" seems to be used as fallback in MapReduceChildJVM#getChildJavaOpts. Is this not enough? {code} private static String getChildJavaOpts(JobConf jobConf, boolean isMapTask) { String userClasspath = ""; String adminClasspath = ""; if (isMapTask) { userClasspath = jobConf.get( JobConf.MAPRED_MAP_TASK_JAVA_OPTS, jobConf.get( JobConf.MAPRED_TASK_JAVA_OPTS, JobConf.DEFAULT_MAPRED_TASK_JAVA_OPTS) ); {code} > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property "mapred.child.java.opts"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258158#comment-14258158 ] Hadoop QA commented on MAPREDUCE-6205: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12689020/MAPREDUCE-6205.patch against trunk revision 4f18018. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:red}-1 findbugs{color}. The patch appears to introduce 16 new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: org.apache.hadoop.ha.TestZKFailoverControllerStress Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5091//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5091//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5091//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5091//console This message is automatically generated. > Update the value of the new version properties of the deprecated property > "mapred.child.java.opts" > -- > > Key: MAPREDUCE-6205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: trunk >Reporter: sam liu >Assignee: sam liu >Priority: Minor > Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch > > > In current hadoop code, the old property "mapred.child.java.opts" is > deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and > MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the > deprecated property "mapred.child.java.opts", hadoop won't automatically > update its new versions properties > MRJobConfig.MAP_JAVA_OPTS("mapreduce.map.java.opts") and > MRJobConfig.REDUCE_JAVA_OPTS("mapreduce.reduce.java.opts"). As hadoop will > update the new version properties for many other deprecated properties, we > also should support such feature on the old property > "mapred.child.java.opts", otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)