[jira] [Updated] (MAPREDUCE-6284) Add Task Attempt State API to MapReduce Application Master REST API
[ https://issues.apache.org/jira/browse/MAPREDUCE-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated MAPREDUCE-6284: -- Summary: Add Task Attempt State API to MapReduce Application Master REST API (was: Add a 'task attempt state' to MapReduce Application Master REST API) > Add Task Attempt State API to MapReduce Application Master REST API > --- > > Key: MAPREDUCE-6284 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6284 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Ryu Kobayashi >Assignee: Ryu Kobayashi >Priority: Minor > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6284.1.patch, MAPREDUCE-6284.1.patch, > MAPREDUCE-6284.2.patch, MAPREDUCE-6284.3.patch, MAPREDUCE-6284.3.patch, > MAPREDUCE-6284.4.patch, MAPREDUCE-6284.5.patch > > > It want to 'task attempt state' on the 'App state' similarly REST API. > GET http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > PUT http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6284) Add a 'task attempt state' to MapReduce Application Master REST API
[ https://issues.apache.org/jira/browse/MAPREDUCE-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533976#comment-14533976 ] Tsuyoshi Ozawa commented on MAPREDUCE-6284: --- +1, checking this in. Note that this feature is alpha lik [YARN's Application State Change API|http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Application_State_API] as described in the documentation. > Add a 'task attempt state' to MapReduce Application Master REST API > --- > > Key: MAPREDUCE-6284 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6284 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Ryu Kobayashi >Assignee: Ryu Kobayashi >Priority: Minor > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6284.1.patch, MAPREDUCE-6284.1.patch, > MAPREDUCE-6284.2.patch, MAPREDUCE-6284.3.patch, MAPREDUCE-6284.3.patch, > MAPREDUCE-6284.4.patch, MAPREDUCE-6284.5.patch > > > It want to 'task attempt state' on the 'App state' similarly REST API. > GET http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > PUT http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6079) Renaming JobImpl#username to reporterUserName
[
https://issues.apache.org/jira/browse/MAPREDUCE-6079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated MAPREDUCE-6079:
-
Target Version/s: 2.8.0
Affects Version/s: 2.7.0
Labels: BB2015-05-RFC (was: BB2015-05-TBR)
Hadoop Flags: Reviewed
> Renaming JobImpl#username to reporterUserName
> -
>
> Key: MAPREDUCE-6079
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6079
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Affects Versions: 2.7.0
>Reporter: Tsuyoshi Ozawa
>Assignee: Tsuyoshi Ozawa
> Labels: BB2015-05-RFC
> Attachments: MAPREDUCE-6079.1.patch, MAPREDUCE-6079.2.patch
>
>
> On MAPREDUCE-6033, we found the bug because of confusing field names
> {{userName}} and {{username}}. We should change the names to distinguish them
> easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6079) Renaming JobImpl#username to reporterUserName
[
https://issues.apache.org/jira/browse/MAPREDUCE-6079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533970#comment-14533970
]
Akira AJISAKA commented on MAPREDUCE-6079:
--
+1 pending Jenkins, thanks Tsuyoshi.
> Renaming JobImpl#username to reporterUserName
> -
>
> Key: MAPREDUCE-6079
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6079
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Tsuyoshi Ozawa
>Assignee: Tsuyoshi Ozawa
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6079.1.patch, MAPREDUCE-6079.2.patch
>
>
> On MAPREDUCE-6033, we found the bug because of confusing field names
> {{userName}} and {{username}}. We should change the names to distinguish them
> easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-3383) Duplicate job.getOutputValueGroupingComparator() in ReduceTask
[ https://issues.apache.org/jira/browse/MAPREDUCE-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Shahid Khan updated MAPREDUCE-3383: Labels: BB2015-05-RFC (was: BB2015-05-TBR) > Duplicate job.getOutputValueGroupingComparator() in ReduceTask > -- > > Key: MAPREDUCE-3383 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3383 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.23.1 >Reporter: Binglin Chang >Assignee: Binglin Chang > Labels: BB2015-05-RFC > Attachments: MAPREDUCE-3383.patch > > > This is probably just a small error by mistake. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-3383) Duplicate job.getOutputValueGroupingComparator() in ReduceTask
[ https://issues.apache.org/jira/browse/MAPREDUCE-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533963#comment-14533963 ] Mohammad Shahid Khan commented on MAPREDUCE-3383: - +1 Changes looks good no need of additional test case as only the code quality is improbed > Duplicate job.getOutputValueGroupingComparator() in ReduceTask > -- > > Key: MAPREDUCE-3383 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3383 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.23.1 >Reporter: Binglin Chang >Assignee: Binglin Chang > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-3383.patch > > > This is probably just a small error by mistake. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6269) improve JobConf to add option to not share Credentials between jobs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533958#comment-14533958 ] zhihai xu commented on MAPREDUCE-6269: -- update the patch based on the latest code. > improve JobConf to add option to not share Credentials between jobs. > > > Key: MAPREDUCE-6269 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6269 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client >Reporter: zhihai xu >Assignee: zhihai xu > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6269.000.patch > > > Improve JobConf to add constructor to avoid sharing Credentials between jobs. > By default the Credentials will be shared to keep the backward compatibility. > We can add a new constructor with a new parameter to decide whether to share > Credentials. Some issues reported in cascading is due to corrupted credentials > at > https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e > If we add this support in JobConf, it will benefit all job clients. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6269) improve JobConf to add option to not share Credentials between jobs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated MAPREDUCE-6269: - Attachment: MAPREDUCE-6269.000.patch > improve JobConf to add option to not share Credentials between jobs. > > > Key: MAPREDUCE-6269 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6269 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client >Reporter: zhihai xu >Assignee: zhihai xu > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6269.000.patch > > > Improve JobConf to add constructor to avoid sharing Credentials between jobs. > By default the Credentials will be shared to keep the backward compatibility. > We can add a new constructor with a new parameter to decide whether to share > Credentials. Some issues reported in cascading is due to corrupted credentials > at > https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e > If we add this support in JobConf, it will benefit all job clients. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6269) improve JobConf to add option to not share Credentials between jobs.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated MAPREDUCE-6269: - Attachment: (was: MAPREDUCE-6269.000.patch) > improve JobConf to add option to not share Credentials between jobs. > > > Key: MAPREDUCE-6269 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6269 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client >Reporter: zhihai xu >Assignee: zhihai xu > Labels: BB2015-05-TBR > > Improve JobConf to add constructor to avoid sharing Credentials between jobs. > By default the Credentials will be shared to keep the backward compatibility. > We can add a new constructor with a new parameter to decide whether to share > Credentials. Some issues reported in cascading is due to corrupted credentials > at > https://github.com/Cascading/cascading/commit/45b33bb864172486ac43782a4d13329312d01c0e > If we add this support in JobConf, it will benefit all job clients. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6020) Too many threads blocking on the global JobTracker lock from getJobCounters, optimize getJobCounters to release global JobTracker lock before access the per job cou
[ https://issues.apache.org/jira/browse/MAPREDUCE-6020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533948#comment-14533948 ] Ameya Sakhalkar commented on MAPREDUCE-6020: Looks good to me. > Too many threads blocking on the global JobTracker lock from getJobCounters, > optimize getJobCounters to release global JobTracker lock before access the > per job counter in JobInProgress > - > > Key: MAPREDUCE-6020 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6020 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 0.23.10 >Reporter: zhihai xu >Assignee: zhihai xu > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6020.branch1.patch > > > Too many threads blocking on the global JobTracker lock from getJobCounters, > optimize getJobCounters to release global JobTracker lock before access the > per job counter in JobInProgress. It may be a lot of JobClients to call > getJobCounters in JobTracker at the same time, Current code will lock the > JobTracker to block all the threads to get counter from JobInProgress. It is > better to unlock the JobTracker when get counter from > JobInProgress(job.getCounters(counters)). So all the theads can run parallel > when access its own job counter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6174) Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533937#comment-14533937
]
Gera Shegalov commented on MAPREDUCE-6174:
--
Hi [~eepayne], thank you for working on the patch. We can check what checkstyle
problems a patch introduced as follows.
Run
{code}
$ mvn clean checkstyle:checkstyle-aggregate
{code}
before and after applying the patch.
Save the resulting {{target/test/checkstyle-errors.xml}} after each run. Then
we can diff both files. The diff will be concise if we get rid of line numbers.
{code}
]$ diff <(cat ~/checkstyle-errors-trunk.xml | sed 's/line=\"[0-9]*\"//g') <(cat
./target/test/checkstyle-errors.xml | sed 's/line=\"[0-9]*\"//g')
46768a46769,46772
> name="/Users/gshegalov/workspace/hadoop-common/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/IFileWrappedMapOutput.java">
> source="com.puppycrawl.tools.checkstyle.checks.coding.HiddenFieldCheck"/>
> source="com.puppycrawl.tools.checkstyle.checks.coding.HiddenFieldCheck"/>
>
46770d46773
<
46911d46913
<
{code}
> Combine common stream code into parent class for InMemoryMapOutput and
> OnDiskMapOutput.
> ---
>
> Key: MAPREDUCE-6174
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6174
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
>Affects Versions: 3.0.0, 2.6.0
>Reporter: Eric Payne
>Assignee: Eric Payne
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6174.002.patch, MAPREDUCE-6174.003.patch,
> MAPREDUCE-6174.v1.txt
>
>
> Per MAPREDUCE-6166, both InMemoryMapOutput and OnDiskMapOutput will be doing
> similar things with regards to IFile streams.
> In order to make it explicit that InMemoryMapOutput and OnDiskMapOutput are
> different from 3rd-party implementations, this JIRA will make them subclass a
> common class (see
> https://issues.apache.org/jira/browse/MAPREDUCE-6166?focusedCommentId=14223368&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14223368)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6079) Renaming JobImpl#username to reporterUserName
[
https://issues.apache.org/jira/browse/MAPREDUCE-6079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated MAPREDUCE-6079:
--
Attachment: MAPREDUCE-6079.2.patch
[~ajisakaa] Thank you for pinging me. Fixed the point.
> Renaming JobImpl#username to reporterUserName
> -
>
> Key: MAPREDUCE-6079
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6079
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Tsuyoshi Ozawa
>Assignee: Tsuyoshi Ozawa
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6079.1.patch, MAPREDUCE-6079.2.patch
>
>
> On MAPREDUCE-6033, we found the bug because of confusing field names
> {{userName}} and {{username}}. We should change the names to distinguish them
> easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6273) HistoryFileManager should check whether summaryFile exists to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state
[
https://issues.apache.org/jira/browse/MAPREDUCE-6273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533889#comment-14533889
]
zhihai xu commented on MAPREDUCE-6273:
--
update the patch based o latest code base.
> HistoryFileManager should check whether summaryFile exists to avoid
> FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state
>
>
> Key: MAPREDUCE-6273
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6273
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6273.000.patch
>
>
> HistoryFileManager should check whether summaryFile exists to avoid
> FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state,
> I saw the following error message:
> {code}
> 2015-02-17 19:13:45,198 ERROR
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Error while trying to
> move a job to done
> java.io.FileNotFoundException: File does not exist:
> /user/history/done_intermediate/agd_laci-sluice/job_1423740288390_1884.summary
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:65)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:55)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1878)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1819)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1771)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:527)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:85)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:356)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at sun.reflect.GeneratedConstructorAccessor29.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1181)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1169)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1159)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:270)
> at
> org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:237)
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:230)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1457)
> at org.apache.hadoop.fs.Hdfs.open(Hdfs.java:318)
> at org.apache.hadoop.fs.Hdfs.open(Hdfs.java:59)
> at
> org.apache.hadoop.fs.AbstractFileSystem.open(AbstractFileSystem.java:621)
> at org.apache.hadoop.fs.FileContext$6.next(FileContext.java:789)
> at org.apache.hadoop.fs.FileContext$6.next(FileContext.java:785)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.open(FileContext.java:785)
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.getJobSummary(HistoryFileManager.java:953)
>
[jira] [Updated] (MAPREDUCE-6273) HistoryFileManager should check whether summaryFile exists to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state
[
https://issues.apache.org/jira/browse/MAPREDUCE-6273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated MAPREDUCE-6273:
-
Attachment: MAPREDUCE-6273.000.patch
> HistoryFileManager should check whether summaryFile exists to avoid
> FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state
>
>
> Key: MAPREDUCE-6273
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6273
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6273.000.patch
>
>
> HistoryFileManager should check whether summaryFile exists to avoid
> FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state,
> I saw the following error message:
> {code}
> 2015-02-17 19:13:45,198 ERROR
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Error while trying to
> move a job to done
> java.io.FileNotFoundException: File does not exist:
> /user/history/done_intermediate/agd_laci-sluice/job_1423740288390_1884.summary
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:65)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:55)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1878)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1819)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1771)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:527)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:85)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:356)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at sun.reflect.GeneratedConstructorAccessor29.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1181)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1169)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1159)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:270)
> at
> org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:237)
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:230)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1457)
> at org.apache.hadoop.fs.Hdfs.open(Hdfs.java:318)
> at org.apache.hadoop.fs.Hdfs.open(Hdfs.java:59)
> at
> org.apache.hadoop.fs.AbstractFileSystem.open(AbstractFileSystem.java:621)
> at org.apache.hadoop.fs.FileContext$6.next(FileContext.java:789)
> at org.apache.hadoop.fs.FileContext$6.next(FileContext.java:785)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.open(FileContext.java:785)
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.getJobSummary(HistoryFileManager.java:953)
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.ac
[jira] [Updated] (MAPREDUCE-6273) HistoryFileManager should check whether summaryFile exists to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state
[
https://issues.apache.org/jira/browse/MAPREDUCE-6273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated MAPREDUCE-6273:
-
Attachment: (was: MAPREDUCE-6273.000.patch)
> HistoryFileManager should check whether summaryFile exists to avoid
> FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state
>
>
> Key: MAPREDUCE-6273
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6273
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Labels: BB2015-05-TBR
>
> HistoryFileManager should check whether summaryFile exists to avoid
> FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state,
> I saw the following error message:
> {code}
> 2015-02-17 19:13:45,198 ERROR
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Error while trying to
> move a job to done
> java.io.FileNotFoundException: File does not exist:
> /user/history/done_intermediate/agd_laci-sluice/job_1423740288390_1884.summary
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:65)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:55)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1878)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1819)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1771)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:527)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:85)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:356)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at sun.reflect.GeneratedConstructorAccessor29.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1181)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1169)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1159)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:270)
> at
> org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:237)
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:230)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1457)
> at org.apache.hadoop.fs.Hdfs.open(Hdfs.java:318)
> at org.apache.hadoop.fs.Hdfs.open(Hdfs.java:59)
> at
> org.apache.hadoop.fs.AbstractFileSystem.open(AbstractFileSystem.java:621)
> at org.apache.hadoop.fs.FileContext$6.next(FileContext.java:789)
> at org.apache.hadoop.fs.FileContext$6.next(FileContext.java:785)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.open(FileContext.java:785)
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.getJobSummary(HistoryFileManager.java:953)
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.access$400(HistoryFileManager.java:82)
>
[jira] [Updated] (MAPREDUCE-6342) Make POM project names consistent
[ https://issues.apache.org/jira/browse/MAPREDUCE-6342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith updated MAPREDUCE-6342: -- Labels: BB2015-05-RFC (was: BB2015-05-TBR) > Make POM project names consistent > - > > Key: MAPREDUCE-6342 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6342 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Reporter: Rohith >Assignee: Rohith >Priority: Minor > Labels: BB2015-05-RFC > Attachments: MAPREDUCE-6342.patch > > > This is track MR changes for POM changes by name -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-3384) Add warning message for org.apache.hadoop.mapreduce.lib.reduce.LongSumReducer
[ https://issues.apache.org/jira/browse/MAPREDUCE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533820#comment-14533820 ] Akira AJISAKA commented on MAPREDUCE-3384: -- I'm thinking adding warning message only in org.apache.hadoop.mapreduce.lib.reduce.LongSumReducer is inconsistent. We need to add warning messages in the following classes as well. * org.apache.hadoop.mapred.lib.LongSumReducer * org.apache.hadoop.mapreduce.lib.reduce.IntSumReducer > Add warning message for org.apache.hadoop.mapreduce.lib.reduce.LongSumReducer > - > > Key: MAPREDUCE-3384 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3384 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: JiangKai >Priority: Minor > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-3384.patch > > > When we call the function reduce() of LongSumReducer,the result may overflow. > We should send a warning message to users if overflow occurs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-3182) loadgen ignores -m command line when writing random data
[ https://issues.apache.org/jira/browse/MAPREDUCE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-3182: - Target Version/s: 2.8.0 (was: 2.6.0) Status: Open (was: Patch Available) > loadgen ignores -m command line when writing random data > > > Key: MAPREDUCE-3182 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3182 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation, mrv2, test >Affects Versions: 2.3.0, 0.23.0 >Reporter: Jonathan Eagles >Assignee: Chen He > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-3182.patch > > > If no input directories are specified, loadgen goes into a special mode where > random data is generated and written. In that mode, setting the number of > mappers (-m command line option) is overridden by a calculation. Instead, it > should take into consideration the user specified number of mappers and fall > back to the calculation. In addition, update the documentation as well to > match the new behavior in the code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-3182) loadgen ignores -m command line when writing random data
[ https://issues.apache.org/jira/browse/MAPREDUCE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-3182: - Component/s: documentation Priority: Minor (was: Major) > loadgen ignores -m command line when writing random data > > > Key: MAPREDUCE-3182 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3182 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation, mrv2, test >Affects Versions: 0.23.0, 2.3.0 >Reporter: Jonathan Eagles >Assignee: Chen He >Priority: Minor > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-3182.patch > > > If no input directories are specified, loadgen goes into a special mode where > random data is generated and written. In that mode, setting the number of > mappers (-m command line option) is overridden by a calculation. Instead, it > should take into consideration the user specified number of mappers and fall > back to the calculation. In addition, update the documentation as well to > match the new behavior in the code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-3182) loadgen ignores -m command line when writing random data
[
https://issues.apache.org/jira/browse/MAPREDUCE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533783#comment-14533783
]
Akira AJISAKA commented on MAPREDUCE-3182:
--
Mostly looks good to me. Two comments:
1. For mapred/GenericMRLoadGenerator, would you document that the number of map
tasks specified by -m option is overridden when -indir is not specified?
2.
{code}
+"RandomWriter will be used to create input directory and data if
\"-indir\"" +
{code}
{code}
+"RandomWriter will be used to create input directory and data if [-indir]"
+
{code}
Is there any reason to use different form for -indir?
> loadgen ignores -m command line when writing random data
>
>
> Key: MAPREDUCE-3182
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3182
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, test
>Affects Versions: 0.23.0, 2.3.0
>Reporter: Jonathan Eagles
>Assignee: Chen He
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-3182.patch
>
>
> If no input directories are specified, loadgen goes into a special mode where
> random data is generated and written. In that mode, setting the number of
> mappers (-m command line option) is overridden by a calculation. Instead, it
> should take into consideration the user specified number of mappers and fall
> back to the calculation. In addition, update the documentation as well to
> match the new behavior in the code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation
[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533631#comment-14533631 ] zhihai xu commented on MAPREDUCE-6302: -- Also linked to YARN-3446 for discussion, YARN-1680 is for CapacityScheduler and YARN-3446 is for FairScheduler. It will be good to merge common functionality into AbstractYarnScheduler. > deadlock in a job between map and reduce cores allocation > -- > > Key: MAPREDUCE-6302 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6302 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: mai shurong >Assignee: Karthik Kambatla >Priority: Critical > Attachments: AM_log_head10.txt.gz, AM_log_tail10.txt.gz, > log.txt, mr-6302-prelim.patch, queue_with_max163cores.png, > queue_with_max263cores.png, queue_with_max333cores.png > > > I submit a big job, which has 500 maps and 350 reduce, to a > queue(fairscheduler) with 300 max cores. When the big mapreduce job is > running 100% maps, the 300 reduces have occupied 300 max cores in the queue. > And then, a map fails and retry, waiting for a core, while the 300 reduces > are waiting for failed map to finish. So a deadlock occur. As a result, the > job is blocked, and the later job in the queue cannot run because no > available cores in the queue. > I think there is the similar issue for memory of a queue . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation
[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533619#comment-14533619 ] Wangda Tan commented on MAPREDUCE-6302: --- Linked to YARN-1680, one is for more accurate calculation, one is to prevent inaccurate calculation. > deadlock in a job between map and reduce cores allocation > -- > > Key: MAPREDUCE-6302 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6302 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: mai shurong >Assignee: Karthik Kambatla >Priority: Critical > Attachments: AM_log_head10.txt.gz, AM_log_tail10.txt.gz, > log.txt, mr-6302-prelim.patch, queue_with_max163cores.png, > queue_with_max263cores.png, queue_with_max333cores.png > > > I submit a big job, which has 500 maps and 350 reduce, to a > queue(fairscheduler) with 300 max cores. When the big mapreduce job is > running 100% maps, the 300 reduces have occupied 300 max cores in the queue. > And then, a map fails and retry, waiting for a core, while the 300 reduces > are waiting for failed map to finish. So a deadlock occur. As a result, the > job is blocked, and the later job in the queue cannot run because no > available cores in the queue. > I think there is the similar issue for memory of a queue . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation
[
https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533618#comment-14533618
]
Wangda Tan commented on MAPREDUCE-6302:
---
[~kasha],
Thanks for working on this.
Just take a look at your patch, overall approach looks good, some comments
about configuration:
{{MR_JOB_REDUCER_FORCE_PREEMPT_DELAY_SEC}}
It is actually not REDUCER_FORCE_PREEMPT_DELAY, it is timeout of mapper
allocation to start reducer preemption, I suggest to rename it to be:
mapreduce.job.mapper.timeout-to-start-reducer-preemption-ms. I think it's
better to use ms instead of sec to better control it.
In addition, do you think should we add a value to let user choose to disable
this? For example, -1.
And could you add some tests?
> deadlock in a job between map and reduce cores allocation
> --
>
> Key: MAPREDUCE-6302
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6302
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.6.0
>Reporter: mai shurong
>Assignee: Karthik Kambatla
>Priority: Critical
> Attachments: AM_log_head10.txt.gz, AM_log_tail10.txt.gz,
> log.txt, mr-6302-prelim.patch, queue_with_max163cores.png,
> queue_with_max263cores.png, queue_with_max333cores.png
>
>
> I submit a big job, which has 500 maps and 350 reduce, to a
> queue(fairscheduler) with 300 max cores. When the big mapreduce job is
> running 100% maps, the 300 reduces have occupied 300 max cores in the queue.
> And then, a map fails and retry, waiting for a core, while the 300 reduces
> are waiting for failed map to finish. So a deadlock occur. As a result, the
> job is blocked, and the later job in the queue cannot run because no
> available cores in the queue.
> I think there is the similar issue for memory of a queue .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6302) deadlock in a job between map and reduce cores allocation
[ https://issues.apache.org/jira/browse/MAPREDUCE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6302: -- Status: Patch Available (was: Open) > deadlock in a job between map and reduce cores allocation > -- > > Key: MAPREDUCE-6302 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6302 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.0 >Reporter: mai shurong >Assignee: Karthik Kambatla >Priority: Critical > Attachments: AM_log_head10.txt.gz, AM_log_tail10.txt.gz, > log.txt, mr-6302-prelim.patch, queue_with_max163cores.png, > queue_with_max263cores.png, queue_with_max333cores.png > > > I submit a big job, which has 500 maps and 350 reduce, to a > queue(fairscheduler) with 300 max cores. When the big mapreduce job is > running 100% maps, the 300 reduces have occupied 300 max cores in the queue. > And then, a map fails and retry, waiting for a core, while the 300 reduces > are waiting for failed map to finish. So a deadlock occur. As a result, the > job is blocked, and the later job in the queue cannot run because no > available cores in the queue. > I think there is the similar issue for memory of a queue . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6251) JobClient needs additional retries at a higher level to address not-immediately-consistent dfs corner cases
[ https://issues.apache.org/jira/browse/MAPREDUCE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated MAPREDUCE-6251: --- Attachment: MAPREDUCE-6251.8.patch ...add line failed to stage for last patch. > JobClient needs additional retries at a higher level to address > not-immediately-consistent dfs corner cases > --- > > Key: MAPREDUCE-6251 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6251 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.6.0 >Reporter: Craig Welch >Assignee: Craig Welch > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6251.0.patch, MAPREDUCE-6251.1.patch, > MAPREDUCE-6251.2.patch, MAPREDUCE-6251.3.patch, MAPREDUCE-6251.4.patch, > MAPREDUCE-6251.6.patch, MAPREDUCE-6251.7.patch, MAPREDUCE-6251.8.patch > > > The JobClient is used to get job status information for running and completed > jobs. Final state and history for a job is communicated from the application > master to the job history server via a distributed file system - where the > history is uploaded by the application master to the dfs and then > scanned/loaded by the jobhistory server. While HDFS has strong consistency > guarantees not all Hadoop DFS's do. When used in conjunction with a > distributed file system which does not have this guarantee there will be > cases where the history server may not see an uploaded file, resulting in the > dreaded "no such job" and a null value for the RunningJob in the client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6251) JobClient needs additional retries at a higher level to address not-immediately-consistent dfs corner cases
[ https://issues.apache.org/jira/browse/MAPREDUCE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533513#comment-14533513 ] Craig Welch commented on MAPREDUCE-6251: Uploaded patch to fix mistake in mapred-default.xml which caused TestNonExistentJob to fail. TestLineRecordReader passes on my box. Whitespace error is bogus (generally, and in this case specifically as it is a line moved not changed by the patch), but tried to fix it anyway. > JobClient needs additional retries at a higher level to address > not-immediately-consistent dfs corner cases > --- > > Key: MAPREDUCE-6251 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6251 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.6.0 >Reporter: Craig Welch >Assignee: Craig Welch > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6251.0.patch, MAPREDUCE-6251.1.patch, > MAPREDUCE-6251.2.patch, MAPREDUCE-6251.3.patch, MAPREDUCE-6251.4.patch, > MAPREDUCE-6251.6.patch, MAPREDUCE-6251.7.patch > > > The JobClient is used to get job status information for running and completed > jobs. Final state and history for a job is communicated from the application > master to the job history server via a distributed file system - where the > history is uploaded by the application master to the dfs and then > scanned/loaded by the jobhistory server. While HDFS has strong consistency > guarantees not all Hadoop DFS's do. When used in conjunction with a > distributed file system which does not have this guarantee there will be > cases where the history server may not see an uploaded file, resulting in the > dreaded "no such job" and a null value for the RunningJob in the client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6251) JobClient needs additional retries at a higher level to address not-immediately-consistent dfs corner cases
[ https://issues.apache.org/jira/browse/MAPREDUCE-6251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated MAPREDUCE-6251: --- Attachment: MAPREDUCE-6251.7.patch > JobClient needs additional retries at a higher level to address > not-immediately-consistent dfs corner cases > --- > > Key: MAPREDUCE-6251 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6251 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver, mrv2 >Affects Versions: 2.6.0 >Reporter: Craig Welch >Assignee: Craig Welch > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6251.0.patch, MAPREDUCE-6251.1.patch, > MAPREDUCE-6251.2.patch, MAPREDUCE-6251.3.patch, MAPREDUCE-6251.4.patch, > MAPREDUCE-6251.6.patch, MAPREDUCE-6251.7.patch > > > The JobClient is used to get job status information for running and completed > jobs. Final state and history for a job is communicated from the application > master to the job history server via a distributed file system - where the > history is uploaded by the application master to the dfs and then > scanned/loaded by the jobhistory server. While HDFS has strong consistency > guarantees not all Hadoop DFS's do. When used in conjunction with a > distributed file system which does not have this guarantee there will be > cases where the history server may not see an uploaded file, resulting in the > dreaded "no such job" and a null value for the RunningJob in the client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6279) AM should explicity exit JVM after all services have stopped
[ https://issues.apache.org/jira/browse/MAPREDUCE-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533509#comment-14533509 ] Hudson commented on MAPREDUCE-6279: --- SUCCESS: Integrated in Hadoop-trunk-Commit #7763 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7763/]) MAPREDUCE-6279. AM should explicity exit JVM after all services have stopped. Contributed by Eric Payne (jlowe: rev f30065c8b6099372f57015b505434120fe83c2b0) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestMRAppMaster.java > AM should explicity exit JVM after all services have stopped > > > Key: MAPREDUCE-6279 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6279 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 2.5.0 >Reporter: Jason Lowe >Assignee: Eric Payne > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6279.v1.txt, MAPREDUCE-6279.v2.txt, > MAPREDUCE-6279.v3.patch, MAPREDUCE-6279.v4.patch > > > Occasionally the MapReduce AM can "get stuck" trying to shut down. > MAPREDUCE-6049 and MAPREDUCE-5888 were specific instances that have been > fixed, but this can also occur with uber jobs if the task code inadvertently > leaves non-daemon threads lingering. > We should explicitly shutdown the JVM after the MapReduce AM has unregistered > and all services have been stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6279) AM should explicity exit JVM after all services have stopped
[ https://issues.apache.org/jira/browse/MAPREDUCE-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-6279: -- Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks, Eric! I committed this to trunk and branch-2. > AM should explicity exit JVM after all services have stopped > > > Key: MAPREDUCE-6279 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6279 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 2.5.0 >Reporter: Jason Lowe >Assignee: Eric Payne > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6279.v1.txt, MAPREDUCE-6279.v2.txt, > MAPREDUCE-6279.v3.patch, MAPREDUCE-6279.v4.patch > > > Occasionally the MapReduce AM can "get stuck" trying to shut down. > MAPREDUCE-6049 and MAPREDUCE-5888 were specific instances that have been > fixed, but this can also occur with uber jobs if the task code inadvertently > leaves non-daemon threads lingering. > We should explicitly shutdown the JVM after the MapReduce AM has unregistered > and all services have been stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6279) AM should explicity exit JVM after all services have stopped
[ https://issues.apache.org/jira/browse/MAPREDUCE-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-6279: -- Labels: (was: BB2015-05-TBR) > AM should explicity exit JVM after all services have stopped > > > Key: MAPREDUCE-6279 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6279 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 2.5.0 >Reporter: Jason Lowe >Assignee: Eric Payne > Attachments: MAPREDUCE-6279.v1.txt, MAPREDUCE-6279.v2.txt, > MAPREDUCE-6279.v3.patch, MAPREDUCE-6279.v4.patch > > > Occasionally the MapReduce AM can "get stuck" trying to shut down. > MAPREDUCE-6049 and MAPREDUCE-5888 were specific instances that have been > fixed, but this can also occur with uber jobs if the task code inadvertently > leaves non-daemon threads lingering. > We should explicitly shutdown the JVM after the MapReduce AM has unregistered > and all services have been stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6279) AM should explicity exit JVM after all services have stopped
[ https://issues.apache.org/jira/browse/MAPREDUCE-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533458#comment-14533458 ] Jason Lowe commented on MAPREDUCE-6279: --- +1 lgtm. Committing this. > AM should explicity exit JVM after all services have stopped > > > Key: MAPREDUCE-6279 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6279 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 2.5.0 >Reporter: Jason Lowe >Assignee: Eric Payne > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6279.v1.txt, MAPREDUCE-6279.v2.txt, > MAPREDUCE-6279.v3.patch, MAPREDUCE-6279.v4.patch > > > Occasionally the MapReduce AM can "get stuck" trying to shut down. > MAPREDUCE-6049 and MAPREDUCE-5888 were specific instances that have been > fixed, but this can also occur with uber jobs if the task code inadvertently > leaves non-daemon threads lingering. > We should explicitly shutdown the JVM after the MapReduce AM has unregistered > and all services have been stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533396#comment-14533396 ] Hudson commented on MAPREDUCE-6356: --- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #188 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/188/]) MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. (aajisaka: rev d335071398fdb14153c854b740814b4fd385b658) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/resources/log4j.properties * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/resources/log4j.properties > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6353) Divide by zero error in MR AM when calculating available containers
[
https://issues.apache.org/jira/browse/MAPREDUCE-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533368#comment-14533368
]
Karthik Kambatla commented on MAPREDUCE-6353:
-
Patch looks reasonable to me. A couple of nits:
# The import removal looks unrelated. Let us leave it in.
# {{getLimitIfAny}} doesn't really say much about what the method does. Rename
it to a more descriptive name?
> Divide by zero error in MR AM when calculating available containers
> ---
>
> Key: MAPREDUCE-6353
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6353
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
>Reporter: Anubhav Dhoot
>Assignee: Anubhav Dhoot
> Attachments: MAPREDUCE-6353.001.patch
>
>
> When running a sleep job with zero CPU vcores i see the following exception
> 2015-04-30 06:41:06,954 ERROR [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: ERROR IN
> CONTACTING RM.
> java.lang.ArithmeticException: / by zero
> at
> org.apache.hadoop.mapreduce.v2.app.rm.ResourceCalculatorUtils.computeAvailableContainers(ResourceCalculatorUtils.java:38)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator$ScheduledRequests.assign(RMContainerAllocator.java:947)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator$ScheduledRequests.access$200(RMContainerAllocator.java:840)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:247)
> at
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$1.run(RMCommunicator.java:282)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6358) Document missing properties in mapred-default.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-6358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533105#comment-14533105 ] Ray Chiang commented on MAPREDUCE-6358: --- Link to YARN equivalent. > Document missing properties in mapred-default.xml > - > > Key: MAPREDUCE-6358 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6358 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 2.7.0 >Reporter: Ray Chiang >Assignee: Ray Chiang > > The following properties are currently not defined in mapred-default.xml. > These properties should either be > A) documented in mapred-default.xml OR > B) listed as an exception (with comments, e.g. for internal use) in the > TestMapreduceConfigFields unit test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6354) shuffle handler should log connection info
[ https://issues.apache.org/jira/browse/MAPREDUCE-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532904#comment-14532904 ] Chang Li commented on MAPREDUCE-6354: - That whitespace issue is not caused by my change. [~jlowe] could you help review my patch > shuffle handler should log connection info > -- > > Key: MAPREDUCE-6354 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6354 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6354.2.patch, MAPREDUCE-6354.3.patch, > MAPREDUCE-6354.4.patch, MAPREDUCE-6354.5.patch, MAPREDUCE-6354.6.patch, > MAPREDUCE-6354.patch > > > currently, shuffle handler only log connection info in debug mode, we want to > log that info in a more concise way -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532885#comment-14532885 ] Hudson commented on MAPREDUCE-6356: --- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2136 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2136/]) MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. (aajisaka: rev d335071398fdb14153c854b740814b4fd385b658) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/resources/log4j.properties > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6354) shuffle handler should log connection info
[
https://issues.apache.org/jira/browse/MAPREDUCE-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532876#comment-14532876
]
Hadoop QA commented on MAPREDUCE-6354:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 34s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 33s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 27s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 17s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 24m 27s | Tests passed in
hadoop-common. |
| {color:green}+1{color} | mapreduce tests | 0m 19s | Tests passed in
hadoop-mapreduce-client-shuffle. |
| | | 62m 38s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731183/MAPREDUCE-6354.6.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 8e991f4 |
| whitespace |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5660/artifact/patchprocess/whitespace.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5660/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-mapreduce-client-shuffle test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5660/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5660/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5660/console |
This message was automatically generated.
> shuffle handler should log connection info
> --
>
> Key: MAPREDUCE-6354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6354
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Chang Li
>Assignee: Chang Li
> Attachments: MAPREDUCE-6354.2.patch, MAPREDUCE-6354.3.patch,
> MAPREDUCE-6354.4.patch, MAPREDUCE-6354.5.patch, MAPREDUCE-6354.6.patch,
> MAPREDUCE-6354.patch
>
>
> currently, shuffle handler only log connection info in debug mode, we want to
> log that info in a more concise way
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532811#comment-14532811 ] Hudson commented on MAPREDUCE-6356: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #187 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/187/]) MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. (aajisaka: rev d335071398fdb14153c854b740814b4fd385b658) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/resources/log4j.properties * hadoop-mapreduce-project/CHANGES.txt > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532769#comment-14532769 ] Hudson commented on MAPREDUCE-6356: --- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #177 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/177/]) MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. (aajisaka: rev d335071398fdb14153c854b740814b4fd385b658) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/log4j.properties * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/resources/log4j.properties > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532737#comment-14532737 ] Hudson commented on MAPREDUCE-6356: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #2118 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2118/]) MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. (aajisaka: rev d335071398fdb14153c854b740814b4fd385b658) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/resources/log4j.properties * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/resources/log4j.properties > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6354) shuffle handler should log connection info
[ https://issues.apache.org/jira/browse/MAPREDUCE-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated MAPREDUCE-6354: Attachment: MAPREDUCE-6354.6.patch > shuffle handler should log connection info > -- > > Key: MAPREDUCE-6354 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6354 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li > Attachments: MAPREDUCE-6354.2.patch, MAPREDUCE-6354.3.patch, > MAPREDUCE-6354.4.patch, MAPREDUCE-6354.5.patch, MAPREDUCE-6354.6.patch, > MAPREDUCE-6354.patch > > > currently, shuffle handler only log connection info in debug mode, we want to > log that info in a more concise way -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532446#comment-14532446 ] Hudson commented on MAPREDUCE-6356: --- FAILURE: Integrated in Hadoop-trunk-Commit #7757 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7757/]) MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. (aajisaka: rev d335071398fdb14153c854b740814b4fd385b658) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/resources/log4j.properties * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/resources/log4j.properties * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/resources/log4j.properties > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6356: - Labels: (was: BB2015-05-RFC) Hadoop Flags: Reviewed > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6356: - Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) +1, committed this to trunk and branch-2. Thanks [~brahmareddy] for the contribution. > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6356: - Affects Version/s: 2.7.0 Hadoop Flags: (was: Reviewed) > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 2.7.0 >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Fix For: 2.8.0 > > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532394#comment-14532394
]
Hadoop QA commented on MAPREDUCE-6356:
--
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 9s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 7 new or modified test files. |
| {color:green}+1{color} | javac | 7m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 19s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | mapreduce tests | 9m 7s | Tests passed in
hadoop-mapreduce-client-app. |
| {color:green}+1{color} | mapreduce tests | 0m 44s | Tests passed in
hadoop-mapreduce-client-common. |
| {color:green}+1{color} | mapreduce tests | 1m 35s | Tests passed in
hadoop-mapreduce-client-core. |
| {color:green}+1{color} | mapreduce tests | 5m 39s | Tests passed in
hadoop-mapreduce-client-hs. |
| {color:green}+1{color} | mapreduce tests | 105m 41s | Tests passed in
hadoop-mapreduce-client-jobclient. |
| {color:green}+1{color} | mapreduce tests | 10m 17s | Tests passed in
hadoop-mapreduce-client-nativetask. |
| {color:green}+1{color} | mapreduce tests | 0m 24s | Tests passed in
hadoop-mapreduce-client-shuffle. |
| | | 148m 32s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731078/MAPREDUCE-6356-002.patch
|
| Optional Tests | javac unit |
| git revision | trunk / 449e442 |
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-common test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-hs test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
|
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| hadoop-mapreduce-client-nativetask test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/artifact/patchprocess/testrun_hadoop-mapreduce-client-nativetask.txt
|
| hadoop-mapreduce-client-shuffle test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/artifact/patchprocess/testrun_hadoop-mapreduce-client-shuffle.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/console |
This message was automatically generated.
> Misspelling of threshold in log4j.properties for tests
> --
>
> Key: MAPREDUCE-6356
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Minor
> Labels: BB2015-05-RFC
> Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch
>
>
> log4j.properties file for test contains misspelling "log4j.threshhold".
> We should use "log4j.threshold" correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6284) Add a 'task attempt state' to MapReduce Application Master REST API
[ https://issues.apache.org/jira/browse/MAPREDUCE-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532277#comment-14532277 ] Tsuyoshi Ozawa commented on MAPREDUCE-6284: --- OK, I'm checking... > Add a 'task attempt state' to MapReduce Application Master REST API > --- > > Key: MAPREDUCE-6284 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6284 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Ryu Kobayashi >Assignee: Ryu Kobayashi >Priority: Minor > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6284.1.patch, MAPREDUCE-6284.1.patch, > MAPREDUCE-6284.2.patch, MAPREDUCE-6284.3.patch, MAPREDUCE-6284.3.patch, > MAPREDUCE-6284.4.patch, MAPREDUCE-6284.5.patch > > > It want to 'task attempt state' on the 'App state' similarly REST API. > GET http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > PUT http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6284) Add a 'task attempt state' to MapReduce Application Master REST API
[ https://issues.apache.org/jira/browse/MAPREDUCE-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532265#comment-14532265 ] Ryu Kobayashi commented on MAPREDUCE-6284: -- [~ozawa] I fixed the checkstyle. and any more... > Add a 'task attempt state' to MapReduce Application Master REST API > --- > > Key: MAPREDUCE-6284 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6284 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Ryu Kobayashi >Assignee: Ryu Kobayashi >Priority: Minor > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6284.1.patch, MAPREDUCE-6284.1.patch, > MAPREDUCE-6284.2.patch, MAPREDUCE-6284.3.patch, MAPREDUCE-6284.3.patch, > MAPREDUCE-6284.4.patch, MAPREDUCE-6284.5.patch > > > It want to 'task attempt state' on the 'App state' similarly REST API. > GET http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > PUT http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6284) Add a 'task attempt state' to MapReduce Application Master REST API
[
https://issues.apache.org/jira/browse/MAPREDUCE-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532246#comment-14532246
]
Hadoop QA commented on MAPREDUCE-6284:
--
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 36s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 3 new or modified test files. |
| {color:green}+1{color} | javac | 7m 42s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 55s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | site | 2m 59s | Site still builds. |
| {color:green}+1{color} | checkstyle | 1m 25s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 5s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 47s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 58s | Tests passed in
hadoop-mapreduce-client-app. |
| {color:green}+1{color} | mapreduce tests | 1m 43s | Tests passed in
hadoop-mapreduce-client-core. |
| {color:green}+1{color} | yarn tests | 0m 22s | Tests passed in
hadoop-yarn-server-web-proxy. |
| | | 57m 6s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731115/MAPREDUCE-6284.5.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle site |
| git revision | trunk / 449e442 |
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5658/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5658/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| hadoop-yarn-server-web-proxy test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5658/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5658/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5658/console |
This message was automatically generated.
> Add a 'task attempt state' to MapReduce Application Master REST API
> ---
>
> Key: MAPREDUCE-6284
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6284
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Ryu Kobayashi
>Assignee: Ryu Kobayashi
>Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-6284.1.patch, MAPREDUCE-6284.1.patch,
> MAPREDUCE-6284.2.patch, MAPREDUCE-6284.3.patch, MAPREDUCE-6284.3.patch,
> MAPREDUCE-6284.4.patch, MAPREDUCE-6284.5.patch
>
>
> It want to 'task attempt state' on the 'App state' similarly REST API.
> GET http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state
> PUT http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532191#comment-14532191 ] Akira AJISAKA commented on MAPREDUCE-6356: -- Submitted jenkins job: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5659/ > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Labels: BB2015-05-RFC > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6356) Misspelling of threshold in log4j.properties for tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532190#comment-14532190 ] Akira AJISAKA commented on MAPREDUCE-6356: -- Many tests are failing with java.lang.NoClassDefFoundError. Looks like the jars in the repository used by the tests are stale. > Misspelling of threshold in log4j.properties for tests > -- > > Key: MAPREDUCE-6356 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6356 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Minor > Labels: BB2015-05-RFC > Attachments: MAPREDUCE-6356-002.patch, MAPREDUCE-6356.patch > > > log4j.properties file for test contains misspelling "log4j.threshhold". > We should use "log4j.threshold" correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6284) Add a 'task attempt state' to MapReduce Application Master REST API
[ https://issues.apache.org/jira/browse/MAPREDUCE-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryu Kobayashi updated MAPREDUCE-6284: - Attachment: MAPREDUCE-6284.5.patch > Add a 'task attempt state' to MapReduce Application Master REST API > --- > > Key: MAPREDUCE-6284 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6284 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Ryu Kobayashi >Assignee: Ryu Kobayashi >Priority: Minor > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6284.1.patch, MAPREDUCE-6284.1.patch, > MAPREDUCE-6284.2.patch, MAPREDUCE-6284.3.patch, MAPREDUCE-6284.3.patch, > MAPREDUCE-6284.4.patch, MAPREDUCE-6284.5.patch > > > It want to 'task attempt state' on the 'App state' similarly REST API. > GET http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > PUT http:///proxy/ _id>/ws/v1/mapreduce/jobs//tasks//attempts//state > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
