[ https://issues.apache.org/jira/browse/MAPREDUCE-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Vary updated MAPREDUCE-6761: ---------------------------------- Attachment: MAPREDUCE-6761.2.patch Thanks [~rchiang], [~templedf] for the reviews. Added a TestCluster testcase, to check the ServiceLoader handling About the for/iterator issue: First I was leaning more to the iterator version, since when I checked the source code of the ServiceLoader, the hasNext() only loaded the configuration files, and the next() instantiated the actual objects (The exception is thrown only in the next() method). So if there was a line with an error in the configuration file, it was still possible to get the next element, and continue with the parsing. If the following lines were correct, those could be used too. After [~templedf] comment, I checked the documentation too, and found this: {noformat} Its hasNext and next methods can therefore throw a ServiceConfigurationError if a provider-configuration file violates the specified format [..] If such an error is thrown then subsequent invocations of the iterator will make a best effort to locate and instantiate the next available provider, but in general such recovery cannot be guaranteed. {noformat} Which basically means, it is possible that the getNext could throw an exception, and the iterator might be stuck - this could cause infinite loop when trying to iterate through the services. So based on these information, I decided to stay on the safe side, and do not use features found in the implementation, but only stated as “best effort” in the documentation, since these might be changed without any notification. This means, that the parsing of the configuration files will be done until the first erroneous element found - as it was before MAPREDUCE-6473. And I can use the foreach loop with a try/catch to handle the exception. The problem with the MAPREDUCE-6473 was that when the initProviderList throws an exception it was not catched at all, so the Providers loaded before the error ware not used, and the Cluster creation is failed. Thanks again, Peter PS: I love and hate your nitpicking infrastructure "You should put an '.' at the end of your sentence in the comment" :) > Regression when handling providers - invalid configuration > ServiceConfiguration causes Cluster initialization failure > --------------------------------------------------------------------------------------------------------------------- > > Key: MAPREDUCE-6761 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6761 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 > Affects Versions: 3.0.0-alpha2 > Reporter: Peter Vary > Assignee: Peter Vary > Labels: supportability > Attachments: MAPREDUCE-6761.2.patch, MAPREDUCE-6761.patch > > > When a rouge org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider > defines a provider that is not on classpath, then the initialization is > failed with the following exception: > java.util.ServiceConfigurationError: > org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider: Provider > org.apache.hadoop.mapred.YarnClientProtocolProvider not found > at java.util.ServiceLoader.fail(ServiceLoader.java:239) > at java.util.ServiceLoader.access$300(ServiceLoader.java:185) > at > java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372) > at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) > at java.util.ServiceLoader$1.next(ServiceLoader.java:480) > at org.apache.hadoop.mapreduce.Cluster.initProviderList(Cluster.java:84) > at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:114) > at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:108) > at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:101) > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:477) > at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:455) > This regression is caused by MAPREDUCE-6473 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org