[
https://issues.apache.org/jira/browse/SQOOP-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Szabo updated SQOOP-1301:
--------------------------------
Fix Version/s: (was: 1.4.7)
1.5.0
> 'run' method in Sqoop.java is not thread-safe
> ---------------------------------------------
>
> Key: SQOOP-1301
> URL: https://issues.apache.org/jira/browse/SQOOP-1301
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.4
> Reporter: Ajay Chitre
> Fix For: 1.5.0
>
>
> It seems the ‘run’ method in Sqoop is not thread-safe. When multiple Sqoop
> jobs are triggered exactly at the same time, they end up stepping on each
> other. Here’s what seems to be the problem: The ‘run’ method calls
> tool.parseArguments which in turn calls
> ConfigurationHelper.parseGenericOptions which in turn creates a new instance
> of GenericOptionsParser. The constructor for GenericOptionsParser is not
> thread-safe because it ends up calling ‘buildGeneralOptions’ which uses
> OptionBuilder.withArgName. This method uses instance variables thereby
> making it thread unsafe.
> The way we’ve got around it is by creating a ‘Lock’ object. This seems to be
> working. If there’s a better way, please let us know. If not, please
> consider adding this feature. We can create a patch if there’s an interest.
> Thanks.
> Ajay Chitre ([email protected])
> Virendra Singh ([email protected])
> Anyway, here’s what we’ve done:
> 1) Created a class that extends Sqoop.java
> public class DlSqoop extends Sqoop {
> 2) Created a Lock object:
> private static Lock monitor = new ReentrantLock();
> 3) Overridden ‘run’ method as follows:
> public int run(String [] args) {
> if (options.getConf() == null) {
> // Configuration wasn't initialized until after the ToolRunner
> // got us to this point. ToolRunner gave Sqoop itself a Conf
> // though.
> options.setConf(getConf());
> }
> try {
> monitor.lock();
> options = tool.parseArguments(args, null, options, false);
> tool.appendArgs(this.childPrgmArgs);
> tool.validateOptions(options);
> } catch (Exception e) {
> // Couldn't parse arguments.
> // Log the stack trace for this exception
> LOG.debug(e.getMessage(), e);
> // Print exception message.
> System.err.println(e.getMessage());
> // Print the tool usage message and exit.
> ToolOptions toolOpts = new ToolOptions();
> tool.configureOptions(toolOpts);
> tool.printHelp(toolOpts);
> return 1; // Exit on exception here.
> } finally {
> monitor.unlock();
> }
> return tool.run(options);
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
