[ https://issues.apache.org/jira/browse/SQOOP-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Attila Szabo updated SQOOP-1301: -------------------------------- Fix Version/s: (was: 1.4.7) 1.5.0 > 'run' method in Sqoop.java is not thread-safe > --------------------------------------------- > > Key: SQOOP-1301 > URL: https://issues.apache.org/jira/browse/SQOOP-1301 > Project: Sqoop > Issue Type: Bug > Affects Versions: 1.4.4 > Reporter: Ajay Chitre > Fix For: 1.5.0 > > > It seems the ‘run’ method in Sqoop is not thread-safe. When multiple Sqoop > jobs are triggered exactly at the same time, they end up stepping on each > other. Here’s what seems to be the problem: The ‘run’ method calls > tool.parseArguments which in turn calls > ConfigurationHelper.parseGenericOptions which in turn creates a new instance > of GenericOptionsParser. The constructor for GenericOptionsParser is not > thread-safe because it ends up calling ‘buildGeneralOptions’ which uses > OptionBuilder.withArgName. This method uses instance variables thereby > making it thread unsafe. > The way we’ve got around it is by creating a ‘Lock’ object. This seems to be > working. If there’s a better way, please let us know. If not, please > consider adding this feature. We can create a patch if there’s an interest. > Thanks. > Ajay Chitre (achi...@cisco.com) > Virendra Singh (virsi...@cisco.com) > Anyway, here’s what we’ve done: > 1) Created a class that extends Sqoop.java > public class DlSqoop extends Sqoop { > 2) Created a Lock object: > private static Lock monitor = new ReentrantLock(); > 3) Overridden ‘run’ method as follows: > public int run(String [] args) { > if (options.getConf() == null) { > // Configuration wasn't initialized until after the ToolRunner > // got us to this point. ToolRunner gave Sqoop itself a Conf > // though. > options.setConf(getConf()); > } > try { > monitor.lock(); > options = tool.parseArguments(args, null, options, false); > tool.appendArgs(this.childPrgmArgs); > tool.validateOptions(options); > } catch (Exception e) { > // Couldn't parse arguments. > // Log the stack trace for this exception > LOG.debug(e.getMessage(), e); > // Print exception message. > System.err.println(e.getMessage()); > // Print the tool usage message and exit. > ToolOptions toolOpts = new ToolOptions(); > tool.configureOptions(toolOpts); > tool.printHelp(toolOpts); > return 1; // Exit on exception here. > } finally { > monitor.unlock(); > } > return tool.run(options); > } -- This message was sent by Atlassian JIRA (v6.4.14#64029)