how to add default hadoop conf property value to SqoopOptions.java

[Suhas Satish]

Hi
I can successfully run the following query in Sqoop -

/opt/mapr/sqoop/sqoop-1.4.4/bin/sqoop import -D
hadoop.job.history.user.location=none -connect jdbc:mysql://10.10.1.10/TestDB
--username mapr --password mapr  --table t1 --hive-import



But I am trying to make this property as the default value within
Sqoop. Whats the best way to do this in the code?


hadoop.job.history.user.location=none

I made the following modifications for this in SqoopOptions.java (diff
attached) but the property doesnt seem to get recognized by sqoop and
I still get the stdout log msg which says -

"14/01/22 15:00:36 INFO mapred.JobClient: Creating job's user history
location directory at t1/_logs

"

With the property correctly set, _logs directory should not even be
created.



diff --git a/src/java/org/apache/sqoop/SqoopOptions.java
b/src/java/org/apache/sqoop/SqoopOptions.java
index 01805f9..4612973 100644
--- a/src/java/org/apache/sqoop/SqoopOptions.java
+++ b/src/java/org/apache/sqoop/SqoopOptions.java
@@ -62,6 +62,8 @@
   // Default hive and hcat locations.
   public static final String DEF_HIVE_HOME = "/usr/lib/hive";
   public static final String DEF_HCAT_HOME = "/usr/lib/hcatalog";
+  public static final String DEF_HIST_LOGS = "none";
+  public static final String PROP_HADOOP_JOB_HISTORY_LOGS_LOCATION =
"hadoop.job.history.user.location";

   public static final boolean METASTORE_PASSWORD_DEFAULT = false;

@@ -186,6 +188,7 @@ public String toString() {
   // class to use.
   @StoredAsProperty("codegen.jar.file") private String existingJarFile;

+  @StoredAsProperty("hadoop.job.history.user.location") private String
hadoopJobHistLogsLoc;
   @StoredAsProperty("mapreduce.num.mappers") private int numMappers;
   @StoredAsProperty("enable.compression") private boolean useCompression;
   @StoredAsProperty("compression.codec") private String compressionCodec;
@@ -837,7 +840,12 @@ public static String getHCatHomeDefault() {
     }
     return hcatHome;
   }
-
+  public static String getHadoopJobHistLogsLoc() {
+    return DEF_HIST_LOGS;
+  }
+  private static void setHadoopJobHistLogsLoc(Configuration conf, String
path) {
+    conf.setStrings(PROP_HADOOP_JOB_HISTORY_LOGS_LOCATION, path);
+  }
   private void initDefaults(Configuration baseConfiguration) {
     // first, set the true defaults if nothing else happens.
     // default action is to run the full pipeline.
@@ -886,6 +894,8 @@ private void initDefaults(Configuration
baseConfiguration) {
     } else {
       this.conf = baseConfiguration;
     }
+    setHadoopJobHistLogsLoc(this.conf, DEF_HIST_LOGS);
+    this.hadoopJobHistLogsLoc = getHadoopJobHistLogsLoc();

     this.extraArgs = null;

diff1
Description: Binary data