Good day. I have a question about sqoop in CDH environment (version 5.4.1)
1. I have created table in Oracle DB CREATE TABLE TEST_SQOOP_EXPORT ( col VARCHAR2(255) ) 2. I have created simple linux text file input_file.txt with 8 rows: A B C D E F G H and copy it to hdfs folder /tmp/sqoop2/impala 3. I have run simple sqoop script sqoop export --connect jdbc:oracle:thin:@//****:****/**** --username **** --password **** --table TEST_SQOOP_EXPORT --export-dir /tmp/sqoop2/impala --input-lines-terminated-by "\n" --verbose &> /tmp/sqoop_test.log Below you may see the content of sqoop_test.log file Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 15/06/05 16:57:45 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.4.1 15/06/05 16:57:45 DEBUG tool.BaseSqoopTool: Enabled debug logging. 15/06/05 16:57:45 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 15/06/05 16:57:45 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory 15/06/05 16:57:45 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory 15/06/05 16:57:45 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory 15/06/05 16:57:45 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop! 15/06/05 16:57:45 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled. 15/06/05 16:57:45 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory 15/06/05 16:57:45 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:oracle:thin:@ 15/06/05 16:57:45 DEBUG manager.OracleManager$ConnCache: Instantiated new connection cache. 15/06/05 16:57:45 INFO manager.SqlManager: Using default fetchSize of 1000 15/06/05 16:57:45 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.OracleManager@7eaf8890 15/06/05 16:57:45 INFO tool.CodeGenTool: Beginning code generation 15/06/05 16:57:45 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM TEST_SQOOP_EXPORT t WHERE 1=0 15/06/05 16:57:45 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM TEST_SQOOP_EXPORT t WHERE 1=0 15/06/05 16:57:46 DEBUG manager.OracleManager: Creating a new connection for jdbc:oracle:thin:@//****:****/****, using username: **** 15/06/05 16:57:46 DEBUG manager.OracleManager: No connection paramenters specified. Using regular API for making connection. 15/06/05 16:57:46 INFO manager.OracleManager: Time zone has been set to GMT 15/06/05 16:57:46 DEBUG manager.SqlManager: Using fetchSize for next query: 1000 15/06/05 16:57:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM TEST_SQOOP_EXPORT t WHERE 1=0 15/06/05 16:57:46 DEBUG manager.SqlManager: Found column COL of type [12, 255, 0] 15/06/05 16:57:46 DEBUG manager.OracleManager$ConnCache: Caching released connection for jdbc:oracle:thin:@//****:****/****/**** 15/06/05 16:57:46 DEBUG orm.ClassWriter: selected columns: 15/06/05 16:57:46 DEBUG orm.ClassWriter: COL 15/06/05 16:57:47 DEBUG orm.ClassWriter: Writing source file: /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/TEST_SQOOP_EXPORT.java 15/06/05 16:57:47 DEBUG orm.ClassWriter: Table name: TEST_SQOOP_EXPORT 15/06/05 16:57:47 DEBUG orm.ClassWriter: Columns: COL:12, 15/06/05 16:57:47 DEBUG orm.ClassWriter: sourceFilename is TEST_SQOOP_EXPORT.java 15/06/05 16:57:47 DEBUG orm.CompilationManager: Found existing /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/ 15/06/05 16:57:47 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce 15/06/05 16:57:47 DEBUG orm.CompilationManager: Returning jar file path /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.6.0-cdh5.4.1.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core.jar 15/06/05 16:57:47 DEBUG orm.CompilationManager: Current sqoop classpath = <****> 15/06/05 16:57:47 DEBUG orm.CompilationManager: Adding source file: /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/TEST_SQOOP_EXPORT.java 15/06/05 16:57:47 DEBUG orm.CompilationManager: Invoking javac with args: 15/06/05 16:57:47 DEBUG orm.CompilationManager: -sourcepath 15/06/05 16:57:47 DEBUG orm.CompilationManager: /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/ 15/06/05 16:57:47 DEBUG orm.CompilationManager: -d 15/06/05 16:57:47 DEBUG orm.CompilationManager: /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/ 15/06/05 16:57:47 DEBUG orm.CompilationManager: -classpath 15/06/05 16:57:47 DEBUG orm.CompilationManager: <****> Note: /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/TEST_SQOOP_EXPORT.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 15/06/05 16:57:48 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/TEST_SQOOP_EXPORT.jar 15/06/05 16:57:48 DEBUG orm.CompilationManager: Scanning for .class files in directory: /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32 15/06/05 16:57:48 DEBUG orm.CompilationManager: Got classfile: /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/TEST_SQOOP_EXPORT.class -> TEST_SQOOP_EXPORT.class 15/06/05 16:57:48 DEBUG orm.CompilationManager: Finished writing jar file /tmp/sqoop-user/compile/c0e189cabdeb91a71ab686220f485b32/TEST_SQOOP_EXPORT.jar 15/06/05 16:57:48 INFO mapreduce.ExportJobBase: Beginning export of TEST_SQOOP_EXPORT 15/06/05 16:57:48 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 15/06/05 16:57:48 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Using InputFormat: class org.apache.sqoop.mapreduce.ExportInputFormat 15/06/05 16:57:49 DEBUG db.DBConfiguration: Securing password into job credentials store 15/06/05 16:57:49 DEBUG manager.OracleManager$ConnCache: Got cached connection for jdbc:oracle:thin:@//****:****/****/**** 15/06/05 16:57:49 INFO manager.OracleManager: Time zone has been set to GMT 15/06/05 16:57:49 DEBUG manager.OracleManager$ConnCache: Caching released connection for jdbc:oracle:thin:@//****:****/****/**** 15/06/05 16:57:49 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 15/06/05 16:57:49 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 15/06/05 16:57:49 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/sqoop-1.4.5-cdh5.4.1.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/var/lib/sqoop/ojdbc6.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/sqoop-1.4.5-cdh5.4.1.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/sqoop-1.4.5-cdh5.4.1.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/slf4j-api-1.7.5.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/jackson-databind-2.3.1.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/commons-jexl-2.1.1.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/parquet-hadoop.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/jackson-annotations-2.3.0.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/commons-codec-1.4.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/snappy-java-1.0.4.1.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/parquet-encoding.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/opencsv-2.3.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/xz-1.0.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/parquet-column.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/parquet-common.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/jackson-core-2.3.1.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/commons-compress-1.4.1.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/parquet-jackson.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/hsqldb-1.8.0.10.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/avro-mapred-hadoop2.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/jackson-core-asl-1.8.8.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/avro.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/kite-data-hive.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/ant-contrib-1.0b3.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/kite-data-mapreduce.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/kite-data-core.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/parquet-generator.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/commons-io-1.4.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/paranamer-2.3.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/commons-logging-1.1.3.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/parquet-avro.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/kite-hadoop-compatibility.jar 15/06/05 16:57:49 DEBUG mapreduce.JobBase: Adding to job classpath: file:/usr/lib/sqoop/lib/parquet-format.jar 15/06/05 16:57:49 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 15/06/05 16:57:49 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 15/06/05 16:57:50 INFO input.FileInputFormat: Total input paths to process : 1 15/06/05 16:57:50 DEBUG mapreduce.ExportInputFormat: Target numMapTasks=4 15/06/05 16:57:50 DEBUG mapreduce.ExportInputFormat: Total input bytes=16 15/06/05 16:57:50 DEBUG mapreduce.ExportInputFormat: maxSplitSize=4 15/06/05 16:57:50 INFO input.FileInputFormat: Total input paths to process : 1 15/06/05 16:57:50 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library 15/06/05 16:57:50 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev d7351f78bf01039a8cd4fea891924e0369aee90c] 15/06/05 16:57:50 DEBUG mapreduce.ExportInputFormat: Generated splits: 15/06/05 16:57:50 DEBUG mapreduce.ExportInputFormat: Paths:/tmp/sqoop2/impala/input_file.txt:0+4 Locations:****; 15/06/05 16:57:50 DEBUG mapreduce.ExportInputFormat: Paths:/tmp/sqoop2/impala/input_file.txt:4+4 Locations:****; 15/06/05 16:57:50 DEBUG mapreduce.ExportInputFormat: Paths:/tmp/sqoop2/impala/input_file.txt:8+4 Locations:****; 15/06/05 16:57:50 DEBUG mapreduce.ExportInputFormat: Paths:/tmp/sqoop2/impala/input_file.txt:12+4 Locations:****; 15/06/05 16:57:50 INFO mapreduce.JobSubmitter: number of splits:4 15/06/05 16:57:50 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative 15/06/05 16:57:50 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1900264151_0001 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471079/ant-eclipse-1.0-jvm1.2.jar <- /home/user/ant-eclipse-1.0-jvm1.2.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar as file:/tmp/hadoop-user/mapred/local/1433523471079/ant-eclipse-1.0-jvm1.2.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471080/jackson-core-asl-1.8.8.jar <- /home/user/jackson-core-asl-1.8.8.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/jackson-core-asl-1.8.8.jar as file:/tmp/hadoop-user/mapred/local/1433523471080/jackson-core-asl-1.8.8.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471081/xz-1.0.jar <- /home/user/xz-1.0.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/xz-1.0.jar as file:/tmp/hadoop-user/mapred/local/1433523471081/xz-1.0.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471082/parquet-jackson.jar <- /home/user/parquet-jackson.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/parquet-jackson.jar as file:/tmp/hadoop-user/mapred/local/1433523471082/parquet-jackson.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471083/parquet-generator.jar <- /home/user/parquet-generator.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/parquet-generator.jar as file:/tmp/hadoop-user/mapred/local/1433523471083/parquet-generator.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471084/paranamer-2.3.jar <- /home/user/paranamer-2.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/paranamer-2.3.jar as file:/tmp/hadoop-user/mapred/local/1433523471084/paranamer-2.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471085/slf4j-api-1.7.5.jar <- /home/user/slf4j-api-1.7.5.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/slf4j-api-1.7.5.jar as file:/tmp/hadoop-user/mapred/local/1433523471085/slf4j-api-1.7.5.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471086/commons-codec-1.4.jar <- /home/user/commons-codec-1.4.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/commons-codec-1.4.jar as file:/tmp/hadoop-user/mapred/local/1433523471086/commons-codec-1.4.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471087/ojdbc6.jar <- /home/user/ojdbc6.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/var/lib/sqoop/ojdbc6.jar as file:/tmp/hadoop-user/mapred/local/1433523471087/ojdbc6.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471088/jackson-annotations-2.3.0.jar <- /home/user/jackson-annotations-2.3.0.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/jackson-annotations-2.3.0.jar as file:/tmp/hadoop-user/mapred/local/1433523471088/jackson-annotations-2.3.0.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471089/commons-io-1.4.jar <- /home/user/commons-io-1.4.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/commons-io-1.4.jar as file:/tmp/hadoop-user/mapred/local/1433523471089/commons-io-1.4.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471090/opencsv-2.3.jar <- /home/user/opencsv-2.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/opencsv-2.3.jar as file:/tmp/hadoop-user/mapred/local/1433523471090/opencsv-2.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471091/snappy-java-1.0.4.1.jar <- /home/user/snappy-java-1.0.4.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/snappy-java-1.0.4.1.jar as file:/tmp/hadoop-user/mapred/local/1433523471091/snappy-java-1.0.4.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471092/parquet-format.jar <- /home/user/parquet-format.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/parquet-format.jar as file:/tmp/hadoop-user/mapred/local/1433523471092/parquet-format.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471093/jackson-core-2.3.1.jar <- /home/user/jackson-core-2.3.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/jackson-core-2.3.1.jar as file:/tmp/hadoop-user/mapred/local/1433523471093/jackson-core-2.3.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471094/parquet-common.jar <- /home/user/parquet-common.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/parquet-common.jar as file:/tmp/hadoop-user/mapred/local/1433523471094/parquet-common.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471095/kite-data-core.jar <- /home/user/kite-data-core.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/kite-data-core.jar as file:/tmp/hadoop-user/mapred/local/1433523471095/kite-data-core.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471096/parquet-avro.jar <- /home/user/parquet-avro.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/parquet-avro.jar as file:/tmp/hadoop-user/mapred/local/1433523471096/parquet-avro.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471097/kite-hadoop-compatibility.jar <- /home/user/kite-hadoop-compatibility.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/kite-hadoop-compatibility.jar as file:/tmp/hadoop-user/mapred/local/1433523471097/kite-hadoop-compatibility.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471098/parquet-column.jar <- /home/user/parquet-column.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/parquet-column.jar as file:/tmp/hadoop-user/mapred/local/1433523471098/parquet-column.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471099/ant-contrib-1.0b3.jar <- /home/user/ant-contrib-1.0b3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/ant-contrib-1.0b3.jar as file:/tmp/hadoop-user/mapred/local/1433523471099/ant-contrib-1.0b3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471100/parquet-hadoop.jar <- /home/user/parquet-hadoop.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/parquet-hadoop.jar as file:/tmp/hadoop-user/mapred/local/1433523471100/parquet-hadoop.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471101/commons-logging-1.1.3.jar <- /home/user/commons-logging-1.1.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/commons-logging-1.1.3.jar as file:/tmp/hadoop-user/mapred/local/1433523471101/commons-logging-1.1.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471102/commons-jexl-2.1.1.jar <- /home/user/commons-jexl-2.1.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/commons-jexl-2.1.1.jar as file:/tmp/hadoop-user/mapred/local/1433523471102/commons-jexl-2.1.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471103/parquet-encoding.jar <- /home/user/parquet-encoding.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/parquet-encoding.jar as file:/tmp/hadoop-user/mapred/local/1433523471103/parquet-encoding.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471104/avro.jar <- /home/user/avro.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/avro.jar as file:/tmp/hadoop-user/mapred/local/1433523471104/avro.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471105/kite-data-hive.jar <- /home/user/kite-data-hive.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/kite-data-hive.jar as file:/tmp/hadoop-user/mapred/local/1433523471105/kite-data-hive.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471106/hsqldb-1.8.0.10.jar <- /home/user/hsqldb-1.8.0.10.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/hsqldb-1.8.0.10.jar as file:/tmp/hadoop-user/mapred/local/1433523471106/hsqldb-1.8.0.10.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471107/kite-data-mapreduce.jar <- /home/user/kite-data-mapreduce.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/kite-data-mapreduce.jar as file:/tmp/hadoop-user/mapred/local/1433523471107/kite-data-mapreduce.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471108/jackson-mapper-asl-1.8.8.jar <- /home/user/jackson-mapper-asl-1.8.8.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/jackson-mapper-asl-1.8.8.jar as file:/tmp/hadoop-user/mapred/local/1433523471108/jackson-mapper-asl-1.8.8.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471109/commons-compress-1.4.1.jar <- /home/user/commons-compress-1.4.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/commons-compress-1.4.1.jar as file:/tmp/hadoop-user/mapred/local/1433523471109/commons-compress-1.4.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471110/jackson-databind-2.3.1.jar <- /home/user/jackson-databind-2.3.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/jackson-databind-2.3.1.jar as file:/tmp/hadoop-user/mapred/local/1433523471110/jackson-databind-2.3.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471111/avro-mapred-hadoop2.jar <- /home/user/avro-mapred-hadoop2.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/lib/avro-mapred-hadoop2.jar as file:/tmp/hadoop-user/mapred/local/1433523471111/avro-mapred-hadoop2.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Creating symlink: /tmp/hadoop-user/mapred/local/1433523471112/sqoop-1.4.5-cdh5.4.1.jar <- /home/user/sqoop-1.4.5-cdh5.4.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: Localized file:/usr/lib/sqoop/sqoop-1.4.5-cdh5.4.1.jar as file:/tmp/hadoop-user/mapred/local/1433523471112/sqoop-1.4.5-cdh5.4.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471079/ant-eclipse-1.0-jvm1.2.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471080/jackson-core-asl-1.8.8.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471081/xz-1.0.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471082/parquet-jackson.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471083/parquet-generator.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471084/paranamer-2.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471085/slf4j-api-1.7.5.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471086/commons-codec-1.4.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471087/ojdbc6.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471088/jackson-annotations-2.3.0.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471089/commons-io-1.4.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471090/opencsv-2.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471091/snappy-java-1.0.4.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471092/parquet-format.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471093/jackson-core-2.3.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471094/parquet-common.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471095/kite-data-core.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471096/parquet-avro.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471097/kite-hadoop-compatibility.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471098/parquet-column.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471099/ant-contrib-1.0b3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471100/parquet-hadoop.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471101/commons-logging-1.1.3.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471102/commons-jexl-2.1.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471103/parquet-encoding.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471104/avro.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471105/kite-data-hive.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471106/hsqldb-1.8.0.10.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471107/kite-data-mapreduce.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471108/jackson-mapper-asl-1.8.8.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471109/commons-compress-1.4.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471110/jackson-databind-2.3.1.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471111/avro-mapred-hadoop2.jar 15/06/05 16:57:51 INFO mapred.LocalDistributedCacheManager: file:/tmp/hadoop-user/mapred/local/1433523471112/sqoop-1.4.5-cdh5.4.1.jar 15/06/05 16:57:51 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 15/06/05 16:57:51 INFO mapreduce.Job: Running job: job_local1900264151_0001 15/06/05 16:57:51 INFO mapred.LocalJobRunner: OutputCommitter set in config null 15/06/05 16:57:51 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.sqoop.mapreduce.NullOutputCommitter 15/06/05 16:57:52 INFO mapred.LocalJobRunner: Waiting for map tasks 15/06/05 16:57:52 INFO mapred.LocalJobRunner: Starting task: attempt_local1900264151_0001_m_000000_0 15/06/05 16:57:52 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 15/06/05 16:57:52 INFO mapred.MapTask: Processing split: Paths:/tmp/sqoop2/impala/input_file.txt:0+4 15/06/05 16:57:52 INFO Configuration.deprecation: map.input.file is deprecated. Instead, use mapreduce.map.input.file 15/06/05 16:57:52 INFO Configuration.deprecation: map.input.start is deprecated. Instead, use mapreduce.map.input.start 15/06/05 16:57:52 INFO Configuration.deprecation: map.input.length is deprecated. Instead, use mapreduce.map.input.length 15/06/05 16:57:52 DEBUG mapreduce.CombineShimRecordReader: ChildSplit operates on: hdfs://****:****/tmp/sqoop2/impala/input_file.txt 15/06/05 16:57:52 DEBUG db.DBConfiguration: Fetching password from job credentials store 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Instructing auto-progress thread to quit. 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Waiting for progress thread shutdown... 15/06/05 16:57:52 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Progress thread shutdown detected. 15/06/05 16:57:52 DEBUG mapreduce.AsyncSqlOutputFormat: Committing transaction of 0 statements 15/06/05 16:57:52 INFO mapred.LocalJobRunner: Starting task: attempt_local1900264151_0001_m_000001_0 15/06/05 16:57:52 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 15/06/05 16:57:52 INFO mapred.MapTask: Processing split: Paths:/tmp/sqoop2/impala/input_file.txt:4+4 15/06/05 16:57:52 DEBUG mapreduce.CombineShimRecordReader: ChildSplit operates on: hdfs://****:****/tmp/sqoop2/impala/input_file.txt 15/06/05 16:57:52 DEBUG db.DBConfiguration: Fetching password from job credentials store 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Instructing auto-progress thread to quit. 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Waiting for progress thread shutdown... 15/06/05 16:57:52 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Progress thread shutdown detected. 15/06/05 16:57:52 DEBUG mapreduce.AsyncSqlOutputFormat: Committing transaction of 0 statements 15/06/05 16:57:52 INFO mapred.LocalJobRunner: Starting task: attempt_local1900264151_0001_m_000002_0 15/06/05 16:57:52 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 15/06/05 16:57:52 INFO mapred.MapTask: Processing split: Paths:/tmp/sqoop2/impala/input_file.txt:8+4 15/06/05 16:57:52 DEBUG mapreduce.CombineShimRecordReader: ChildSplit operates on: hdfs://****:****/tmp/sqoop2/impala/input_file.txt 15/06/05 16:57:52 DEBUG db.DBConfiguration: Fetching password from job credentials store 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Instructing auto-progress thread to quit. 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Waiting for progress thread shutdown... 15/06/05 16:57:52 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Progress thread shutdown detected. 15/06/05 16:57:52 DEBUG mapreduce.AsyncSqlOutputFormat: Committing transaction of 0 statements 15/06/05 16:57:52 INFO mapred.LocalJobRunner: Starting task: attempt_local1900264151_0001_m_000003_0 15/06/05 16:57:52 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 15/06/05 16:57:52 INFO mapred.MapTask: Processing split: Paths:/tmp/sqoop2/impala/input_file.txt:12+4 15/06/05 16:57:52 DEBUG mapreduce.CombineShimRecordReader: ChildSplit operates on: hdfs://****:****/tmp/sqoop2/impala/input_file.txt 15/06/05 16:57:52 DEBUG db.DBConfiguration: Fetching password from job credentials store 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Instructing auto-progress thread to quit. 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Waiting for progress thread shutdown... 15/06/05 16:57:52 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false 15/06/05 16:57:52 DEBUG mapreduce.AutoProgressMapper: Progress thread shutdown detected. 15/06/05 16:57:52 DEBUG mapreduce.AsyncSqlOutputFormat: Committing transaction of 0 statements 15/06/05 16:57:52 INFO mapred.LocalJobRunner: map task executor complete. 15/06/05 16:57:52 WARN mapred.LocalJobRunner: job_local1900264151_0001*java.lang.Exception: java.io.IOException: java.lang.ClassNotFoundException: TEST_SQOOP_EXPORT* at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) Caused by: java.io.IOException: java.lang.ClassNotFoundException: TEST_SQOOP_EXPORT at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:70) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: TEST_SQOOP_EXPORT at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:274) at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:66) ... 10 more 15/06/05 16:57:52 INFO mapreduce.Job: Job job_local1900264151_0001 running in uber mode : false 15/06/05 16:57:52 INFO mapreduce.Job: map 0% reduce 0% 15/06/05 16:57:52 INFO mapreduce.Job: Job job_local1900264151_0001 failed with state FAILED due to: NA 15/06/05 16:57:53 INFO mapreduce.Job: Counters: 0 15/06/05 16:57:53 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 15/06/05 16:57:53 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 3.5336 seconds (0 bytes/sec) 15/06/05 16:57:53 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 15/06/05 16:57:53 INFO mapreduce.ExportJobBase: Exported 0 records. 15/06/05 16:57:53 ERROR tool.ExportTool: Error during export: Export job failed! I don't understand how I may resolve this error! Show me the right way please!
