Hi Abraham, I posted the details on the error and our findings today( sent to the group ) , not sure if it is a sqoop bug. If you can take a look at it and give your insights that would be great. Please let me know if you cannot find it. i will be able to send it to you.
Thanks, Sai On Thu, Dec 4, 2014 at 11:33 AM, Abraham Elmahrek <[email protected]> wrote: > Hey there, > > Could you provide an example snippet of the content you're exporting? > > -Abe > > On Tue, Dec 2, 2014 at 2:49 PM, sai chaitanya tirumerla < > [email protected]> wrote: > >> Hi, >> >> My sqoop export command is as follows >> >> sqoop export \ >> --connect "xxxxxx" \ >> --username "xxxxxx" \ >> --password "xxxxxxxxx" \ >> --table "abcd" \ >> --columns >> accnt_duns_num,accnt_firm_name,business_name,ceo_first_name,ceo_full_name,ceo_last_name,ceo_middle_name,country_name_50,county_name_45,fax_nbr_nt_fmtd,first_exec_first_name,first_exec_last_name,first_exec_mid_name,mail_addr_2,mail_addr_64,phone_16,primary_addr_1,primary_addr_2,sec_exec_first_name,sec_exec_last_name,sec_exec_mid_name,second_name,state_prov_name_50,third_exec_first_name,third_exec_last_name,third_exec_mid_name >> \ >> --export-dir "$S3_PATH/abcd" \ >> --escaped-by '\\' --enclosed-by '"' --fields-terminated-by ',' \ >> --update-key accnt_duns_num >> >> Here is the full error stack trace for the error above >> >> 14/12/02 22:38:28 INFO tool.CodeGenTool: Beginning code generation >> 14/12/02 22:38:28 INFO manager.SqlManager: Executing SQL statement: >> SELECT t.* FROM [t_cde_duns_master] AS t WHERE 1=0 >> 14/12/02 22:38:28 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is >> /home/hadoop >> Note: >> /tmp/sqoop-hadoop/compile/f782b6393504f22cee53e80c8b63e1c1/t_cde_duns_master.java >> uses or overrides a deprecated API. >> Note: Recompile with -Xlint:deprecation for details. >> 14/12/02 22:38:30 INFO orm.CompilationManager: Writing jar file: >> /tmp/sqoop-hadoop/compile/f782b6393504f22cee53e80c8b63e1c1/t_cde_duns_master.jar >> 14/12/02 22:38:30 INFO mapreduce.ExportJobBase: Beginning export of >> t_cde_duns_master >> 14/12/02 22:38:30 INFO Configuration.deprecation: mapred.jar is >> deprecated. Instead, use mapreduce.job.jar >> 14/12/02 22:38:31 INFO guice.EmrFSBaseModule: Consistency disabled, using >> com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem as FileSystem >> implementation. >> 14/12/02 22:38:32 INFO fs.EmrFileSystem: Using >> com.amazon.ws.emr.hadoop.fs.s3n.S3NativeFileSystem as filesystem >> implementation >> 14/12/02 22:38:32 INFO Configuration.deprecation: >> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use >> mapreduce.reduce.speculative >> 14/12/02 22:38:33 INFO s3n.S3NativeFileSystem: listStatus >> s3://datamasking/bfix/odsg/cde/dbo/t_cde_duns_master/masked with recursive >> false >> 14/12/02 22:38:33 INFO s3n.S3NativeFileSystem: Opening >> 's3://datamasking/bfix/odsg/cde/dbo/t_cde_duns_master/masked/43679ac8-bb94-47da-9474-0c62782ebf2d-000000.gz' >> for reading >> 14/12/02 22:38:33 INFO Configuration.deprecation: >> mapred.map.tasks.speculative.execution is deprecated. Instead, use >> mapreduce.map.speculative >> 14/12/02 22:38:33 INFO Configuration.deprecation: mapred.map.tasks is >> deprecated. Instead, use mapreduce.job.maps >> 14/12/02 22:38:33 INFO client.RMProxy: Connecting to ResourceManager at / >> 10.37.70.235:9022 >> 14/12/02 22:38:34 INFO s3n.S3NativeFileSystem: listStatus >> s3://datamasking/bfix/odsg/cde/dbo/t_cde_duns_master/masked with recursive >> false >> 14/12/02 22:38:34 INFO input.FileInputFormat: Total input paths to >> process : 6 >> 14/12/02 22:38:34 INFO s3n.S3NativeFileSystem: listStatus >> s3://datamasking/bfix/odsg/cde/dbo/t_cde_duns_master/masked with recursive >> false >> 14/12/02 22:38:34 INFO input.FileInputFormat: Total input paths to >> process : 6 >> 14/12/02 22:38:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library >> from the embedded binaries >> 14/12/02 22:38:34 INFO lzo.LzoCodec: Successfully loaded & initialized >> native-lzo library [hadoop-lzo rev 77cfa96225d62546008ca339b7c2076a3da91578] >> 14/12/02 22:38:35 INFO mapreduce.JobSubmitter: number of splits:3 >> 14/12/02 22:38:35 INFO mapreduce.JobSubmitter: Submitting tokens for job: >> job_1415156847621_2303 >> 14/12/02 22:38:35 INFO impl.YarnClientImpl: Submitted application >> application_1415156847621_2303 >> 14/12/02 22:38:35 INFO mapreduce.Job: The url to track the job: >> http://10.37.70.235:9046/proxy/application_1415156847621_2303/ >> 14/12/02 22:38:35 INFO mapreduce.Job: Running job: job_1415156847621_2303 >> 14/12/02 22:38:42 INFO mapreduce.Job: Job job_1415156847621_2303 running >> in uber mode : false >> 14/12/02 22:38:42 INFO mapreduce.Job: map 0% reduce 0% >> 14/12/02 22:38:49 INFO mapreduce.Job: Task Id : >> attempt_1415156847621_2303_m_000000_0, Status : FAILED >> Error: java.io.IOException: Can't export data, please check task tracker >> logs >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) >> at >> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:775) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) >> Caused by: java.io.IOException: >> com.microsoft.sqlserver.jdbc.SQLServerException: The value is not set for >> the parameter number 2. >> at >> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220) >> at >> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46) >> at >> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646) >> at >> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89) >> at >> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112) >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:84) >> ... 10 more >> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The value is >> not set for the parameter number 2. >> at >> com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190) >> at >> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.buildParamTypeDefinitions(SQLServerPreparedStatement.java:260) >> at >> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.buildPreparedStrings(SQLServerPreparedStatement.java:219) >> at >> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doPrepExec(SQLServerPreparedStatement.java:612) >> at >> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatementBatch(SQLServerPreparedStatement.java:1282) >> at >> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtBatchExecCmd.doExecute(SQLServerPreparedStatement.java:1209) >> at >> com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696) >> at >> com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715) >> at >> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180) >> at >> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155) >> at >> com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:1173) >> at >> org.apache.sqoop.mapreduce.AsyncSqlOutputFormat$AsyncSqlExecThread.run(AsyncSqlOutputFormat.java:231) >> >> 14/12/02 22:38:49 INFO mapreduce.Job: Task Id : >> attempt_1415156847621_2303_m_000002_0, Status : FAILED >> Error: java.io.IOException: Can't export data, please check task tracker >> logs >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112) >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) >> at >> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:775) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) >> Caused by: java.io.IOException: >> com.microsoft.sqlserver.jdbc.SQLServerException: The value is not set for >> the parameter number 2. >> at >> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220) >> at >> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46) >> at >> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646) >> at >> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89) >> at >> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112) >> at >> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:84) >> ... 10 more >> >> >> Any help would be appreciated. >> >> Thanks >> >> >> >> >> >> On Tue, Dec 2, 2014 at 2:43 PM, sai chaitanya tirumerla < >> [email protected]> wrote: >> >>> Hi, >>> >>> I am trying to export s3 data into sql server but it keeps on failing >>> with error : >>> >>> "The value is not set for the parameter number 2" >>> >>> Not sure what the error is about everything is specified within the >>> sqoop export command though. >>> >>> The full error stack trace >>> >>> >>> >>> >> >
