Eric, Priya - FYI, https://issues.apache.org/jira/browse/SQOOP-450 will fix this issue.
Regards, Kathleen On Tue, Nov 8, 2011 at 5:15 PM, Priya <[email protected]> wrote: > I have the same issue using Sqoop 1.3.0-cdh3u1 > The import works, however I see some junk characters at the end of > each part imported, like below: > cter_set_client */ > cter_set_results */ > tion_connection */ > > > neral_ci */ > ode */ > is there a solution? > If I exclude the --direct option, data import is as expected. > Thanks! > > On Sep 19, 5:15 pm, Kate Ting <[email protected]> wrote: > > [To: [email protected]; Bcc: [email protected]] > > > > Eric - to rule out permission issues andmysqldumpissues: > > > > 1. Does username 'sqoop' have permission to access table1? > > 2. Can you ssh into one of the node machines, runmysqldump, and then > > paste the output? > > > > Regards, Kate > > > > > > > > > > > > > > > > On Sat, Sep 17, 2011 at 1:05 PM, eric hernandez <[email protected]> > wrote: > > > Kate, > > > I am sqooping from a MySQL Server version: 5.0.58-enterprise-gpl MySQL > > > Enterprise Server > > > > > Currently i am using on all data nodes > > >mysqldump Ver 10.13 Distrib 5.5.15, for Linux (x86_64) > > > > > But I tried also using > > > mysql-5.0.77-4.el5_6.6.x86_64 mysqldumpfrom the centos 5.5 repo. > > > > > Both fail. > > > > > -Eric > > > > > On Sat, Sep 17, 2011 at 11:43 AM, Kate Ting <[email protected]> wrote: > > > > >> Eric, what is your mysql version and yourmysqldumpversion? > > > > >> Regards, Kate > > > > >> On Fri, Sep 16, 2011 at 7:30 PM, eric hernandez < > [email protected]> > > >> wrote: > > >> > Kate, > > > > >> > sqoop import --connect jdbc:mysql://192.168.0.100:3307/test--verbose -m > > >> > 1 > > >> > --username sqoop --password sanitized --hive-overwrite --direct > --table > > >> > table1 --hive-import --create-hive-table --hive-table table1 > > >> > --fields-terminated-by '\t' --lines-terminated-by '\n' --append > > > > >> > Please note this works fine if i remove the --direct option. I am > also > > >> > limiting it to 1 mapper because if not the output of the failure is > very > > >> > long. > > > > >> > On Fri, Sep 16, 2011 at 3:01 PM, Kate Ting <[email protected]> > wrote: > > > > >> >> Eric - what is the exact Sqoop command that you ran (including, if > > >> >> applicable, contents of the options-file)? > > > > >> >> Regards, Kate > > > > >> >> On Fri, Sep 16, 2011 at 9:27 AM, eric hernandez > > >> >> <[email protected]> > > >> >> wrote: > > >> >> > Yes i havemysqldumpon all nodes. > > > > >> >> > Verbose output > > > > >> >> > 11/09/16 09:22:42 DEBUG tool.BaseSqoopTool: Enabled debug > logging. > > >> >> > 11/09/16 09:22:42 WARN tool.BaseSqoopTool: Setting your password > on > > >> >> > the > > >> >> > command-line is insecure. Consider using -P instead. > > >> >> > 11/09/16 09:22:42 DEBUG sqoop.ConnFactory: Loaded manager > factory: > > >> >> > com.cloudera.sqoop.manager.DefaultManagerFactory > > >> >> > 11/09/16 09:22:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory: > > >> >> > com.cloudera.sqoop.manager.DefaultManagerFactory > > >> >> > 11/09/16 09:22:42 DEBUG manager.DefaultManagerFactory: Trying > with > > >> >> > scheme: > > >> >> > jdbc:mysql: > > >> >> > 11/09/16 09:22:42 INFO manager.MySQLManager: Preparing to use a > MySQL > > >> >> > streaming resultset. > > >> >> > 11/09/16 09:22:42 DEBUG sqoop.ConnFactory: Instantiated > ConnManager > > >> >> > com.cloudera.sqoop.manager.DirectMySQLManager@7ad81784 > > >> >> > 11/09/16 09:22:42 INFO tool.CodeGenTool: Beginning code > generation > > >> >> > 11/09/16 09:22:42 DEBUG manager.SqlManager: No connection > paramenters > > >> >> > specified. Using regular API for making connection. > > >> >> > 11/09/16 09:22:43 DEBUG manager.SqlManager: Using fetchSize for > next > > >> >> > query: > > >> >> > -2147483648 > > >> >> > 11/09/16 09:22:43 INFO manager.SqlManager: Executing SQL > statement: > > >> >> > SELECT > > >> >> > t.* FROM `table1` AS t LIMIT 1 > > >> >> > 11/09/16 09:22:43 DEBUG manager.SqlManager: Using fetchSize for > next > > >> >> > query: > > >> >> > -2147483648 > > >> >> > 11/09/16 09:22:43 INFO manager.SqlManager: Executing SQL > statement: > > >> >> > SELECT > > >> >> > t.* FROM `table1` AS t LIMIT 1 > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: selected columns: > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: id > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: application_id > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: event_id > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: response_id > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: target_id > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: mode > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: date_created > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: Writing source file: > > >> >> > > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b/table1.java > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: Table name: table1 > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: Columns: id:4, > > >> >> > application_id:4, > > >> >> > event_id:4, response_id:4, target_id:4, mode:1, date_created:93, > > >> >> > 11/09/16 09:22:43 DEBUG orm.ClassWriter: sourceFilename is > > >> >> > table1.java > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: Found existing > > >> >> > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b/ > > >> >> > 11/09/16 09:22:43 INFO orm.CompilationManager: HADOOP_HOME is > > >> >> > /usr/lib/hadoop > > >> >> > 11/09/16 09:22:43 INFO orm.CompilationManager: Found hadoop core > jar > > >> >> > at: > > >> >> > /usr/lib/hadoop/hadoop-0.20.2-cdh3u1-core.jar > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: Adding source > file: > > >> >> > > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b/table1.java > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: Invoking javac > with > > >> >> > args: > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: -sourcepath > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: > > >> >> > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b/ > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: -d > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: > > >> >> > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b/ > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: -classpath > > >> >> > 11/09/16 09:22:43 DEBUG orm.CompilationManager: > > > > >> >> > > /etc/hadoop/conf:/usr/java/jdk1.6.0_21/lib/tools.jar:/usr/lib/hadoop:/usr/lib/hadoop/hadoop-core-0.20.2-cdh3u1.jar:/usr/lib/hadoop/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.0.1.jar:/usr/lib/hadoop/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop/lib/commons-net-1.4.1.jar:/usr/lib/hadoop/lib/core-3.1.1.jar:/usr/lib/hadoop/lib/hadoop-fairscheduler-0.20.2-cdh3u1.jar:/usr/lib/hadoop/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop/lib/hue-plugins-1.2.0-cdh3u1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.5.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.5.2.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/jetty-6.1.26.jar:/usr/lib/hadoop/lib/jetty-servlet-tester-6.1.26.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/junit-4.5.jar:/usr/lib/hadoop/lib/kfs-0.2.2.jar:/usr/lib/hadoop/lib/log4j-1.2.15.jar:/usr/lib/hadoop/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop/lib/oro-2.0.8.jar:/usr/lib/hadoop/lib/servlet-api-2.5-20081211.jar:/usr/lib/hadoop/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop/lib/jsp-2.1/jsp-api-2.1.jar:/usr/lib/sqoop/conf:/etc/zookeeper::/usr/lib/sqoop/lib/ant-contrib-1.0b3.jar:/usr/lib/sqoop/lib/ant-eclipse-1.0-jvm1.2.jar:/usr/lib/sqoop/lib/avro-1.5.1.jar:/usr/lib/sqoop/lib/avro-ipc-1.5.1.jar:/usr/lib/sqoop/lib/avro-mapred-1.5.1.jar:/usr/lib/sqoop/lib/commons-io-1.4.jar:/usr/lib/sqoop/lib/ivy-2.0.0-rc2.jar:/usr/lib/sqoop/lib/jackson-core-asl-1.7.3.jar:/usr/lib/sqoop/lib/jackson-mapper-asl-1.7.3.jar:/usr/lib/sqoop/lib/jopt-simple-3.2.jar:/usr/lib/sqoop/lib/mysql-connector-java-5.1.15-bin.jar:/usr/lib/sqoop/lib/paranamer-2.3.jar:/usr/lib/sqoop/lib/snappy-java-1.0.3-rc2.jar:/usr/lib/sqoop/sqoop-1.3.0-cdh3u1.jar:/usr/lib/sqoop/sqoop-test-1.3.0-cdh3u1.jar::/usr/lib/hadoop/hadoop-0.20.2-cdh3u1-core.jar:/usr/lib/sqoop/sqoop-1.3.0-cdh3u1.jar > > >> >> > 11/09/16 09:22:44 INFO orm.CompilationManager: Writing jar file: > > >> >> > > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b/table1.jar > > >> >> > 11/09/16 09:22:44 DEBUG orm.CompilationManager: Scanning for > .class > > >> >> > files in > > >> >> > directory: > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b > > >> >> > 11/09/16 09:22:44 DEBUG orm.CompilationManager: Got classfile: > > >> >> > > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b/table1.class > > >> >> > -> > > >> >> > table1.class > > >> >> > 11/09/16 09:22:44 DEBUG orm.CompilationManager: Finished writing > jar > > >> >> > file > > >> >> > > /tmp/sqoop-root/compile/4da62fc9c254b3faa3ba5115ef61783b/table1.jar > > >> >> > 11/09/16 09:22:44 DEBUG tool.ImportTool: Using temporary folder: > > >> >> > 16092244516table1 > > >> >> > 11/09/16 09:22:44 INFO manager.DirectMySQLManager: Beginning > > >> >> >mysqldump > > >> >> > fast > > >> >> > path import > > >> >> > 11/09/16 09:22:44 INFO mapreduce.ImportJobBase: Beginning import > of > > >> >> > table1 > > >> >> > 11/09/16 09:22:44 DEBUG manager.SqlManager: Using fetchSize for > next > > >> >> > query: > > >> >> > -2147483648 > > >> >> > 11/09/16 09:22:44 INFO manager.SqlManager: Executing SQL > statement: > > >> >> > SELECT > > >> >> > t.* FROM `table1` AS t LIMIT 1 > > >> >> > 11/09/16 09:22:44 DEBUG mapreduce.MySQLDumpImportJob: Using > > >> >> > InputFormat: > > >> >> > class com.cloudera.sqoop.mapreduce.MySQLDumpInputFormat > > >> >> > 11/09/16 09:22:44 DEBUG mapreduce.JobBase: Adding to job > classpath: > > >> >> > file:/usr/lib/sqoop/sqoop-1.3.0-cdh3u1.jar > > >> >> > 11/09/16 09:22:44 DEBUG mapreduce.JobBase: Adding to job > classpath: > > >> >> > file:/usr/lib/sqoop/lib/mysql-connector-java-5.1.15-bin.jar > > >> >> > 11/09/16 09:22:44 DEBUG mapreduce.JobBase: Adding to job > classpath: > > >> >> > file:/usr/lib/sqoop/sqoop-1.3.0-cdh3u1.jar > > >> >> > 11/09/16 09:22:44 DEBUG mapreduce.JobBase: Adding to job > classpath: > > >> >> > file:/usr/lib/sqoop/sqoop-1.3.0-cdh3u1.jar > > >> >> > 11/09/16 09:22:44 DEBUG mapreduce.JobBase: Adding to job > classpath: > > >> >> > file:/usr/lib/sqoop/lib/jackson-mapper-asl-1.7.3.jar > > >> >> > 11/09/16 09:22:44 DEBUG > > > > ... > > > > read more ยป > > -- > NOTE: The mailing list [email protected] is deprecated in favor of > Apache Sqoop mailing list [email protected]. Please > subscribe to it by sending an email to > [email protected]. >
