> On 五月 3, 2017, 12:15 p.m., Boglarka Egyed wrote: > > Hi Ying, > > > > Thanks for the updated diff however the manual test case added by you still > > fails with the following: > > > > 2017-05-03 13:53:39,234 (main) [ERROR - > > org.apache.sqoop.tool.ImportTool.run(ImportTool.java:630)] Import failed: > > Cannot convert SQL type 2009 > > 2017-05-03 13:53:39,234 (main) [ERROR - > > org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:185)] Got exception running > > Sqoop: java.lang.IllegalArgumentException: Cannot convert SQL type 2009 > > java.lang.IllegalArgumentException: Cannot convert SQL type 2009 > > at org.apache.sqoop.manager.ConnManager.toAvroType(ConnManager.java:223) > > at org.apache.sqoop.manager.ConnManager.toAvroType(ConnManager.java:277) > > at > > org.apache.sqoop.orm.AvroSchemaGenerator.toAvroType(AvroSchemaGenerator.java:166) > > at > > org.apache.sqoop.orm.AvroSchemaGenerator.toAvroSchema(AvroSchemaGenerator.java:136) > > at > > org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:96) > > at > > org.apache.sqoop.mapreduce.DataDrivenImportJob.generateAvroSchema(DataDrivenImportJob.java:156) > > at > > org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:110) > > at > > org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264) > > at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) > > at org.apache.sqoop.manager.Db2Manager.importTable(Db2Manager.java:72) > > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520) > > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) > > at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > > at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:42) > > at > > com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:228) > > at > > com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:243) > > at > > org.apache.sqoop.manager.db2.DB2XmlTypeParquetImportManualTest.runDb2Test(DB2XmlTypeParquetImportManualTest.java:241) > > at > > org.apache.sqoop.manager.db2.DB2XmlTypeParquetImportManualTest.testParquetDb2Import(DB2XmlTypeParquetImportManualTest.java:212) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > > at > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > > at > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > > at > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > > at > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > > at > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > > at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) > > 2017-05-03 13:53:39,234 (main) [ERROR - > > com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:230)] > > Got exception running Sqoop: java.lang.IllegalArgumentException: Cannot > > convert SQL type 2009 > > java.lang.IllegalArgumentException: Cannot convert SQL type 2009 > > at org.apache.sqoop.manager.ConnManager.toAvroType(ConnManager.java:223) > > at org.apache.sqoop.manager.ConnManager.toAvroType(ConnManager.java:277) > > at > > org.apache.sqoop.orm.AvroSchemaGenerator.toAvroType(AvroSchemaGenerator.java:166) > > at > > org.apache.sqoop.orm.AvroSchemaGenerator.toAvroSchema(AvroSchemaGenerator.java:136) > > at > > org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:96) > > at > > org.apache.sqoop.mapreduce.DataDrivenImportJob.generateAvroSchema(DataDrivenImportJob.java:156) > > at > > org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:110) > > at > > org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264) > > at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) > > at org.apache.sqoop.manager.Db2Manager.importTable(Db2Manager.java:72) > > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520) > > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) > > at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > > at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:42) > > at > > com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:228) > > at > > com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:243) > > at > > org.apache.sqoop.manager.db2.DB2XmlTypeParquetImportManualTest.runDb2Test(DB2XmlTypeParquetImportManualTest.java:241) > > at > > org.apache.sqoop.manager.db2.DB2XmlTypeParquetImportManualTest.testParquetDb2Import(DB2XmlTypeParquetImportManualTest.java:212) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > > at > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > > at > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > > at > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > > at > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > > at > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > > at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) > > 2017-05-03 13:53:39,235 (main) [ERROR - > > org.apache.sqoop.manager.db2.DB2XmlTypeParquetImportManualTest.runDb2Test(DB2XmlTypeParquetImportManualTest.java:244)] > > Got IOException during import: java.io.IOException: Failure during job; > > return status 1 > > java.io.IOException: Failure during job; return status 1 > > at > > com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:237) > > at > > com.cloudera.sqoop.testutil.ImportJobTestCase.runImport(ImportJobTestCase.java:243) > > at > > org.apache.sqoop.manager.db2.DB2XmlTypeParquetImportManualTest.runDb2Test(DB2XmlTypeParquetImportManualTest.java:241) > > at > > org.apache.sqoop.manager.db2.DB2XmlTypeParquetImportManualTest.testParquetDb2Import(DB2XmlTypeParquetImportManualTest.java:212) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > > at > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > > at > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > > at > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > > at > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > > at > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > > at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) > > > > Is there any extra setting which should be made before running maybe? I ran > > the command below: > > > > ant clean test -Dthirdparty=true -Dsqoop.thirdparty.lib.dir=3rdparty > > -Dsqoop.test.db2.connectstring.host_url=connectstring > > -Dsqoop.test.db2.connectstring.database=databasename > > -Dsqoop.test.db2.connectstring.username=username > > -Dsqoop.test.db2.connectstring.password=password > > -Dtestcase=DB2XmlTypeParquetImportManualTest > > > > Thanks, > > Bogi
Hi Bogi, The error seems happened UT run using invalidate import argvs, but I can not reproduce in my environment. I have update an new patch to LOG the import argvs, it will LOG import args under string of "runImport with argv is" - Ying ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58477/#review173726 ----------------------------------------------------------- On 五月 8, 2017, 7:35 a.m., Ying Cao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58477/ > ----------------------------------------------------------- > > (Updated 五月 8, 2017, 7:35 a.m.) > > > Review request for Sqoop, Boglarka Egyed, Attila Szabo, and Szabolcs Vasas. > > > Bugs: SQOOP-3173 > https://issues.apache.org/jira/browse/SQOOP-3173 > > > Repository: sqoop-trunk > > > Description > ------- > > SQOOP-3173 : support DB2 xml data type when sqoop import with parquet > > > Diffs > ----- > > src/java/org/apache/sqoop/manager/ConnManager.java 1811ce0 > > src/test/org/apache/sqoop/manager/db2/DB2XmlTypeParquetImportManualTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/58477/diff/3/ > > > Testing > ------- > > UT passed by manually > > This is third-part test, I ran it by : > > ant clean test -Dthirdparty=true -Dsqoop.thirdparty.lib.dir=3rdparty > -Dsqoop.test.db2.connectstring.host_url=connectstring > -Dsqoop.test.db2.connectstring.database=databasename > -Dsqoop.test.db2.connectstring.username=username > -Dsqoop.test.db2.connectstring.password=password > -Dtestcase=DB2XmlTypeParquetImportManualTest > > > Thanks, > > Ying Cao > >
