Thanks Richa. The issue was suppose to be fixed in Hive 0.12 version as per the jira https://issues.apache.org/jira/browse/HIVE-7164.
Even raised a ticket in sqoop jira [SQOOP-2840] for this . Thanks Biswa On Tue, Mar 1, 2016 at 9:56 AM, Richa Sharma <mailtorichasha...@gmail.com> wrote: > Hi, > > The values should still persist if partition column data type in Hive is a > string. > > I am checking HCatalog documentation for support of int data type in > partition column. > > Cheers > Richa > > On Tue, Mar 1, 2016 at 3:06 PM, Biswajit Nayak <biswa...@altiscale.com> > wrote: > >> Hi Richa, >> >> Thats a work around. But how to handle the columns with INT type. >> Changing the type will be the last option for me. >> >> Regards >> Biswa >> >> >> >> On Tue, Mar 1, 2016 at 9:31 AM, Richa Sharma <mailtorichasha...@gmail.com >> > wrote: >> >>> Hi Biswajit >>> >>> The answer is in the last line of the error message. Change the data >>> type of partition column to string in hive and try again. >>> >>> Hope it helps ! >>> >>> Richa >>> >>> 16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException running >>> export job: java.io.IOException: The table provided default.emp_details1 >>> uses unsupported partitioning key type for column salary : int. Only >>> string fields are allowed in partition columns in Catalog >>> >>> >>> On Tue, Mar 1, 2016 at 2:19 PM, Biswajit Nayak <biswa...@altiscale.com> >>> wrote: >>> >>>> Hi All, >>>> >>>> I am trying to do a SQOOP export from hive( integer type partition) to >>>> mysql through HCAT and it fails with the following error. >>>> >>>> Versions:- >>>> >>>> Hadoop :- 2.7.1 >>>> Hive :- 1.2.0 >>>> Sqoop :- 1.4.5 >>>> >>>> Table in Hive :- >>>> >>>> >>>> hive> use default; >>>> OK >>>> Time taken: 0.028 seconds >>>> hive> describe emp_details1; >>>> OK >>>> id int >>>> name string >>>> deg string >>>> dept string >>>> salary int >>>> >>>> # Partition Information >>>> # col_name data_type comment >>>> >>>> salary int >>>> Time taken: 0.125 seconds, Fetched: 10 row(s) >>>> hive> >>>> >>>> hive> select * from emp_details1; >>>> OK >>>> 1201 gopal 50000 >>>> 1202 manisha 50000 >>>> 1203 kalil 50000 >>>> 1204 prasanth 50000 >>>> 1205 kranthi 50000 >>>> 1206 satish 50000 >>>> Time taken: 0.195 seconds, Fetched: 6 row(s) >>>> hive> >>>> >>>> >>>> Conf added to Hive metastore site.xml >>>> >>>> >>>> [alti-test-01@hdpnightly271-ci-91-services ~]$ grep -A5 -B2 -i >>>> "hive.metastore.integral.jdo.pushdown" /etc/hive-metastore/hive-site.xml >>>> </property> >>>> <property> >>>> <name>hive.metastore.integral.jdo.pushdown</name> >>>> <value>TRUE</value> >>>> </property> >>>> >>>> </configuration> >>>> [alti-test-01@hdpnightly271-ci-91-services ~]$ >>>> >>>> >>>> The issue remains same >>>> >>>> >>>> [alti-test-01@hdpnightly271-ci-91-services ~]$ /opt/sqoop-1.4.5/bin/sqoop >>>> export --connect jdbc:mysql://localhost:3306/test --username hive >>>> --password ********* --table employee --hcatalog-database default >>>> --hcatalog-table emp_details1 >>>> Warning: /opt/sqoop-1.4.5/bin/../../hbase does not exist! HBase imports >>>> will fail. >>>> Please set $HBASE_HOME to the root of your HBase installation. >>>> Warning: /opt/sqoop-1.4.5/bin/../../accumulo does not exist! Accumulo >>>> imports will fail. >>>> Please set $ACCUMULO_HOME to the root of your Accumulo installation. >>>> Warning: /opt/sqoop-1.4.5/bin/../../zookeeper does not exist! Accumulo >>>> imports will fail. >>>> Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation. >>>> 16/02/12 08:04:00 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5 >>>> 16/02/12 08:04:00 WARN tool.BaseSqoopTool: Setting your password on the >>>> command-line is insecure. Consider using -P instead. >>>> 16/02/12 08:04:00 INFO manager.MySQLManager: Preparing to use a MySQL >>>> streaming resultset. >>>> 16/02/12 08:04:00 INFO tool.CodeGenTool: Beginning code generation >>>> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: SELECT >>>> t.* FROM `employee` AS t LIMIT 1 >>>> 16/02/12 08:04:01 INFO manager.SqlManager: Executing SQL statement: SELECT >>>> t.* FROM `employee` AS t LIMIT 1 >>>> 16/02/12 08:04:01 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is >>>> /opt/hadoop >>>> Note: >>>> /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.java >>>> uses or overrides a deprecated API. >>>> Note: Recompile with -Xlint:deprecation for details. >>>> 16/02/12 08:04:07 INFO orm.CompilationManager: Writing jar file: >>>> /tmp/sqoop-alti-test-01/compile/1b0d4b1c30f167eb57ef488232ab49c8/employee.jar >>>> 16/02/12 08:04:07 INFO mapreduce.ExportJobBase: Beginning export of >>>> employee >>>> 16/02/12 08:04:08 INFO mapreduce.ExportJobBase: Configuring HCatalog for >>>> export job >>>> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Configuring HCatalog >>>> specific details for job >>>> 16/02/12 08:04:08 INFO manager.SqlManager: Executing SQL statement: SELECT >>>> t.* FROM `employee` AS t LIMIT 1 >>>> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column names >>>> projected : [id, name, deg, salary, dept] >>>> 16/02/12 08:04:08 INFO hcat.SqoopHCatUtilities: Database column name - >>>> info map : >>>> id : [Type : 4,Precision : 11,Scale : 0] >>>> name : [Type : 12,Precision : 20,Scale : 0] >>>> deg : [Type : 12,Precision : 20,Scale : 0] >>>> salary : [Type : 4,Precision : 11,Scale : 0] >>>> dept : [Type : 12,Precision : 10,Scale : 0] >>>> >>>> 16/02/12 08:04:10 INFO hive.metastore: Trying to connect to metastore with >>>> URI thrift://hive-hdpnightly271-ci-91.test.altiscale.com:9083 >>>> 16/02/12 08:04:10 INFO hive.metastore: Connected to metastore. >>>> 16/02/12 08:04:11 INFO hcat.SqoopHCatUtilities: HCatalog full table schema >>>> fields = [id, name, deg, dept, salary] >>>> 16/02/12 08:04:12 ERROR tool.ExportTool: Encountered IOException running >>>> export job: java.io.IOException: The table provided default.emp_details1 >>>> uses unsupported partitioning key type for column salary : int. Only >>>> string fields are allowed in partition columns in Catalog >>>> >>>> >>>> Stuck with this issue. Any one had conquered this before. >>>> >>>> Regards >>>> Biswa >>>> >>>> >>> >> >
