Hello Hive Users,
There is a strange situation I am facing.
I have a string column in my Hive table ( its IP address). I am creating a
UDF where I am taking this string column and converting it into Long value.
Its a simple UDF. Following is my code :
package com.practice.hive.udf;
public class IPtoINT extends UDF {
public static LongWritable execute(Text addr) {
String[] addrArray = addr.toString().split("\\.");
long num = 0;
for (int i=0;i<addrArray.length;i++) {
int power = 3-i;
num += ((Integer.parseInt(addrArray[i])%256 *
Math.pow(256,power)));
}
return new LongWritable(num);
}
}
After creating jar, I am running the following commands:
$ hive
hive > add jar /home/hadoop/Desktop/HiveData/IPtoINT.jar;
hive > create temporary function ip2int as 'com.practice.hive.udf.IPtoINT';
hive > select ip2int(ip1) from sample_data;
But running the above, is giving me the following error:
java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
{"ip1":"1.0.144.36","ip2":16814116,"country":"Thailand","key":null}
at
org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native
Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{"ip1":"1.0.144.36","ip2":16814116,"country":"Thailand","key":null}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:550)
at
org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
execute method public org.apache.hadoop.io.Text
com.musigma.hive.udf.ip2int.evaluate(org.apache.hadoop.io.Text) on object
com.musigma.hive.udf.ip2int@19a4d79 of class com.musigma.hive.udf.ip2int
with arguments {1.0.144.36:org.apache.hadoop.io.Text} of size 1
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:848)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:181)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.evaluate(ExprNodeGenericFuncEvaluator.java:163)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:76)
at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at
org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:531)
... 9 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:824)
... 18 more
Caused by: java.lang.NumberFormatException: For input string: "1"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:441)
at java.lang.Long.<init>(Long.java:702)
at com.musigma.hive.udf.ip2int.evaluate(ip2int.java:11)
... 23 more
If I am running the HIVE UDF like --- select ip2int("102.134.123.1") from
sample_data; Its not giving any error.
Strange thing is, its numberformat exception. I am not able to use the
string into int in my java. Very strange issue.
Can someone please tell me what stupid mistake I am doing ?
Is there any other UDF that does string to int/long coversion ?
Regards,
Praveenesh
