[ 
https://issues.apache.org/jira/browse/HIVE-758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12804938#action_12804938
 ] 

SeanM commented on HIVE-758:
----------------------------

This UDAF works well but I've encountered two gotchyas:

*strong*Nested queries with a where clause that filter out records will throw 
an exception, even if there are no null values in the table whatsoever*strong*
{noformat} 
Hive> SELECT hbase_put("test", rowid, "data", colfamily, value, 0) FROM ( 
SELECT * FROM some_table WHERE value = "some_value") t1;

java.lang.RuntimeException: Error while closing operators
        at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:232)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method 
public boolean 
org.apache.hadoop.hive.contrib.udaf.hbase.UDAFHbasePut$UDAFHbasePutEvaluator.iterate(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,int)
  on object 
org.apache.hadoop.hive.contrib.udaf.hbase.udafhbaseput$udafhbaseputevalua...@70e35d5
 of class 
org.apache.hadoop.hive.contrib.udaf.hbase.UDAFHbasePut$UDAFHbasePutEvaluator 
with arguments {null, null, null, null, null, null} of size 6
        at 
org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:799)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:462)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:470)
        at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:211)
        ... 4 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute 
method public boolean 
org.apache.hadoop.hive.contrib.udaf.hbase.UDAFHbasePut$UDAFHbasePutEvaluator.iterate(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,int)
  on object 
org.apache.hadoop.hive.contrib.udaf.hbase.udafhbaseput$udafhbaseputevalua...@70e35d5
 of class 
org.apache.hadoop.hive.contrib.udaf.hbase.UDAFHbasePut$UDAFHbasePutEvaluator 
with arguments {null, null, null, null, null, null} of size 6
        at 
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:661)
        at 
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.iterate(GenericUDAFBridge.java:167)
        at 
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:110)
        at 
org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:768)
        ... 12 more
Caused by: java.lang.IllegalArgumentException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at 
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:638)
        ... 15 more


It's just a hunch, but it seems like the UDAFs iterate() is being called with 
null values for rows that were filtered out?
{noformat} 


*strong*Null values*strong*
The UDAF is very sensitive to null values. If using mapped or array types, or 
any field that may possibly be null, use an if construct for safety:
{noformat}
if (some_field IS NULL, "", some_field)
{noformat}




> function to load data from hive to hbase
> ----------------------------------------
>
>                 Key: HIVE-758
>                 URL: https://issues.apache.org/jira/browse/HIVE-758
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Contrib
>            Reporter: Raghotham Murthy
>            Priority: Minor
>         Attachments: hive-758.1.patch, hive-758.2.patch
>
>
> supoprt a query like: SELECT hbase_put('hive_hbase_table', rowid, colfamily, 
> col, value, ts) FROM src;

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to