Unxpected exception: Unxpected exception: org.apache.hadoop.io.LongWritable cannot be cast to java.lang.Long

[Dan Fan]

Dear hive users:


Quick question about hive longwritable convert to long.

I have a generic udf called protected_column, which works well as following:


Select protect_column(auction_id_64, ‘auction_id_64’, vp_bitmap) from table ;

And works well when I run

Select * from ( select protect_column(auction_id_64, ‘auction_id_64', 
vp_bitmap) from tableA union all select protect_column(auction_id_64, 
‘auction_id_64', vp_bitmap) from tableB) tmp ;


However, the generic udf would fail if I use it in the join.




SELECT *
FROM   (SELECT Protect_column(auction_id_64, 'auction_id_64', vp_bitmap)
        FROM   view_agg_dw_impressions_tmp t
               LEFT OUTER JOIN table_A pub
                            ON ( pub.id = t.publisher_id )
               LEFT OUTER JOIN table_B inv
                            ON ( inv.id = t.inventory_source_id )
        WHERE  t.buyer_member_id = 1541
               AND dh = '2014-09-19 12') feed;



... 8 more

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unxpected 
exception: Unxpected exception: org.apache.hadoop.io.LongWritable cannot be 
cast to java.lang.Long

at 
org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:234)

at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:847)

at 
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)

at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:847)

at 
org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:136)

at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:847)

at 
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)

at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:847)

at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:519)


It fails if I run the following too.



SELECT Protect_column(auction_id_64, 'auction_id_64', vp_bitmap)
FROM   (SELECT auction_id_64, vp_bitmap
        FROM   view_agg_dw_impressions_tmp t
               LEFT OUTER JOIN table_A pub
                            ON ( pub.id = t.publisher_id )
               LEFT OUTER JOIN table_B inv
                            ON ( inv.id = t.inventory_source_id )
        WHERE  t.buyer_member_id = 1541
               AND dh = '2014-09-19 12') feed;


Any one can help explaining why hive wanna convert longwritable to long and why 
the udf works fine on select query and union all query but not join queries ?


Thanks


Dan