Hi there, We hit a possible issue with Pig (version 0.9.1) and HBaseStorage where we try to LOAD multiple sets of data and UNION them. Here's a simple example that shows the problem:
HBase Data (use hbase shell to create table and add rows): create 'test', {NAME => 'data', VERSIONS => 1} put 'test', '11111', 'data:value', '1' put 'test', '11112', 'data:value', '2' put 'test', '11113', 'data:value', '3' put 'test', '22221', 'data:value', '4' put 'test', '22222', 'data:value', '5' put 'test', '22223', 'data:value', '6' Pig Statements (create file test.pig): load1 = LOAD 'hbase://test' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('data:*','-loadKey -gte 11110 -lte 22220') AS (key:chararray, map:map[]); load2 = LOAD 'hbase://test' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('data:*','-loadKey -gte 22220 -lte 33330') AS (key:chararray, map:map[]); result = UNION load1, load2; dump result; Run Script: pig -x local test.pig Result: (11111,[value#1]) (11112,[value#2]) (11113,[value#3]) (11111,[value#1]) (11112,[value#2]) (11113,[value#3]) The result should be the following: (11111,[value#1]) (11112,[value#2]) (11113,[value#3]) (22221,[value#4]) (22222,[value#5]) (22223,[value#6]) If we dump load1 or load2 we see the results we expect, but when the UNION is performed, it does not put the expected data together. Is this a known issue with Pig/HBaseStorage or are we not using them as we should? If it's a usage problem, what would be the proper way of loading multiple sets of data and union them? Thanks in advance. Eduardo.