AFAIK you cannot use a variable as a map key. What you can do is transform the map to a tuple of (key,value). There must be some readily available UDFs to do that.
prodid_to_class_id = LOAD 'structured/prod_id_to_class_id_map.txt' as (prod_to_class_map: map[]); prodid_class_id = FOREACH prodid_to_class_id GENERATE flatten(MapToBag(prod_to_class_map)) as (key,value); raw_meds_data = JOIN raw_customer_data on prod_id, prodid_class_id on key using 'replicated'; meds_to_gen_c_mapped = FOREACH raw_meds_data GENERATE raw_customer_data::cust_id, prodid_class_id::value as class_id; On Wed, Sep 10, 2014 at 6:39 AM, Sameer Tilak <[email protected]> wrote: > Hi All,I have some issue with using map data structure. Any help will be > great. > Product is too fine-scale so we turn that into classes. I have a file that > has a map stored in the following format: > // Loading the raw data.raw_customer_data = LOAD 'cust_clean.tsv' AS > (cust_id:chararray, session_id:chararray, ordering_date:chararray, prod_id: > chararray); > I want to turn product id to class_id using the below map. Basically given > a customer info with prod_id, I want to turn that into customer info with > class_id.prodid_to_class_id = LOAD 'structured/prod_id_to_class_id_map.txt' > as (prod_to_class_map: map[]); > > prod_id_to_class_id_map.txt has the following format:prod_id#class_id > [524145#1204022AC][526457#1202465BC][2133411#1201978CC][6203089#121537AC] > > meds_to_gen_c_mapped = FOREACH raw_meds_data GENERATE fake_pat_id, > fake_enc_id, prodid_to_class_id:: prod_to_class_map#'prod_id'; > I get error that prodid_to_class_id:: prod_to_class_map does not exist. > Also, prodid_to_class_id:: prod_to_class_map#'prod_id' is this correct way > to deference when I want to use raw_customer_data:: prod_id as the key in > the map. -- Regards, Abhishek Agarwal
