Hi, is there a way to join data from the JSON metadata file with each csv record? There is no key between csv and JSON data.
for example partial JSON data: (1. column value) Drop (2. column value) Passthru (3. column value) Encrypt CSV records: 1;2;3; 3;2;1; .... CSV joined with JSON data (1, drop), (2, passthru), (3, encrypt) (3, drop), (2, passthru), (1, encrypt) Thanks On Sun, Dec 6, 2015 at 11:47 PM, John Smith <[email protected]> wrote: > Hi, > > I would like to ask you if there is any possibility to write pig flow > using control (metadata file). > > As a source I want to use one json (metadata) file and another csv file > with user data. > > Based on the description insde json file i want to process csv file > > For example: > JSON > ... > fields : [ {name: name, encryption: yes ... }, {...} .... ]} > > ... > CSV > name, age, insurance, .... > John S, 39, yes > > > Pig script reads json file and based on the file it processes CSV eg. if > there is flag for the first field "name" saying encryption: yes it will > somehow encrypt the value inside column Name (csv file). > > > Is it possible to control flow/ of pig script using another metadata file? > > > Thank you! >
