Hi,
is there a way to join data from the JSON metadata file with each csv
record? There is no key between csv and JSON data.
for example partial JSON data:
(1. column value) Drop
(2. column value) Passthru
(3. column value) Encrypt
CSV records:
1;2;3;
3;2;1;
....
CSV joined with JSON data
(1, drop), (2, passthru), (3, encrypt)
(3, drop), (2, passthru), (1, encrypt)
Thanks
On Sun, Dec 6, 2015 at 11:47 PM, John Smith <lenov...@gmail.com> wrote:
> Hi,
>
> I would like to ask you if there is any possibility to write pig flow
> using control (metadata file).
>
> As a source I want to use one json (metadata) file and another csv file
> with user data.
>
> Based on the description insde json file i want to process csv file
>
> For example:
> JSON
> ...
> fields : [ {name: name, encryption: yes ... }, {...} .... ]}
>
> ...
> CSV
> name, age, insurance, ....
> John S, 39, yes
>
>
> Pig script reads json file and based on the file it processes CSV eg. if
> there is flag for the first field "name" saying encryption: yes it will
> somehow encrypt the value inside column Name (csv file).
>
>
> Is it possible to control flow/ of pig script using another metadata file?
>
>
> Thank you!
>
