I wrote a simple shell script to generate the individual pig scripts and a
control script that executed them when it found a pig script and file to
load... Worked okay.
On Dec 6, 2015 5:47 PM, "John Smith" <[email protected]> wrote:
> Hi,
>
> I would like to ask you if there is any possibility to write pig flow using
> control (metadata file).
>
> As a source I want to use one json (metadata) file and another csv file
> with user data.
>
> Based on the description insde json file i want to process csv file
>
> For example:
> JSON
> ...
> fields : [ {name: name, encryption: yes ... }, {...} .... ]}
>
> ...
> CSV
> name, age, insurance, ....
> John S, 39, yes
>
>
> Pig script reads json file and based on the file it processes CSV eg. if
> there is flag for the first field "name" saying encryption: yes it will
> somehow encrypt the value inside column Name (csv file).
>
>
> Is it possible to control flow/ of pig script using another metadata file?
>
>
> Thank you!
>
