Re: Pig storage and load functions and Cache

2012-10-17 Thread Dmitriy Ryaboy
I am not sure I understand the question. You are trying to decide how to store results of your computation? Text (PigStorage, the default) is probably easiest to work with, but there are many other options depending on what you want to do with the data. TextLoader is a loader not a storer. BinStora

Re: Pig storage and load functions and Cache

2012-10-15 Thread abhishek dodda
hi Dmitriy, Thanks for the information. Can you share your views on the below query. BinStorage() PigDump() PigStorage() TextLoader() Load or storing in which of the above format.Will optimize the queries.Considering i have text files. Regards Abhi On Mon, Oct 8, 2012 at 12:10 AM, Dmitriy R

Re: Pig storage and load functions and Cache

2012-10-07 Thread Dmitriy Ryaboy
Pig has multi-query execution optimization built-in. If you compute multiple relations in your script that share parent relations, those parent relations will be computed only once. You don't have to do anything to make that happen. If you prefer to handle your own caching, you would have to handl

Pig storage and load functions and Cache

2012-10-05 Thread Abhishek
BinStorage() PigDump() PigStorage() TextLoader() Load or storing in which of the above format.Will optimize the queries. Can cache be any where in pig.How can the cache be use ful in pig. Regards Abhi