Dear Wiki user, You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The "LoadStoreMigrationGuide" page has been changed by PradeepKamath. http://wiki.apache.org/pig/LoadStoreMigrationGuide?action=diff&rev1=34&rev2=35 -------------------------------------------------- The methods in the old !LoadFunc have been split among a !LoadFunc abstract class which has the main methods for loading data and 3 new interfaces * !LoadMetadata has methods to deal with metadata - most implementation of loaders don't need to implement this unless they interact with some metadata system. The getSchema() method in this interface provides a way for loader implementations to communicate the schema of the data back to pig. The other methods are concerned with other types of metadata like partition keys and statistics - * !LoadPushDown has methods to push operations from pig runtime into loader implementations - currently only projections .i.e the pushProjection() method is called by Pig to communicate to the loader what exact fields are required in the pig script. The implementation can chose to honor the request or respond that it will not honor the request and return all fields in the data + * !LoadPushDown has methods to push operations from pig runtime into loader implementations - currently only projections .i.e the pushProjection() method is called by Pig to communicate to the loader what exact fields are required in the pig script. The implementation can chose to honor the request or respond that it will not honor the request and return all fields in the data. (Irrespective of whether the implementation can or cannot return only the required fields, if the implementation also implements getSchema(), the schema returned in getSchema() should be for the entire tuple of data.) * !LoadCaster has methods to convert byte arrays to specific types. A loader implementation should implement this if casts (implicit or explicit) from !DataByteArray fields to other types need to be supported. The main change is that the new !LoadFunc API is based on a !InputFormat to read the data. Implementations can choose to use existing !InputFormats like !TextInputFormat or implement a new one.
