Dear Wiki user, You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The "Pig070IncompatibleChanges" page has been changed by PradeepKamath. http://wiki.apache.org/pig/Pig070IncompatibleChanges?action=diff&rev1=34&rev2=35 -------------------------------------------------- || Change || Section || Impact || Steps to address || Comments || || Load/Store interface changes || Changes to the Load and Store Functions || High || [[LoadStoreMigrationGuide || Load Store Migration Guide]] [[Pig070LoadStoreHowTo || Pig 0.7.0 Load Store Guide]]|| || || Data compression becomes load/store function specific || Handling Compressed Data || Unknown but hopefully low || If compression is needed, the underlying Input/Output format would need to support it || || - || Bzip compressed files in PigStorage format can no longer have .bz extension || Handling Compressed Data || Low || 1. Rename existing .bz files to .bz2 files. 2. Update scripts to read/write files with bz2 extension || This change is due to the fact that Text{Input/Output}Format only supports bz2 extension || || Switching to Hadoop's local mode || Local Mode || Low || None || Main change is 10-20x performance slowdown. Also, local mode now uses the same UDF interfaces to execute UDFs as the MR mode. || || Removing support for Load-Stream or Stream-Store optimization || Streaming || Low to None || None || This feature was never documented so it is unlikely it was ever used || || We no longer support serialization and decerialization via load/store functions || Streaming || Unknown but hopefully low to medium || Implement new PigToStream and StreamToPig interfaces for non-standard serialization || LoadStoreRedesignProposal || @@ -30, +29 @@ With Pig 0.7.0 the read/write functionality is taking over by Hadoop's Input/OutputFormat and how compression is handled or whether it is handled at all depends on the Input/OutputFormat used by the loader/store function. - The main input format that supports compression is TextInputFormat. It supports bzip files with .bz2 extension and gzip files with .gz extension. '''Note that it does not support .bz files'''. PigStorage is the only loader that comes with Pig that is derived from TextInputFormat which means it will be able to handle .bz2 and .gz files. Other loaders such as BinStorage will no longer support compression. + The main input format that supports compression is TextInputFormat. PigStorage is the only loader that comes with Pig that is derived from TextInputFormat which means it will be able to handle .bz2 and .gz files. Other loaders such as BinStorage will no longer support compression. On the store side, TextOutputFormat also supports compression but the store function needs do to additional work to enable it. Again, PigStorage will support compressions while other functions will not.
