Hive has built-in support for gzipped text files so you don't need to do anything special with them (meaning you just load them like regular files). However, there is one important limitation with this approach these files are not splittable. Depending on the size of your log files and how many of them you have perday that may not be a problem. The other, slightly more complicated approach is to use sequence files, which support splittable compression. You can find more details here: https://cwiki.apache.org/confluence/display/Hive/CompressedStorage
On Mon, Nov 11, 2013 at 5:09 PM, Raj Hadoop <hadoop...@yahoo.com> wrote: > Hi, > > I have a web log files (text format). I want to load these files to a Hive > table in compressed format. How do I do it ? > > Should I compress the text file (using any Linux utilities) and then > create the Hive table? > > Can any one provide me the Hive syntax for loading the compressed file? > > Thanks, > Raj >
