?????? Using snappy compresscodec in hive
Hi Gopal, Thanks for your reply! One more question, does the effect of using pure-java version is the same as that of using SnappyCodec? Or, in other words, is there any difference between these two methods, about the compression result and effect? Looking forward to your reply and help. Best, Zhefu Peng -- -- ??: "Gopal Vijayaraghavan"; : 2018??7??24??(??) 10:53 ??: "user@hive.apache.org"; ????: Re: Using snappy compresscodec in hive > "TBLPROPERTIES ("orc.compress"="Snappy"); " That doesn't use the Hadoop SnappyCodec, but uses a pure-java version (which is slower, but always works). The Hadoop snappyCodec needs libsnappy installed on all hosts. Cheers, Gopal
?????? Using snappy compresscodec in hive
Hi Gopal, Thanks for your reply! One more question, does the effect of using pure-java version is the same as that of using SnappyCodec? Or, in other words, is there any difference between these two methods, about the compression result and effect? Looking forward to your reply and help. Best, Zhefu Peng -- -- ??: "Gopal Vijayaraghavan"; : 2018??7??24??(??) 10:53 ??: "user@hive.apache.org"; ????: Re: Using snappy compresscodec in hive > "TBLPROPERTIES ("orc.compress"="Snappy"); " That doesn't use the Hadoop SnappyCodec, but uses a pure-java version (which is slower, but always works). The Hadoop snappyCodec needs libsnappy installed on all hosts. Cheers, Gopal
Re: Using snappy compresscodec in hive
> "TBLPROPERTIES ("orc.compress"="Snappy"); " That doesn't use the Hadoop SnappyCodec, but uses a pure-java version (which is slower, but always works). The Hadoop snappyCodec needs libsnappy installed on all hosts. Cheers, Gopal
Using snappy compresscodec in hive
Hi, Here is a confusion I encountered these days: I don't install or build snappy on my hadoop cluster, but when I tested and compared about the compression ratio of Parquet and ORC storage format. During the test, I can set the way of compression for two storage format, for example, using "TBLPROPERTIES ("orc.compress"="Snappy"); " or "set parquet.compression=snappy;", both these commands would work. However, when I just want to compress the textfile format with snappy compression, it says that "can not find or access the snappy library". I wonder why this situation happen, and, I really doubt that whether the ORC or Parquet file using "Snappy" compression. But, the storage really becomes smaller, and diff from the "gzip" or "zlib" compression. Looking forward to your reply and help. Best, Zhefu Peng
Using snappy compresscodec in hive
Hi, Here is a confusion I encounter these days: I don't install or build snappy on my hadoop cluster, but when I tested and compared about the compression ratio of Parquet and ORC storage format. During the test, I can set the way of compression for two storage format, for example, using "TBLPROPERTIES ("orc.compress"="Snappy"); " or "set parquet.compression=snappy;", both these commands would work. However, when I just want to compress the textfile format with snappy compression, it says that "can not find or access the snappy library". I wonder why this situation happen, and, I really doubt that whether the ORC or Parquet file using "Snappy" compression. But, the storage really becomes smaller, and diff from the "gzip" or "zlib" compression. Looking forward to your reply and help. Best, Zhefu Peng
Using snappy compresscodec in hive
Hi, Here is a confusion I encounter these days: I don't install or build snappy on my hadoop cluster, but when I tested and compared about the compression ratio of Parquet and ORC storage format. During the test, I can set the way of compression for two storage format, for example, using "TBLPROPERTIES ("orc.compress"="Snappy"); " or "set parquet.compression=snappy;", both these commands would work. However, when I just want to compress the textfile format with snappy compression, it says that "can not find or access the snappy library". I wonder why this situation happen, and, I really doubt that whether the ORC or Parquet file using "Snappy" compression. But, the storage really becomes smaller, and diff from the "gzip" or "zlib" compression. Looking forward to your reply and help. Best, Zhefu Peng