Hi Adam,
https://hbase.apache.org/book.html#data.block.encoding.types contains a
detailed description of compression in HBase. The final solution for us was to
configure the snappy codec in hbase-site.xml:
<property>
<name>hbase.io.compress.snappy.codec</name>
<value>org.apache.hadoop.hbase.io.compress.aircompressor.SnappyCodec</value>
</property>
It seems to me that Hadoop completely covers this internally itself. Apart from
the fact that there are no more native libraries in hadoop/lib/native, we have
not changed anything in the Hadoop configuration.
Best regards
Udo
> Am 02.05.2024 um 16:38 schrieb Adam Sjøgren <[email protected]>:
>
> Udo writes:
>
>> Then I saw that the region servers had problems with Snappy
>> compression. I'm not sure, but I believe the native Snappy libs were
>> part of the previous Hadoop distribution, at least they are not
>> included in the current one. After copying them over it seems to work
>> now. But what is the recommended way to enable snappy compression in
>> Hbase now?
>
> I am very interested in this as well, as I have been having problems
> upgrading Hadoop past 3.2.4 with HBase due to snappy compression not
> working¹.
>
> How/what did you copy over?
>
>
> Best regards,
>
> Adam
>
>
> ¹ https://lists.apache.org/thread/jpm5m6wv8odg69worhdh187h1hy2vr9p
>
> --
> "Ni kan skratta om ni vill Adam Sjøgren
> Håna oss, vi rör oss, ni står still" [email protected]
>
