Oh I did not know that the version was HDP. I just thought the set up process was through some HDP links. Thanks.
On Tue, Sep 12, 2017 at 10:44 PM, Ted Yu <yuzhih...@gmail.com> wrote: > Ram: > As Arul indicated, HDP is being used where there're a lot of backports to > 1.1.2 baseline. > > Arul: > I suggest going thru vendor's channel first. > > Cheers > > On Tue, Sep 12, 2017 at 10:11 AM, ramkrishna vasudevan < > ramkrishna.s.vasude...@gmail.com> wrote: > > > Why Master also accepts MaxDirectMemory config is probably because in 1.1 > > onwards we treat HMaster as HregionServer but it does the region > managment > > also. Am not very sure in 1.1.2 is HMaster allowed to host regions ? If > so > > you need to configure MaxDirectMemory, if not probably we can see how we > > can avoid it. We need to raise a JIRA for that. > > > > Coming to the size of MaxDirectMemory less than bucket cache - I am > > wondering was there a bug previously? Because assume you need 25G offheap > > bucket cache then atleast 25G MaxDirectMemory is a must. Ideally you may > > need some delta more than 25G. > > > > 0.98 is obsolete now so its better we go with how 1.1.2 works. But if you > > feel there is a documentation that could help I think it is better we > > provide one so that users like you are not affected. > > > > Regards > > Ram > > > > > > > > > > On Tue, Sep 12, 2017 at 10:22 PM, Arul Ramachandran <arkup...@gmail.com> > > wrote: > > > > > Thank you, Ram. > > > > > > >> So are you trying to use bucket cache feature in offheap mode with > > > 1.1.2? > > > > > > Yes. > > > > > > >> So even in 0.98 you were using bucket cache in offheap mode? > > > > > > Yes, but it is a different hbase cluster and it run 0.98. The one I am > > > trying to setup offheap cache is hbase 1.1.2 > > > > > > > > > -arul > > > > > > > > > On Tue, Sep 12, 2017 at 9:34 AM, ramkrishna vasudevan < > > > ramkrishna.s.vasude...@gmail.com> wrote: > > > > > > > Hi > > > > > > > > So are you trying to use bucket cache feature in offheap mode with > > 1.1.2? > > > > If so then it is needed that the MaxDirectMemory is greater than the > > > > offheap bucket cache size. > > > > > > > > If you are not using in offheap mode then probably there is no need > for > > > > MaxDirectMemory to be greater than bucket cache size. > > > > > > > > >>in Hbase 0.98, I had to set -XX:MaxDirectMemorySize less than > > > > hbase.bucket.cache.size > > > > So even in 0.98 you were using bucket cache in offheap mode? > > > > > > > > Regards > > > > Ram > > > > > > > > On Tue, Sep 12, 2017 at 9:40 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > > > Looks like the config you meant should be hbase.bucketcache.size > > > > > > > > > > As the refguide says: > > > > > > > > > > A float that EITHER represents a percentage of total heap memory > size > > > to > > > > > give to the cache (if < 1.0) OR, it is the total capacity in > > megabytes > > > of > > > > > BucketCache. Default: 0.0 > > > > > > > > > > If you specify the size as capacity, -XX:MaxDirectMemorySize should > > be > > > > > bigger than the capacity. > > > > > > > > > > For #2, did you encounter some error ? > > > > > > > > > > Cheers > > > > > > > > > > On Tue, Sep 12, 2017 at 8:52 AM, Arul Ramachandran < > > arkup...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > In HBase 1.1.2, I am setting up bucket cache. I set > MaxDirectMemory > > > > size > > > > > > greater than hbase.bucket.cache.size - only then it would work. > > > > > > > > > > > > 1) Does HBASE_REGIONSERVER_OPTS -XX:MaxDirectMemorySize needs to > be > > > > > greater > > > > > > than hbase.bucket.cache.size? > > > > > > 2) It seems with hbase 1.1.2, HBASE_MASTER_OPTS also needs the > > > > > > -XX:MaxDirectMemorySize setting? > > > > > > > > > > > > IIRC, in Hbase 0.98, I had to set -XX:MaxDirectMemorySize less > than > > > > > > hbase.bucket.cache.size --and-- I did not have to set > > > > > > -XX:MaxDirectMemorySize for HBASE_MASTER_OPTS. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > Arul > > > > > > > > > > > > > > > > > > > > >
