Hi All,
Week 2 progress update is available at
https://cwiki.apache.org/confluence/display/GORA/%5BGORA-532%5D+Apache+Gora+Benchmark+Module+Weekly+Report
I have one question that I would like my mentors to advise on, I am still
working it but thought it would be good to report it because it is HBase
specific.
So the problem has to do with an OutOfMemory error when inserting 1M +
record in HBase. This happens when I try to run the actual benchmark by
first loading HBase with 1 million plus records. It works perfectly for
MongoDB but not HBase
So I am assuming this problem is specific to HBase. The stack trace is
given below.
Exception in thread "Thread-1" java.lang.OutOfMemoryError: GC overhead
limit exceeded
at
java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300)
at java.lang.StringCoding.encode(StringCoding.java:344)
at java.lang.String.getBytes(String.java:918)
at org.apache.hadoop.hbase.util.Bytes.toBytes(Bytes.java:733)
at
org.apache.gora.hbase.util.HBaseByteInterface.toBytes(HBaseByteInterface.java:225)
at
org.apache.gora.hbase.store.HBaseStore.addPutsAndDeletes(HBaseStore.java:383)
at
org.apache.gora.hbase.store.HBaseStore.addPutsAndDeletes(HBaseStore.java:348)
at org.apache.gora.hbase.store.HBaseStore.put(HBaseStore.java:319)
at org.apache.gora.hbase.store.HBaseStore.put(HBaseStore.java:84)
at
org.apache.gora.benchmark.GoraBenchmarkClient.insert(GoraBenchmarkClient.java:141)
at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:148)
at
com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:461)
at com.yahoo.ycsb.ClientThread.run(Client.java:269)
The insert implementation of the module available at
https://github.com/sneceesay77/gora/tree/GORA-532/gora-benchmark in
GoraBenchmarkClient.java is very straight forward. I have had a brief look
at HBaseStore.java put() implementation but could not find an issue with
that.
If I solve this problem, then I will do run more workloads to verify that
the module is stable for the basic implementation. Then I will go ahead and
work on suggestions made by Renato last week.
Please let me know what your thoughts are.
Thank you.
**Sheriffo Ceesay**
