Hi, Sheriffo. You can try reusing the Persistent instances [1] to insert the data. I don't know all the backends, but they should be reusable, at least in mongoDB and HBase.
[1] - https://github.com/sneceesay77/gora/blob/GORA-532/gora-benchmark/src/main/java/org/apache/gora/benchmark/GoraBenchmarkClient.java#L130 Regards, Alfonso Nishikawa El lun., 10 jun. 2019 a las 21:14, Alfonso Nishikawa (< alfonso.nishik...@gmail.com>) escribió: > Hi, Sheriffo. > > I really don't know how to solve it, but are you setting any Xmx / Xms > configuration values? > > Regards, > > Alfonso NIshikawa > > > El sáb., 8 jun. 2019 a las 16:02, Sheriffo Ceesay (<sneceesa...@gmail.com>) > escribió: > >> Hi All, >> >> Week 2 progress update is available at >> >> https://cwiki.apache.org/confluence/display/GORA/%5BGORA-532%5D+Apache+Gora+Benchmark+Module+Weekly+Report >> >> I have one question that I would like my mentors to advise on, I am still >> working it but thought it would be good to report it because it is HBase >> specific. >> >> So the problem has to do with an OutOfMemory error when inserting 1M + >> record in HBase. This happens when I try to run the actual benchmark by >> first loading HBase with 1 million plus records. It works perfectly for >> MongoDB but not HBase >> >> So I am assuming this problem is specific to HBase. The stack trace is >> given below. >> >> Exception in thread "Thread-1" java.lang.OutOfMemoryError: GC overhead >> limit exceeded >> >> >> >> at >> java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300) >> >> >> >> at java.lang.StringCoding.encode(StringCoding.java:344) >> >> >> >> >> at java.lang.String.getBytes(String.java:918) >> >> >> >> >> at org.apache.hadoop.hbase.util.Bytes.toBytes(Bytes.java:733) >> >> >> >> >> at >> >> org.apache.gora.hbase.util.HBaseByteInterface.toBytes(HBaseByteInterface.java:225) >> >> >> >> at >> >> org.apache.gora.hbase.store.HBaseStore.addPutsAndDeletes(HBaseStore.java:383) >> >> >> >> at >> >> org.apache.gora.hbase.store.HBaseStore.addPutsAndDeletes(HBaseStore.java:348) >> >> >> >> at org.apache.gora.hbase.store.HBaseStore.put(HBaseStore.java:319) >> >> >> >> >> at org.apache.gora.hbase.store.HBaseStore.put(HBaseStore.java:84) >> >> >> >> >> at >> >> org.apache.gora.benchmark.GoraBenchmarkClient.insert(GoraBenchmarkClient.java:141) >> >> >> >> at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:148) >> >> >> >> >> at >> com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:461) >> >> >> >> at com.yahoo.ycsb.ClientThread.run(Client.java:269) >> >> The insert implementation of the module available at >> https://github.com/sneceesay77/gora/tree/GORA-532/gora-benchmark in >> GoraBenchmarkClient.java is very straight forward. I have had a brief look >> at HBaseStore.java put() implementation but could not find an issue with >> that. >> >> If I solve this problem, then I will do run more workloads to verify that >> the module is stable for the basic implementation. Then I will go ahead >> and >> work on suggestions made by Renato last week. >> >> Please let me know what your thoughts are. >> >> >> Thank you. >> >> >> >> **Sheriffo Ceesay** >> >
