[
https://issues.apache.org/jira/browse/GORA-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485415#comment-13485415
]
raf shin commented on GORA-173:
-------------------------------
The problem resolved. The problem was not related to Gora HBaseStore. It was
related to our own code.
Two writer threads removed the same object from a Queue because the remove
operation was not synchronized.
Thread1 HBaseStore.put(key, aWebPage)
Thread2 HBaseStore.put(key, theSameWebPagePointer)
so it caused a ConcurrentModificationException.
We synchronized removing from the queue and it resolved the issue. So i think
the issue should be closed.
> Iterating over StatefulHashMap in HBaseStore throws
> ConcurrentModificationException
> -----------------------------------------------------------------------------------
>
> Key: GORA-173
> URL: https://issues.apache.org/jira/browse/GORA-173
> Project: Apache Gora
> Issue Type: Bug
> Affects Versions: 0.1.1-incubating
> Reporter: raf shin
>
> We are using just one thread working with HBaseStore in Nutch.
> HBaseStore.put throws the following exception. I think the same problem was
> in CassandraStore that is
> solved now. But the problem in HBaseStore remains. We used the same solution
> for HBaseStore but
> it didn't work.
> java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
> at java.util.HashMap$EntryIterator.next(HashMap.java:834)
> at java.util.HashMap$EntryIterator.next(HashMap.java:832)
> at org.apache.gora.hbase.store.HBaseStore.put(HBaseStore.java:267)
> at
> org.nise.hiva.fetcher.FetcherReducer$WriteThread.run(FetcherReducer.java:1557)
> Line 266 in HBaseStore:
> StatefulHashMap<Utf8, ?> map = (StatefulHashMap<Utf8, ?>) o;
> for (Entry<Utf8, State> e : map.states().entrySet()) {
> Utf8 mapKey = e.getKey();
> switch (e.getValue()) {
> case DIRTY:
> byte[] qual = Bytes.toBytes(mapKey.toString());
> byte[] val = toBytes(map.get(mapKey),
> field.schema().getValueType());
> put.add(hcol.getFamily(), qual, val);
> hasPuts = true;
> break;
> case DELETED:
> qual = Bytes.toBytes(mapKey.toString());
> hasDeletes = true;
> delete.deleteColumn(hcol.getFamily(), qual);
> break;
> }
> ***************************************************************************
> Our solution was :
> Set<Utf8> keys = map.keySet();
> Utf8 [] arr = (Utf8[])keys.toArray();
> for(Utf8 key: arr){....}
> But it didn't work and we throw the same exception.
> Our version is 0.1 but i checked the trunk and did't see much change in that
> code so the trunk code
> may throw the same exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
