BTW, I am using iterator.remove() to remove elements from FastByIDMap & FastIDSet. Will this be a problem?
I have moved the filtering logic into normal Map and Set. Will test if this problem persists after this change. Weiwei 2009/12/6 Sean Owen <[email protected]> > If he's using the normal class APIs, then shouldn't matter what the > heck is being done to the class. I can't spot a problem that would > fill up the hash table. > > I don't think he's messing with the implementation or data structures > directly? > > But yes I am still quite puzzled, I haven't seen this, can't reproduce > it even with some tests that guess at pathological patterns, and can't > see anything wrong with the logic. I'm stumped on this one. > > On Sat, Dec 5, 2009 at 7:54 PM, Ted Dunning <[email protected]> wrote: > > Is it possible that weiwei is filling the fastMap explicitly? > > > > On Sat, Dec 5, 2009 at 1:31 AM, Sean Owen <[email protected]> wrote: > > > >> That all sounds fine, should be no problem with this usage pattern. I > >> don't think the issue is where you think it is -- The problem seemed > >> to be with the FastMap used to cache recommendations. Is there > >> anything interesting about how this is used? I just added a couple > >> unit tests for the cache to try to reproduce it but didn't see > >> anything. > >> > >> You can remove CachingRecommender to see what happens but that would > >> be a shame since it's just avoiding the problem. If there's an issue > >> I'm keen to track it down. > >> > >> I can't figure out what the issue is. Maybe I can add some asserts to > >> the code that could trigger some signs of a problem. > >> > >> 2009/12/5 施伟伟 <[email protected]>: > >> > Thanks for reply. > >> > It does not happen all the time. I saw this twice and could not easily > >> > reproduce it. > >> > Next time I will try testing it with jpda enabled. > >> > > >> > I am not sure if I am doing something wrong in the data model. > >> > First I fill the FastMap with data read from external data source. > >> > Then I iterate all entries in FastMap and find a set of items that I > >> don't > >> > need. > >> > Finally I iterate the entries again to remove those items from > FastMap. > >> If > >> > the item set of any userID becomes empty, the userID will be removed > from > >> > FastMap too. > >> > > >> > Maybe I should do these outside mahout using ordinary Map? > >> > > >> > Thanks, > >> > Weiwei > >> > > >> > > > > > > > > -- > > Ted Dunning, CTO > > DeepDyve > > >
