In general, you have to increase a JVM's heap size if you're running anything that needs non-trivial memory. I think the default heap size is still 32M or 64M, which is quite small for these purposes. So I am not surprised if you must increase the heap size, in general.
It is still surprising to me that the 1M data set would work with a default heap size but your much smaller data set would not. Does your data set contain a bunch of different item IDs? Because the slope-one diff storage scales with the square of the number of items. But, I doubt you do, and the size of the diff storage is capped anyway. I think there's something else wrong here, like your input is actually not what you think it is, or something else you're doing is consuming a great deal of memory. I would debug and/or profile to see where the memory is used. For example you've not shown where the OutOfMemoryError is thrown. Sean On Tue, Nov 29, 2011 at 2:21 PM, Tan Shern Shiou <shernshiou....@mnc.com.my>wrote: > I am sorry if I didnt make myself clear. > I have 2 datasets here. > 1. Grouplens 1M > 2. My own with 10,000 ratings (very small test data) > > The taste-web running fine with default heapsize. But when I load with my > own dataset (10,000+), it crash after slopone recommendation. I need to > change to 2048M heapsize for it to run. > > It doesnt make sense. What could be the problem? I am afraid if I scale > this to production, it may not run. :( > > > On 29/11/2011 10:15 PM, Sean Owen wrote: > >> This doesn't make sense -- it runs with 1M ratings, but not with 10000? >> are >> you sure you have your numbers right? 10000 is a tiny data set. >> >> Heap size should not be 2048, but something like 2048M. >> >> On Tue, Nov 29, 2011 at 2:03 PM, Tan Shern Shiou >> <shernshiou....@mnc.com.my>**wrote: >> >> Currently I am running into some problem. My taste-web run fined with >>> Grouplens 1M without changing MAVEN_OPS heapsize. >>> >>> However, my datapoints has only 1200++ users with 10000++ ratings can >>> only >>> run if I change the heapsize to 2048. Any suggestion to solve this when I >>> need to scale up my apps. Thanks. >>> >>> >>> > > -- >
