I tried this as you recommended, sadly, it doesn't do better. :(
On Wednesday, November 5, 2014 4:48:10 PM UTC+8, Amit Murthy wrote: > > Could you try: > > @everywhere gc() > @everywhere Base.flush_gc_msgs() > @everywhere gc() > > > > On Wed, Nov 5, 2014 at 2:07 PM, Seth Yuan <seth...@gmail.com <javascript:> > > wrote: > >> Hi Amit, >> >> Thanks for the response, I tried Base.flush_gc_msgs(), the result is >> shown below: >> >> >> <https://lh5.googleusercontent.com/-bEVLUYu-DPk/VFnhFUh28CI/AAAAAAAABk0/WNl3ukSetUk/s1600/mem.png> >> It did free some memory on some workers, but not quite. I think the >> acceptable number for workers would be around 100MB (like the master is). >> >> So what's wrong with DArrays? >> >> >> On Wednesday, November 5, 2014 2:32:42 PM UTC+8, Amit Murthy wrote: >>> >>> Distributed objects are currently gc'ed asynchronously. >>> >>> You may force this using an internal function flush_gc_msgs like this: >>> >>> @everywhere Base.flush_gc_msgs() >>> @everywhere gc() >>> >>> >>> The above is an internal function and is not meant for general use. It >>> will be great if you could open an issue on github requesting a feature to >>> force gc of distributed objects. >>> >>> >>> >>> On Wed, Nov 5, 2014 at 9:01 AM, Seth Yuan <seth...@gmail.com> wrote: >>> >>>> I did a test on deallocations, and the behavior is not what I expected. >>>> >>>> The code I ran is: >>>> >>>> @everywhere using Ops >>>> >>>> function test_allocation() >>>> a = drand(10000, 10000) >>>> b = 2a >>>> readline() >>>> end >>>> >>>> test_allocation() >>>> @everywhere gc() >>>> readline() >>>> >>>> Ops module defined the '*' operator (where a new DArray is created). I >>>> can see that master's memory went down after the gc() call, but the >>>> workers' memory remained unchanged. >>>> >>>> >>>> <https://lh5.googleusercontent.com/-qo7SKwkoya0/VFmZrIWL0EI/AAAAAAAABkk/FctwkEcQWnk/s1600/mem.png> >>>> So the question is: "how to really free a DArray? Both master and >>>> workers!" >>>> >>>> >>>> On Thursday, October 16, 2014 10:18:42 PM UTC+8, Amit Murthy wrote: >>>>> >>>>> meant to say "all references from all processes are removed" >>>>> >>>>> On Thu, Oct 16, 2014 at 7:47 PM, Amit Murthy <amit....@gmail.com> >>>>> wrote: >>>>> >>>>>> Yes, it is gc'ed just like any other object, when all references to >>>>>> it from any process is removed. >>>>>> >>>>>> On Thu, Oct 16, 2014 at 2:59 PM, Seth Yuan <seth...@gmail.com> wrote: >>>>>> >>>>>>> Now, I know by reading the documentation how a DArray is created, >>>>>>> but I'm not sure when a DArray is deallocated. >>>>>>> >>>>>>> Is it GC able when the master looses all references of it? Does >>>>>>> somebody know the answer to this question? >>>>>>> >>>>>> >>>>>> >>>>> >>> >
