Jochen, I compiled with the GC flag and can confirm that osmjs's memory usage is definitely still growing - much slower, but that is probably due to the garbage collection processes making the whole thing slow. I ran that US pbf again and it croaked when its memory footprint was about 50% in top.
As you said without any further output it's hard to do any further useful debugging. Martijn On Wed, Nov 9, 2011 at 1:37 AM, Jochen Topf <joc...@remote.org> wrote: > On Wed, Nov 09, 2011 at 12:24:55AM -0700, Martijn van Exel wrote: >> I'm testing my osmium setup, osmjs specifically. I'm feeding osmjs >> small pbf planet extracts (city scale) and run the time.js script, >> that goes OK. Then I feed it a US pbf planet extract, and osmjs says >> 'Killed' after about a minute or so. I tried -l none but that makes >> no difference. -d gives me nothing. >> >> Osmosis says the pbf is OK (I ran $OSMOSIS --rb ../us.pbf --wn and it >> did not give me any errors, I assume that means the file is OK). >> >> osmjs does not seem to have run out of memory. I don't have a lot >> (2GB) but that should not prevent osmjs from running this, should it? > > If you don't use -l or -m osmjs should not use a lot of memory. Maybe there > is a memory leak somewhere. Unfortunately thats hard to see, because the > Javascript engine will take all the memory it gets (up to some limit) and > only do garbage collection once that runs out. So in the "ps" output the > process will always use lots of memory and suddenly crash, when it can't > garbage collect any more. I don't remember what the message was in that > case but thats probably the "Killed" you see. > > You can compile with -DOSMIUM_V8_FORCE_GC to force a garbage collection after > each callback. This will make osmjs really slow, but you'll see the memory > for the process growing over time, which it shouldn't do. > > I run osmjs regularly on a germany extract and sometimes others and don't have > any problem like that. But it could be that the problem only shows up with, > say > very long ways, or objects with many tags or so. There could be something > different about the US extract that triggers this. (Or maybe its just much > bigger...) > > Jochen > -- > Jochen Topf joc...@remote.org http://www.remote.org/jochen/ +49-721-388298 > > -- martijn van exel geospatial omnivore 1109 1st ave #2 salt lake city, ut 84103 801-550-5815 http://oegeo.wordpress.com _______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev