On Tue, Dec 26, 2006 at 11:51:47AM -0700, we recorded a bogon-computron
collision of the <[EMAIL PROTECTED]> flavor, containing:
> I had Xastir running from a terminal window, and found out that it seg
> faulted. I can't find a core dump though.
>
> I also seem to be having problems with the shapefiles. It seemed to
> work pretty fast when I first started playing with them. Now it can
> take up to 2 or 3 minutes to render an image. I thought it was because
> I had created a dbfawk file, and it was taking a while to parse out
> the file, but I removed the dbfawk file, and still get massive delays
> when trying to use shapefiles.
I've got a theory. I ran the VM this weekend for several days. It never
crashed or bogged down as you describe, but when I used a bunch of shapefile
maps it did use a bit more memory than I thought it would have needed. I
betcha dollars to donuts what you're seeing is that xastir is starting to take
up more RAM than has been allocated and eventually the VM starts thrashing.
You can see if that's the case by running "top" in another terminal
window and watching the available memory and swap usage (up near the top of
the display). If you start to see the swap usage go up as you use xastir,
then yes, xastir is starting to take up more RAM than was reserved to it, and
is trying to use swap. If it tries to use swap a lot, then it will slow down
dramatically. If it tries to use so much RAM that it tries to use more swap
than is available, it will crash unceremoniously. I'm keen to know if that's
what's going on.
If it is, and if you have enough RAM on your computer, you could try bumping
up the allocation to the virtual machine. Out-of-the-box the VM is allocated
only 256MB, and in the instance I had running for a few days I found that it
liked to use almost all of that, sometimes coming within 2 or 3 MB of the
available RAM. This was with a fair number of shapefiles in use, while
zooming and panning a little. If I'd added a few more shapefiles that needed
rtrees, especially large maps that had lots of shapes in them, I can imagine
that it would start to have trouble finding room for them. If you're running
with shapefiles that cover very large areas, and viewing them in screens that
only cover a small part of those areas, it's possible that your usage is
causing some hefty memory usage due to all the rtrees. Bumping up to 300 or
512MB (if you can spare it on your computer) would probably help, and more
is better. Stay within the range recommended by vmware and guide your choice
by watching the windows task manager performance meter, though, because if
you make the VM too big then windows could start having trouble finding space
for its own bloat, and then *it* would start to thrash to swap.
I saw no obvious signs that there was a memory leak that would cause rtree
to consume memory without bounds --- if I left xastir alone for a few hours
then memory consumption went down so there were a few dozen MB of free space.
But I can't positively rule out that possibility at this point, but am
reasonably convinced there's none by pointing to a few years of using the
feature without seeing RAM-related crashes or unbounded growth).
Rtree enhances shapefile speed by caching some information about shapefiles,
trading RAM for speed. If you have lots of shapefiles and are doing lots of
zooming and panning you could cause a fair amount of memory bloat. There is
some attempt to throw away rtrees that aren't being used, but that only helps
so much.
(Details: an rtree spatial index is built for each shapefile the
first time it's rendered in such a way that it is not wholly contained in
the current screen view. The rtree stores information about the bounding
box of each shape in the shapefile, and allows fast lookups to answer the
question "which shapes intersect the screen?" Without rtree, that was
answered by reading each shape sequentially from the file and checking
every time the screen is redrawn. An existing spatial index is used
whenever the associated shapefile is again rendered into a window that
doesn't wholly contain it, but the spatial index is NOT accessed if the
entire shapefile is contained in the window. Any rtree spatial index that
hasn't been accessed in an hour is discarded and its memory freed. The
idea is not to bother keeping around indices that aren't being used often,
and only use them when they help (i.e. when the answer to the question
"which shapes intersect the screen?" is not "all of them". Even with this
attempt at minimizing memory footprint, rtrees can wind up taking up a
fair amount of RAM.)
One of the decisions I had to make when generating the virtual machine was
to pick something that would be small enough to get running quickly and would
cater to the lowest common denominator machine. If you have enough RAM, you
should definitely allocate more to the virtual m
