A weekly fatal unable to fork error message from /etc/cron.weekly/apt-
xapian-index on a server with 512MB memory with a web server and
database server running leads to this bug. That cron job calls /usr/sbin
/update-apt-xapian-index, which is a Python script that imports axi and
axi.indexer. Those Python packages are the heart of the problem.
On an idle system with 512M memory and 768M swap, while /etc/cron.weekly
/apt-xapian-index was running, I used free to see memory and swap used.
It rose to then peaked at this:
total used free
Mem: 503528 497736 5792
Swap: 786428 231196 555232
Total used memory is 728M.
After ending, it immediately dropped back to this:
total used free
Mem: 503528 259160 244368
Swap: 786428 224484 561944
Total used memory is 483M.
I conclude that apt-xapian-index consumes the difference, which is 245M.
Running "apt-cache stats" I see at the end "Total space accounted for:
26.0 M".
Therefore, it takes 245M to sort and index 26M of information. This
seems conclusive that the algorithms, containers, and/or functions
chosen are very inefficient. It should definitely not require 10 times
the memory space of what is being indexed.
The solution is to change the sorting algorithm in the python axi and
axi.indexer modules. The first priority is to switch to an algorithm
that consumes a whole lot less memory (i.e., each step of the algorithm
keeps less objects in memory), and it will stop crashing and stop
thrashing (memory swapping to disk).
The second priority is of lesser importance (because renice can solve a
lot of the effect), which is to switch to an algorithm that takes a lot
less time to run (i.e., takes fewer steps to complete), and it will stop
consuming so much CPU for so long.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/655831
Title:
update-apt-xapian-index bogs down system
To manage notifications about this bug go to:
https://bugs.launchpad.net/ac100/+bug/655831/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
