Re: registry.db getting rather obese (> 600MB) and updates very slow. Anything to do?

2020-06-22 Thread Ryan Schmidt
On Jun 22, 2020, at 17:33, Dave Horsfall wrote:

> On Sun, 21 Jun 2020, Jeffrey Walton wrote:
> 
>> Another PowerMac 10.5 datapoint, if it is helpful:
> 
> On my MacBook Pro (Sierra), it's
> 
> -rw-r--r--  1 root  admin  62558208 Jun 22 08:09 
> /opt/local/var/macports/registry/registry.db
> 
> And yeah, as Ryan noted later I got lotsa ports installed :-)

That's only 60MB so not very big compared to Ken's 628MB.


>> My MacPorts update recipe is as follows.
>> 
>>   /opt/local/bin/port selfupdate
>>   /opt/local/bin/port upgrade outdated
>>   /opt/local/bin/port clean inactive
>>   /opt/local/bin/port -f uninstall inactive
> 
> Is this the "official" care and feeding of MacPorts?

The official routine is:

sudo port selfupdate
sudo port upgrade outdated

Running "sudo port reclaim" should take care of cleaning up things that are no 
longer needed. (It could of course still be improved to handle additional 
cases.) MacPorts should prompt you about running reclaim if you haven't done so 
in awhile.


> My routine (no real reason) is:
> 
> Sunday: port -u uninstall # Clean out saved ports

This is the same as "sudo port uninstall inactive". I advocate removing this 
confusing second usage of the -u flag. https://trac.macports.org/ticket/33933


> Monday: port selfupdate   # Update itself
> and port -p upgrade   # And play dumb by answering the 
> questions
> 
> I haven't timed it (but the build takes a heck of a long time sometimes)
> and I'm asked if I want to do a "clean" after two weeks.

Ken's observation is about the amount of time it takes to deal with the 
registry after a port installation is finished, not the amount of time it takes 
to build.


> Oh, I use "-p" because I come across the occasional broken port ("guile"
> springs to mind) and I really want to see what can be built after that, 
> instead of waiting for a fix.

You definitely should not use -p with upgrade or install. It will proceed to 
build ports despite failures to upgrade their dependencies, which you 
absolutely do not want to do. I've explained why many times before on this 
list. We really shouldn't allow -p to be used with upgrade or install or should 
at least print a much more stern warning.



Re: registry.db getting rather obese (> 600MB) and updates very slow. Anything to do?

2020-06-22 Thread Dave Horsfall

On Sun, 21 Jun 2020, Jeffrey Walton wrote:


Another PowerMac 10.5 datapoint, if it is helpful:


On my MacBook Pro (Sierra), it's

-rw-r--r--  1 root  admin  62558208 Jun 22 08:09 
/opt/local/var/macports/registry/registry.db

And yeah, as Ryan noted later I got lotsa ports installed :-)


My MacPorts update recipe is as follows.

   /opt/local/bin/port selfupdate
   /opt/local/bin/port upgrade outdated
   /opt/local/bin/port clean inactive
   /opt/local/bin/port -f uninstall inactive


Is this the "official" care and feeding of MacPorts?  My routine (no real 
reason) is:


Sunday: port -u uninstall   # Clean out saved ports
Monday: port selfupdate # Update itself
and port -p upgrade # And play dumb by answering the questions

I haven't timed it (but the build takes a heck of a long time sometimes)
and I'm asked if I want to do a "clean" after two weeks.

Oh, I use "-p" because I come across the occasional broken port ("guile"
springs to mind) and I really want to see what can be built after that, 
instead of waiting for a fix.


-- Dave


Re: registry.db getting rather obese (> 600MB) and updates very slow. Anything to do?

2020-06-22 Thread Ryan Schmidt
On Jun 21, 2020, at 13:17, Ken Cunningham wrote:

> On one of my systems, 10.5 PPC, things are taking a long time when 
> uninstalling a port - like 4-5 minutes of heavy tcl interpreter use.
> 
> The registry.db is pretty huge:
> 
> $ ls -la
> total 1286264
> drwxr-xr-x 7 root  admin238 21 Jun 11:06 .
> drwxr-xr-x@   15 root  admin510 18 Dec  2018 ..
> -rw-r--r-- 1 root  admin  0 21 Jun 10:59 .registry.lock
> -rw-rw-r-- 1 root  admin  0  1 Oct  2015 .turd_MacPorts
> drwxr-xr-x  2954 root  admin 100436 21 Jun 10:59 portfiles
> drwxr-xr-x   185 root  admin   6290 13 Jun 19:51 portgroups
> -rw-r--r-- 1 root  admin  658564096 21 Jun 11:06 registry.db
> 
> 
> As you can see, this install has been going along for 5 years, so I guess 
> some bitrot could occur. I’m sure I could blow out the installation and start 
> fresh, but that’s not terribly trivial on 10.5 PPC with no functional 
> buildbot. 
> 
> Is there something else I can do to put that registry.db on a diet?

Since the registry contains information about every port you have installed, 
and every file installed by those ports, it's natural that it will be larger 
the more ports you have installed. On our Catalina buildbot worker, for 
example, since they keep all ports installed, the registry is 1GB. If you don't 
have many ports installed, then that size of registry would be unexpected.

The registry doesn't immediately shrink if you uninstall a port, but after 
enough free space builds up in the registry MacPorts will perform the SQLite 
"vacuum" procedure to reclaim space. MacPorts does not print any status 
information to tell you when this is happening.

It is normal that it can take a long time for MacPorts to write the modified 
registry to disk after you've installed or uninstalled something if it is large 
because you have many ports installed, or if MacPorts happens to perform the 
vacuum procedure. How long would depend on the speed of your disk. The 300MB 
registry on my main Mac takes a few seconds to rewrite but that's on a fast SSD.


On Jun 22, 2020, at 00:28, Ken Cunningham wrote:

> I decided to do the obvious, backed up all the built software, and 
> reinstalled everything.

Has that helped? Do you now have a much smaller registry with the same set of 
ports installed? If so that would be unexpected and might indicate a problem in 
how we're cleaning up the registry during normal use.