Here's some more information about the blacklist diff files that you might want to take into consideration as you tweak your maintenance strategy.
The squidGuard blacklist dated 11/26 contained 21 urls.????????.diff files, the earliest dated 20010814 and the most recent dated 20011125. These files contain the adds and deletions made to the database on that day. I combined all of those diff files, sorted and deduped. I was surprised to see that those 21 urls.????????.diff files contained 5,092 duplicate entries! I looked at the sorted file before it was deduped. Here are some examples from that file: -12.151.19.5/members -12.151.19.5/members -12.151.19.5/members +12.151.19.5/members +12.151.19.5/members -128.121.10.76/a3/sexxxpics -128.121.10.76/a3/sexxxpics -128.121.10.76/a3/sexxxpics +128.121.10.76/a3/sexxxpics +128.121.10.76/a3/sexxxpics +128.121.10.76/a3/sexxxpics -128.121.10.76/a3/xxasian1 -128.121.10.76/a3/xxasian1 -128.121.10.76/a3/xxasian1 +128.121.10.76/a3/xxasian1 +128.121.10.76/a3/xxasian1 +128.121.10.76/a3/xxasian1 -128.121.10.76/a3/xxxgirls -128.121.10.76/a3/xxxgirls -128.121.10.76/a3/xxxgirls +128.121.10.76/a3/xxxgirls +128.121.10.76/a3/xxxgirls +128.121.10.76/a3/xxxgirls -128.241.79.40/teen -128.241.79.40/teen -128.241.79.40/teen -128.241.79.40/teen +128.241.79.40/teen +128.241.79.40/teen +128.241.79.40/teen +128.241.79.40/teen -130.94.139.201/amanda/petite -130.94.139.201/amanda/petite -130.94.139.201/amanda/petite -130.94.139.201/amanda/petite -130.94.139.201/amanda/petite +130.94.139.201/amanda/petite +130.94.139.201/amanda/petite +130.94.139.201/amanda/petite +130.94.139.201/amanda/petite +130.94.139.201/amanda/petite -140.99.105.202/members -140.99.105.202/members -140.99.105.202/members -140.99.105.202/members -140.99.105.202/members -140.99.105.202/members -140.99.105.202/members -140.99.105.202/members -140.99.105.202/members +140.99.105.202/members +140.99.105.202/members +140.99.105.202/members +140.99.105.202/members +140.99.105.202/members +140.99.105.202/members +140.99.105.202/members +140.99.105.202/members +140.99.105.202/members -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 -194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 +194.19.2.13/~u222900693 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 -194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 +194.19.2.13/~u251601155 Is your confidence level going up on the robot? Look at that last example. The robot was run 21 times, and in those 21 runs it decided to ADD <194.19.2.13/~u251601155> twelve times. It also decided to DELETE <194.19.2.13/~u251601155> thirteen times. Oh well, I guess it's the best we have.
