It is not perfect and it could be streamlined greatly. YMWV
=====
Overview:
As of this writing, Apt-move currently has two options.
a) Mirror - mirror "everything" which puts undue load on the public debian repository servers and wastes a great deal of network bandwidth and local hard drive space for packages you will never use. (Pesonally I believe this feature should NEVER be used and deleted from the Apt-move package.)
Or
b) Sync - "mirror what is installed on the server" machine. Usually you do not want every single package ever installed on any client to be installed on the server. If anything you want the exact oppsite, a minimum of number of packages loaded on the server, a sub-set of packages that the local network might require.
Neither of these solutions is a perfect for a group of locally networked machines with varying installed package sets.
Apt-move does have the Exclude packages capability, but when your trying to exclude 17,100 packages out of 18,600, an 'include' list is much more logical because it is only 1,400 to 1,800 packages long and easier to maintain with the varying client package requirments.
What this patch does is reprogram the apt-move "sync" function to use a local network master packages file instead of polling the local server for deciding which packages to mirror.
This patch also provides the functionality to combine the package lists from all local network client machines into a single combined package listing that Apt-move sync will use to decide what files to mirror locally.
Applying implimenting an Apt-move local repository with this patch applied reduces the amount of bandwith required on your local networks as well as reduces the amount of bandwidth required by the public debian apt repository servers.
Important! If you run a single debian installation on your entire local network, you should not be running apt-move nor applying this patch. You will be causing more bandwidth waste of your own network connection as well as that of the debian repository if not supporting more than one debian installation.
=====
Requires:
1) apt-move must already be operational and accessable to all machines (or multiple Debian based partitions for multiboot systems) on the local network.
2) A public samba share located on the apt-move server.
3) All server and client machines must use the same distribution name(s) in their /etc/apt-get/sources.list files. (ex: woody or stable not both.)
4) The server's /etc/apt/sources.list should be a superset of all client's sources.list (How-To not described here.) The client repositories can vary, but the server sources.list should include all repositories used by each client. Becareful of the order of repositories. The one listed first will overide any subsequent repository if the package name is the same in both except when the latter has a newer version number.
Suggests:
1) smb4k on the client local network machines
2) cron on the server
3) ntpdate on the server
4) vncserver (not vnc4server!!) on the server if headless
5) xvnc4viewer on any client machine used to maintain the server if headless.
Assumes:
1) you have a permanent server. Runs 24 hrs a day and is not normally used as a desktop machine. (or desktop that doubles as a server that not rebooted or left shutdown for extended periods of time.)
2) You have numerous debian installations or debian client machines on your network that you keep updated regularly.
3) Only one Apt-move repository server is operational on the local network.
Ignores:
1) Doesn't explain how permissions of the samba share can interfere with proper operation and updating. If you do not know how to deal with file and samba permissions you are not ready to install this patch as written.
2) Does not describe alternate solutions for getting the client packages list files onto server. ftp, NFS, ssh, telnet, tty, etc, etc, etc.
3) Does not explain Apt-move functionality in general.
=====
The patch process:
Identify or create a samba share and make it publically available. I suggest creating an "apt-move" folder in this share to keep it from getting mixed with other files this share might be used for.
(ex: /var/share is my public samba share. The folder /var/share/apt-move becomes the public folder of package listings from all clients and my local server.)
On the server where Apt-move is installed run as root: cp /usr/bin/apt-move /usr/bin/apt-move.net
edit /usr/bin/apt-move.net and/or apply the following diff
--- apt-move 2004-11-21 18:04:26.000000000 -0500
+++ apt-move.net 2005-04-09 13:14:29.000000000 -0400
@@ -1573,3 +1573,4 @@
pf=$TMPHOME/makeselect
- dpkg --get-selections | sort -sb -k 2,2 > $pf-tmp1
+ #dpkg --get-selections | sort -sb -k 2,2 > $pf-tmp1
+ cat /var/share/apt-move/selections.network | sort -sb -k 2,2 > $pf-tmp1
echo install | join -j2 2 -o 2.1 - $pf-tmp1 > $f
Note: Change or edit "/var/share/apt-move/" above to point at the server's samba share path to the public Apt-move folder.
In the /var/share/apt-move/ folder on the apt-move server run:
cd /var/share/apt-move
dpkg --get-selections >> selections.serverAt each clent machine, run as any user:
dpkg --get-selections >> selections.<machine's nickname>Copy the client machine's selections.<machine's nickname> file to the server's samba share. (ex: on my client machines with smb4k this path is /home/<user>/smb4k/<server name>/share/apt-move/)
Once you have the array of package selection files copied to your server's samba share apt-move folder.
Create a script on the server named "make-selections" based on the following code as a guideline (the suggested location is the server's samba share apt-move folder):
#!/bin/bash
## Author: Luther Pollok <[EMAIL PROTECTED]>
#samba share path on the server
mypath="/var/share/apt-move"outputfile="selections.network"
rm -f $mypath/$outputfile
# Where "machine nickname" is the name chosen when the file was
# created for each client machine or multiboot installation. file1="selections.server"
file2="selections.<machine nickname #1>"
file3="selections.<machine nickname #2>"
file4="selections.<machine nickname #3>"
file5="selections.<machine nickanme #4>"
# etc
# etc
# etc ## note: the following code section was originally written
## as a single line.
## Splitting it by adding " \" + CrLf was for clarity and
## has not been tested as written here. ## Warning, delete the "# etc \" lines in the final script
## in this next section in all cases!!!!!!!!
## These are there only to show how the script can be
## expanded to support more client machines.
## You have been warned. cat \
$mypath/$file1 \
$mypath/$file2 \
$mypath/$file3 \
$mypath/$file4 \
$mypath/$file5 \
# etc \ <<-- make sure you delete this line
# etc \ <<-- make sure you delete this line
# etc \ <<-- make sure you delete this line
| sort -u > $mypath/$outputfileexit 0
Then run "make-selections" script.
You can now manually run apt-move.net -d <distibution name> sync
=====
No point going this far without adding in a little automation. ;-)
To make this a once a day automatically updating apt package repository, create a file at /usr/bin/ named "apt-move-updater" and use the following code as a guideline. Edit or customize as needed:
#!/bin/bash
## Author: Luther Pollok <[EMAIL PROTECTED]>
## Optional
#echo "Updating Time"
#/etc/init.d/ntpdate restart ## Optional
#echo "Updating Master list"
#/<path-to>/make-selectionsecho "updating package lists"
## using dselect to update apt is preferred
## over apt-get update
# apt-get updatedselect update
## note: "unstable", "testing" and/or "stable" entries must
## match what was configured in /etc/apt-get/sources.list
## on the server machine.
## All client machines must be also configured to use
## the same names of distributions in
## their sources.list as well.
## i.e. You can't use "woody" or "sarge" on a client machine and
## "stable" or "testing" on the server and expect it to work.## Also assumes that /usr/bin/ is in the root user's path
echo "synching debian unstable"
apt-move.net -d unstable sync
#echo "synching debian testing"
#apt-move.net -d testing sync
#echo "synching debian stable"
#apt-move.net -d stable sync
echo "cleaning apt-get archives"
## Enable the apt-get clean line when your convinced that
## your local apt-move repository server is operating correctly.#apt-get clean
exit 0
At this point I suggest that you copy any and all deb packages that you currently have on each client machine into the server's /var/cache/apt/archive/ folder to keep from downloading the same file again for filling the local repository. This is not required, but it will save you a lot of time and bandwidth. (Don't forget that you now have a public samba share on the server, hint, hint, hint!!!)
Test this file now by manually running it as root: /usr/bin/apt-move-updater
Note: The first time you run it will take a long time because it is filling your local repository with the complete set of packages based on your new selections.network master list. Any subsequent runs will be relatively quicker.
If this works, add a apt-move-update line to the server's /etc/crontab file:
10 4 * * * root /usr/bin/apt-move-updater
Note: This assumes your machine is running as local time not GMT. change "10 4" above to the correct GMT that represents 4:10am local time to your server. We don't want every machine in the world to hit these servers at the exact same time. No one will be able to download if the servers become overloaded because of this patch.
Optional: If the above line was added to crontab and ntpdate was not added to apt-move-updater then add a ntpdate line to /etc/crontab
5 4 * * * root /etc/init.d/ntpdate restart
Note: As above for "10 4" set the ntpdate command to operate shortly before the apt-move updater line.
....And your done with this patch.
=====
Notes:
1) The folder "/var/share/apt-move" can be anywhere on the server machine and can be called anything you like, but all code described above needs to be changed to match where the server's samba share and the apt-move public folder resides.
2) I keep "make-selections" in the samba share folder. This allows any machine on the network to update the selections.network file, but it does not need to reside there. So for example you could put make-selections in /usr/bin/ and add a line to apt-move-updater to always update the selections.network file before running the apt-move sync commands.
3) As each machine's packages list changes significantly, you will need to copy the updated selections.<machine's nickname file> over to the server and re-run the make-selections script.
4) You may want to manually run "apt-move-updater" or run "apt-move.net -d <distribution name> sync" the first time to get some packages loaded the server during testing.
5) Whenever you see your client machine downloading from the internet instead of your local server, copy the deb packages over to the server's /var/cache/apt/archive and then run "deslect update" and "apt-move update" on the server to prevent additional client machines from doing the same. This usually happens when a package was uploaded after the last time a apt-move-updater script.
Bugs:
1) Package lines created by "dpkg --get-selections" with "hold", "deinstall", etc, as package status indicators end up as duplicate entries in the resultant selections.network file. I hadn't figured out how to remove the dups, but I believe it will involve inserting a pipe for grep between cat and sort in either make-selections or apt-move.net files. I have not observed any unusualy operation when these duplicated entries are included. It was assumed that apt-move already knew how to deal with these dup entries, so it was not researched any further.
enjoy
martin02 Orlando, FL [EMAIL PROTECTED]
-- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
