Re: Goal: a specialized inventory of installed packages
On Sun 05 Apr 2020 at 13:17:37 (-0500), Richard Owlett wrote:
> On 04/05/2020 12:00 PM, David Wright wrote:
> > On Sun 05 Apr 2020 at 10:30:41 (-0500), Richard Owlett wrote:
> >
> > > I currently have a configuration of Stretch that meets most of my needs.
> >
> > > I'm setting out to do an _extremely_ custom *minimal* install of Buster.
> > > The desired inventory shall list *ONLY* top level packages.
> > > [ E.G. if gfortran was purposely installed, the ~dozen packages
> > > installed because they were tagged as depends, recommends, or suggests
> > > wold *NOT* be listed. ]
> > >
> > > Is there a suitable tool? [Synaptic's History menu item is closest
> > > I've found].
> >
> > I assume that what you're after is a command like
> >apt-get install a b c d …
> > where the list is the shortest that would install the said system.
> >
> > In which case, I would presume the answer is no.
> >
> > But it's not too onerous to write a script to parse the status file
> > (dpkg-query does the grunt work) and determine which packages are
> > not a [pre-]depends/recommends/suggests of any other. You may or
> > may not want to prune the resulting list, removing the names of any
> > packages that the d-i has no option but to install itself.
> >
> > How kindly apt will take to be presented with such a list of packages
> > for installation at one sitting, I don't know.
>
> I wasn't aiming for tool to automajically recreate an existing system.
> I was going more for a tool to allow me to visualize specifying what
> my new system would/should look like.
> 'apt-mark ...' seems to be aimed at my goals.
> Thanks.
Well, it does appear from your response, "I'll just do what the
installer thinks is an absolute minimal install and run 'apt-mark
showmanual' against it", that you're still enjoying doing installs,
so you can probably iterate towards a solution by by trying your
minimal list and seeing what results (comparing it with your
"reference" installation).
And it does appear from your other response that minimalism is more
important than function, if you exclude Recommends.
But in any case, I was assuming you wanted a reverse tool, to work out
which packages on your system are top-level, in the sense that they
are not dependent on any other package that you've installed.
I've just cannibalised a function that I use for kind of similar
purposes, because with being kept at home it seemed a good time to try
out bash's associative arrays. (I've always dropped into Python in the
past.) You might want to try it out: its default is to include
Recommends, but you can generate a longer list by using strict Depends.
It doesn't handle alternatives ( | is treated as , ) and it ignores
Pre-Depends. It also works ab initio, as if you had to drive the
d-i selection yourself. But one could modify it to merge in a
dpkg-query listing from a system that's untouched post-install.
It uses trash files (as I often do) rather than pipes so that you can
inspect partial results. As I say, it was just to get my hands dirty
with bash associative arrays. Strictly, they're not functionally
required here, but they're meant to be fast to search than a list
of an indexed array. Anyway:
function my-packages-toplevels {
[ "$1" = "-?" ] && printf '%s\n' "Usage:$FUNCNAME [Depends-only]
lists the packages installed on this system that are not dependent (and,
if lacking any argument, not recommended) by any other package." >&2 &&
return 1
local -A Depsrecs
local Pkg Format=' ${Depends} ${Recommends} \n'
[ -n "$1" ] && Format=' ${Depends} \n'
local Unique1="$(mktemp ${Uniquetrash:-/tmp}/$FUNCNAME-"$(date
+%s)"-1some-)"
local Unique2="$(mktemp ${Uniquetrash:-/tmp}/$FUNCNAME-"$(date
+%s)"-2all-)"
dpkg-query -W --showformat "$Format" | sed -e 's/([^)]\+)//g;s/,/ /g;s/|/
/g;s/[ ]\+/ /g;s/ /\n/g;' | sort -u | tail -n +2 >| "$Unique1"
while IFS=$'\n' read -r Pkg; do
Depsrecs["$Pkg"]="yes"
done < "$Unique1"
dpkg-query -W --showformat '${Package}\n' | sort >| "$Unique2"
while IFS=$'\n' read Pkg; do
[[ ! -n "${Depsrecs[$Pkg]}" ]] && printf '%s\n' "$Pkg"
done < "$Unique2"
return 0
}
Cheers,
David.
Re: Goal: a specialized inventory of installed packages
On Du, 05 apr 20, 11:59:17, Richard Owlett wrote: > > I did a test run. I think I see the pattern of which packages it marks as > manual. > E.G. It shows systemd related items as "manual". But for *MY* purposes I > would class them as "auto". That will not be a problem in practice. I'll > just do what the installer thinks is an absolute minimal install and run > 'apt-mark showmanual' against it. I'll save that list. Then create a script > to ignore those lines when 'apt-mark showmanual' is run against my actual > system. Something I used to do for similar purposes[1] was to mark everything as auto installed and then mark as manual the packages I was actually using (typically things like mutt, GUI/TUI browser, etc.). This was done using aptitude's interactive mode (not sure if apt-mark even existed at the time), as I could easily see what aptitude wanted to keep or remove. If you intend to also *remove* packages based on this approach do mind the settings for "APT::AutoRemove::RecommendsImportant" and "APT::AutoRemove::SuggestsImportant". [1] I stopped doing this because I found the optimization was not worth the time spent. Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Re: Goal: a specialized inventory of installed packages
On Du, 05 apr 20, 11:52:32, The Wanderer wrote: > > From what I've seen, it looks as if debian-installer also flags some > packages as manually installed, during initial install of the Debian > system. I don't know which ones do and don't get that treatment. At least the packages installed during the 'debootstrap' phase. Possibly also those installed by tasks (if one is selected) as well as linux image(s) and bootloader (grub). Kind regards, Andrei -- http://wiki.debian.org/FAQsFromDebianUser signature.asc Description: PGP signature
Re: Goal: a specialized inventory of installed packages
On 04/05/2020 01:29 PM, Marco Möller wrote: Once you have your list of packages in a text file, for each package one line, you could apply the list like this: apt install $(< mylist.txt) Good to know that is known to work. Consider to first do a simulation run for finding problems in the list: -s Consider to use the following flag in order to not draw in a maybe gigantic tail of not really wanted stuff: --no-install-recommends That had been my intention for that reason. A few years ago someone suggested additional methods to install a minimum number of packages. I'll have to search my old emails - possibly on a semi-retired machine. If you from time to time would like to include some package from the maybe gigantic tail of recommends, then simply add it individually to the list! I maintain manually the list, which you are about to generate now, and have very good experience starting with the following complete command: apt install -s --no-install-recommends $(< mylist.txt) When installing the basic Debian system from the official media, you may want to do this in mode "expert install", That has been my procedure. and you may want to know that all offered graphical Desktop Environments are full fledged installs coming with many (IMHO too much!) applications, You're preaching to the choir ;/ while afterwards going for a manual install of the desktop environment outside of the Debian installer will allow you to go for a fine grained "minimal" Desktop Environment installation only coming with the really needed functionality and without the applications. I therefore recommend to have your personally selected packages for the wished Desktop Environment (KDE, Gnome, XFCE, etc.) included in your personal list, and when arriving at the Debian installer's "tasksel" menu do not only deactivate all the listed Desktop Environments, which become offered, but even do deactivate the head entry “Debian desktop environment” itself, otherwise this would install a full featured GNOME desktop although below in the list you have had deactivated GNOME. I usually only select here “standard system utilities” and “print server”, and then have all the needed packages for "my" Desktop Environment in my list. I've a collection of experimental preseed.cfg files aimed at that. If following this path, then be aware that for a laptop essential or nice to have tools like wpasupplicant or network-manager (and many more!) will not become automatically installed for you! Setting up your personal list also concerning the Desktop Environment manually will thus require some extended effort. Unfortunately, the Debian installer is only offering to install full fledged Desktop Environments, but not minimal versions of it. Since I saw a site that would create a custom preseed.cfg in response to a series of questions resembling those the standard installer asks. That site is obsolete as it predates Squeeze IIRC. In any case I would do it in TCL/Tk as a educational coding exercise. Good luck! Marco. Thank you.
Re: Goal: a specialized inventory of installed packages
Once you have your list of packages in a text file, for each package one line, you could apply the list like this: apt install $(< mylist.txt) Consider to first do a simulation run for finding problems in the list: -s Consider to use the following flag in order to not draw in a maybe gigantic tail of not really wanted stuff: --no-install-recommends If you from time to time would like to include some package from the maybe gigantic tail of recommends, then simply add it individually to the list! I maintain manually the list, which you are about to generate now, and have very good experience starting with the following complete command: apt install -s --no-install-recommends $(< mylist.txt) When installing the basic Debian system from the official media, you may want to do this in mode "expert install", and you may want to know that all offered graphical Desktop Environments are full fledged installs coming with many (IMHO too much!) applications, while afterwards going for a manual install of the desktop environment outside of the Debian installer will allow you to go for a fine grained "minimal" Desktop Environment installation only coming with the really needed functionality and without the applications. I therefore recommend to have your personally selected packages for the wished Desktop Environment (KDE, Gnome, XFCE, etc.) included in your personal list, and when arriving at the Debian installer's "tasksel" menu do not only deactivate all the listed Desktop Environments, which become offered, but even do deactivate the head entry “Debian desktop environment” itself, otherwise this would install a full featured GNOME desktop although below in the list you have had deactivated GNOME. I usually only select here “standard system utilities” and “print server”, and then have all the needed packages for "my" Desktop Environment in my list. If following this path, then be aware that for a laptop essential or nice to have tools like wpasupplicant or network-manager (and many more!) will not become automatically installed for you! Setting up your personal list also concerning the Desktop Environment manually will thus require some extended effort. Unfortunately, the Debian installer is only offering to install full fledged Desktop Environments, but not minimal versions of it. Good luck! Marco.
Re: Goal: a specialized inventory of installed packages
On 04/05/2020 12:00 PM, David Wright wrote: On Sun 05 Apr 2020 at 10:30:41 (-0500), Richard Owlett wrote: I currently have a configuration of Stretch that meets most of my needs. I'm setting out to do an _extremely_ custom *minimal* install of Buster. The desired inventory shall list *ONLY* top level packages. [ E.G. if gfortran was purposely installed, the ~dozen packages installed because they were tagged as depends, recommends, or suggests wold *NOT* be listed. ] Is there a suitable tool? [Synaptic's History menu item is closest I've found]. I assume that what you're after is a command like apt-get install a b c d … where the list is the shortest that would install the said system. In which case, I would presume the answer is no. But it's not too onerous to write a script to parse the status file (dpkg-query does the grunt work) and determine which packages are not a [pre-]depends/recommends/suggests of any other. You may or may not want to prune the resulting list, removing the names of any packages that the d-i has no option but to install itself. How kindly apt will take to be presented with such a list of packages for installation at one sitting, I don't know. Cheers, David. I wasn't aiming for tool to automajically recreate an existing system. I was going more for a tool to allow me to visualize specifying what my new system would/should look like. 'apt-mark ...' seems to be aimed at my goals. Thanks.
Re: Goal: a specialized inventory of installed packages
On Sun 05 Apr 2020 at 10:30:41 (-0500), Richard Owlett wrote: > I currently have a configuration of Stretch that meets most of my needs. > I'm setting out to do an _extremely_ custom *minimal* install of Buster. > The desired inventory shall list *ONLY* top level packages. > [ E.G. if gfortran was purposely installed, the ~dozen packages > installed because they were tagged as depends, recommends, or suggests > wold *NOT* be listed. ] > > Is there a suitable tool? [Synaptic's History menu item is closest > I've found]. I assume that what you're after is a command like apt-get install a b c d … where the list is the shortest that would install the said system. In which case, I would presume the answer is no. But it's not too onerous to write a script to parse the status file (dpkg-query does the grunt work) and determine which packages are not a [pre-]depends/recommends/suggests of any other. You may or may not want to prune the resulting list, removing the names of any packages that the d-i has no option but to install itself. How kindly apt will take to be presented with such a list of packages for installation at one sitting, I don't know. Cheers, David.
Re: Goal: a specialized inventory of installed packages
On 04/05/2020 10:52 AM, The Wanderer wrote: On 2020-04-05 at 11:30, Richard Owlett wrote: I moved from WindowsXP when Squeeze was the current release. In the first year I did *many* installs from scratch to determine what I wanted in a final system (made much use of preseeding). I currently have a configuration of Stretch that meets most of my needs. As the installation was performed using the default Debian installer, there is a plethora of packages of no interest installed. As some important packages are not installed by default, apt and Synaptic were used to install them. I'm setting out to do an _extremely_ custom *minimal* install of Buster. The desired inventory shall list *ONLY* top level packages. [ E.G. if gfortran was purposely installed, the ~dozen packages installed because they were tagged as depends, recommends, or suggests wold *NOT* be listed. ] This sounds like a job for apt's "manual" vs. "auto" status flag. 'apt-mark showmanual' will list all the packages which are flagged as being manually installed. *YES* THANK YOU! Specifying a package explicitly to 'apt-get install' or similar flags it as manually installed, if you actually go through and complete the installation (cancelling the install also cancels the flag). Any packages which it pulls in as dependencies - whether "Depends:" "Recommends", or "Suggests:" - get flagged as "auto" instead. From what I've seen, it looks as if debian-installer also flags some packages as manually installed, during initial install of the Debian system. I don't know which ones do and don't get that treatment. I did a test run. I think I see the pattern of which packages it marks as manual. E.G. It shows systemd related items as "manual". But for *MY* purposes I would class them as "auto". That will not be a problem in practice. I'll just do what the installer thinks is an absolute minimal install and run 'apt-mark showmanual' against it. I'll save that list. Then create a script to ignore those lines when 'apt-mark showmanual' is run against my actual system. I don't know if that's good enough for you, but it's very likely the best you're going to get. It looks like a good match so far. Thanks again.
Re: Goal: a specialized inventory of installed packages
On 2020-04-05 at 11:30, Richard Owlett wrote: > I moved from WindowsXP when Squeeze was the current release. In the > first year I did *many* installs from scratch to determine what I > wanted in a final system (made much use of preseeding). > > I currently have a configuration of Stretch that meets most of my > needs. As the installation was performed using the default Debian > installer, there is a plethora of packages of no interest installed. > As some important packages are not installed by default, apt and > Synaptic were used to install them. > > I'm setting out to do an _extremely_ custom *minimal* install of > Buster. The desired inventory shall list *ONLY* top level packages. [ > E.G. if gfortran was purposely installed, the ~dozen packages > installed because they were tagged as depends, recommends, or > suggests wold *NOT* be listed. ] This sounds like a job for apt's "manual" vs. "auto" status flag. 'apt-mark showmanual' will list all the packages which are flagged as being manually installed. Specifying a package explicitly to 'apt-get install' or similar flags it as manually installed, if you actually go through and complete the installation (cancelling the install also cancels the flag). Any packages which it pulls in as dependencies - whether "Depends:" "Recommends", or "Suggests:" - get flagged as "auto" instead. From what I've seen, it looks as if debian-installer also flags some packages as manually installed, during initial install of the Debian system. I don't know which ones do and don't get that treatment. I don't know if that's good enough for you, but it's very likely the best you're going to get. -- The Wanderer The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw signature.asc Description: OpenPGP digital signature
