Re: how to clone apt repository to newest only?
I live in South Korea. Most of the government systems in Korea operate in a closed environment and are not connected to the internet. This is because they are vulnerable to security. Anyway, I decided to use the update dvd image. Alternatively, it would be good to create the image directly using jigdo. Thanks to all of you for your help. 2023년 12월 27일 (수) 오전 7:33, Andrew M.A. Cater 님이 작성: > On Tue, Dec 26, 2023 at 04:49:13PM -0500, Roy J. Tellason, Sr. wrote: > > On Tuesday 26 December 2023 09:34:00 am Andrew M.A. Cater wrote: > > > Living offline is not really feasible anymore - there are too many > security > > > updates needed. > > (snip) > > > Linux distributions do update and you should ideally be running the > latest > > > most up to date security patches. > > > > I must be missing something here. If one is running a system that's NOT > net-connected, why is security so important an issue? > > > > You always have to hope that it remains not connected :) > > Remembering that each point update introduces fixes which may clear > previous problems, it is always worth keeping the system up to date. > > Given the inadvertent upstream kernel problems we gained during the 12.3 > release which resulted in 12.4 and that we then needed 12.5 relatively > immediately to solve problems that some users had - if you'd _only_ > had the 12.4 medium, you might have had problems which could only have > been fixed by being net connected to pick up the appropriate kernel. > > Just because you have a (relatively) isolated system doesn't mean that > your system shouldn't be consistent, patched and up to date which will > allow you to be sure that known vulnerabilites have been addressed. > > There's nothing like the joy of inheriting a system tucked away somewhere > that hasn't been updated or rebooted in five years and not knowing what > you might expect when logging in, what services are running or what will > happen if you have to reboot. Marginally better because you know about it > then finding the system that everything depends on is undocumented, > running on a system with dead disks in the RAID and that has just > been bounced by the unscheduled power outage when the UPS failed .. > > > -- > > Member of the toughest, meanest, deadliest, most unrelenting -- and > > ablest -- form of life in this section of space, a critter that can > > be killed but can't be tamed. --Robert A. Heinlein, "The Puppet Masters" > > - > > Sounds like a project manager imposing random requirements :) > > All the very best, as ever, > > Andy Cater > (amaca...@debian.org) > > > Information is more dangerous than cannon to a society ruled by lies. > --James > > M Dakin > > > >
Heinlein and requirements (was Re: how to clone apt repository to newest only?)
On 2023-12-26 at 17:33, Andrew M.A. Cater wrote: > On Tue, Dec 26, 2023 at 04:49:13PM -0500, Roy J. Tellason, Sr. wrote: >> -- >> Member of the toughest, meanest, deadliest, most unrelenting -- and >> ablest -- form of life in this section of space, a critter that can >> be killed but can't be tamed. --Robert A. Heinlein, "The Puppet Masters" >> - > > Sounds like a project manager imposing random requirements :) You want to talk about random (or not so much) requirements found in Heinlein? >>> A human being should be able to change a diaper, plan an >>> invasion, butcher a hog, conn a ship, design a building, write a >>> sonnet, balance accounts, build a wall, set a bone, comfort the >>> dying, take orders, give orders, cooperate, act alone, solve >>> equations, analyze a new problem, pitch manure, program a >>> computer, cook a tasty meal, fight efficiently, die gallantly. >>> Specialization is for insects. I can probably do... somewhere in the range from four to ten of those, depending on one's definitions. -- The Wanderer The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. -- George Bernard Shaw signature.asc Description: OpenPGP digital signature
Re: how to clone apt repository to newest only?
On Tue, Dec 26, 2023 at 04:49:13PM -0500, Roy J. Tellason, Sr. wrote: > On Tuesday 26 December 2023 09:34:00 am Andrew M.A. Cater wrote: > > Living offline is not really feasible anymore - there are too many security > > updates needed. > (snip) > > Linux distributions do update and you should ideally be running the latest > > most up to date security patches. > > I must be missing something here. If one is running a system that's NOT > net-connected, why is security so important an issue? > You always have to hope that it remains not connected :) Remembering that each point update introduces fixes which may clear previous problems, it is always worth keeping the system up to date. Given the inadvertent upstream kernel problems we gained during the 12.3 release which resulted in 12.4 and that we then needed 12.5 relatively immediately to solve problems that some users had - if you'd _only_ had the 12.4 medium, you might have had problems which could only have been fixed by being net connected to pick up the appropriate kernel. Just because you have a (relatively) isolated system doesn't mean that your system shouldn't be consistent, patched and up to date which will allow you to be sure that known vulnerabilites have been addressed. There's nothing like the joy of inheriting a system tucked away somewhere that hasn't been updated or rebooted in five years and not knowing what you might expect when logging in, what services are running or what will happen if you have to reboot. Marginally better because you know about it then finding the system that everything depends on is undocumented, running on a system with dead disks in the RAID and that has just been bounced by the unscheduled power outage when the UPS failed .. > -- > Member of the toughest, meanest, deadliest, most unrelenting -- and > ablest -- form of life in this section of space, a critter that can > be killed but can't be tamed. --Robert A. Heinlein, "The Puppet Masters" > - Sounds like a project manager imposing random requirements :) All the very best, as ever, Andy Cater (amaca...@debian.org) > Information is more dangerous than cannon to a society ruled by lies. --James > M Dakin >
Re: how to clone apt repository to newest only?
On Tue, 26 Dec 2023 16:49:13 -0500 "Roy J. Tellason, Sr." wrote: > I must be missing something here. If one is running a system that's > NOT net-connected, why is security so important an issue? Physical access, especially a multi-user system. Think a college science lab. -- Does anybody read signatures any more? https://charlescurley.com https://charlescurley.com/blog/
Re: how to clone apt repository to newest only?
On Tuesday 26 December 2023 09:34:00 am Andrew M.A. Cater wrote: > Living offline is not really feasible anymore - there are too many security > updates needed. (snip) > Linux distributions do update and you should ideally be running the latest > most up to date security patches. I must be missing something here. If one is running a system that's NOT net-connected, why is security so important an issue? -- Member of the toughest, meanest, deadliest, most unrelenting -- and ablest -- form of life in this section of space, a critter that can be killed but can't be tamed. --Robert A. Heinlein, "The Puppet Masters" - Information is more dangerous than cannon to a society ruled by lies. --James M Dakin
Re: how to clone apt repository to newest only?
On 20/12/2023 13:05, 이 강우 wrote: how to clone apt repository to newest only? If you are asking about partial mirror then some of the following links might be useful: - https://wiki.debian.org/DebianRepository/Setup#Debian_Repository_Mirroring_Tools - https://www.debian.org/mirror/ftpmirror.en.html - https://wiki.debian.org/DebianRepository/Setup - for apt-cacher-ng: https://www.unix-ag.uni-kl.de/~bloch/acng/html/howtos.html#howto-importiso
Re: how to clone apt repository to newest only?
On Tue, Dec 26, 2023 at 10:19:26PM +0900, 이강우(KangWoo Lee) wrote: > The reason I'm asking for this feature is that > > For example, I want to install the most recent packages when installing an > OS in a specific closed network environment. > > Of course, I could use a recently created DVD iso file, but I would need to > have an internet connection to apply files that have been updated since > this ISO was created, so I only want to copy and apply the most recent > packages. > > Is there any way to do this? > > OK - this is a little harder to explain :) Living offline is not really feasible anymore - there are too many security updates needed. . If you really want to live in a closed network environment - you can't really do that with DNF either. If you're using Red Hat proper, then Red Hat will normally expect you to run an Internet connected Satellite server. Linux distributions do update and you should ideally be running the latest most up to date security patches. Debian produces updates pretty well every day for one package or another. The default settings for apt in Debian include a line for debian-security for just this reason. Approximately once every two months, we produce a point release for Debian stable which will pull together package fixes and security updates up to that point from the state of the previous point release. We do produce media that will give you just those updates to apply to a running system - almost nobody does this, and the update media itself is rarely, if ever, tested - it is used by very few people, if any. If you were installing a system today - 26th December - you could install from the base media released as part of Debian 12.4 - but there were almost immediate updates provided in stable-updates to deal with kernel issues, for example. Those wouldn't be on the media until 12.5 which is currently being discussed to take place in February 2025. At that point, we will have new media - and the smaller update media to allow you to update from 12.4. The canonical way to do disconnected mirroring is to have a Debian mirror connected to the Internet somewhere and to allow that to do daily updates. You can then take the daily updates and gateway them into your closed network (or disconnect the mirror from the Internet and allow it to connect to an "internal" copy of the mirror before disconnecting the "external-allowed" copy and reconnecting it to the Internet.) The Debian suggested mirroring scripts use rsync and produce logs so it is not difficult to extract daily updates. Setting up a full Debian mirror is not particularly hard - all architectures with a mirror of Debian CD images will fit within 6TB or so. I wrote up some outline instructions on a blog syndicated to Planet Debian, for example: http://flosslinuxblog.blogspot.com/2020/02/rebuilding-mirror-software-mirroring-of.html Note, I have rearranged the addresses on this reply so that it goes first to the debian-user mailing list. Follow up to the list, please. With every good wish, as ever, Andy Cater [amaca...@debian.org] > > > 2023년 12월 25일 (월) 오후 11:05, Andrew M.A. Cater 님이 작성: > > > On Mon, Dec 25, 2023 at 12:21:29PM +, �� wrote: > > [Copied to the poster because they may not be subscribed] > > > > > how to clone apt repository to newest only? > > > Fedora/Red Hat will organize the repository by copying only the most > > recent packages from that distribution if you give it the "reposync > > --newest-only" option, but Debian doesn't seem to be able to do that. > > > > > > What can I do? > > > > > > > > Hi > > > > By default, apt will check the dates on the package manifests and bring you > > up to date based on that. > > > > If you install from nothing then the installer will do the same assuming > > that you have an internet connection. > > > > reposync is really a Red Hat ecosystem specific command, I think. > > > > (already answered on the list: can I suggest that you subscribe to the > > list) > > > > Andy > > (amaca...@debian.org) > > > >
Re: how to clone apt repository to newest only?
Am Dienstag, 26. Dezember 2023, 14:19:26 CET schrieb 이강우(KangWoo Lee): Suggestion: First of all: Not all packages on the install DVD are as new as in the repo. So the installer is always upgrading packages during the installation process. What you can do, is downloading all the packages you need with another debian system, but not install it. This can be done by synaptic, aptitude or apt-get. For example with apt-get this shoule be apt-get -d --reinstall install packagename1 packagename2 ... packagename_whatever. Then you will find all packages below /var/cache/apt/archives/ and can copy them to the other debian system using rsync. Now these can be installed using apt-get on the other computer. If I remember correctly, there is also an option apt-get -d --reinstall install world which would redownload, but NOT install all installed packages. However, of the second one I am not sure if this is working, maybe someone else can confirm or deny this. third suggestion: You can create a list with all installed packages, then edit it and download these files with apt- get as described above. However, oif you want two identical systems, and only one is connected to the internet and the other is airgapped, but connected to the first one, you can simply rsync the whole system to the other or using harrdrive cloning tools (like clonezilla or similar) Hope this helps a little bit. Best regards Hans > The reason I'm asking for this feature is that > > For example, I want to install the most recent packages when installing an > OS in a specific closed network environment. > > Of course, I could use a recently created DVD iso file, but I would need to > have an internet connection to apply files that have been updated since > this ISO was created, so I only want to copy and apply the most recent > packages. > > Is there any way to do this? > > 2023년 12월 25일 (월) 오후 11:05, Andrew M.A. Cater 님이 작성: > > On Mon, Dec 25, 2023 at 12:21:29PM +, �� wrote: > > [Copied to the poster because they may not be subscribed] > > > > > how to clone apt repository to newest only? > > > Fedora/Red Hat will organize the repository by copying only the most > > > > recent packages from that distribution if you give it the "reposync > > --newest-only" option, but Debian doesn't seem to be able to do that. > > > > > What can I do? > > > > Hi > > > > By default, apt will check the dates on the package manifests and bring > > you > > up to date based on that. > > > > If you install from nothing then the installer will do the same assuming > > that you have an internet connection. > > > > reposync is really a Red Hat ecosystem specific command, I think. > > > > (already answered on the list: can I suggest that you subscribe to the > > list) > > > > Andy > > (amaca...@debian.org)
Re: how to clone apt repository to newest only?
On Tue, Dec 26, 2023 at 10:19:26PM +0900, 이강우(KangWoo Lee) wrote: > For example, I want to install the most recent packages when installing an > OS in a specific closed network environment. > > Of course, I could use a recently created DVD iso file, but I would need to > have an internet connection to apply files that have been updated since > this ISO was created, so I only want to copy and apply the most recent > packages. > > Is there any way to do this? There are many ways, and I can't even name all of them. I'll just describe one way. Assume that all of your systems are running the same release, on the same architecture. You might have one system (same release, same architecture, same set of packages) that DOES have Internet access. Maybe it's a laptop that you take to another location, or whatever. So, periodically, you take the laptop to your grandma's house, or whatever it is. While there, you do an "apt-get update" and an "apt-get dist-upgrade". This brings in new package lists (in the /var/lib/apt/lists/ directory), and possibly some new *.deb package files (in /var/cache/apt/archives/). After that's done, you bring the laptop to the isolated network. Then you rsync the /var/lib/apt/lists/ directory to all the other computers. Next, you share your /var/cache/apt/archives/ via NFS, and mount it on all of the other computers. Finally, you do an "apt-get dist-upgrade" on all the other computers. They'll use the lists that you copied to them, and the NFS-shared package archive. Once that's all done, unmount the /var/cache/apt/archives/ directory from the other systems, and you're all set for a while. If the laptop starts to fill up, you can do "apt-get autoclean", but be very careful NEVER to run "apt-get clean".
Re: how to clone apt repository to newest only?
The reason I'm asking for this feature is that For example, I want to install the most recent packages when installing an OS in a specific closed network environment. Of course, I could use a recently created DVD iso file, but I would need to have an internet connection to apply files that have been updated since this ISO was created, so I only want to copy and apply the most recent packages. Is there any way to do this? 2023년 12월 25일 (월) 오후 11:05, Andrew M.A. Cater 님이 작성: > On Mon, Dec 25, 2023 at 12:21:29PM +, �� wrote: > [Copied to the poster because they may not be subscribed] > > > how to clone apt repository to newest only? > > Fedora/Red Hat will organize the repository by copying only the most > recent packages from that distribution if you give it the "reposync > --newest-only" option, but Debian doesn't seem to be able to do that. > > > > What can I do? > > > > > Hi > > By default, apt will check the dates on the package manifests and bring you > up to date based on that. > > If you install from nothing then the installer will do the same assuming > that you have an internet connection. > > reposync is really a Red Hat ecosystem specific command, I think. > > (already answered on the list: can I suggest that you subscribe to the > list) > > Andy > (amaca...@debian.org) > >
Re: how to clone apt repository to newest only?
As Andrew did, I also CC'd.. :) On 12/25/23, 이 강우 wrote: > how to clone apt repository to newest only? > Fedora/Red Hat will organize the repository by copying only the most recent > packages from that distribution if you give it the "reposync --newest-only" > option, but Debian doesn't seem to be able to do that. > > What can I do? Hi.. This is Draft Email #2 for me for this thread. The first email is very long. I chopped off all of the tips and am only focusing on the following questions for now. Am starting this time with an apt query: $ apt-cache search reposync Got a potential hit! The package is called dnf-plugins-core. It looks interesting (to me). Its description is: Description-en: Core plugins for DNF, the Dandified Yum package manager This package enhances DNF with builddep, config-manager, copr, debug, debuginfo-install, download, needs-restarting, groups-manager, repoclosure, repograph, repomanage, reposync, changelog and repodiff commands. It's the only package that references reposync. I'll be downloading and poking at it as a personal Debian development learning adventure by comparing it to wget and rsync as referenced further below. If dnf-plugins-core does not work for some reason, here are some questions that might help Debian Users help you What are you actually trying to do? Might also be asked as.. what were you doing in the past? What exact command(s) were you using? Internet searching on "reposync" alone makes it look like you're trying to do what I have found that wget does. It worked for an LS (Linux From Scratch) short webpage of only links today. Wget also worked on a Debian repository related webpage that included child directories. Running "man rsync" references "URL" a few times, too, but I've not been successful with it in the past. This thread is a reminder of that feature so I'll be playing with it again later. It's always good to know more than one way to accomplish all Linux tasks. :) My other questions that will help Debian Users help you are. Which Debian directory are you asking about? Or is it even tied to debian(dot)org? If you're [pinging] a webpage that is not Debian and it's not too personal, what webpage are you trying to sync? A #1 question I have is... Where are the files you are trying to duplicate (the source files), and where are you duplicating those to (the intended target directory/destination)? Another way to ask that is: Are you duplicating from one personal computer to another, or are you trying to pull from an online Internet server's repository to download onto a personal computer? Or are you maybe even trying to do yet something else that is not mentioned above? Your answer(s) might make a difference in the command and flags you could use. As an example, that massively long Draft Email #1 I wrote earlier included this useful tip I just learned for my own usage today: $ wget -c --recursive --no-parent https://www.linuxfromscratch.org/lfs/downloads/development/ That lead came from StackOverflow: https://stackoverflow.com/questions/273743/using-wget-to-recursively-fetch-a-directory-with-arbitrary-files-in-it Just test drove it, and it did work as hoped. That "--no-parent" is telling wget to focus only on the current directory, e.g. for me the LFS "development" download webpage along with any possible child directories found there. Be aware that there can still be some extra junk come in, depending on what webpage is being tapped. The more text content and less HTML, the better. Wget does work as expected, does keep digging into child directories, too, because I just tested a Debian repository related webpage under /debian/dists. That's all I have for now. Just let us know... An aside to wget and rsync Developers: Thank you for your work! Between the two of your packages, it's a multi-times daily thing going on between us. Cindy :) -- Talking Rock, Pickens County, Georgia, USA * runs with a jingle-jingle *
Re: how to clone apt repository to newest only?
On Mon, Dec 25, 2023 at 12:21:29PM +, �� wrote: [Copied to the poster because they may not be subscribed] > how to clone apt repository to newest only? > Fedora/Red Hat will organize the repository by copying only the most recent > packages from that distribution if you give it the "reposync --newest-only" > option, but Debian doesn't seem to be able to do that. > > What can I do? > > Hi By default, apt will check the dates on the package manifests and bring you up to date based on that. If you install from nothing then the installer will do the same assuming that you have an internet connection. reposync is really a Red Hat ecosystem specific command, I think. (already answered on the list: can I suggest that you subscribe to the list) Andy (amaca...@debian.org)
how to clone apt repository to newest only?
how to clone apt repository to newest only? Fedora/Red Hat will organize the repository by copying only the most recent packages from that distribution if you give it the "reposync --newest-only" option, but Debian doesn't seem to be able to do that. What can I do?
Re: how to clone apt repository to newest only?
On Wed, Dec 20, 2023 at 06:05:46AM +, �� wrote: > how to clone apt repository to newest only? > Fedora/Red Hat will organize the repository by copying only the most recent > packages from that distribution if you give it the "reposync --newest-only" > option, but Debian doesn't seem to be able to do that. > > What can I do? > By default, apt will check the dates on the package manifests and bring you up to date based on that. If you install from nothing then the installer will do the same assuming that you have an internet connection. reposync is really a Red Hat ecosystem specific command, I think. Andy
how to clone apt repository to newest only?
how to clone apt repository to newest only? Fedora/Red Hat will organize the repository by copying only the most recent packages from that distribution if you give it the "reposync --newest-only" option, but Debian doesn't seem to be able to do that. What can I do?
