Re: Why Debian packaging structure is so difficult
On 09/01/2023 13:20, Sadhu Santh wrote: If not, why the upstream structure is not made simpler? If I remember correctly, current repository layout minimizes traffic at the moment of new release. .deb packages are already synced as a part of testing suite. Almost certainly it is explained on some official page or in a manual, but I can not figure out where I saw it. For tools aware of Debian repository structure see https://www.debian.org/mirror/ftpmirror https://wiki.debian.org/DebianRepository/Setup#Debian_Repository_Mirroring_Tools Personally I use apt-cacher-ng, so I can not comment reliability of other tools.
Re: Why Debian packaging structure is so difficult
On Fri, Jan 13, 2023 at 01:59:50PM +0530, Sadhu Santh wrote: > > > > I am hosting a local Debian mirror for my LAN. This helps in low internet > > > bandwidth use. > > > > > > I keep only the required distributions (past five years and testing > > > release). > > > > > > Compared to other distributions such ArchLinux/RockyLinux (single line > > > Rsync > > > can do the job), the partial mirroring over Rsync in Debian is a complex > > > process. > > > > > > I use ftpsync, which can run on any Linux distribution. > > > Ftpsync is also available in Debian - the ftpsync.conf file will effectively do your mirroring for you > > > Is there any simple set of Rsync commands to sync a particular version of > > > the platform (e.g. x86) of Debian? If not, why the upstream structure is > > > not > > > made simpler? > > > > > > Thanks for your guidance on the matter. > > > The best solution, IMHO, is to mirror using a tool that is aware of the > > specific structure of apt repositories. Personally, I have had > > excellent success with apt-cacher-ng, which functions much like a squid > > proxy. > > Another possibility seems like it would be apt-mirror. However, I have > > never used it and so I cannot give a specific or detailed > > recommendation. > > > Thanks for your kind suggestions. > > I suppose acngwill only function if the operating system is Debian or a > close relative. > My current archive arrangement was created on CentOS a few years ago, and > ftpsync worked. > http://flosslinuxblog.blogspot.com/2020/02/rebuilding-mirror-software-mirroring-of.html might help - it's how I set up my mirror. The post after that is how I was mirroring EPEL and setting up Apache - I now use nginx. It's very straightforward, I use this daily. The two stage rsync is to ensure that the metadata is working so that Debian packages can be validated against checksums and so on. For Rocky, I now use the Rocky mirroring script: for EPEL, I'm still using the Fedora script. > > After CentOS was withdrawn, we reverted to Debian on the majority of > machines. > > Repository synchronisation for CentOS and related systems is frequently > completed with a single line of rsync command via cron. > > So why use a different tool to achieve synchronisation. > See above: ftpsync will mostly work. Critically, keeping timestamps and lockfiles means that you only download when upstream has changed. > is thereany plans to make the repository structure simpler, or did I miss > something? > > Best Regards, > > All the very best, as ever, Andy Cater
Re: Why Debian packaging structure is so difficult
On 09/01/23 17:48, Roberto C. Sánchez wrote: On Mon, Jan 09, 2023 at 11:50:23AM +0530, Sadhu Santh wrote: Hi, I am hosting a local Debian mirror for my LAN. This helps in low internet bandwidth use. I keep only the required distributions (past five years and testing release). Compared to other distributions such ArchLinux/RockyLinux (single line Rsync can do the job), the partial mirroring over Rsync in Debian is a complex process. I use ftpsync, which can run on any Linux distribution. Is there any simple set of Rsync commands to sync a particular version of the platform (e.g. x86) of Debian? If not, why the upstream structure is not made simpler? Thanks for your guidance on the matter. The best solution, IMHO, is to mirror using a tool that is aware of the specific structure of apt repositories. Personally, I have had excellent success with apt-cacher-ng, which functions much like a squid proxy. Another possibility seems like it would be apt-mirror. However, I have never used it and so I cannot give a specific or detailed recommendation. Thanks for your kind suggestions. I suppose acngwill only function if the operating system is Debian or a close relative. My current archive arrangement was created on CentOS a few years ago, and ftpsync worked. After CentOS was withdrawn, we reverted to Debian on the majority of machines. Repository synchronisation for CentOS and related systems is frequently completed with a single line of rsync command via cron. So why use a different tool to achieve synchronisation. is thereany plans to make the repository structure simpler, or did I miss something? Best Regards,
Re: Why Debian packaging structure is so difficult
On Mon, 9 Jan 2023 07:35:18 -0700 Charles Curley wrote: > On Mon, 9 Jan 2023 07:18:53 -0500 > Roberto C. Sánchez wrote: > > > The best solution, IMHO, is to mirror using a tool that is aware of > > the specific structure of apt repositories. Personally, I have had > > excellent success with apt-cacher-ng, which functions much like a > > squid proxy. You configure apt-cacher-ng with the sources you like, > > then you configure the clients on your network with the same sources > > but tell them to use the apt-cacher-ng proxy. > > I second using apt-cacher-ng (acng). I've never configured acng itself, > just the clients, and that is simple and easily scripted. > > The only thing it doesn't do is cache https only repositories. Those are > few and far between, and the reason it doesn't is inherent in https, so > I doubt any similar program will cache them. There are ways for clients > to deal with that limitation. In my limited experience, HTTPS-only repositories are actually fairly common: I've had to enable, at one point or another, apt-cacher-ng workarounds (such as adding them to a PassThroughPattern) for a bunch of repositories, including those of WineHQ, Xpra, VScodium, and the Tor Project. -- Celejar
Re: Why Debian packaging structure is so difficult
On Mon, 9 Jan 2023 07:18:53 -0500 Roberto C. Sánchez wrote: > The best solution, IMHO, is to mirror using a tool that is aware of > the specific structure of apt repositories. Personally, I have had > excellent success with apt-cacher-ng, which functions much like a > squid proxy. You configure apt-cacher-ng with the sources you like, > then you configure the clients on your network with the same sources > but tell them to use the apt-cacher-ng proxy. I second using apt-cacher-ng (acng). I've never configured acng itself, just the clients, and that is simple and easily scripted. The only thing it doesn't do is cache https only repositories. Those are few and far between, and the reason it doesn't is inherent in https, so I doubt any similar program will cache them. There are ways for clients to deal with that limitation. -- Does anybody read signatures any more? https://charlescurley.com https://charlescurley.com/blog/
Re: Why Debian packaging structure is so difficult
On Mon, Jan 09, 2023 at 11:50:23AM +0530, Sadhu Santh wrote: > Hi, > > I am hosting a local Debian mirror for my LAN. This helps in low internet > bandwidth use. > > I keep only the required distributions (past five years and testing > release). > > Compared to other distributions such ArchLinux/RockyLinux (single line Rsync > can do the job), the partial mirroring over Rsync in Debian is a complex > process. > > I use ftpsync, which can run on any Linux distribution. > > Is there any simple set of Rsync commands to sync a particular version of > the platform (e.g. x86) of Debian? If not, why the upstream structure is not > made simpler? > > Thanks for your guidance on the matter. The best solution, IMHO, is to mirror using a tool that is aware of the specific structure of apt repositories. Personally, I have had excellent success with apt-cacher-ng, which functions much like a squid proxy. You configure apt-cacher-ng with the sources you like, then you configure the clients on your network with the same sources but tell them to use the apt-cacher-ng proxy. Whenever a package is downloaded, apt-cacher-ng keeps it around and serves it from the local cache for subsequent requests. You are able to configure how much disk space it uses, how many pervious versions of packages to keep around, etc. Another possibility seems like it would be apt-mirror. However, I have never used it and so I cannot give a specific or detailed recommendation. Regards, -Roberto -- Roberto C. Sánchez
Why Debian packaging structure is so difficult
Hi, I am hosting a local Debian mirror for my LAN. This helps in low internet bandwidth use. I keep only the required distributions (past five years and testing release). Compared to other distributions such ArchLinux/RockyLinux (single line Rsync can do the job), the partial mirroring over Rsync in Debian is a complex process. I use ftpsync, which can run on any Linux distribution. Is there any simple set of Rsync commands to sync a particular version of the platform (e.g. x86) of Debian? If not, why the upstream structure is not made simpler? Thanks for your guidance on the matter. Regards, SS
