Hi Darik, > On Nov 8, 2021, at 6:55 PM, Darik Horn <dajh...@gmail.com> wrote: > > Hi all, > > The FreeDOS repo ISO file that was published today can be reduced from 632MB > to 579MB by repacking it. This adds 53MB of media headroom, which should > reduce pressure to bump packages in future point releases.
Just a note regarding the FreeDOS repo ISO... The Repo ISO image is mostly provided just as a convenience. The creation and publishing of that ISO is part of a fully automated process that is handled by the repository management utilities. Once per day in the early morning hours, the management utilities check to see if any packages have been added or updated. When it notices that occurred, it gathers all of the latest versions for each package and generates a ISO out of them. The creation of that ISO is not very complicated. However, it is not just a simple copy and burn image. It also goes through and generates listing and other metadata files for programs like FDNPKG and FDIMPLES. It also generates a bunch of “flat” html pages for that image as well. There are several “packages” that are not in the repository at present that may be added later. There are various reasons for their current exclusion. For example, someday PC/GEOS[1] may be added. I have not discussed it with it’s developers for quite a long while. But, the last time I did, they did not feel it was ready for general public use/testing. That may be different now. Anyhow, the full version of that is around 120MB. I only see the total amount of software in the repository growing over time. Assuming we continue to provide a convenient download ISO that contains all of those packages, we will need a long term solution. That would probably involve splitting package groups onto separate images or possible a dvd iso. > > Original (as of 2021-11-08): > https://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/repositories/latest/cdrom.iso > > <https://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/repositories/latest/cdrom.iso> > > Repack: > https://drive.google.com/drive/folders/1b7IE-EsK5R1ROmXTaEvYNi9-kfVvH5wi?usp=sharing > > <https://drive.google.com/drive/folders/1b7IE-EsK5R1ROmXTaEvYNi9-kfVvH5wi?usp=sharing> > > The procedure is: > > 1. Unzip each FDN package. > 2. Further unpack each ZIP and 7Z source file. > 3. Zip+Store each SOURCES/* tree like it has LFN. > 4. Rezip each FDN package like usual. > 5. Optimize each package with AdvanceCOMP. > > The size at each step is: > > 660,869,928 bytes, original > 629,362,932 bytes, rezipped with InfoZip 3.0 > 605,766,821 bytes, optimized with AdvZip 2.1 > > Everything stays in zip format 2.0 and remains compatible with PKUnzip 2.04g > and FDNPKG. > > Supposing that other ARCHIVER/* formats were implemented by the FreeDOS > packaging system, the cdrom.iso size could be: > > 627,403,406 bytes, ARJ (-jm) > 557,334,645 bytes, ZIP/BZ2 (-9) > 495,380,042 bytes, RAR2 (-m5 -mdE -mm) > 449,729,467 bytes, 7Z/LZMA2 (-mx9 -ms=on -mqs=on -md=1m) The test results you’ve provided are interesting. They definitely show an improvement over the current packages. It would be better to recompress the packages when they are added to the repository. There are several benefits to doing it at that point. As you have noted, it would reduce the space required on the repo ISO. It would also reduce the space used on the server and download size/time for individual packages. Recompression is something that will be considered for the next version of the repository management utilities. The current version of the repo utils is something like version 3 or 4. I haven’t really been counting. Nowadays, the management of the repo is fully automated. It makes things much easier to manage. For example, we can just push an updated package on the server and walk away. It will do a bunch of verification stuff on the package and repository itself. Locate the older version. Version out the different copies. Adjust some symlinks for updater programs like FDNPKG. Create metadata and index files. Create/update static browsable HTML files from customizable templates. Maintains an RSS feed. Builds repository status vs previous OS release comparison charts. Publishes a package ISO. And a couple other things. There are plans for a new version of the repo management utilities. There are a number of new features and things that can be improved. Just for example… Image and screen snapshots for packages. Improved directory versioning hierarchy. External software metadata (HTML pages for software not in the repo and link to an external website). And many many more. > > UnRAR 2.50 is particularly impressive because it is 16-bit, fast, and will > extract on an 8088 with 640KB of memory. It also has an output size > competitive with the kind of 7-Zip 19 archive that can be extracted by 7ZDEC. > > For fun, I also replaced SLICER with RAR and reduced the number of > installation floppies from eight to three. These repacked floppy images are > also in the linked folder. SLICER is NOT a compression program. It is more like TAR. But even then, that comparison is only valid with glance at some of the features. SLICER is a small keyword/tag based archiver, file splitting and installer hybrid program. There is support for compression built into the SLICER archive file format. However at present, SLICER does not have support for doing compression. Eventually, it will get it. But, probably not for a while. I spent a good deal of time considering the many different needs a of Floppy Edition. Also, what could be done with existing utilities. SLICER is far from perfect. But, it is the only archiver that meets all of the required needs for the Floppy Edition. Here are just some some of the key reasons it was created and is being used. SLICER is tag/keyword based. This off-loads a good deal of work that is needed from the batch based installer. The installer gathers a little information regarding the system and passes those keywords to SLICER which extracts/installs the appropriate files for the system. For example, if the installer sees it is running on a 286 there is no point (with some exceptions, and it can be overridden) and possible detrimental to install software requiring a 386. This in combination with tags related to video, virtual machine, etc are all passed along and provide a variety of installation possibilities. While it is possible to kind-of do something similar to tags/keywords with TAR or another archiver, that causes several problems. Mostly, it would add a lot of overhead to the batch installer. And, it would require multiple extraction passes and disk swaps. One for each individual tag/keyword. SLICER is small. Less than 1/10th the size of RAR. While, this does not matter as much for a 1.44mb diskette. It matters a lot if a user only has a 160K drive to work with. With RAR being roughly 353K, that is not going to happen. All that being said. By no means do I love SLICER. It’s really weird and rather a pain to use. For 99.9% of things, other utilities like RAR, TAR, GZIP, etc are much easier to use. :-) Jerome —— [1] PC/GEOS sources and event CI builds can be downloaded from their GitHub page at https://github.com/bluewaysw/pcgeos/releases <https://github.com/bluewaysw/pcgeos/releases> Also, you can find and chat with them on Slack at https://bluewaysw.slack.com/ <https://bluewaysw.slack.com/> and their website https://blog.bluewaysw.de <https://blog.bluewaysw.de/>
_______________________________________________ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
