Re: [Freedos-user] FreeDOS and LFN

[Jerome Shidel]

Hi Joseph,
 
> I do notice that the dir command doesn’t seem to display them, and I saw 
> nothing in the help for dir about lfn’s.

I keep thinking about doing a new directory lister. I’ve never been fond of the 
DOS one. It’s been decades since the last version of my D.EXE program. If I 
ever find the time, I’d make a new one with LFN support. Someday. Hopefully.

:-)

Jerome

___
Freedos-user mailing list
Freedos-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-user

Re: [Freedos-user] Floppy Installer repack of FreeDOS 1.3-RC5

[Jerome Shidel]

> On Dec 18, 2021, at 4:19 PM, Darik Horn  wrote:
> 
>> That starts to get complicated when trying to just use a program like UNZIP. 
>> At nearly 200K for just the binary,
> 
> The unzip-5.52 binary in the repack is 50,229 bytes.  Which versions
> are you using?

The latest version we have 6.0. 
https://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/repositories/latest/pkg-html/unzip.html
 


It is a 1.6m package with Sources.
It is 132K compressed without sources.
The Binary is 192.5K uncompressed.

It is not 8086 compatible and requires DPMI. So That is another 20.8K. 
So, just for the current UnZIP and CWSDPMI binaries about 220K is required.

> Compare versus 27,814 + 39,910 = 67,724 bytes for slicer and gzip in
> FreeDOS 1.3-RC5.
> 

I think that could be a little mis-leading. 

For this reason… At present, both UnZip and Gzip are installed on 386+ 
machines with the FloppyEdition. So, their overall size is not very important.
What does matter is the amount of space required on the boot/install diskette.
Whether UnZip needs 220k or 50k, that is still bigger than 27k needed by 
Slicer. Gzip, is not on the boot disk. It is first in the archive and extracted 
by Slicer. Slicer then begins using it automatically. So, the size of GZip
is not relevant. Actually, a few kbytes could be squeezed out of the archive
by splitting Gzip into 2 pieces. The first piece would be just the 8086 EXE.
The second piece could then be compressed with the 386 version and 
everything else in the current Gzip package. 


> 
>> But, for your tests. They should be split at the same size 59K for user 
>> diskette compatibility.
> 
> Why is this the right size?  The current slice size of 60,416 bytes
> doesn't have a filesystem overhead, which seems intentional, but it
> isn't apparent how the end-user benefits.

In Part, the RBE only needs to create one Archive (using DOS), then
it can just spread it out across the different media quickly using Linux.
It is also less code in the RBE. The User benefits by not only getting 
a single 1.44mb version. 

The 59k slices spread nicely across the different media.

720k only 3k remaining.
1.2mb only 4.5k remaining.
1.44mb only 6.5k remaining.

Intent is also to eventually do a little restructuring of the boot disk
and installer to allow provide 360k version as well. But, that is an 
extremely low priority. It would have around 1-2k free space 
remaining.

The smaller slice size also allows the usage of not 100% healthy 
disks that have had bad sectors mapped out. You would need to
know how to do that and manually copy the slices. But, it is still
an option.

> 
> I'm getting better results with larger files by doing this...
> 
> In a 1.44MB image, a default "format a:" has 1,457,664 bytes usable,
> less directory overhead and cluster slack of a few kilobytes.  (eg:
> The FreeDOS boot disk.)
> 
> If I tune the image with "mformat -c4 -r1 a:" and put one big file
> inside, then I get an additional 12,800 bytes of payload space, am
> guaranteed to get zero directory overhead and zero cluster slack, and
> get maximum throughput from the floppy drive.
> 
> This savings won't always matter, but it is the reason why the
> ZipSplit build has one fewer disk than the Slicer build.
> 
>> I don’t feel a couple percentage points better compression work the time and 
>> testing to
>> make large changes to the FloppyEdition installer. But, it’s not like it 
>> cannot be changed.
> 
> This is reasonable given how late it is in the release cycle, and the
> gzip enhancement is indeed good.  

I think the level of compression provided with pass-through Gzip is 
“good enough”. An overall performance is on par with MS-DOS.
A recent test install of MS-DOS onto my Pentium-Pro took roughly 
5-6 minutes to copy and extract the DOS files from floppy. Installing
RC5 from floppy to an Intel Atom based Netbook took about 7-8 
minutes to extract the 5 diskettes of FreeDOS packages. There are 
numerous hardware difference between the two machines. So, it
was not a benchmark speed test. But, does show it takes a similar
about of time per disk. 

There would need to be a very large improvement in compression
to make it worthwhile in adding additional overhead or just reworking
the portion of the RBE that creates the FloppyEdition.

The RBE was a lot of work to create. It fully automates the creation of
a release. Basically, anyone can follow the directions in the wiki and
create a HOST VM for the RBE. Use git to checkout the RBE. Then
run install as sudo/superuser to configure the system automatically.
Once that is done, to create a release… type “make” and hit enter.
Go to lunch, come back you’ve got a FreeDOS release with the 
latest packages, in all the various formats that is ready to send out
the door. 

By no means is the RBE perfect. This is version 3 and it may have a 
couple minor 

Re: [Freedos-user] Floppy Installer repack of FreeDOS 1.3-RC5

[Darik Horn]

> That starts to get complicated when trying to just use a program like UNZIP. 
> At nearly 200K for just the binary,

The unzip-5.52 binary in the repack is 50,229 bytes.  Which versions
are you using?

Compare versus 27,814 + 39,910 = 67,724 bytes for slicer and gzip in
FreeDOS 1.3-RC5.


> But, for your tests. They should be split at the same size 59K for user 
> diskette compatibility.

Why is this the right size?  The current slice size of 60,416 bytes
doesn't have a filesystem overhead, which seems intentional, but it
isn't apparent how the end-user benefits.

I'm getting better results with larger files by doing this...

In a 1.44MB image, a default "format a:" has 1,457,664 bytes usable,
less directory overhead and cluster slack of a few kilobytes.  (eg:
The FreeDOS boot disk.)

If I tune the image with "mformat -c4 -r1 a:" and put one big file
inside, then I get an additional 12,800 bytes of payload space, am
guaranteed to get zero directory overhead and zero cluster slack, and
get maximum throughput from the floppy drive.

This savings won't always matter, but it is the reason why the
ZipSplit build has one fewer disk than the Slicer build.


> I don’t feel a couple percentage points better compression work the time and 
> testing to
> make large changes to the FloppyEdition installer. But, it’s not like it 
> cannot be changed.

This is reasonable given how late it is in the release cycle, and the
gzip enhancement is indeed good.  Nevertheless, this is a fun
optimization job, and I'm getting slightly better results using
standard tools.


> There have been requests to add very useful programs like EDIT to that boot 
> diskette.

This might be doable; I will look at it.


___
Freedos-user mailing list
Freedos-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-user

Re: [Freedos-user] Floppy Installer repack of FreeDOS 1.3-RC5

[Jerome Shidel]

> On Dec 18, 2021, at 11:43 AM, Darik Horn  wrote:
> 
> Jerome,
> 
>> Did your repack include ALL packages included in the FloppyEdition not just 
>> i386.
> 
> Yes, the repack contains 1,177 files.
> 
> 
>> For example, VirtualBox and VMWare include FDNet as well as the set for 386s.
> 
> Note that these things have empty categories in the official build, per below.

Technically correct. 

When the installer sees it is running on a VM, it includes a tag for that VM.
That tag doesn’t have any correlation in the archive at present. 

But, when the installer sees it is running on VirtualBox or VMWare, it also
throws in the Network Tag. Which includes FDNet. And possible other
network based programs in the future. 

In theory, we could include a SOURCE tag that would then include the
appropriate sources. But, that would needlessly bloat the FloppyEdition. All 
the sources are available on the CD/USB media and online. So, there is no need
to waist space with them on the diskettes.

The TAG system and hardware detection, may be expanded in the future to 
include things like Mouse, Joystick, Printer, etc. 

Each program is also tagged with it’s name. That way if someone for some reason
wanted to extract just that program, they can. But the installer doesn’t use 
those 
TAGS either. 

I just remembered, there is one inefficiency in the archive that is not related 
to
SLICER itself and would account for more of the %5 difference to ZIP. Basically,
the RBE creates a fake package called FDINST specifically for machine that 
cannot support 386 or better programs. FDINST is part of FDNPKG. At present,
the RBE includes two packages FDINST and FDNPKG. At some point, it will
be told to handle that differently. At present, the files related to FDINST are 
in the
the archive twice. It is not a SLICER issue, just something I haven’t got 
around to
make the RBE build the archive more efficiently.

> 
>> Does it make any accommodations for different hardware?
> 
> Yes, the hardware detection logic is unchanged.
> 
> 
>> Or just install everything wether or not it is supported by the target 
>> platform?
> 
> In the RC4 repack, yes, the @LISTFILE method is equivalent to the
> "slicer /g" switch.
> 
> In the RC5 repack, no, it just does the same thing as "/g *".
> 
> //
> 
> Two unexpected results from the second experiment are:
> 
> * The tarball method isn't worthwhile, despite using a solid archive format.
> * The InfoZIP zipsplit method is better than the PKZip spanning method.

This made me think of another thought…

For maximum diskette capacity compatibility (and not requiring the end user to 
re-split an archive),
slicer is using 59K slices. That size wastes only a little space on 360K,720K, 
1.2M, 1.44M. It would also 
be workable on smaller sizes. But no-one is going to use a 120K diskette. It’s 
doubtful anyone will ever
try to use 360K or smaller diskettes. But, if they are lucky enough to have the 
hardware and diskettes, a 
user may want to stick the release on a couple 2.88M diskettes.

But, for your tests. They should be split at the same size 59K for user 
diskette compatibility.

Also, you be sure to consideration the extra space required for the archiver. 
Some are quite large. Slicer
gets around that problem. The first thing it extracts is GZIP which is 
uncompressed in the archive. 
Everything after that is Gzip compressed. 

That starts to get complicated when trying to just use a program like UNZIP. At 
nearly 200K for just the binary,
it won’t fit on the 720k boot diskette. At present, that diskette has only 120K 
free and that may be reduced
somewhat as well. There have been requests to add very useful programs like 
EDIT to that boot diskette.

> 
> //
> 
> In the official FreeDOS 1.3-RC5 floppy edition, %TTAGS% for things
> like networking and virtualization are empty.  When I manually unpack
> FREEDOS.SAF, I get these categories and file counts:
> 
> 186: 919
> 286: 1027
> 386: 1155
> 486: 1155
> 586: 1155
> 686: 1155
> 8086: 919
> 8088: 919
> AMBHELP: 11
> AMBREAD: 7
> APPEND: 14
> ASSIGN: 17
> ATTRIB: 11
> CALLVER: 6
> CGA: 807
> CHKDSK: 6
> CHOICE: 33
> COMP: 7
> CPIDOS: 27
> CTMOUSE: 34
> CWSDPMI: 14
> DEBUG: 17
> DEFAULT: 0
> DEFRAG: 7
> DELTREE: 11
> DEVLOAD: 9
> DISKCOMP: 7
> DISKCOPY: 19
> DISPLAY: 10
> EDIT: 10
> EDLIN: 40
> EGA: 1177
> EXE2BIN: 13
> FC: 26
> FDAPM: 9
> FDHELPER: 26
> FDIMPLES: 16
> FDINST: 5
> FDISK: 26
> FDNET: 20
> FDNPKG: 21
> FDXMS: 13
> FDXMS286: 12
> FIND: 27
> FORMAT: 13
> FREECOM: 48
> FREEDOS: 1177
> GRAPHICS: 11
> GZIP: 11
> HIMEMX: 10
> JEMM: 24
> KERNEL: 26
> KEYB: 10
> KEYB_LAY: 25
> LABEL: 13
> LBACACHE: 13
> MEM: 19
> MIRROR: 16
> MKEYB: 10
> MODE: 8
> MORE: 33
> MOVE: 19
> NANSI: 8
> NETWORK: 20
> NLSFUNC: 11
> PRINT: 7
> RECOVER: 7
> REPLACE: 9
> SHARE: 6
> SHSUCDX: 8
> SHSUFDRV: 12
> SORT: 21
> SWSUBST: 7
> TREE: 22
> UDVD2: 7
> UNDELETE: 11
> UNFORMAT: 12
> UNZIP: 11
> V8POWER: 144
> VGA: 1177
> XCOPY: 19
> ZIP: 18

Interesting. 

I don’t feel a couple 

Re: [Freedos-user] Floppy Installer repack of FreeDOS 1.3-RC5

[Darik Horn]

Jerome,

> Interesting. There are other compression methods to try. For example, lzma, 
> bzip2, arj, etc.

These codecs were excluded in the first repack series.  Results were:

1. LMZA1

Not significantly better than ZIP while using a dictionary size
appropriate for small-memory machines.  And 16-bit utilities are not
in distro.

2. ARJ

Belov's implementation from http://arj.sourceforge.net/ has
performance similar to ZIP.  I didn't try the official non-free
encoder because I couldn't get a license for it.

3. BZIP2

Way too slow.


> Overall, I think using GZip for the pass-through compression works well 
> enough, supports 8086 and uses little space.

Deflate32 as implemented by GZip or InfoZIP is definitely the
sweet-spot for baseline installation targets, especially with zopfli
post-processing.

If I do a third series, then I will try LZ4 and multivolume 7Z, but
I'm also tempted to implement a RAR2 encoder for unrarlib and/or
libarchive.


___
Freedos-user mailing list
Freedos-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-user

Re: [Freedos-user] Floppy Installer repack of FreeDOS 1.3-RC5

[Darik Horn]

Jerome,

> Did your repack include ALL packages included in the FloppyEdition not just 
> i386.

Yes, the repack contains 1,177 files.


> For example, VirtualBox and VMWare include FDNet as well as the set for 386s.

Note that these things have empty categories in the official build, per below.


> Does it make any accommodations for different hardware?

Yes, the hardware detection logic is unchanged.


> Or just install everything wether or not it is supported by the target 
> platform?

In the RC4 repack, yes, the @LISTFILE method is equivalent to the
"slicer /g" switch.

In the RC5 repack, no, it just does the same thing as "/g *".

//

Two unexpected results from the second experiment are:

* The tarball method isn't worthwhile, despite using a solid archive format.
* The InfoZIP zipsplit method is better than the PKZip spanning method.

//

In the official FreeDOS 1.3-RC5 floppy edition, %TTAGS% for things
like networking and virtualization are empty.  When I manually unpack
FREEDOS.SAF, I get these categories and file counts:

186: 919
286: 1027
386: 1155
486: 1155
586: 1155
686: 1155
8086: 919
8088: 919
AMBHELP: 11
AMBREAD: 7
APPEND: 14
ASSIGN: 17
ATTRIB: 11
CALLVER: 6
CGA: 807
CHKDSK: 6
CHOICE: 33
COMP: 7
CPIDOS: 27
CTMOUSE: 34
CWSDPMI: 14
DEBUG: 17
DEFAULT: 0
DEFRAG: 7
DELTREE: 11
DEVLOAD: 9
DISKCOMP: 7
DISKCOPY: 19
DISPLAY: 10
EDIT: 10
EDLIN: 40
EGA: 1177
EXE2BIN: 13
FC: 26
FDAPM: 9
FDHELPER: 26
FDIMPLES: 16
FDINST: 5
FDISK: 26
FDNET: 20
FDNPKG: 21
FDXMS: 13
FDXMS286: 12
FIND: 27
FORMAT: 13
FREECOM: 48
FREEDOS: 1177
GRAPHICS: 11
GZIP: 11
HIMEMX: 10
JEMM: 24
KERNEL: 26
KEYB: 10
KEYB_LAY: 25
LABEL: 13
LBACACHE: 13
MEM: 19
MIRROR: 16
MKEYB: 10
MODE: 8
MORE: 33
MOVE: 19
NANSI: 8
NETWORK: 20
NLSFUNC: 11
PRINT: 7
RECOVER: 7
REPLACE: 9
SHARE: 6
SHSUCDX: 8
SHSUFDRV: 12
SORT: 21
SWSUBST: 7
TREE: 22
UDVD2: 7
UNDELETE: 11
UNFORMAT: 12
UNZIP: 11
V8POWER: 144
VGA: 1177
XCOPY: 19
ZIP: 18


___
Freedos-user mailing list
Freedos-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-user

[Freedos-user] FreeDOS and LFN

[Joseph Norton]

Hi listers: I realize this is probably old hat, but, I’m curious about the issue of LFN support on FreeDOS. FreeDOS is touted to support long file names. However, the support for them is commented out, but, of course, easily enabled in fdauto.bat. I do notice that the dir command doesn’t seem to display them, and I saw nothing in the help for dir about lfn’s. What are the considerations in using FreeDOS with LFN support?If no one feels like an explanation, maybe I can be directed to some documentation on the advisability or lack thereof of using the long file name support? Thanks!

___
Freedos-user mailing list
Freedos-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-user