Re: [Freedos-devel] Another implementation of GRAFTABL
Hi, On Wed, 13 Jul 2022 at 00:22, Steve Nickolas wrote: > On Tue, 12 Jul 2022, Aitor Santamaría wrote: > (I think the PC Convertible is, in fact, the only reason there's even an > option - it supports loadable charsets, but only in 8x8, so installing a > full charset, 8x8+8x14+8x16, would be a waste.) > It's so long that I last touched DISPLAY, but I seem to remember that the subdriver may decide where to cut the codepages. This way, one could write a Convertible sub-driver that cuts them to 8x8 (probably same for CGA). Aitor ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On Tue, 12 Jul 2022, Aitor Santamaría wrote: Once solved the "legal issues", one way to go could be to enrich DISPLAY instead of making a new driver. DISPLAY is organised as having "sub-drivers": DISPLAY CON = (EGA,437,2) where "EGA" is used to select the sub-driver, 437 is to declare the codepage that is hardwired in the hardware, and 2 is the number of how many room for CPIs you want. Ideally, you could add a "CGA" sub-driver, so that instead of GRAFTABL, one could issue DISPLAY CON=(CGA,437,1) and you save from the TSR stuff, the codepage loading stuff and the interrupts hooking stuff. Just an idea that lingered in my mind for a lot of time, although I never had an enormous interest to care about CGA. Aitor I don't think there's any use for DISPLAY.SYS outside of EGA, VGA, and the PC Convertible. There isn't a way to override the text charset, which is what DISPLAY.SYS needs. (I think the PC Convertible is, in fact, the only reason there's even an option - it supports loadable charsets, but only in 8x8, so installing a full charset, 8x8+8x14+8x16, would be a waste.) -uso.___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
Hello Steve, On Tue, 12 Jul 2022 at 12:19, Steve Nickolas wrote: > I've been working on this for my own project, and it's only useful for the > rare CGA or Tandy users out there (which is probably why MS-DOS 6 > relegated it to the Supplemental disk and PC DOS 6 removed it). > > GRAFTABL hooks INT1F to provide the upper half of the charset in graphics > mode on a CGA or Tandy. (DISPLAY.SYS does this and more on more modern > display adapters.) The current version seems to be incomplete - only > supports a single hardwired codepage? - so I figured I'd offer my own, > which is my first successful attempt at a TSR that actually does > something. > Once solved the "legal issues", one way to go could be to enrich DISPLAY instead of making a new driver. DISPLAY is organised as having "sub-drivers": DISPLAY CON = (EGA,437,2) where "EGA" is used to select the sub-driver, 437 is to declare the codepage that is hardwired in the hardware, and 2 is the number of how many room for CPIs you want. Ideally, you could add a "CGA" sub-driver, so that instead of GRAFTABL, one could issue DISPLAY CON=(CGA,437,1) and you save from the TSR stuff, the codepage loading stuff and the interrupts hooking stuff. Just an idea that lingered in my mind for a lot of time, although I never had an enormous interest to care about CGA. Aitor ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Re: Another implementation of GRAFTABL
On Tue, 12 Jul 2022, Wilhelm Spiegl wrote: Hi, I just looked at my website http://home.mnet-online.de/willybilly/fdhelp-108a-internet/en/hhstndrd/base/graftabl.htm and noticed that the documentation for graftabl from 2007 is extremely short, strictly speaking: there is none. As I never learned what graftabl is really good for, could you please write and add at least a short and understandable documentation and some examples for it. Would be very nice and helpful for me and the FD help. Thx for doing so It serves a very specific purpose, and isn't very useful these days (which is why it was relegated to supplemental with MS-DOS 6 and omitted with PC DOS 6.1). Basically, when you enter graphics mode on a CGA, it gets the first half of its character set from F000:FA6E, and the other half from wherever INT1F points. INT1F by default points to nothing in particular so you'll get garbage if you try to output 80-FF. GRAFTABL loads this missing half of the character set into RAM, and points INT1F to it. On an EGA or VGA, DISPLAY.SYS and MODE do everything that GRAFTABL does and more, so it makes no sense to install it there. Also, it doesn't do any good in text mode. When it was introduced in PC DOS 3.0, it only contained one character set. This became 4 (437, 860, 863, 865) in 3.3, 5 (850 was added) in 4.0, and 6 (852 was added) in 5.0. Because in the time between MS-DOS 5 and 6 it became more common to have up-to-date PCs, with VGA, GRAFTABL became even less useful than it was, and was relegated to the Supplemental disk starting with version 6.0. GRAFTABL is the same as GRAFTABL 437, probably for backward compatibility with MS-DOS 3.0-3.21. -uso. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Re: Re: Another implementation of GRAFTABL
I just noticed that i already wrote an updated version for the next version of help, see:http://home.mnet-online.de/willybilly/HTMLhelp-110/en/hhstndrd/base/graftabl.htmIf there is something wrong with it, please send a mail to my mail addressThx--Diese Nachricht wurde von meinem Android Mobiltelefon mit mail.com Mail gesendet.Am 12.07.22, 22:42 schrieb Wilhelm Spiegl : Hi, I just looked at my website http://home.mnet-online.de/willybilly/fdhelp-108a-internet/en/hhstndrd/base/graftabl.htm and noticed that the documentation for graftabl from 2007 is extremely short, strictly speaking: there is none. As I never learned what graftabl is really good for, could you please write and add at least a short and understandable documentation and some examples for it. Would be very nice and helpful for me and the FD help. Thx for doing so Willi -- Diese Nachricht wurde von meinem Android Mobiltelefon mit mail.com Mail gesendet. Am 12.07.22, 21:52 schrieb Steve Nickolas : On Tue, 12 Jul 2022, C. Masloch wrote: > Your technique is the default way to do this, most TSRs ever written did > it that way. The SMC is just a small optimisation over it. Codegolfing is good, when done right. :D > If you really want to continue to use the free software release of > Microsoft's Debug, you may want to look into the following: > > * Some notes on building the original sources, probably not needed if > you already figured out how to manage that. [1] Yeah, I'd actually managed to roll a few things before then, though I think I may have referenced that later. > * A fix to the CALL 5 bug (that was preserved all the way into the > latest Microsoft Debug versions) [2] Yep, started from that codebase. I think John Elliott might have been the first one to actually roll MSDOS.SYS. > * Some discussion on additional bugs in this Debug (might be fixed by > later MS-DOS Debug) at [3], in particular the fact that it doesn't set > itself up as self-parented and also fails to create a proper child PSP > for the client when just assembling into the code segment. The child PSP > problem is not relevant when loading a program into the debugger. > > * For historical reference, there are some versions of the original > 86-DOS Monitor and Debug out there, some of which explicitly note that > copyright does not apply to them. > > Other than that, do have a look at FreeDOS Debug [4] or my repo of its > history going back decades [5], as well as my fork, lDebug [6]. Another > fork with fewer changes is debug.gen [7]. There is also Enhanced Debug > in the same family but that one is not free software, it's not > redistributable. > > Regards, > ecm Yeah, I'm aware of the PC DOS guy's fork of FreeDOS DEBUG. It's interesting that in an operating system that is mostly GPL, that one component happened to end up X11-licensed, and MS picked the same license when they opened up MS-DOS 2.11... I managed to get EDLIN and DEBUG to roll with WASM with some help from someone on Libera, and it was a bit of an undertaking. EDLIN was easier, I think. But the code is a bit nasty - delves into MS-DOS internals to get the current directory instead of just calling GETPWD (AH=47), and uses FCBs for everything instead of "Xenix functions". Like, it's a transitional version before all the "Xenix" stuff got fully brought in or something. Eugh. -uso. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Re: Another implementation of GRAFTABL
Hi, I just looked at my website http://home.mnet-online.de/willybilly/fdhelp-108a-internet/en/hhstndrd/base/graftabl.htmand noticed that the documentation for graftabl from 2007 is extremely short, strictly speaking: there is none.As I never learned what graftabl is really good for, could you please write and add at least a short and understandable documentation and some examples for it. Would be very nice and helpful for me and the FD help.Thx for doing soWilli--Diese Nachricht wurde von meinem Android Mobiltelefon mit mail.com Mail gesendet.Am 12.07.22, 21:52 schrieb Steve Nickolas : On Tue, 12 Jul 2022, C. Masloch wrote: > Your technique is the default way to do this, most TSRs ever written did > it that way. The SMC is just a small optimisation over it. Codegolfing is good, when done right. :D > If you really want to continue to use the free software release of > Microsoft's Debug, you may want to look into the following: > > * Some notes on building the original sources, probably not needed if > you already figured out how to manage that. [1] Yeah, I'd actually managed to roll a few things before then, though I think I may have referenced that later. > * A fix to the CALL 5 bug (that was preserved all the way into the > latest Microsoft Debug versions) [2] Yep, started from that codebase. I think John Elliott might have been the first one to actually roll MSDOS.SYS. > * Some discussion on additional bugs in this Debug (might be fixed by > later MS-DOS Debug) at [3], in particular the fact that it doesn't set > itself up as self-parented and also fails to create a proper child PSP > for the client when just assembling into the code segment. The child PSP > problem is not relevant when loading a program into the debugger. > > * For historical reference, there are some versions of the original > 86-DOS Monitor and Debug out there, some of which explicitly note that > copyright does not apply to them. > > Other than that, do have a look at FreeDOS Debug [4] or my repo of its > history going back decades [5], as well as my fork, lDebug [6]. Another > fork with fewer changes is debug.gen [7]. There is also Enhanced Debug > in the same family but that one is not free software, it's not > redistributable. > > Regards, > ecm Yeah, I'm aware of the PC DOS guy's fork of FreeDOS DEBUG. It's interesting that in an operating system that is mostly GPL, that one component happened to end up X11-licensed, and MS picked the same license when they opened up MS-DOS 2.11... I managed to get EDLIN and DEBUG to roll with WASM with some help from someone on Libera, and it was a bit of an undertaking. EDLIN was easier, I think. But the code is a bit nasty - delves into MS-DOS internals to get the current directory instead of just calling GETPWD (AH=47), and uses FCBs for everything instead of "Xenix functions". Like, it's a transitional version before all the "Xenix" stuff got fully brought in or something. Eugh. -uso. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On at 2022-07-12 21:59 +0200, C. Masloch wrote: On at 2022-07-12 15:45 -0400, Steve Nickolas wrote: The new "new2F": new2F: cmp ah, 0xB0 ; is it ours? je .2 .1: jmp 0:0 .2: cmp al, 1 ja .1 ; >1 - chain mov al, 0xFF ; needs set for both functions. je .3 ; 1 - where are we? iret ; 0 - are we here? .3: mov word ds:[bx], entry mov ds:[bx+2], cs iret old2F equ .1+1 ...probably as codegolfed as it's going to get. Found another byte to save: Replace "je .3" then "iret" by "jne .3" then put the ".3" label before the "iret". You only need the one iret at the very end, and the jump after "mov al, 0FFh" should branch to it. Regards, ecm ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On Tue, 12 Jul 2022, C. Masloch wrote: On at 2022-07-12 15:45 -0400, Steve Nickolas wrote: I haven't uploaded a copy of the new source anywhere yet - it'll probably be in the next DOSLITE source batch along with my work on a few other DOS commands, but I don't want to replace the copy I've already uploaded, and DOSLITE isn't ready to go onto anything like github or gitlab yet. I strongly suggest to keep source history. If you'd rather not, you do not need to publish that right now but I do recommend you keep it privately at least. I started in 2010 [1] and it has been a great experience that often proved useful. Yeah. "Upload early, upload often" is very much at play here, something I learned when I lost the hard drive I was keeping a fansub project on. Because I uploaded the raw on BitTorrent and had been maintaining timestamped uploads of the scripts on my Web server, nothing was lost. Several revisions of GRAFTABL that I haven't released, plus the one I did, are archived in a folder along with other tools like ATTRIB, CHOICE, DELTREE, FIND, LABEL, SORT and TREE, plus my hacks on EDLIN and MORE. But I wanted to get a bit closer to my goal before putting them into "proper" revision control - just a few loose commands doesn't feel worth making a project on Github or its many clones. (Or into a Subversion or something on my own server.) Not quite: The "ds:" segment override prefixes are unneeded actually. NASM will happily emit them, but they eat memory completely unnecessarily. "mov word [bx], entry" already defaults to ds as its segment. (Unlike addresses including "[bp]" which default to ss.) I didn't notice those at first because I assumed you were using es. That's what your comment in your caller says: .21: mov ax, 0xB001 ; So where is it? mov bx, whence ; ES:BX gets this. int 0x2F However, the Interrupt List [2] does say it uses ds:bx so your interrupt handler is correct, your caller's comment is incorrect. Oopsie. XD Maybe I wrote "ES" because a lot of stuff that uses BX with a segment uses ES. I almost never use ES myself - my code generally runs in a 64K space. Removing the redundant segment escapes does save a few bytes. -uso. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On at 2022-07-12 15:45 -0400, Steve Nickolas wrote: I haven't uploaded a copy of the new source anywhere yet - it'll probably be in the next DOSLITE source batch along with my work on a few other DOS commands, but I don't want to replace the copy I've already uploaded, and DOSLITE isn't ready to go onto anything like github or gitlab yet. I strongly suggest to keep source history. If you'd rather not, you do not need to publish that right now but I do recommend you keep it privately at least. I started in 2010 [1] and it has been a great experience that often proved useful. As I said, I managed to move the CP437 table into the spot reserved for the resident table (except for the first 3 bytes), which saved good memory, and with C. Masloch's optimizations and by simplifying the version check the file size and resident footprint went down even further. (In an earlier incarnation, I had to jigger around 'jmp far 0', but it turns out 'jmp 0:0' as mentioned was the solution to that.) Yes, good catch, it isn't immediately obvious why NASM won't allow a "far" keyword there. The resident footprint went down 1 paragraph, from 1344 to 1328 - now only 128 bytes larger than the official version. (I wonder if any of the PSP can be just up and deallocated?) You can deallocate the PSP (either with some tricks on the parent stack or just split Memory Control Blocks manually), but it will leave a memory "hole" in its place. This is seldom useful. (Bret's and my method relocate the PSP precisely to avoid creating fragmentation holes when allocating the final resident memory.) The new "new2F": new2F: cmp ah, 0xB0 ; is it ours? je .2 .1: jmp 0:0 .2: cmp al, 1 ja .1 ; >1 - chain mov al, 0xFF ; needs set for both functions. je .3 ; 1 - where are we? iret ; 0 - are we here? .3: mov word ds:[bx], entry mov ds:[bx+2], cs iret old2F equ .1+1 ...probably as codegolfed as it's going to get. Not quite: The "ds:" segment override prefixes are unneeded actually. NASM will happily emit them, but they eat memory completely unnecessarily. "mov word [bx], entry" already defaults to ds as its segment. (Unlike addresses including "[bp]" which default to ss.) I didn't notice those at first because I assumed you were using es. That's what your comment in your caller says: .21: mov ax, 0xB001 ; So where is it? mov bx, whence ; ES:BX gets this. int 0x2F However, the Interrupt List [2] does say it uses ds:bx so your interrupt handler is correct, your caller's comment is incorrect. Regards, ecm [1]: https://hg.pushbx.org/ecm/ldebug/rev/5eef5847c863 [2]: https://fd.lod.bz/rbil/interrup/video/2fb001.html#5274 ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On Tue, 12 Jul 2022, C. Masloch wrote: Your technique is the default way to do this, most TSRs ever written did it that way. The SMC is just a small optimisation over it. Codegolfing is good, when done right. :D If you really want to continue to use the free software release of Microsoft's Debug, you may want to look into the following: * Some notes on building the original sources, probably not needed if you already figured out how to manage that. [1] Yeah, I'd actually managed to roll a few things before then, though I think I may have referenced that later. * A fix to the CALL 5 bug (that was preserved all the way into the latest Microsoft Debug versions) [2] Yep, started from that codebase. I think John Elliott might have been the first one to actually roll MSDOS.SYS. * Some discussion on additional bugs in this Debug (might be fixed by later MS-DOS Debug) at [3], in particular the fact that it doesn't set itself up as self-parented and also fails to create a proper child PSP for the client when just assembling into the code segment. The child PSP problem is not relevant when loading a program into the debugger. * For historical reference, there are some versions of the original 86-DOS Monitor and Debug out there, some of which explicitly note that copyright does not apply to them. Other than that, do have a look at FreeDOS Debug [4] or my repo of its history going back decades [5], as well as my fork, lDebug [6]. Another fork with fewer changes is debug.gen [7]. There is also Enhanced Debug in the same family but that one is not free software, it's not redistributable. Regards, ecm Yeah, I'm aware of the PC DOS guy's fork of FreeDOS DEBUG. It's interesting that in an operating system that is mostly GPL, that one component happened to end up X11-licensed, and MS picked the same license when they opened up MS-DOS 2.11... I managed to get EDLIN and DEBUG to roll with WASM with some help from someone on Libera, and it was a bit of an undertaking. EDLIN was easier, I think. But the code is a bit nasty - delves into MS-DOS internals to get the current directory instead of just calling GETPWD (AH=47), and uses FCBs for everything instead of "Xenix functions". Like, it's a transitional version before all the "Xenix" stuff got fully brought in or something. Eugh. -uso. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
I haven't uploaded a copy of the new source anywhere yet - it'll probably be in the next DOSLITE source batch along with my work on a few other DOS commands, but I don't want to replace the copy I've already uploaded, and DOSLITE isn't ready to go onto anything like github or gitlab yet. As I said, I managed to move the CP437 table into the spot reserved for the resident table (except for the first 3 bytes), which saved good memory, and with C. Masloch's optimizations and by simplifying the version check the file size and resident footprint went down even further. (In an earlier incarnation, I had to jigger around 'jmp far 0', but it turns out 'jmp 0:0' as mentioned was the solution to that.) The resident footprint went down 1 paragraph, from 1344 to 1328 - now only 128 bytes larger than the official version. (I wonder if any of the PSP can be just up and deallocated?) The new "new2F": new2F:cmp ah, 0xB0; is it ours? je.2 .1: jmp 0:0 .2: cmp al, 1 ja.1 ; >1 - chain mov al, 0xFF; needs set for both functions. je.3 ; 1 - where are we? iret ; 0 - are we here? .3: mov word ds:[bx], entry mov ds:[bx+2], cs iret old2F equ .1+1 ...probably as codegolfed as it's going to get. The original xchg ah, al cmp ax, 0x0200 has been replaced with "cmp al, 0x02" in a size optimization. The code following the freeing of the environment has also been replaced to close everything first and do things more efficiently. So far, so good... -uso. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On at 2022-07-12 15:15 -0400, Steve Nickolas wrote: You do not use an IBM Interrupt Sharing Protocol (IISP) [11] header for your interrupt hook. Therefore, you could optimise this part a bit, from: old2F: dd 0x ... .1: jmp far [cs:old2F] Into this: .1: jmp 0:0 old2F equ $ - 4 This is some self-modifying code (SMC) to stash the downlink into an immediate far jump instruction, instead of using the indirect far jump to refer to a different memory location in your code segment. The dollar sign is used in the equate to denote the current assembly position after the 5-byte instruction; it is offset by minus four so as to address the far pointer in the instruction's encoding. (As mentioned, you cannot do this if you use a standard IISP header, because that has a "jmp short $+18" (EBh 10h) instruction right in front of the downlink field.) I believe it was from analyzing some MS-DOS 2.0 code that I got the technique I used here. Your technique is the default way to do this, most TSRs ever written did it that way. The SMC is just a small optimisation over it. If you do use the IISP then the "jmp far [cs:.next]" way is the correct way. (You can also do "pushf" then "call far [cs:.next]" if you want to call instead of chain. Or chain by way of "retf", though that doesn't really have any advantages.) Next, you're using "or al, al" to check a register for zero. However, it is more idiomatic [12] to use "test al, al" instead, which right away hints (to a reader or even to the processor) that no change of the register occurs. Also not sure where I got the "or" idiom, but I know that it doesn't alter the register data. It doesn't modify it indeed, same as "and reg, reg". I just think test is the preferable choice, not that it matters much. (Regarding a here-deleted convo regarding EMS and XMS memory) I've been thinking about how to implement other DOS commands - I'm basically trying to create a compact reimplementation of the MS-DOS 6.21 userland, more or less, and I know I'm in way over my head - and trying to figure out XMS and EMS has been something I've been wracking my brain over for an idea to implement XCOPY, by creating a series of "packets" with commands like "create directory", "open file", "write to file", etc., and store as many of these "packets" in RAM, including XMS and EMS, as possible before dumping the data out to disk. Certainly possible. Also, I've been hacking on MS-DOS 2 DEBUG trying to figure out how to plug in MS-DOS 4 functions (EMS support and read/write on BIGFAT devices) without success, but that's another matter ;p If you really want to continue to use the free software release of Microsoft's Debug, you may want to look into the following: * Some notes on building the original sources, probably not needed if you already figured out how to manage that. [1] * A fix to the CALL 5 bug (that was preserved all the way into the latest Microsoft Debug versions) [2] * Some discussion on additional bugs in this Debug (might be fixed by later MS-DOS Debug) at [3], in particular the fact that it doesn't set itself up as self-parented and also fails to create a proper child PSP for the client when just assembling into the code segment. The child PSP problem is not relevant when loading a program into the debugger. * For historical reference, there are some versions of the original 86-DOS Monitor and Debug out there, some of which explicitly note that copyright does not apply to them. Other than that, do have a look at FreeDOS Debug [4] or my repo of its history going back decades [5], as well as my fork, lDebug [6]. Another fork with fewer changes is debug.gen [7]. There is also Enhanced Debug in the same family but that one is not free software, it's not redistributable. Regards, ecm [1]: http://www.bttr-software.de/forum/forum_entry.php?id=15572=0=last_answer=DESC=all [2]: http://www.seasip.info/DOS/ [3]: https://stackoverflow.com/questions/72024176/how-can-i-fix-my-dosbox-it-freezes-and-auto-closed-whenever-i-use-dos-debugs-g?noredirect=1#comment127385645_72024176 [4]: https://github.com/Baron-von-Riedesel/DOS-debug/ [5]: https://hg.pushbx.org/ecm/fddebug [6]: https://pushbx.org/ecm/web/#projects-ldebug [7]: https://github.com/lpproj/debug.gen ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On Tue, 12 Jul 2022, C. Masloch wrote: (Only replied-to portions copied) Further, you calculate the size of the PSP block to keep resident by using test, add, and shifts: mov dx, trans ; Allow everything preceding the test dx, 0x000F ; transient portion of GRAFTABL to jz.20 ; remain resident. (Rounded up to add dx, 0x10; the next paragraph, because MS-DOS .20: mov cl, 4 ; wants the size in paragraphs.) shr dx, cl mov ax, 0x3100 int 0x21; TSR EXIT CODE 0 It is more efficient to change this like so: mov dx, trans add dx, 15 mov cl, 4 shr dx, cl The "add dx, 15" makes the shr round up in its division. (I expect that the addition won't overflow here.) However, you can easily change it so that the addition is done by the assembler at build time, using "mov dx, trans + 15". Prolly the 65C02 programmer in me. xD Another problem (which is also little known) is that your use of interrupt 21h service 31h will retain all of your currently open process handles, as well as the entire system's System File Table (SFT) entries associated with these. This is not a problem by default because all your handles will be DUPlicated from the parent's, for your stdin, stdout, stderr, stdaux, and stdprn. That means they will share the same SFT entries as already used by the shell. However, if the user runs your program with output redirection (either to a file or a character device such as NUL), as in "graftabl > nul", then you will leak the SFT entry which was reserved for your process to use. I assume that the people involved in the design of this DOS service expected that TSRs would generally want to keep around their PSPs, so that they could swap processes and then use their own handles as preserved in their process handle tables. However, in practice most TSRs never re-use their PSPs after the DOS TSR termination handling is done. So in that case, as it is for your application, you should explicitly free all handles before terminating. Here's how I solved it [7] in FDAPM (and with equivalent code in FreeDOS SHARE): xor bx, bx ; = 0 mov cx, word [32h] ; get amount of handles .loop: mov ah, 3Eh int 21h ; close it inc bx ; next handle loop .loop ; loop for all process handles --> Hm. Probably a good point. In your executable entrypoint you have a near jump to skip the buffer later used for the table data: entry:jmp trans db1021 dup 0x00 It is implied by the size of the buffer that the jump must be near, so it takes 3 bytes, and then you add another 1021 bytes to get a total of 1024 bytes. (I think the "dup" syntax is only supported by recent versions of NASM, but that's not important.) However, I'd prefer to use some calculation to get NASM to reliably fill the buffer, such as: entry: jmp trans times 1024 - ($ - entry) db 0 Alternatively, you could use my fill macro [10] like this: entry: fill 1024, 0, jmp trans (Also, as I think you already suggested in this thread, for optimising the transient executable size you could put one of the tables into this buffer to save 1 KiB at the end of the executable. (Just the jump needs to stay, you could re-initialise it to hold the correct 3byte for the table start later.) Or stash some of the messages in there, as long as they're shorter than the 1 KiB size.) (I've more recently done this with the CP437 table, because it's the default.) You do not use an IBM Interrupt Sharing Protocol (IISP) [11] header for your interrupt hook. Therefore, you could optimise this part a bit, from: old2F:dd0x ... .1: jmp far [cs:old2F] Into this: .1: jmp 0:0 old2F equ $ - 4 This is some self-modifying code (SMC) to stash the downlink into an immediate far jump instruction, instead of using the indirect far jump to refer to a different memory location in your code segment. The dollar sign is used in the equate to denote the current assembly position after the 5-byte instruction; it is offset by minus four so as to address the far pointer in the instruction's encoding. (As mentioned, you cannot do this if you use a standard IISP header, because that has a "jmp short $+18" (EBh 10h) instruction right in front of the downlink field.) I believe it was from analyzing some MS-DOS 2.0 code that I got the technique I used here. Next, you're using "or al, al" to check a register for zero. However, it is more idiomatic [12] to use "test al, al" instead, which right away hints (to a reader or even to the processor) that no change of the register occurs. Also not sure where I got the "or" idiom, but I know that it doesn't alter the
Re: [Freedos-devel] Another implementation of GRAFTABL
On at 2022-07-12 20:01 +0200, Jose Senna wrote: Bret Johnson said: > The way it works is to make a "copy" of > itself at the top of conventional memory, > terminates itself (using a normal DOS > terminate process, which includes deleting > the original PSP), and then continues > running from the "copy". The "copy" decides > where the best place in memory is to load > the TSR (which can even be in one or more > "memory holes" left by some other program > or in upper memory),allocates appropriate > memory block(s), and then installs itself > in the allocated memory. But when it terminates, the system would not return to the parent of the utility, (COMMAND or whatever called the utility) ? So, how can the copy continue running after termination ? Good question! As I independently invented this method, I will answer quickly. You can read my reference implementation in my TSR example [1]. Basically, it creates a new PSP (the system structure that makes up a DOS process) in a temporary memory block, then bends that memory block to "belong to itself" (as memory blocks used for PSPs usually do). This means that when we terminate with service 4Ch, the temporary allocation is not freed because it doesn't belong to the original process. The actual "jump" over to the new temporary process is achieved by twisting the parent process field and the Parent Return Address (aka interrupt 22h) of the original process. Their original contents (which refer to the shell or whatever other process EXECed our program) are transferred into the new temporary PSP, so that upon its termination it will actually return to the shell. The old original PSP is instead modified to refer to the new PSP as its parent, and the original PSP's PRA is also modified to jump into the code of the relocated process. (Additionally, there is a field in the *parent* PSP which indicates what stack to use when returning from DOS process termination to the parent.) In part this was inspired by FreeDOS Debug (and before this actual shells for DOS, too, I believe even MS-DOS COMMAND.COM does this). It sets up its parent as itself, and its PRA to return into its own resident code. DOS detects the special condition of a self-parented process in its termination handling, and will not free memory nor close the file handles in this case. (Just as for service 31h TSR termination actually, it also neither frees memory nor closes handles, as I mentioned in my longer reply earlier.) That allows the shell (and the debugger, which touts itself a kind of shell [2]) to retain control after it was aborted with Control-C or upon a critical error when "Abort" was selected. Regards, ecm [1]: https://hg.pushbx.org/ecm/tsr/file/daca203fa216/transien.asm#l988 [2]: https://hg.pushbx.org/ecm/ldebug/file/d8d009634290/source/debug.asm#l1319 ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
Hi list, On at 2022-07-12 16:29 +, Bret Johnson wrote: For TSR's, there are additional things you can do to reduce memory. You can look at the source code for my PRTSCR program (available at http://bretjohnson.us) that uses a BUNCH of tricks. For example, it doesn't even use the DOS TSR interrupt. The way it works is to make a "copy" of itself at the top of conventional memory, terminates itself (using a normal DOS terminate process, which includes deleting the original PSP), and then continues running from the "copy". The "copy" decides where the best place in memory is to load the TSR (which can even be in one or more "memory holes" left by some other program or in upper memory), allocates appropriate memory block(s), and then installs itself in the allocated memory. I learned that technique from ECM a long time ago. It's much more complicated than a "normal" TSR installation, but is much more efficient in terms of ultimate memory use. I developed this optimal installation handling first for RxANSI, based on Henrik Haftmann's ANSI, which I forked starting in 2008 [1]. I eventually adapted it into the TSR example [2] and then also for some other TSRs such as lClock [3]. To note is that the resident block installed this way doesn't have a PSP at all, just an MCB with itself as the owner and a DOS v4+ style MCB name for MEM type programs. Other notes on your TSR and application: Your GRAFTABL uses interrupt 21h service 31h to stay resident. You do free the environment, but to be on the safe side it is better to also clear the process's environment field to zero. Example code is in my TSR example [4]: xor ax, ax xchg ax, word [ cs:2Ch ]; set PSP field to zero mov es, ax mov ah, 49h int 21h ; Free our environment Further, you calculate the size of the PSP block to keep resident by using test, add, and shifts: mov dx, trans ; Allow everything preceding the test dx, 0x000F ; transient portion of GRAFTABL to jz.20 ; remain resident. (Rounded up to add dx, 0x10; the next paragraph, because MS-DOS .20: mov cl, 4 ; wants the size in paragraphs.) shr dx, cl mov ax, 0x3100 int 0x21; TSR EXIT CODE 0 It is more efficient to change this like so: mov dx, trans add dx, 15 mov cl, 4 shr dx, cl The "add dx, 15" makes the shr round up in its division. (I expect that the addition won't overflow here.) However, you can easily change it so that the addition is done by the assembler at build time, using "mov dx, trans + 15". Moreover, while it is not supported in the most obvious way (like "mov dx, (trans + 15) / 16") you can teach NASM to do the entire calculation (shift or division and all) by calculating a scalar length of the program (as opposed to a relocatable symbol like created by your "trans:" label). There's an example of this in the DVORAK TSR (under GNU GPL v2+) by Donald Bindner [5] that goes as follows: even 16 end_of_resident equ ($-$$ + 0100h) ... mov dx, end_of_resident/16 ; number of paragraphs to keep mov ax, 3100h ; terminate resident w/ 0 return code int 21h As you can see, NASM allows to divide the scalar value. (Rounding up is not needed in this particular calculation because they already aligned the position of the end_of_resident equate to a 16-byte boundary.) Here is another example [6], in FDAPM (by Eric Auer) which I extended with some but not all of my TSR ideas: mov dx, (eofTSR - start + 256 + 15) >> 4 ; +256 for PSP, start is at offset 100h mov ax,3101h; go TSR, errorlevel 1 int 21h In my TSRs and other applications I use more sophisticated calculations, some much more so. These are often based on different sections or segments of the program. I generally calculate deltas to address different parts and hardcode resulting values into the program at build time. However, the suboptimal way of calculating the amount of paragraphs at run time is a very common oversight. Another problem (which is also little known) is that your use of interrupt 21h service 31h will retain all of your currently open process handles, as well as the entire system's System File Table (SFT) entries associated with these. This is not a problem by default because all your handles will be DUPlicated from the parent's, for your stdin, stdout, stderr, stdaux, and stdprn. That means they will share the same SFT entries as already used by the shell. However, if the user runs your program with output redirection (either to a file or a character device such as NUL), as in "graftabl > nul", then you will leak the SFT entry which was reserved for your process to use.
Re: [Freedos-devel] Another implementation of GRAFTABL
Hello Steve Nickolas, I suppose I could step on zero page (PSP) but I don't know how safe that is. The graphics table is written at CS:0100-04FF (this could be why Microsoft's version is incompatible with mine, while mine is compatible with Microsoft's). The command line buffer starting at PSP:0x0080 is almost certainly safe to mess with. Consider this: PSP:0x0080 is the default Disk Transfer Area (DTA) for FCB operations --- such as int 0x21, ah = 0x14 and 0x15 --- where the DTA gets overwritten with arbitrary data. The "default" FCBs at PSP:0x005c and PSP:0x006c are _probably_ also safe to overwrite (though admittedly I am now less sure about this). Thank you! -- https://gitlab.com/tkchia ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
Bret Johnson said: > The way it works is to make a "copy" of > itself at the top of conventional memory, > terminates itself (using a normal DOS > terminate process, which includes deleting > the original PSP), and then continues > running from the "copy". The "copy" decides > where the best place in memory is to load > the TSR (which can even be in one or more > "memory holes" left by some other program > or in upper memory),allocates appropriate > memory block(s), and then installs itself > in the allocated memory. But when it terminates, the system would not return to the parent of the utility, (COMMAND or whatever called the utility) ? So, how can the copy continue running after termination ? ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On Wed, 13 Jul 2022, TK Chia wrote: Hello Bret Johnson, [me:] GRAFTABL basically just needs to install a 1024-byte glyph table somewhere in conventional memory, and point the int 0x1f vector at it. (The glyph table can then be used to display "extended ASCII" characters in a CGA graphics mode.) There is no need to even install any resident code. I guess I misspoke --- it seems that Steve's implementation of GRAFTABL hooks int 0x2f, rather than int 0x1f, and does involve a little bit of code. Thank you! It hooks both. -uso.___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On Wed, 13 Jul 2022, TK Chia wrote: Well, this is GRAFTABL we are discussing here, so it probably does qualify as one of the "simplest TSRs". :-) GRAFTABL basically just needs to install a 1024-byte glyph table somewhere in conventional memory, and point the int 0x1f vector at it. (The glyph table can then be used to display "extended ASCII" characters in a CGA graphics mode.) There is no need to even install any resident code. Thank you! It actually does need some resident code so that another GRAFTABL can detect it, but it's very minimal. Basically, "if AX=B000, return AL=FF; if AX=B001, return AL=FF and copy a far pointer to the graphics table into DS:BX; otherwise chain into the old handler." (It's "new2F" in the source.) This change probably happened with the introduction of codepages in MS-DOS 3.3. FWIW, MS-DOS 6.22 MEM.EXE reports a memory footprint of 1344 bytes, vs. 1200 for Microsoft's version. That's not a big difference, but it's something, and I don't think "uso's code is sloppier" explains it all. -uso. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
Hello Bret Johnson, [me:] GRAFTABL basically just needs to install a 1024-byte glyph table somewhere in conventional memory, and point the int 0x1f vector at it. (The glyph table can then be used to display "extended ASCII" characters in a CGA graphics mode.) There is no need to even install any resident code. I guess I misspoke --- it seems that Steve's implementation of GRAFTABL hooks int 0x2f, rather than int 0x1f, and does involve a little bit of code. Thank you! -- https://gitlab.com/tkchia ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
Hello Bret Johnson, [Steve Nickolas:] Stack allocation could be implicit. Don't know how NASM is handling this, as I wouldn't touch that one with a barge pole... ;-) Check the list file when assembling this.. In TSR's, the stack is NEVER implicit. It is something you must explicitly handle somewhere in the TSR. For all but the simplest TSR's, the TSR should set up a stack for itself to use while it is processing things. Well, this is GRAFTABL we are discussing here, so it probably does qualify as one of the "simplest TSRs". :-) GRAFTABL basically just needs to install a 1024-byte glyph table somewhere in conventional memory, and point the int 0x1f vector at it. (The glyph table can then be used to display "extended ASCII" characters in a CGA graphics mode.) There is no need to even install any resident code. Thank you! -- https://gitlab.com/tkchia ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
> Stack allocation could be implicit. Don't know how NASM is handling > this, as I wouldn't touch that one with a barge pole... ;-) > Check the list file when assembling this.. In TSR's, the stack is NEVER implicit. It is something you must explicitly handle somewhere in the TSR. For all but the simplest TSR's, the TSR should set up a stack for itself to use while it is processing things. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
For TSR's, there are additional things you can do to reduce memory. You can look at the source code for my PRTSCR program (available at http://bretjohnson.us) that uses a BUNCH of tricks. For example, it doesn't even use the DOS TSR interrupt. The way it works is to make a "copy" of itself at the top of conventional memory, terminates itself (using a normal DOS terminate process, which includes deleting the original PSP), and then continues running from the "copy". The "copy" decides where the best place in memory is to load the TSR (which can even be in one or more "memory holes" left by some other program or in upper memory), allocates appropriate memory block(s), and then installs itself in the allocated memory. I learned that technique from ECM a long time ago. It's much more complicated than a "normal" TSR installation, but is much more efficient in terms of ultimate memory use. PRTSCR also includes the ability for the TSR to allocate memory blocks in Expanded Memory (EMS) or Extended Memory (EMS, though this happens indirectly through the use of DOS Protected Mode Services or DPMS). Using these techniques, you can actually have a complicated TSR that requires LOTS of data but only a small part of the data (and code) requires the use of conventional (or even upper) memory. I'm still experimenting with the EMS & DPMS things so don't think that part is necessarily "good to go", but it is something you can experiment with if you want. I'm also converting the code from A86 to NASM, and the code on the web site is in A86 (actually, A386) format so you would need modify it to work with some other assembler. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On 7/12/2022 9:13 AM, Steve Nickolas wrote: On Tue, 12 Jul 2022, Ralf Quint wrote: On 7/12/2022 3:01 AM, Steve Nickolas wrote: For some reason I don't understand (me am n00b) the footprint is 144 bytes larger than the MS-DOS 5/6 version, but the binary is smaller. Most commonly, these kind of things are due to a different amount of stack or other dynamic memory being allocated... Ralf For what it's worth, I don't allocate anything. Could be an issue in my math (generating the paragraph count), or maybe I'm including something I shouldn't. I've never written TSRs before and I'm a n00b to ASM. (I do keep graftabl in revision control as part of a larger project.) Stack allocation could be implicit. Don't know how NASM is handling this, as I wouldn't touch that one with a barge pole... ;-) Check the list file when assembling this.. Ralf ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On Tue, 12 Jul 2022, Ralf Quint wrote: On 7/12/2022 3:01 AM, Steve Nickolas wrote: For some reason I don't understand (me am n00b) the footprint is 144 bytes larger than the MS-DOS 5/6 version, but the binary is smaller. Most commonly, these kind of things are due to a different amount of stack or other dynamic memory being allocated... Ralf For what it's worth, I don't allocate anything. Could be an issue in my math (generating the paragraph count), or maybe I'm including something I shouldn't. I've never written TSRs before and I'm a n00b to ASM. (I do keep graftabl in revision control as part of a larger project.) -uso. ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On 7/12/2022 3:01 AM, Steve Nickolas wrote: For some reason I don't understand (me am n00b) the footprint is 144 bytes larger than the MS-DOS 5/6 version, but the binary is smaller. Most commonly, these kind of things are due to a different amount of stack or other dynamic memory being allocated... Ralf ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
On Tue, 12 Jul 2022, Jerome Shidel wrote: I haven’t looked at the code for either. But, there are some things in general to consider with TSRs. I would assume the memory footprint difference may be related to optimization of the executables layout and data storage. Basically, when you return to DOS, you release all the memory you don’t need. So, you store all the data you do need to keep in the beginning of the program. At start, jump past it and do all the initialization stuff. Then free everything you don’t need when you call the terminate-but-stay-resident interrupt. You could even use areas of the PSP (program segment prefix) to store some data. For example, once processed, you could move data into the PSP command line buffer. Another thing, is to free the env segment. Maybe even Shuffle your own data and code around at run-time. Lots of tricks like that. I do use some of these tricks. There is code to free the environment, and the JMP at the beginning is zapped (it is organized to put the full resident portion in one place, starting at CS:0100). I suppose I could step on zero page (PSP) but I don't know how safe that is. The graphics table is written at CS:0100-04FF (this could be why Microsoft's version is incompatible with mine, while mine is compatible with Microsoft's). I could actually probably store most of CP437 up at the top to reduce .COM size. -uso.___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] Another implementation of GRAFTABL
Hi, > On Jul 12, 2022, at 6:19 AM, Steve Nickolas wrote: > [..] > For some reason I don't understand (me am n00b) the footprint is 144 bytes > larger than the MS-DOS 5/6 version, but the binary is smaller. I haven’t looked at the code for either. But, there are some things in general to consider with TSRs. I would assume the memory footprint difference may be related to optimization of the executables layout and data storage. Basically, when you return to DOS, you release all the memory you don’t need. So, you store all the data you do need to keep in the beginning of the program. At start, jump past it and do all the initialization stuff. Then free everything you don’t need when you call the terminate-but-stay-resident interrupt. You could even use areas of the PSP (program segment prefix) to store some data. For example, once processed, you could move data into the PSP command line buffer. Another thing, is to free the env segment. Maybe even Shuffle your own data and code around at run-time. Lots of tricks like that. :-) Jerome ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
[Freedos-devel] Another implementation of GRAFTABL
I've been working on this for my own project, and it's only useful for the rare CGA or Tandy users out there (which is probably why MS-DOS 6 relegated it to the Supplemental disk and PC DOS 6 removed it). GRAFTABL hooks INT1F to provide the upper half of the charset in graphics mode on a CGA or Tandy. (DISPLAY.SYS does this and more on more modern display adapters.) The current version seems to be incomplete - only supports a single hardwired codepage? - so I figured I'd offer my own, which is my first successful attempt at a TSR that actually does something. I admit that the font data was taken from a copy of EGA.CPI many years ago (I think it was the one from Win95, and I extracted it back in 2000 when I was reverse-engineering the format). If that's a problem, feel free to replace it with something else, but I used it with the understanding that this kind of font data can't actually be copyrighted; take that as you may. (IANAL) The code should be pretty well organized. The command-line help is also extracted from MS-DOS if that's a problem - feel free to rewrite it, it's UIUC-license (essentially the same as 3-clause BSD). https://6.buric.co/graftabl.a86 Just use "nasm -o graftabl.com graftabl.a86" to build it. Use "nasm -DHELP -o graftabl.com graftabl.a86" to add the command line help. My implementation is based on RBIL and poking at the binary black-box style. For some reason I don't understand (me am n00b) the footprint is 144 bytes larger than the MS-DOS 5/6 version, but the binary is smaller. -uso. (My preference for UIUC licensing is, admittedly, a bit strange; originally I preferred BSD but I think UIUC is a little safer against the kind of rules-lawyering that affected PINE. It's basically the BSD license with the first paragraph replaced with that of the MIT/X11 license. Again - IANAL, I just don't really care what others do as long as they don't claim they wrote it.) ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
