[Simh] SIMH test suite / target CPU speed control
While discussing SIMH futures, Vince Mulhollon wrote: I use a bash script that automates: wget's some relevant papertape or whatever of diagnostic routine runs a short expect script on simh pdp8 or whatever that executes the diagnostic routine and saves the disk images etc. md5sum the resulting output, and compare to the golden correct answer. One potential problem would be accounting for any variation in the execution behavior, e.g. if the time-of-day clock was read from the host or if network interactions occur (assuming networking is enabled). That would prevent you from getting repeatable modification dates on the SIMH target's disk file systems, and also the content of log files e..g the OpenVMS OPERATOR.LOG. So, the resultant disk images would differ in non-trivial ways. Even if you could seed the time-of-day clock before running your SIMH test suite, any other process running on your host OS (and these days, there are many), would cause jitter in the elapsed execution time, again affecting timestamps. Even under DOS, I once observed a Toshiba Tecra laptop (Pentium 233) which had about a 10% variation in the number of simple increment operations it could do between the 55msec BIOS timer ticks - my conclusion was that the effects of all the background hardware ISRs implemented on a modern PC in the BIOS ar e non-trivial, e.g. battery management, network, USB, and who knows what else, even under good old DOS. For over a decade I've dreamed of a simh with real time execution, so it runs and I/O operates at real time speeds, or perhaps a configurable multiple thereof. I've always believed the current idle loop detection system is an architectural and scalability dead end, the solution to a PDP-8 running at 100 times real speed and hogging the host CPU 100% is to force the PDP-8 to run at configurable real speed and ignore that its hogging a whopping 1% of the host CPU in the idle loop. The last time I looked, the arcade machine emulator MAME achieves this using the x86 Performance Counter functionality, and by knowing the clock cycle cost of each machine language instruction. It then lets the emulated CPU run for some number of expected cycles, then idles itself until it's ready to do another chunk of instructions. For example, if you can do 100msec worth of emulated instructions in 1msec on your shiny new multi-GHz CPU-equipped PC, you simply idle for 99msec. To the observing user, if the time quanta is set right, it's not observable that it's not truly realtime. For MAME, I think this is normally synchronized to the video refresh rate since that's what someone playing an arcade game is observing. One of the challenges is retrofitting machine cycle counting into SIMH. Some of those complex VAX instructions like the CRC instruction could take a variable amount of time to complete. If you truly want to be cycle-accurate, you need to know all the gory implementation details. If you wanted a near enough is good enough approximation and had an instruction which took between, say, 6 and 10 cycles to execute, you might split the difference and say that 8 cycles is an OK average. Jason ___ Simh mailing list Simh@trailing-edge.com http://mailman.trailing-edge.com/mailman/listinfo/simh
Re: [Simh] SIMH test suite / target CPU speed control
On Fri, Sep 03, 2010 at 08:20:49AM -0400, Armistead, Jason wrote: One potential problem would be accounting for any variation in the execution behavior, e.g. if the time-of-day clock was read from the host or if network interactions occur (assuming networking is enabled). Yes sir, we can tell here whom spends most of their time in the oldies (paper tape, etc) and whom spends most of their time in the moderns that had those fancy real time clock things... There are two ways to work around this with modern OS. One is to check for results on the line printer etc. So, my expect script installs fortran from papertape and then enters, compiles, and runs a test fortran program that verifies if 2+2 really does equal 2*2, etc. This is a good test of overall machine operation and can also handle some pretty complicated tests. Perhaps you can even skip verification of the executable output, in that merely being able to install a compiler and compile anything without error messages is strong indication that all is well. Another way to do it is just tough it out the hard way without simple md5sum of the whole works. So, if I run it ten times on my golden machine and compare the results and list the differences, those ten lists of differences should be vaguely similar in their differences. More specifically, feed the disk images into a dump program, feed the ascii dumps into diff against a similar dump of the golden test result, the resulting diif/patch file against any date/time and network access should be about 150K and 12345 lines. Within 5% of normal patch file size is probably correct, but a patch thats fifty times bigger than ever seen before is almost certainly indicating a problem. If you wanted a near enough is good enough approximation and had an instruction which took between, say, 6 and 10 cycles to execute, you might split the difference and say that 8 cycles is an OK average. Getting within an order of magnitude would probably be accurate enough at using enough of the host CPU that you're not wasting time yet also not hogging the CPU. I'd be happy even just seeing it with I/O... If my 110 baud console is outputting multiple pages of text per second, theres something wrong. At least on the PDP8 it should be pretty easy to do some kind of leaky token bucket that returns the busy flag if you've printed more than ten characters per second or whatever. Of simh could emulate disk drive seek times by simply sleeping for an appropriate amount of time. Although its tremendously host dependent, it would be fun to do some control system theory to make a self controlled system that constantly corrects the # of emulated instructions per second equal to 50% of CPU load or some other configurable percentage. My laptop gets uncomfortably hot in some simh experiments.. ___ Simh mailing list Simh@trailing-edge.com http://mailman.trailing-edge.com/mailman/listinfo/simh
Re: [Simh] SIMH test suite / target CPU speed control
In article 3d3fc9427c810d4c9ce9d722d942f0ae071ef...@uusmneg3.na.utcmail.com, Armistead, Jason jason.armist...@otis.com writes: One potential problem would be accounting for any variation in the execution behavior, e.g. if the time-of-day clock was read from the host or if network in teractions occur (assuming networking is enabled). Properly written unit tests can avoid this problem by providing test doubles for the subsystem that introduces variation (clock, network, disk, etc.). The last time I looked at the source code for SIMH, I was surprised that it had zero unit tests (still the case in version 3.8-1). That makes it hard for others to contribute (how do I know I didn't break something at a distance?) and difficult to know with any certainty that any of these emulations are correct. -- The Direct3D Graphics Pipeline -- DirectX 9 draft available for download http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/ Legalize Adulthood! http://legalizeadulthood.wordpress.com ___ Simh mailing list Simh@trailing-edge.com http://mailman.trailing-edge.com/mailman/listinfo/simh
