Hi JPR, i'm glad the 'Bat Signal' still works. On Fri, Oct 11, 2019 at 1:34 AM JPR via 4D_Tech <4d_tech@lists.4d.com> wrote:
> Do you like Fairy Tales? > I thought that was the demo... > > So what you are saying is that if you are deploying 4D Server in a VM > there is not need to try and use preemptive mode. You get no benefit from > doing that. It’s a waste of time? > Wrong. It will be a benefit in terms of speed, even not as large as you > dream, because the system will optimise the use of each possible core. And > a benefit in term of smoothness of execution, because no process will > freeze the others. > OK - but assuming I'm running on a VM isn't that speed and usability benefit coming from moving the processing into a Worker? > > The whole benefit of preemptive mode is to allow 4D to assign a process > to a core. > Wrong. What I explained is that 4D does NOT assign a process to a core, > but assigns a process to a THREAD, and the System assigns cores to Threads. And once more I display my sketchy understanding of how the hardware works. > > But in the case of a VM the actual, physical cores are managed by the VM. > Wrong. The VM talks to the System, and the System manages cores. In fact, > just imagine that a machine can run several VM at the same time. > Ibid. If you have been at the 2019 Tour, remember J_OtherObjects example, where I > show that if you use cooperative processes, with one executing an > excruciating method, then the Forms on use are kind of frozen. Then, by > switching to preemptive mode, the same awfully blocking method doesn't > impact on interface while running. > True (in fact I just pulled it out to look at again). But we are running that on our machines where the _threads_ can be assigned by the _system_ to various cores and I can see that in the activity viewer. In fact running on 'bare metal' shows a lot of variation in the activity of the various cores. I went rummaging around in the knowledgebase to see if I could find what I'm remembering. I think it was one of the keynotes from the '16 Summit and I couldn't find it. But I did find Laurent Ensault's session on the subject: https://kb.4d.com/assetid=77547 I rewatched it and ran the example database. The demo spawns 4 cooperative and 4 preemptive processes then runs an intensive method 1mil times in each. I made a small change to his demo - I added the start and end ticks to the process record in addition to the duration. Why? Because I wanted to see the total time spent on processing the 'job'. I do this by taking the end of the last process to finish and subtracting the start of the first process to begin. I also moved the demo db to v17.3. As luck would have it I have an AWS instance with v17.3 installed so I am able to run the demo locally and on AWS. On my laptop: Cooperative 1: 376 ticks Cooperative 2: 376 ticks Cooperative 3: 375 ticks Cooperative 4: 367 ticks Total run time: 395 ticks Preemp 1: 118 ticks Preemp 2: 119 ticks Preemp 3: 117 ticks Preemp 4: 119 ticks Total run time: 119 ticks // that's what we want to see! On AWS: Cooperative 1: 740 ticks Cooperative 2: 706 ticks Cooperative 3: 793 ticks Cooperative 4: 731 ticks Total run time: 821 ticks Preemp 1: 881 ticks Preemp 2: 824 ticks Preemp 3: 857 ticks Preemp 4: 848 ticks Total run time: 881 ticks Your first thought might be, "hold on, why is the total run time almost the same as each process' run time?" The very first line of the method sets a variable with the tick count. Because Laurent included the IDLE command in the demo method those methods yield the processor back to 4D on each iteration. (The video talks about the details on this.) So Coop1 starts and then yields back to 4D which starts Coop2 and so on. In contrast Preemptive processes simply start and run without having to 'yield back' to 4D. So what we see is that each them gets started pretty much at the same time and complete pretty close together as well. This probably wouldn't be the case if they weren't doing exactly the same work and if 4D were actually working instead of doing a demo. Total run time, as I said, is the tick count for the last process to finish minus the tick count of the first one to start. Or the total amount of time spent processing by all 4 processes. This illustrates the point I recalled from back then: when you run on a VM the advantage of the the OS to distribute threads to the actual cores is abstracted away. They may be distributed to virtual cores but that doesn't impact the total processing time. In the case of AWS apparently this VM overhead actually causes it to run slower but this is a small sample set. I have no idea why and frankly don't have the expertise to figure it out. But it's there. FYI - the AWS instance is an m5ad.xlarge: 4 cores, 16gb RAM. It sorta sucks. I'd love to hear from anyone actually running 4D server on AWS with good performance. -- Kirk Brooks San Francisco, CA ======================= What can be said, can be said clearly, and what you can’t say, you should shut up about *Wittgenstein and the Computer * ********************************************************************** 4D Internet Users Group (4D iNUG) Archive: http://lists.4d.com/archives.html Options: https://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **********************************************************************
