On 8/24/20, Rob Cliffe <rob.cli...@btinternet.com> wrote: > > Are you suggesting something I could do in Python that would achieve my > aim of *replacing* one program by another
No, it's not possible with the Windows API. Implementing POSIX exec would require extensive use of undocumented NT runtime library functions, system calls, and PEB data in order to reset the address space and handle table and reinitialize the process, and it would probably also require the help of a kernel driver to modify the _EPROCESS record in kernel space. > (1) "*spawn*": I see that there are some os.spawn* functions. The POSIX way to create a new process is fork (clone), which is optionally followed by exec (image overlay). Cloning the current process is fast and efficient. The Windows way to create a new process is spawn, as implemented by WinAPI CreateProcessW, which is relatively slow and laborious. In Python, the preferred way to call the latter is with subprocess.Popen, or helper functions such as subprocess.run. Spawing a Windows process is a lot of work. It involves creating a new process object (a large _EPROCESS record in the kernel) and address space (a VAD tree and page tables); allocating and initializing a process environment block (PEB; a large block of memory that includes process flags, parameters, environment variables, loader data, activation contexts, and much more); optionally inheriting handles for kernel objects from the parent process; mapping an executable file as the process image; and creating an initial thread that starts at the OS entrypoint function (a queued APC that calls LdrInitializeThunk, which resumes at RtlUserThreadStart). The latter initializes the process; registers with the session server csrss.exe via its LPC port; loads and initializes dependent DLLs, which may require multiple activation contexts and LPC message transactions with the SxS fusion loader in the session server; possibly (if it's a console app) connects to or spawns the console-session host conhost.exe; and calls the image entrypoint such as __wmainCRTStartup. The latter initializes the C runtime before calling the application's entrypoint (e.g. wmain). For Python, the latter calls Py_Main, which initializes the interpreter before compiling and executing the main script. > (2) "*wait*": Wait for what? How? Functions such as os.system and subprocess.run wait on (i.e. synchronize on) the process via WinAPI WaitForSingleObject[Ex], which waits for the process to terminate. If the child process uses console I/O and inherits the parent's console, then it's important for the parent to wait before resuming its own interactive console UI. Otherwise both parent and child will compete for the console. > (3) "*proxy the exit status*": Sorry, I have no idea what this means. Generally the parent process needs to know whether the child process succeeded or failed. Typically the exit status (aka exit code) for success is 0, and all other values indicate failure. Commonly an exit status of 1 is used for failure. In Windows, the exit status for a process is returned by GetExitCodeProcess. By "proxy the exit status", I mean that a program that spawns and waits for another program will usually return the exit status from the spawned child as its own exit status. Consider the following snippet from the source code for the py.exe launcher, which spawns python.exe for the selected version of Python: ok = CreateProcessW(NULL, cmdline, NULL, NULL, TRUE, 0, NULL, NULL, &si, &pi); if (!ok) error(RC_CREATE_PROCESS, L"Unable to create process using '%ls'", cmdline); AssignProcessToJobObject(job, pi.hProcess); CloseHandle(pi.hThread); WaitForSingleObjectEx(pi.hProcess, INFINITE, FALSE); ok = GetExitCodeProcess(pi.hProcess, &rc); if (!ok) error(RC_CREATE_PROCESS, L"Failed to get exit code of process"); debug(L"child process exit code: %d\n", rc); exit(rc); This C code includes the steps that I discussed: CreateProcessW, WaitForSingleObjectEx, GetExitCodeProcess, and finally exit(rc), where rc is the exit status from the spawned python.exe process. The launcher also creates a kill-on-close, silent-breakaway job object and assigns the child process to it. That way if the launcher is terminated or crashes, the python.exe process will also be terminated, so there's a tight coupling between py.exe and python.exe. The silent-breakaway setting means processes spawned by python.exe are not assigned to the job. You can work with job objects via ctypes or PyWin32's win32job module. I can provide example code for either approach. > from the DOS prompt, it works as expected. You're not running DOS. Most likely the shell you're running is CMD or PowerShell, attached to a console session that's hosted by either conhost.exe (default) or openconsole.exe (an open-source build of conhost.exe that's used by the new Windows Terminal). These are Windows applications. -- https://mail.python.org/mailman/listinfo/python-list