Re: ui/cocoa.m compile error (Cocoa -> SDL)
> On 17 Jan 2022, at 18:19, Philippe Mathieu-Daudé wrote: > > The consensus was UI should not be addressed within QEMU itself, but > via an external helper, eventually using D-Bus. D-Bus support has been > recently merged: > https://www.qemu.org/docs/master/interop/dbus.html Thank you for the info. I'll review the details when I'll start work on this, and hopefully find a solution that will be acceptable for upstream. However, please note that the solution used in qemu-arm-gnuarmeclipse was highly appreciated, and the use case was very intuitive, simply starting the emulator brought the graphical window, which looked like this: - https://eclipse-embed-cdt.github.io So, if the UI should not be addressed within QEMU itself, the details of starting the external helper should be transparent for the users. >> If you think useful, I can contribute the patches back to upstream. > > All contributions are welcomed for review. Even if a contribution is > not merged right away, you can see the mailing list as an archive for > previous attemps, in case a contributor gets stuck or have to switch > to another project, someone else can restart / keep working on earlier > series. Ok, for now I avoided the problem by reverting to Cocoa, but I might have to address it in the future. If someone else needs SDL support on macOS, we can work together to update the patches I used in qemu-arm-gnuarmeclipse to the new upstream code. Perhaps we should log a ticket to keep track of the issue. Regards, Liviu
Re: ui/cocoa.m compile error (Cocoa -> SDL)
> On 17 Jan 2022, at 15:30, Liviu Ionescu wrote: > > I will try to rebuild using --with-cocoa on newer systems, to validate the > rest of the build. I confirm that after reverting to --with-cocoa, both Intel and Apple Silicon builds are able to emulate Thomas' vexpress-a9 project. However this was an expensive change, since I had to install a macOS 10.14 virtual machine, with a full build environment, GitHub self-hosted runners, etc. I think that decisions to deprecate support for older macOS releases should be considered carefully, since using older machines for running builds and tests is not a rare occurrence. Regards, Liviu
Re: ui/cocoa.m compile error (Cocoa -> SDL)
+Markus/Daniel/Marc-Andre On 17/1/22 17:07, Liviu Ionescu wrote: On 17 Jan 2022, at 16:06, Peter Maydell wrote: This is newer versions of macOS being stricter about enforcing that some operations are only permitted on the correct thread. Older versions let QEMU/SDL get away with doing them on the "wrong" thread, which all happened to work. Ah, right, I remember this gave me a lot of headaches when I added the animated LEDs to qemu-system-gnuarmeclipse, and in the end I had to rewrite the SDL code to guarantee that graphical updates happen only on the main thread. It looks like it's "code would need to be updated to fix the problems that newer macOS complains about", which nobody has done. Later this year I plan to add animated LEDs to qemu-system-arm too, so I might face the same problems again. Note, this topic has been discussed 2 years ago while I proposed an UI visualizer: https://wiki.qemu.org/Internships/ProjectIdeas/ArduinoVisualisation The consensus was UI should not be addressed within QEMU itself, but via an external helper, eventually using D-Bus. D-Bus support has been recently merged: https://www.qemu.org/docs/master/interop/dbus.html If you think useful, I can contribute the patches back to upstream. All contributions are welcomed for review. Even if a contribution is not merged right away, you can see the mailing list as an archive for previous attemps, in case a contributor gets stuck or have to switch to another project, someone else can restart / keep working on earlier series.
Re: ui/cocoa.m compile error (Cocoa -> SDL)
> On 17 Jan 2022, at 16:06, Peter Maydell wrote: > > This is newer versions of macOS being stricter about enforcing that > some operations are only permitted on the correct thread. Older versions > let QEMU/SDL get away with doing them on the "wrong" thread, which > all happened to work. Ah, right, I remember this gave me a lot of headaches when I added the animated LEDs to qemu-system-gnuarmeclipse, and in the end I had to rewrite the SDL code to guarantee that graphical updates happen only on the main thread. > It looks like it's "code would need to be updated to fix the > problems that newer macOS complains about", which nobody has done. Later this year I plan to add animated LEDs to qemu-system-arm too, so I might face the same problems again. If you think useful, I can contribute the patches back to upstream. Liviu
Re: ui/cocoa.m compile error (Cocoa -> SDL)
> On 17 Jan 2022, at 14:51, Liviu Ionescu wrote: > > it crashes with ... I can try to investigate, perhaps it is only a small > mistake somewhere. I did some new tests, and I confirm that on a macOS 11.6 running on a MacMini 2020 M1 it throws the exception, as shown before, when testing a build performed on a macOS 11.6. However, on an Intel macOS 11.6, a QEMU 6.2 built --with-sdl on an old macOS 10.13, managed to start, and I got a graphical window, which was a bit surprising: However, the console displayed the following: ilg@wks day16 % /Users/ilg/Downloads/xpack-binaries/qemu-arm/xpack-qemu-arm-6.2.0-1/bin/qemu-system-aarch64 -net none -parallel none -monitor none -M vexpress-a9 -kernel winter.zImage -dtb vexpress-v2p-ca9.dtb 2022-01-17 15:07:38.259 qemu-system-aarch64[62163:1267689] WARNING: NSWindow drag regions should only be invalidated on the Main Thread! This will throw an exception in the future. Called from ( 0 AppKit 0x7ff806a86f50 -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 352 1 AppKit 0x7ff806a91185 -[NSView setFrameSize:] + 2315 2 AppKit 0x7ff806ad3ce8 -[NSTitlebarView setFrameSize:] + 86 3 AppKit 0x7ff806aa1d5c -[NSView setFrame:] + 404 4 AppKit 0x7ff806ad3c8b -[NSTitlebarView resizeWithOldSuperviewSize:] + 95 5 AppKit 0x7ff806ab35ef -[NSView resizeSubviewsWithOldSize:] + 501 6 AppKit 0x7ff806a90ef0 -[NSView setFrameSize:] + 1654 7 AppKit 0x7ff806ab42a1 -[NSTitlebarContainerView setFrameSize:] + 147 8 AppKit 0x7ff806aa1d5c -[NSView setFrame:] + 404 9 AppKit 0x7ff806ab3c76 -[NSView resizeWithOldSuperviewSize:] + 697 10 AppKit 0x7ff806ab35ef -[NSView resizeSubviewsWithOldSize:] + 501 11 AppKit 0x7ff806a90ef0 -[NSView setFrameSize:] + 1654 12 AppKit 0x7ff806ab18e5 -[NSThemeFrame setFrameSize:] + 518 13 AppKit 0x7ff806ab0f0c -[NSWindow _oldPlaceWindow:fromServer:] + 697 14 AppKit 0x7ff806aaf8f3 -[NSWindow _setFrameCommon:display:fromServer:] + 2696 15 libSDL2-2.0.0.dylib 0x00010ba7125d Cocoa_SetWindowSize + 285 16 libSDL2-2.0.0.dylib 0x00010ba41a56 SDL_SetWindowSize_REAL + 198 17 qemu-system-aarch64 0x00010a761b80 qemu-system-aarch64 + 2947968 18 qemu-system-aarch64 0x00010a4ba318 qemu-system-aarch64 + 164632 19 qemu-system-aarch64 0x00010a5a23a9 qemu-system-aarch64 + 1115049 20 qemu-system-aarch64 0x00010aa5a053 qemu-system-aarch64 + 6062163 21 qemu-system-aarch64 0x00010aa59ea3 qemu-system-aarch64 + 6061731 22 qemu-system-aarch64 0x00010ab2224d qemu-system-aarch64 + 6881869 23 ??? 0x000111a25855 0x0 + 4590819413 ) In other words: "WARNING: NSWindow drag regions should only be invalidated on the Main Thread! This will throw an exception in the future.", which is exactly what happened on builds performed on more recent macOS. I will try to rebuild using --with-cocoa on newer systems, to validate the rest of the build. Liviu
Re: ui/cocoa.m compile error (Cocoa -> SDL)
On Mon, 17 Jan 2022 at 12:52, Liviu Ionescu wrote: > > Then perhaps it is a misunderstanding from my part, if I try to start the > vexpress-a9 machine using the line copied from Thomas' presentation, the > console shows, it prints "Guest has not initialized the display yet" using > the PC-like font, than it crashes with: > > > ``` > Exception Note:EXC_CORPSE_NOTIFY > > Application Specific Information: > *** Terminating app due to uncaught exception > 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should > only be invalidated on the Main Thread!' > abort() called > terminating with uncaught exception of type NSException This is newer versions of macOS being stricter about enforcing that some operations are only permitted on the correct thread. Older versions let QEMU/SDL get away with doing them on the "wrong" thread, which all happened to work. We had to do some reworking of the cocoa UI code to handle a similar issue. > If you think that --with-sdl should work on macOS too, I can try > to investigate, perhaps it is only a small mistake somewhere. It looks like it's "code would need to be updated to fix the problems that newer macOS complains about", which nobody has done. (I think this is probably code within QEMU, rather than code within libSDL itself.) -- PMM
Re: ui/cocoa.m compile error (Cocoa -> SDL)
> On 17 Jan 2022, at 13:35, Peter Maydell wrote: > > (2) the "use SDL" option seems to have worked for at least > some people in the past: SDL worked nicely for me too in the past, qemu-system-gnuarmeclipse was configured with SDL for all platforms and it worked unchanged in the last 4-5 years. > here's a bug report from 2019 > https://bugs.launchpad.net/qemu/+bug/1847906 where the > reporter said that cocoa didn't work for him but SDL did, > for instance. Then perhaps it is a misunderstanding from my part, if I try to start the vexpress-a9 machine using the line copied from Thomas' presentation, the console shows, it prints "Guest has not initialized the display yet" using the PC-like font, than it crashes with: ``` Exception Note:EXC_CORPSE_NOTIFY Application Specific Information: *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should only be invalidated on the Main Thread!' abort() called terminating with uncaught exception of type NSException Application Specific Backtrace 0: 0 CoreFoundation 0x7ff8040acf0b __exceptionPreprocess + 242 1 libobjc.A.dylib 0x7ff803e0db9d objc_exception_throw + 48 2 CoreFoundation 0x7ff8040d58ca -[NSException raise] + 9 3 AppKit 0x7ff806a86f31 -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 321 4 AppKit 0x7ff806a91185 -[NSView setFrameSize:] + 2315 5 AppKit 0x7ff806ad3ce8 -[NSTitlebarView setFrameSize:] + 86 6 AppKit 0x7ff806aa1d5c -[NSView setFrame:] + 404 7 AppKit 0x7ff806ad3c8b -[NSTitlebarView resizeWithOldSuperviewSize:] + 95 8 AppKit 0x7ff806ab35ef -[NSView resizeSubviewsWithOldSize:] + 501 9 AppKit 0x7ff806a90ef0 -[NSView setFrameSize:] + 1654 10 AppKit 0x7ff806ab42a1 -[NSTitlebarContainerView setFrameSize:] + 147 11 AppKit 0x7ff806aa1d5c -[NSView setFrame:] + 404 12 AppKit 0x7ff806ab3c76 -[NSView resizeWithOldSuperviewSize:] + 697 13 AppKit 0x7ff806ab35ef -[NSView resizeSubviewsWithOldSize:] + 501 14 AppKit 0x7ff806a90ef0 -[NSView setFrameSize:] + 1654 15 AppKit 0x7ff806ab18e5 -[NSThemeFrame setFrameSize:] + 518 16 AppKit 0x7ff806ab0f0c -[NSWindow _oldPlaceWindow:fromServer:] + 697 17 AppKit 0x7ff806aaf8f3 -[NSWindow _setFrameCommon:display:fromServer:] + 2696 18 libSDL2-2.0.0.dylib 0x0001042c3aea Cocoa_SetWindowSize + 282 19 libSDL2-2.0.0.dylib 0x000104294d76 SDL_SetWindowSize_REAL + 198 ``` If you think that --with-sdl should work on macOS too, I can try to investigate, perhaps it is only a small mistake somewhere. Liviu
Re: ui/cocoa.m compile error (Cocoa -> SDL)
> On 17 Jan 2022, at 13:35, Peter Maydell wrote: > > ... the macOS host support in QEMU is not very well > maintained, so the default is "it doesn't change" BTW, my main development platform is macOS, both Intel and Apple Silicon, so if someone is willing to improve macOS host support in QEMU, I can help with testing. My experience with Cocoa is close to none, but with SDL2 I had a light contact when I implemented the animated LEDs in qemu-system-gnuarmeclipse. Regards, Liviu
Re: ui/cocoa.m compile error (Cocoa -> SDL)
On Mon, 17 Jan 2022 at 11:21, Liviu Ionescu wrote: > I did not check the implementation details, but if Cocoa is > mandatory when building on macOS, why is it even allowed to > choose SDL during configure? Because (1) the macOS host support in QEMU is not very well maintained, so the default is "it doesn't change", and (2) the "use SDL" option seems to have worked for at least some people in the past: here's a bug report from 2019 https://bugs.launchpad.net/qemu/+bug/1847906 where the reporter said that cocoa didn't work for him but SDL did, for instance. -- PMM
Re: ui/cocoa.m compile error (Cocoa -> SDL)
> On 17 Jan 2022, at 11:41, Thomas Huth wrote: > > On 10/01/2022 11.33, Liviu Ionescu wrote: >> I now have successful builds on all platforms, including on macOS 11 with >> Apple Silicon and macOS 10.13 with Intel, but I had to disable Cocoa >> support, and enable SDL support. >> The resulting binaries (qemu-system-arm/aarch64/riscv32/riscv64) start, but >> I could not tell if the lack of Cocoa in the macOS builds has some >> disadvantages or not. >> Are there any emulated Arm/RISC-V machines that use graphics, so I can test >> my macOS binaries with? > > Have a look here: > > https://www.qemu-advent-calendar.org/2018/#day-16 > https://www.qemu-advent-calendar.org/2018/#day-24 Thank you, Thomas. I tried to run the Arm demo, but, as expected, on macOS starting a qemu configured with SDL instead of Cocoa, failed. I did not check the implementation details, but if Cocoa is mandatory when building on macOS, why is it even allowed to choose SDL during configure? Regards, Liviu
Re: ui/cocoa.m compile error (Cocoa -> SDL)
On 10/01/2022 11.33, Liviu Ionescu wrote: I now have successful builds on all platforms, including on macOS 11 with Apple Silicon and macOS 10.13 with Intel, but I had to disable Cocoa support, and enable SDL support. The resulting binaries (qemu-system-arm/aarch64/riscv32/riscv64) start, but I could not tell if the lack of Cocoa in the macOS builds has some disadvantages or not. Are there any emulated Arm/RISC-V machines that use graphics, so I can test my macOS binaries with? Have a look here: https://www.qemu-advent-calendar.org/2018/#day-16 https://www.qemu-advent-calendar.org/2018/#day-24 HTH, Thomas
Re: ui/cocoa.m compile error
> On 10 Jan 2022, at 12:44, Peter Maydell wrote: > > ... On Linux this tends to take the form > of dropping support for older versions of various dependencies > and compilers. My build environment uses modern tools (like GCC 11, etc), but compiled from sources on an older Linux release. The xPack QEMU specific build scripts also compile all QEMU dependencies from sources. Unless you change the system APIs your code depends on (like it happened to Cocoa), I should be able to build future QEMU versions in my environment. The problem is not that I enjoy using very old releases, but my tools are also used in some very conservative enterprise environments, where RedHat 7 is still in wide use (given that support for it ends in 2024). Starting with 2022 I just dropped support for RH7, and moved to RedHat 8, which is roughly equivalent to Ubuntu 18, and I have to support it for a few more years. Thus, where possible, I would appreciate changes in APIs to be additive, and support for older versions to be kept for a while as unmaintained, not removed completely. Thank you, Liviu
Re: ui/cocoa.m compile error
On Fri, 7 Jan 2022 at 23:47, Liviu Ionescu wrote: > > > > > On 8 Jan 2022, at 00:16, Peter Maydell wrote: > > > > ... In this > > specific case, NSPasteboardTypeOwner seems to be part of > > an API introduced in 10.14 (Mojave). > > Thank you for the details, this explains the error. > > > So the upstream answer, I'm afraid, is that you should > > upgrade to a newer macOS version. > > Unfortunately this is not possible. :-( > > I'm producing cross-platform and cross-version binaries, which means unique > binaries based on a bit older release that are expected to run on all more > recent releases. For macOS I just upgraded the base from 10.10 to 10.13. For > Linux I upgraded from Ubuntu 12 (kept for compatibility with RedHat 7) to > Ubuntu 18. I think if your policy for minimum-supported-platforms is different from and goes back further than upstream you're likely to run into this kind of issue from time to time, not just on macOS. On Linux this tends to take the form of dropping support for older versions of various dependencies and compilers. Ubuntu 18 is still in upstream's set of supported distros, but when the new LTS releases later this year then 18 will no longer be in the supported set. thanks -- PMM
Re: ui/cocoa.m compile error (Cocoa -> SDL)
I now have successful builds on all platforms, including on macOS 11 with Apple Silicon and macOS 10.13 with Intel, but I had to disable Cocoa support, and enable SDL support. The resulting binaries (qemu-system-arm/aarch64/riscv32/riscv64) start, but I could not tell if the lack of Cocoa in the macOS builds has some disadvantages or not. Are there any emulated Arm/RISC-V machines that use graphics, so I can test my macOS binaries with? Liviu
Re: ui/cocoa.m compile error
> On 8 Jan 2022, at 00:16, Peter Maydell wrote: > > ... In this > specific case, NSPasteboardTypeOwner seems to be part of > an API introduced in 10.14 (Mojave). Thank you for the details, this explains the error. > So the upstream answer, I'm afraid, is that you should > upgrade to a newer macOS version. Unfortunately this is not possible. :-( I'm producing cross-platform and cross-version binaries, which means unique binaries based on a bit older release that are expected to run on all more recent releases. For macOS I just upgraded the base from 10.10 to 10.13. For Linux I upgraded from Ubuntu 12 (kept for compatibility with RedHat 7) to Ubuntu 18. So, it seems that with the current upstream code, the direct consequence would be that for the macOS builds I cannot enable Cocoa. My next choice would be SDL2; I tested and the build passed, but I did not know how to test if it works. My other binary distribution (the qemu-system-gnuarmeclipse), based on the venerable 2.8, used SDL2 from the very beginning on all platforms and still works on all macOS releases since 10.10, so I guess this would be a more permisive solution. However I don't know if any of the existing Arm and RISC-V emulated platforms can use SDL2 on macOS. Actually I don't even know which of the existing emulated platforms, especially the bare-metal ones, use any graphics at all (in qemu-system-gnuarmeclipse I added pictures of all emulated boards and implemented animated graphical LEDs, all via SDL2; it was a quite convenient feature, highly appreciated by the users). So, to summarise, I'm interested only in qemu-system-arm/aarch64/riscv32/riscv64, and the build must pass on Ubuntu 18 for Intel and Arm Linux, mingw 9 on Linux for Windows (7 and up), macOS 10.13 for Intel and macOS 11.6 for Apple Silicon. The binaries should run on all more recent versions, and are mainly targeted for running unit-tests on CI servers (so no need for graphics), but also for local debugging sessions via GDB and semihosting (where animated LEDs would be a nice to have feature). Thank you in advance for your valuable advice. Liviu
Re: ui/cocoa.m compile error
On Fri, 7 Jan 2022 at 21:56, Liviu Ionescu wrote: > I'm building 6.2.0 on macOS, and on a recent macOS 11.6 with > Apple Silicon the build passes, but on a slightly older macOS > 10.13, which is my base platform for Intel macOS builds, > compiling ui/cocoa.m fails QEMU's supported-hosts policy https://www.qemu.org/docs/master/about/build-platforms.html is that we support the most recent version of macOS, and also the previous version until 2 years after the new version is released. Currently that means Monterey and Big Sur and (since Monterey is only just out) Catalina. Apple's APIs have quite a lot of churn, and there are regular deprecations and new APIs, so while macOS versions older than our officially supported set sometimes will be able to build QEMU, they often won't. (Also from time to time we remove back-compatibility ifdefs that are only needed for building on older no-longer-supported macOS.) In this specific case, NSPasteboardTypeOwner seems to be part of an API introduced in 10.14 (Mojave). So the upstream answer, I'm afraid, is that you should upgrade to a newer macOS version. -- PMM
