On Wed, 2 Aug 2023 21:46:32 GMT, Harshitha Onkar <hon...@openjdk.org> wrote:
>> **Problem:** >> >> On macOS, Robot erroneously produces lowercase letter when mouse is moved >> (manually) in unison with Robot's keyEvents. This issue was originally >> logged by a developer of an on-screen accessibility keyboard - TouchBoard. >> Originally reported at >> https://github.com/adoptium/adoptium-support/issues/710 >> >> This issue is reproducible on JDK versions 22 to 11, but works fine on >> JDK-8. (details below) >> >> This issue is not restricted to the Shift modifier key and causes problems >> with other modifier keys as well and in some scenarios without any external >> mouse movement. >> >> - This works correctly on JDK-8 up to JDK-9+129 when Accessibility APIs >> (AXUIElementCreateSystemWide/ AXUIElementPostKeyboardEvent) were used. Later >> on it was changed to CGEvents. >> >> - With the present code, the issue occurs at >> [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.) >> The flags gets reset or cleared when mouse is moved physically in unison >> with Robot's key events. >> >> - The physical mouse movement causes the event flags to be reset. >> >> **Impact:** >> >> Modifier keys don't work as expected when using Robot with any simultaneous >> physical mouse movement and in case of TouchBoard, this behavior breaks the >> usability of the on-screen a11y keyboard. There is no known workaround for >> this particular use case except for reverting to JDK-8. More details on this >> use case >> [here.](https://github.com/adoptium/adoptium-support/issues/710#issuecomment-1594103280) >> >> **Proposed Fix:** >> >> - In order to avoid resetting of the CGEventFlags here >> [CRobot.m#L295](https://github.com/openjdk/jdk/blob/ac6af6a64099c182e982a0a718bc1b780cef616e/src/java.desktop/macosx/native/libawt_lwawt/awt/CRobot.m#L295.), >> the CGEvent flag state is obtained in `initRobot` (stored in initFlags) >> which is later used within `CRobot_keyEvent`. >> >> - The incoming keyCode is used to determine whether it is a modifier key and >> the corresponding modifierFlagMask is either added or cleared from the >> initFlags based on whether the modifier key was pressed or released. >> >> - Finally, only the required and known flag bits from initFlag are copied >> over to local flag which is used in `CGEventSetFlags()`. >> >> **Testing:** >> >> The newly added test - RobotModifierMaskTest.java tests for Shift, Caps, >> Control, Option and Command keys. >> The test runs in 2 modes - as automated and manual test. >> >> CASE 1 : ... > > Harshitha Onkar has updated the pull request incrementally with one > additional commit since the last revision: > > changed allModifiersMask datatype Hello, I am the developer that originally reported this bug to the Adoptium team. I was reviewing the bug discussions on this page and elsewhere, and I have a suggestion and some questions. ### Did Apple fix this bug? (It still does not work for me on the latest macOS.) I read this comment: > Link: https://bugs.openjdk.org/browse/JDK-8302618 > Harshitha Onkar added a comment - 2023-12-22 14:45 > [~prr] Revisited this issue. It works fine on the latest macOS 14.2 and still > fails on macOS > Ventura. This is true for the Java test case and the native reproducer. > Something must have been fixed on the new version although there wasn't any > update on the > Apple Bug report that was filed. I tested the bug today and I am still able to replicate it on my own computer, using the following set up: An M1 Macbook Pro running Sonoma 14.5, on JDK 19 and JDK 22 (Zulu 22.0.1+8). So, it appears that Apple might not have fixed this issue in the latest MacOS. ### Suggestion, Add unit tests for this bug to the openJDK: It appears that this pull request includes some test code for this issue. I would suggest, at minimum, that unit testing for this issue be added to the openJDK. That way if the Robot functionality breaks in any future versions of macOS (or other operating systems), it will be caught more quickly by the openJDK developers. Apparently this issue existed for years without anyone noticing. It would be sad for the same issue to resurface in the future, without anyone noticing again. Note that unit tests for this issue could be added to the JDK regardless of whether any bug fix is applied in openJDK, using conditional test execution. If there is an express decision to NOT fix this issue in the JDK (and to instead rely on a fix that was or will be supplied by Apple), then if desired, the testing code could be surrounded with a conditional code block to exclude only the past version(s) of macOS that are known to be broken. That way the tests would only execute, 1) On all the existing and past operating system versions where Robot is currently known or expected to work correctly, and 2) On all future operating system releases. ### Option, Apply a bug fix in the openJDK, since Apple has not explicitly responded: It may be useful to apply a fix for this bug in the openJDK, (if there is no detriment to doing so), rather than relying solely on any fix that Apple may (or may not) release. Based on Harshitha Onkar's comment copied above, I have an impression that Apple has not yet responded to this issue. I also learned today that macOS Ventura is the last macOS system that many older Mac models can run. That is because reportedly[[1]](https://www.techradar.com/computing/mac-os/the-latest-macos-ventura-update-has-left-owners-of-old-macs-stranded-in-a-sea-of-problems-raising-a-chorus-of-complaints) [[2]](https://en.wikipedia.org/wiki/MacOS_Sonoma#Supported_hardware), macOS Sonoma only supports Mac computer models from 2018 onwards. So if this bug is not fixed in the JDK, then it will likely never be fixed for many older Mac models. Also as I mentioned above, this bug still exist for me even though I am currently running macOS Sonoma 14.5. If I understand correctly, that means it's possible that Ap ple has not fixed the problem for the latest versions of macOS. ### Thanks Thank you to everyone who worked on this issue. :) ### Just for reference, here are links to some pages discussing this bug: * This pull request. https://github.com/openjdk/jdk/pull/14744 * Bug Report at OpenJDK. https://bugs.openjdk.org/browse/JDK-8302618 * Bug Report at Adoptium. https://github.com/adoptium/adoptium-support/issues/710 * Apple developer forums post. https://forums.developer.apple.com/forums/thread/733161 * An old Stack Overflow report, 2018: https://stackoverflow.com/questions/49050543/java-robot-type-uppercase-lowercase-and-symbols-from-string PS, There is an Apple Developer forum post here: (https://forums.developer.apple.com/forums/thread/733161) where someone tried to report this issue to Apple on that forum. In the same thread, Apple suggested reporting the issue using a different channel called the Apple Feedback Assistant. Apparently that is an official channel for reporting Apple bugs. Feedback Assistant instructions page: https://developer.apple.com/bug-reporting/ Feedback Assistant link: https://feedbackassistant.apple.com/ It is unclear to me if anyone has reported the bug using the Apple Feedback Assistant. I could send a report to Apple myself, but I may not be the best person to file the report. I would have trouble giving them all the needed information because I am not familiar with the topics of Objective-C, the CGEventSourceFlagsState API, or the event flags that have been mentioned as potentially being at the root of the issue. ------------- PR Comment: https://git.openjdk.org/jdk/pull/14744#issuecomment-2153274517 PR Comment: https://git.openjdk.org/jdk/pull/14744#issuecomment-2158088438
