Re: KeyCode in the javafx.scene.robot.Robot interface
Mark, > I iterate over each key in that set, sending that keycode to a text > field. I then do the same for each keycode but with the shift key held. > I don't doubt that it could cause havoc in some setups! On a German or Spanish layout on the Mac BACK_QUOTE will invoke a dead key and throw everything off. Dead keys are miscoded on the Mac but since they aren’t used while typing there hasn’t been much motivation to fix that. If sending a KeyCode generates an InputMethodEvent instead of a KeyEvent you hit a dead key. Send a SPACE to clear the dead key state. That will generate another InputMethodEvent and insert a character into the text control. > In my case, I'm almost always testing whether some UI elements become > enabled or disabled based on validation occurring on a text field of > some kind. A recent case was checking to see if a text field correctly > rejected input that didn't appear to be an email address. Typing '@' > into the field was difficult to make reliable. :) For this sort of testing you can build KeyEvents and fire them yourself. That would allow you to bypass a bunch of fraught areas of the Robot like incorrect focus or accidentally invoking input methods. The internal KeyEventFirer object in JavaFX takes this approach. It uses several internal API’s but illustrate the basics, like sending PRESSED/TYPED/RELEASED trios for KeyCodes that generate characters. I would not mix this approach with a Robot since you would lose control of the ordering of events. Martin
Re: KeyCode in the javafx.scene.robot.Robot interface
Hello! On 15/05/2024 14:49, Martin Fox wrote: > Mark, > > You may already know this but before JavaFX 21 the Mac and Windows Robot code > had some long-standing bugs with non-US keyboards. Linux is in better shape > but you can encounter problems if the user has installed multiple layouts (I > have a PR pending to fix that). That might explain some of the flakey > behavior you’ve been seeing. Your approach might also invoke an unexpected > dead key depending on what modifiers you’re probing. Very possible. I use this base set of keycodes: https://github.com/io7m-com/xoanon/blob/ac7c9c900c7908c5760a74d6cf4056fe3ffb8e92/com.io7m.xoanon.commander/src/main/java/com/io7m/xoanon/commander/internal/XCCommander.java#L235 I iterate over each key in that set, sending that keycode to a text field. I then do the same for each keycode but with the shift key held. I don't doubt that it could cause havoc in some setups! > With one exception JavaFX doesn’t have any facility for querying the keyboard > layout. It relies on the platform code to take in keyboard events and > translate them to JavaFX events on the fly. The exception is the internal > call Toolkit.getKeyCodeForChar which attempts to map from a character to a > KeyCode and is used to match shortcuts specified as KeyCharacterCombinations. > Unfortunately the current implementation on Mac and Linux can’t really be > extended to cover the sort of testing you’re trying to do. Right. > JavaFX needs a better framework for testing text entry and I’ve thought about > how that might look... In my case, I'm almost always testing whether some UI elements become enabled or disabled based on validation occurring on a text field of some kind. A recent case was checking to see if a text field correctly rejected input that didn't appear to be an email address. Typing '@' into the field was difficult to make reliable. :) -- Mark Raynsford | https://www.io7m.com
Re: KeyCode in the javafx.scene.robot.Robot interface
In another thread someone asked whether JavaFX could report which keyboard layout is currently active. I don’t think there’s any useful way of doing this. The Mac organizes layouts using language codes. There are five variants for German and 15 for English. The German variants are ABC-QWERTZ, Austrian, German, German-DIN-2137, and SwissGerman. The English variants are ABC, USExtended, ABC-India, Australian, British, British-PC, Canadian, Colemak, Dvorak, Dvorak-Left, DVORAK-QWERTYCMD, Dvorak-Right, Irish, US, and USInternational-PC. Linux organizes layouts using country codes. When the user searches for a layout they see display names based on language but under the hood it’s all organized by country. On my Ubuntu 22 system there are 10 variants for Great Britain, 25 for the United States, and 20 for Germany. On Windows the user configures some number of input languages and then adds keyboard layouts to each language. On my system I have one language (English) and inside it are four layouts (U.S., French, German, and Spanish). It’s easy to find the user’s current input language but there’s no documented way to retrieve information about the current layout. Folks have reverse-engineered this but I’ve resisted going down that rabbit hole (it involves digging around in the registry). Setting aside the more specialized layouts like Dvorak most of the variants within a given language or country differ in the way they handle dead keys or the position of punctuation and symbol characters. So the variants matter but can’t be reported in any consistent way across platforms. > On May 15, 2024, at 7:49 AM, Martin Fox wrote: > > Mark, > > You may already know this but before JavaFX 21 the Mac and Windows Robot code > had some long-standing bugs with non-US keyboards. Linux is in better shape > but you can encounter problems if the user has installed multiple layouts (I > have a PR pending to fix that). That might explain some of the flakey > behavior you’ve been seeing. Your approach might also invoke an unexpected > dead key depending on what modifiers you’re probing. > > With one exception JavaFX doesn’t have any facility for querying the keyboard > layout. It relies on the platform code to take in keyboard events and > translate them to JavaFX events on the fly. The exception is the internal > call Toolkit.getKeyCodeForChar which attempts to map from a character to a > KeyCode and is used to match shortcuts specified as KeyCharacterCombinations. > Unfortunately the current implementation on Mac and Linux can’t really be > extended to cover the sort of testing you’re trying to do. > > (On any OS the keyboard machinery is designed to take a series of physical > keystrokes and produce a series of characters. Attempting to go in the > opposite direction is fraught with complications particularly for punctuation > and symbols. It’s faster and more reliable to wait for the user to type > something and record what happens. That’s how getKeyCodeForChar is > implemented on the Mac; until the user types something it yields no results. > Linux is currently buggy but the PR to fix it adopts the same approach as the > Mac.) > > JavaFX needs a better framework for testing text entry and I’ve thought about > how that might look. But my priorities might be different than yours. Based > on the bugs I’ve seen the biggest issue is that developers only test on the > platforms and layouts they use. Much of this is due to lack of awareness but > it’s also a big hurdle to have to buy Mac, Windows, and Linux boxes and then > manually switch between layouts to test. My ideal framework would be based on > emulated keyboards so, say, a U.S. Windows developer could generate a key > event stream that corresponds to a Spanish Mac layout. > > That approach wouldn’t involve sending platform events through the OS. > Extending the Robot to make it easier to, say, target punctuation and symbols > via platform events would be nice to have but not essential (IMHO). > > So, no, you’re not missing anything exposed or under the hood. The approach > you’re taking is probably the best that can be done with the current API. > > Martin > >> On May 12, 2024, at 3:16 AM, Mark Raynsford wrote: >> >> Hello! >> >> I maintain a test harness for JavaFX applications: >> >> https://www.github.com/io7m-com/xoanon >> >> I expose an interface that uses the javafx.scene.robot.Robot >> interface internally, but I expose a slightly higher level >> API that allows for (amongst other things) typing text as strings >> on components rather than sending individual key codes. >> >> The problem I immediately run into is that KeyCodes are keyboard >> layout specific, and there doesn't appear to be any way to detect >> what keyboard layout is in use. If I, for example, send the KeyCode for >> the `@` symbol, I'll actually get `"` on some keyboard layouts. This >> can cause some types of tests to fail, even though th
Re: KeyCode in the javafx.scene.robot.Robot interface
Mark, You may already know this but before JavaFX 21 the Mac and Windows Robot code had some long-standing bugs with non-US keyboards. Linux is in better shape but you can encounter problems if the user has installed multiple layouts (I have a PR pending to fix that). That might explain some of the flakey behavior you’ve been seeing. Your approach might also invoke an unexpected dead key depending on what modifiers you’re probing. With one exception JavaFX doesn’t have any facility for querying the keyboard layout. It relies on the platform code to take in keyboard events and translate them to JavaFX events on the fly. The exception is the internal call Toolkit.getKeyCodeForChar which attempts to map from a character to a KeyCode and is used to match shortcuts specified as KeyCharacterCombinations. Unfortunately the current implementation on Mac and Linux can’t really be extended to cover the sort of testing you’re trying to do. (On any OS the keyboard machinery is designed to take a series of physical keystrokes and produce a series of characters. Attempting to go in the opposite direction is fraught with complications particularly for punctuation and symbols. It’s faster and more reliable to wait for the user to type something and record what happens. That’s how getKeyCodeForChar is implemented on the Mac; until the user types something it yields no results. Linux is currently buggy but the PR to fix it adopts the same approach as the Mac.) JavaFX needs a better framework for testing text entry and I’ve thought about how that might look. But my priorities might be different than yours. Based on the bugs I’ve seen the biggest issue is that developers only test on the platforms and layouts they use. Much of this is due to lack of awareness but it’s also a big hurdle to have to buy Mac, Windows, and Linux boxes and then manually switch between layouts to test. My ideal framework would be based on emulated keyboards so, say, a U.S. Windows developer could generate a key event stream that corresponds to a Spanish Mac layout. That approach wouldn’t involve sending platform events through the OS. Extending the Robot to make it easier to, say, target punctuation and symbols via platform events would be nice to have but not essential (IMHO). So, no, you’re not missing anything exposed or under the hood. The approach you’re taking is probably the best that can be done with the current API. Martin > On May 12, 2024, at 3:16 AM, Mark Raynsford wrote: > > Hello! > > I maintain a test harness for JavaFX applications: > > https://www.github.com/io7m-com/xoanon > > I expose an interface that uses the javafx.scene.robot.Robot > interface internally, but I expose a slightly higher level > API that allows for (amongst other things) typing text as strings > on components rather than sending individual key codes. > > The problem I immediately run into is that KeyCodes are keyboard > layout specific, and there doesn't appear to be any way to detect > what keyboard layout is in use. If I, for example, send the KeyCode for > the `@` symbol, I'll actually get `"` on some keyboard layouts. This > can cause some types of tests to fail, even though the code is correct. > > Consider a test where an email address is typed into a text field, and > then the form containing the field is checked to see if the text field > accepted/rejected the text as being a valid email address. On some > keyboard layouts, a test that sends the string "some...@example.com" > will fail because what ends up in the text field is actually > "someone"example.com". > > I provide a workaround for this: At the start of the test suite run, > xoanon will send every KeyCode to a text field one at a time and see > what character appeared in the text field as a result. It then builds a > mapping between characters to KeyCodes, and it can then do the > translations as necessary when running the rest of the test suite: > > https://github.com/io7m-com/xoanon?tab=readme-ov-file#keymap-generation > > Unfortunately, this is still rather flaky. This often seems to fail more > or less at random for reasons that aren't clear to me. > > Is there some way the Robot interface could be extended to allow for > typing strings? Failing that, is there some way JavaFX could advertise > the underlying keyboard map? I feel like some underlying component must > know the information I'm trying to infer, but it doesn't seem to be > exposed anywhere that I've been able to find. > > -- > Mark Raynsford | https://www.io7m.com
KeyCode in the javafx.scene.robot.Robot interface
Hello! I maintain a test harness for JavaFX applications: https://www.github.com/io7m-com/xoanon I expose an interface that uses the javafx.scene.robot.Robot interface internally, but I expose a slightly higher level API that allows for (amongst other things) typing text as strings on components rather than sending individual key codes. The problem I immediately run into is that KeyCodes are keyboard layout specific, and there doesn't appear to be any way to detect what keyboard layout is in use. If I, for example, send the KeyCode for the `@` symbol, I'll actually get `"` on some keyboard layouts. This can cause some types of tests to fail, even though the code is correct. Consider a test where an email address is typed into a text field, and then the form containing the field is checked to see if the text field accepted/rejected the text as being a valid email address. On some keyboard layouts, a test that sends the string "some...@example.com" will fail because what ends up in the text field is actually "someone"example.com". I provide a workaround for this: At the start of the test suite run, xoanon will send every KeyCode to a text field one at a time and see what character appeared in the text field as a result. It then builds a mapping between characters to KeyCodes, and it can then do the translations as necessary when running the rest of the test suite: https://github.com/io7m-com/xoanon?tab=readme-ov-file#keymap-generation Unfortunately, this is still rather flaky. This often seems to fail more or less at random for reasons that aren't clear to me. Is there some way the Robot interface could be extended to allow for typing strings? Failing that, is there some way JavaFX could advertise the underlying keyboard map? I feel like some underlying component must know the information I'm trying to infer, but it doesn't seem to be exposed anywhere that I've been able to find. -- Mark Raynsford | https://www.io7m.com
