Re: KeyCode in the javafx.scene.robot.Robot interface
Hello! On 15/05/2024 14:49, Martin Fox wrote: > Mark, > > You may already know this but before JavaFX 21 the Mac and Windows Robot code > had some long-standing bugs with non-US keyboards. Linux is in better shape > but you can encounter problems if the user has installed multiple layouts (I > have a PR pending to fix that). That might explain some of the flakey > behavior you’ve been seeing. Your approach might also invoke an unexpected > dead key depending on what modifiers you’re probing. Very possible. I use this base set of keycodes: https://github.com/io7m-com/xoanon/blob/ac7c9c900c7908c5760a74d6cf4056fe3ffb8e92/com.io7m.xoanon.commander/src/main/java/com/io7m/xoanon/commander/internal/XCCommander.java#L235 I iterate over each key in that set, sending that keycode to a text field. I then do the same for each keycode but with the shift key held. I don't doubt that it could cause havoc in some setups! > With one exception JavaFX doesn’t have any facility for querying the keyboard > layout. It relies on the platform code to take in keyboard events and > translate them to JavaFX events on the fly. The exception is the internal > call Toolkit.getKeyCodeForChar which attempts to map from a character to a > KeyCode and is used to match shortcuts specified as KeyCharacterCombinations. > Unfortunately the current implementation on Mac and Linux can’t really be > extended to cover the sort of testing you’re trying to do. Right. > JavaFX needs a better framework for testing text entry and I’ve thought about > how that might look... In my case, I'm almost always testing whether some UI elements become enabled or disabled based on validation occurring on a text field of some kind. A recent case was checking to see if a text field correctly rejected input that didn't appear to be an email address. Typing '@' into the field was difficult to make reliable. :) -- Mark Raynsford | https://www.io7m.com
KeyCode in the javafx.scene.robot.Robot interface
Hello! I maintain a test harness for JavaFX applications: https://www.github.com/io7m-com/xoanon I expose an interface that uses the javafx.scene.robot.Robot interface internally, but I expose a slightly higher level API that allows for (amongst other things) typing text as strings on components rather than sending individual key codes. The problem I immediately run into is that KeyCodes are keyboard layout specific, and there doesn't appear to be any way to detect what keyboard layout is in use. If I, for example, send the KeyCode for the `@` symbol, I'll actually get `"` on some keyboard layouts. This can cause some types of tests to fail, even though the code is correct. Consider a test where an email address is typed into a text field, and then the form containing the field is checked to see if the text field accepted/rejected the text as being a valid email address. On some keyboard layouts, a test that sends the string "some...@example.com" will fail because what ends up in the text field is actually "someone"example.com". I provide a workaround for this: At the start of the test suite run, xoanon will send every KeyCode to a text field one at a time and see what character appeared in the text field as a result. It then builds a mapping between characters to KeyCodes, and it can then do the translations as necessary when running the rest of the test suite: https://github.com/io7m-com/xoanon?tab=readme-ov-file#keymap-generation Unfortunately, this is still rather flaky. This often seems to fail more or less at random for reasons that aren't clear to me. Is there some way the Robot interface could be extended to allow for typing strings? Failing that, is there some way JavaFX could advertise the underlying keyboard map? I feel like some underlying component must know the information I'm trying to infer, but it doesn't seem to be exposed anywhere that I've been able to find. -- Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Sat, 2023-12-16 at 12:10 -0800, John Neffenger wrote: > > I would prefer not to have the tyranny of the project default > replaced > with a new tyranny of the app default. :-) > > Wouldn't it be nice to allow developers and end users to enable > hinting > for any JavaFX application just by defining a system property? That is a good point. I had forgotten that most applications don't allow the user to specify their own CSS. What's the expected precedence? I assume it's: * If the CSS specifies a setting, and the corresponding property isn't set, the CSS wins. * If the property specifies a setting, and the CSS isn't set, the property wins. * If the CSS specifies a setting, and the corresponding property specifies a setting, the property wins. > > Of course, I could always do both, as with the way font smoothing > > is > > right now (prism.lcdtext and -fx-font-smoothing-type: gray). :) > > Would you consider splitting it into two enhancement requests? The > first > could propose the global properties, and a second could allow the > developer to set them node by node. I'm pretty discouraged in the sense of getting things upstream. I've watched the project turn away contributors for years now, sadly. I'll definitely be implementing this in separate pieces, so the patches will be available for consideration afterwards. > It helps to go big with any "before" and "after" screenshots: 800 > percent minimum, 400 percent for a retina display, with "smoothing" > disabled for the image zoom. If at all possible, do what you can to > take > the issue out of the realm of opinions, where it's often dismissed > outright. For example, see > <https://github.com/javafxports/openjdk-jfx/issues/229>. I can definitely do this. -- Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Fri, 2023-12-15 at 17:55 -0800, John Neffenger wrote: > On 12/15/23 2:21 PM, Mark Raynsford wrote: > > 2. Add an -fx-font-hinting CSS attribute that, by default, will > > be > > set to false. Setting this value to true will enable hinting > > using whatever is the default (in other words, not setting > > FT_LOAD_NO_HINTING). > > Have you thought of a 'prism.hinting' system property similar to the > current 'prism.lcdtext'? > > Simply being able to toggle off FT_LOAD_NO_HINTING would, I think, > result in text rendering on Linux similar to the JDK (vertical-only > hinting when using FreeType 2.8 or later). > > A system property would allow end users to control the hinting, too, > with the added benefit of being a very simple code change. I have, yes. I actually considered properties for each of these, but I thought it more important to be able to specify them on a node-by-node basis. Of course, I could always do both, as with the way font smoothing is right now (prism.lcdtext and -fx-font-smoothing-type: gray). :) -- Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Fri, 2023-12-15 at 13:40 -0800, Philip Race wrote: > I already gave a brief explanation of why FX text rendering is what > it is. > I will expand on it a bit but a number of the salient points have > already been made by others. > It is unlikely we will make anything other than carefully considered > tweaks, so this is by way of explanation. I'm actually not even asking to change the defaults. The dictated-from- above defaults are the problem. I am inevitably going to fork JavaFX and maintain a patched version even if the changes won't be accepted upstream. These are the changes I intend to make: 1. Add an -fx-font-bitmaps CSS attribute that, by default, will be set to false. Setting this value to true will cause bitmap information to be loaded if present, and ignored otherwise. 2. Add an -fx-font-hinting CSS attribute that, by default, will be set to false. Setting this value to true will enable hinting using whatever is the default (in other words, not setting FT_LOAD_NO_HINTING). 3. Add an extra value to -fx-font-smoothing: none. Setting this will use FT_LOAD_TARGET_MONO and disable effectively disable antialiasing. This is important for my users for small-text console components. I'm just tired of my applications looking like optical migraines when everyone else on the planet (including the UI library in the JDK itself) manages to look better, regardless of the apparent technical consequences for frankly niche cases. > Some people absolutely insist that being true to the design is what > is most important > A high contrast is the most important to someone else. > No jaggies is important to another person. > Hinting is essential to some people in some cases. To other people it > is anathema. etc, etc. And by disallowing developers to change the behaviour in any way, you're deciding this for them. You're effectively pleasing the smallest set of people possible. > And then on top of that hinting is not even designed to be used in > such rotated cases. So we must all pay for ugly text because someone somewhere wants to spin text around, and we don't get to opt-out. > And as the Amble case points out, you are dependent on the quality of > the hinting which is very expensive. Poor hinting is worse than none. So JavaFX, which has no idea which fonts I'm bundling, is clearly more qualified than I am to decide whether I want hinting or not. > And animations using text - not common in Swing, but central to FX - > are very jerky if you use hinting. It's almost as if there's some kind of design flaw, and a ton of work to maintain a feature that I've yet to see anyone actually say that they care about. Take a look at any modern 3D rendering engine with UI support, and you won't see these issues, and they not only perform scaling and rotation but frequently perspective projection onto 3D surfaces. > And "bitmaps" in a font are extremely rare. Maybe in the fonts you use. "Extremely rare" is entirely subjective; there's an entire field of programmer fonts that use carefully tuned bitmaps to stay sharp. Guess we're just not allowed to use those either, because JavaFX says so. > LCD text also doesn't work well with rotations because the 3X > resolution you get is only in one orientation. On that we agree. I have no interest in LCD text on any platform. Guess I'll get working on the fork. -- Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Thu, 2023-12-14 at 23:38 +, Pedro Duque Vieira wrote:
>
> System.setProperty("prism.lcdtext", "false");
>
>
> Javafx font rendering on Mac OS is better than on Windows.
Yeah, I've had a go with this as well. On Windows 10, at least,
the choice is between blurry text, or blurry text with awful colour
fringing.
Not great!
--
Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Thu, 2023-12-14 at 15:02 -0800, John Neffenger wrote: > On 12/14/23 4:39 AM, Mark Raynsford wrote: > > The key point is "keep text aligned to the pixel grid". The thing > > is: > > This isn't affected by hinting or the lack of it whatsoever, at > > least > > with the way text is implemented in Prism and Freetype, as far as I > > can > > tell. > > It seems to have influenced the original choice for JavaFX. Felipe > Heidrich addresses the font scaling issue at 30:54 in his > presentation: > > Hinting (30m 54s into the video) > https://www.youtube.com/watch?v=LCrCni6EVek=1854s > > When using hinting, he says ... > > > You're going to compromise the intent, and again, you're going to > compromise the linearity. So again, if you start scaling, things are > going to start jumping. > > They're going to start jumping because they've got to fix the pixel > grid > here and there, or just like, the stems or the crossbars in the > glyphs > are going to ... it will go like, point 12, point 13, 14, and things > look to be going well, then go 14, 15, kind of gives a jump and > everything kind of doubles. So what he's describing is scaling in terms of changing the underlying font size and regenerating glyphs at each animation step. As in, if you implement zooming text by displaying fonts at size 12.12pt, 12.13pt, 12.14pt, etc. Is this really such a serious problem and urgent use-case that we have to live with awful text rendering to support it? The only applications I can think of that do text zooming like this are web browsers and, even there, there's no expectation of smoothly animating fonts between different sizes (and the layout is going to jump around all over the place anyway, due to other objects on web pages changing sizes). This seems like an extremely bad trade-off. These days, we have GPUs. Render text at a few fixed sizes and linearly interpolate between the rendered images whilst scaling up nodes. I highly doubt users would notice or care about the difference. -- Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Wed, 2023-12-13 at 22:15 -0800, John Neffenger wrote: > > There are some text animations in JavaFX that you won't see in most > user > interfaces: think text rotations, not stock tickers. It's difficult > to > keep text aligned to the pixel grid when the grid is rotating. I don't buy this argument, having looked at the implementation. The key point is "keep text aligned to the pixel grid". The thing is: This isn't affected by hinting or the lack of it whatsoever, at least with the way text is implemented in Prism and Freetype, as far as I can tell. In the Prism/Freetype implementation, when the user wants to render the string "Hello World!", for each character in the string, a glyph is requested from Freetype. The contents of the returned glyph bitmap are packed into a persistent texture atlas on the GPU (the "glyph cache"). When actually rendering the string in the UI, the system constructs a set of polygons where each polygon represents one glyph, and each polygon is textured by having its UV coordinates mapped to the corresponding region of the glyph cache for the glyph. What this means is that it's entirely irrelevant whether hinting is used to produce the original glyph bitmap or not: The hinting just controls what pixels come out of Freetype before being written into the glyph cache. The system blindly selects a region of pixels from the glyph cache to be mapped onto polygons; it doesn't even know if those pixels were produced by a text rendering algorithm that used hinting or not. This argument _might_ apply if text was being directly software rendered to the screen, and the entire text was being re-rendered as full strings aligned to a screen-sized pixel grid every time. I could see in that case that an algorithm might produce output that varied on a frame-to-frame basis dependent on hinting. That's not the case here, though. Case in point: I've worked with a lot of OpenGL applications that do text rendering the same way, all of which used hinting, and all of which did heavy rotation and scaling of the resulting text as part of animations. Noone ever complained about jerky or weird text. In fact, most rendering engines these days don't even bother rendering text on a per-glyph basis; they just tell Freetype (or text library of choice) to render a full string into an image, with hinting, and that image is uploaded directly onto the GPU and displayed as a single textured polygon. Additionally, there are also techniques devoted to preserving sharpness and reducing other artifacts in the face of rotations. They're known as "fat pixel" shaders, and are trivial to implement: https://jorenjoestar.github.io/post/pixel_art_filtering/ They're usually discussed in the context of pixel art, but they work very well with text rendering too for the same reasons. -- Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Thu, 2023-12-14 at 10:23 +0100, John Hendrikx wrote: > > Animated text is extremely rare in productivity applications, so I > think > this really should be an option on a Node by Node basis (there is > already the cache/cacheHint property which are animation related). A > big > wall of text (like in a rich text control, TextArea or some kind of > text > editor) shouldn't have to suffer in readability just in case we may > want > to rotate or scale it... If Browsers had the JavaFX mentality of font > rendering, everyone would switch to the one browser that renders font > properly -- and I can imagine poorly rendered text is one the first > things people notice and can be a big reason for worse JavaFX > adoption. I agree strongly. Purely anecdotally, and obviously I'm only a single sample point (although there are two more lower down!): I started out as a classic Mac user back in the early days of Mac OS 7. I migrated to Windows 2000 and Linux, and have primarily been a Linux user and to a drastically lesser extent a Windows user for my entire career. I mention this because I want to make it clear that I'm not biased towards any particular style of text rendering. I've worked on GTK, QT, and FLTK applications. Some small-scale Cocoa applications when Mac OS X first appeared. I've written Windows applications in the Win32 days. I've written Swing applications. All of these have been used by others to varying degrees. What's the _one_ platform where users complained about the text? JavaFX. Worst is that there's really nothing I can tell them... "Sorry the text is awful, the UI library I used has made the decision for me and won't allow me to change it." It's essentially my only complaint with JavaFX. In all other respects, it's better than every other UI library I've ever used. -- Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Tue, 2023-12-12 at 12:38 -0800, Philip Race wrote: > Usage of hinting in UIs is on the way out. > macOS stopped applying hints ages ago. Apple also canned LCD text. > High DPI displays are obsoleting the raison d'etre of both of these. High DPI displays might be the future, but they're not the present yet and the numbers don't support the idea that the market is changing all that much. Looking at the numbers that exist, the standard 1080p display is still a significant portion of the marketplace: https://gs.statcounter.com/screen-resolution-stats/desktop/worldwide I would love to live in a world where everyone has such a high resolution display that we can turn off AA entirely, but we don't live in that one yet. > A big problem with hinting is that it distorts both the size and the > shape, so UIs do not scale evenly Can you give an example of UIs scaling unevenly, or animations looking jerky? All other non-JavaFX UI applications on my system evidently use hinting, and I don't see anything recognizable there. I'm unclear on how this could affect scaling given that glyphs are rendered once and then stuck into a texture atlas on the GPU; a glyph rendered without hinting is going to be just as static as a glyph rendered with hinting, except that the glyph rendered with hinting is going to look "better" (assuming non-broken hinting in the font file). Do you mean that the glyphs are continuously regenerated during some kind of scaling animation operation? > Another is that a poorly hinted font is far worse than no hinting at all. Which should be decided on a font-by-font basis, which I think the developer bundling the font is best equipped to make decisions about. -- Mark Raynsford | https://www.io7m.com
Re: The crisp fonts saga
On Tue Dec 12 16:03:50 UTC 2023, Thiago Milczarek Sayão wrote: > Fonts started to look better on Ubuntu 23.04 > https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/6190 > May be related to this change. Hello! To be clear: All the tests I performed were against the last "release" version of JavaFX on Central (21.0.1, which came out in 2023-10-17). I patched locally against the current `master` from the jfx repository, and the current `master` is definitely still discarding bitmaps and hinting: https://github.com/openjdk/jfx/blob/master/modules/javafx.graphics/src/main/java/com/sun/javafx/font/freetype/FTFontFile.java#L133 The file actually hasn't been touched in 7 years! -- Mark Raynsford | https://www.io7m.com
The crisp fonts saga
taxia.io7m.com/2023/12/12/dejavu_12_hinting_nobitmaps_mono.png That, to my eyes, looks pretty good. The JavaFX defaults for the same font are not good: https://ataxia.io7m.com/2023/12/12/dejavu_12_nohinting_nobitmaps_normal.png I've tried on multiple systems (all Linux, however), and I've yet to be able to contrive a situation where the JavaFX defaults give better rendering results with any combinations of font sizes, with or without AA. A brief inspection of the JDK's Java2D sources show that it does conditionally use FT_LOAD_NO_HINTING depending on various settings (there are comments about FT_LOAD_NO_HINTING yielding constant-sized glyphs, which supposedly can make things easier in some cases). The Java2D results on the systems I tested are consistently better. So I guess my questions are: * Why do we discard hinting information? * Why do we discard bitmaps? * Would JavaFX accept patches to allow hinting, bitmaps, and FT_LOAD_TARGET_MONO? Ideally this should be developer-controlled, so I would need to come up with a pleasant API for it. My experience has been that most JavaFX applications tend to bundle fonts rather than relying on anything the system has. I suspect that, given that developers are including their own fonts, they are the best equipped to answer questions about hinting and AA, rather than just setting values and hoping that the font they get will work well, so an explicit API might be fine. -- Mark Raynsford | https://www.io7m.com
Re: Possible to deploy patched openjfx locally?
On Tue, 2023-12-12 at 11:29 +0100, John Hendrikx wrote: > Hi, > > Depending on what you want to do, if you're not using modules, you > can > just place a changed source file with your own sources (in their > original package) and it should override it. > Hehe, thanks! I had forgotten that that was even an option. I would normally be using modules, but for the sake of testing purposes, I'll be doing this instead. -- Mark Raynsford | https://www.io7m.com
Possible to deploy patched openjfx locally?
Hello! I'm attempting to experiment with some patches to OpenJFX locally. Is there a straightforward way to publish the resulting jar files to my local ~/.m2 repository? There appears to be a `MAVEN_PUBLISH` property, but this just results in: $ sh gradlew -PMAVEN_PUBLISH=true sdk ... * What went wrong: A problem occurred configuring root project 'jfx'. > Failed to notify project evaluation listener. > Could not find method classifier() for arguments [linux] on task ':base:modularPublicationJarLinux' of type org.gradle.api.tasks.bundling.Jar. > Could not find method classifier() for arguments [linux] on task ':graphics:modularPublicationJarLinux' of type org.gradle.api.tasks.bundling.Jar. > Could not find method classifier() for arguments [linux] on task ':controls:modularPublicationJarLinux' of type org.gradle.api.tasks.bundling.Jar. > Could not find method classifier() for arguments [linux] on task ':swing:modularPublicationJarLinux' of type org.gradle.api.tasks.bundling.Jar. > Could not find method classifier() for arguments [linux] on task ':fxml:modularPublicationJarLinux' of type org.gradle.api.tasks.bundling.Jar. > Could not find method classifier() for arguments [linux] on task ':media:modularPublicationJarLinux' of type org.gradle.api.tasks.bundling.Jar. > Could not find method classifier() for arguments [linux] on task ':web:modularPublicationJarLinux' of type org.gradle.api.tasks.bundling.Jar. -- Mark Raynsford | https://www.io7m.com
Re: My JavaFX Christmas Wishlist
On Wed, 2023-11-15 at 15:37 -0300, Thiago Milczarek Sayão wrote: > - Vulkan rendering Seconded! > - A pay raise for Kevin :) Seconded! -- Mark Raynsford | https://www.io7m.com
Re: My JavaFX Christmas Wishlist
On Wed, 2023-11-15 at 09:47 +0100, Dirk Lemmermann wrote: > > All I want for Christmas is … / what I think is needed for JavaFX > going forward ... Is it sad that all I really want for Christmas is to be able to specify that text under a certain size threshold shouldn't be antialiased? My JavaFX console components look very blurry. :) -- Mark Raynsford | https://www.io7m.com
Re: Vulkan
On 2023-06-22T06:56:48 -0300 Thiago Milczarek Sayão wrote: > Hi, > > Just curiosity. Would a Vulkan implementation offer any improvement over > OpenGL in the case of JavaFX? > > I see that GTK has a Vulkan pipeline. > > Wayland too, it feels so fast and fluid. > > And why metal and not Vulkan since it's cross platform and metal isn't? > > Warning: It may be a stupid question. :) I second, third, and fourth this. I really don't know why any project is pursuing anything other than a Vulkan backend for rendering; the tools are better, the API is not utterly error-prone like OpenGL, and it works on every platform (assuming the presence of MoltenVK for Apple's walled garden). -- Mark Raynsford | https://www.io7m.com pgpiR7Loh5E6E.pgp Description: OpenPGP digital signature
Re: Should javafx.scene.robot.Robot be able to use nested event loops?
On 2023-05-22T07:42:23 -0700 Martin Fox wrote: > Mark, > > Do you have a sample app that exhibits this behavior? I can’t reproduce it on > Mac or Windows. > > I can’t think of anything on the Robot side that would cause this. A Robot > just asks the OS to post an event. The event isn’t targeted at a specific > process or window and the Robot has no control over where it ends up. If it > does get delivered to your app it should behave exactly like a user generated > event. > > If the user can close the window it means there’s an event loop running and > that should be draining the OS queue. So I’m puzzled by what’s going on here. > > Anyway, not expected behavior. Still a mystery. Amusingly, I spent half an hour extracting the smallest code sample that would reproduce it and, of course, it turned out I couldn't reproduce it. As I mentioned, my mail hasn't been reaching the list, and I first ran into this a few weeks back. It's possible that a JavaFX update in the intervening weeks fixed the root cause (I likely wasn't even up-to-date when I ran into it, so it could be any reasonably recent release that fixed it). I'm back working on the original ITs now, so I'll see if I run into it again. -- Mark Raynsford | https://www.io7m.com
Should javafx.scene.robot.Robot be able to use nested event loops?
(Apologies for what might be the third duplicate to the list; for some reason list mail is reaching me, but none of my messages to the list are getting through). Hello! I've been putting together some integration tests using javafx.scene.robot.Robot for an application. I've noticed that if the application creates a Stage and then calls showAndWait() on it, there appears to be no way to get a robot to send events to that stage. If that Stage is opened during an integration test, and then manually closed by the user, all of the events (mouse clicks, keys typed, etc) that _would_ have been sent are immediately applied in the context of the outer event loop. Is this expected behaviour? The various Robot implementation classes are suspiciously short, so I'm guessing that this is something that's just not supported. -- Mark Raynsford | https://www.io7m.com
