[OpenJDK 2D-Dev] what changed to make NSImage://NSListViewTemplate render correctly in JDK 17?

[Alan Snyder]

I notice that NSImage://NSListViewTemplate renders correctly in JDK 17 but not 
in previous JDKs, at least those that I have tried.
In the older JDKs, it renders as lines, whereas in JDK 17 it renders as dots 
and lines (as it does in Finder).
I tested on a recent iMac with Retina display using macOS 11.3.1.

I tried to find a JDK source change that would explain this improvement, but 
without success.
Anyone know?
Could it be an SDK linkage issue?

  Alan


package test;

import java.awt.Image;
import java.awt.Toolkit;
import javax.swing.*;

public class TestNSListViewTemplate
extends JFrame
{
public TestNSListViewTemplate()
{
Image im = 
Toolkit.getDefaultToolkit().getImage("NSImage://NSListViewTemplate");
ImageIcon icon = new ImageIcon(im);
JLabel label = new JLabel(icon);
setContentPane(label);
setSize(800, 800);
setVisible(true);
}

public static void main(String[] args)
{
SwingUtilities.invokeLater(TestNSListViewTemplate::new);
}
}

Re: [OpenJDK 2D-Dev] Heads up : JDK 17 b19 through b22 will use Metal instead of OpenGL for Java 2D rendering on macOS.

[Alan Snyder]

> On May 6, 2021, at 1:45 PM, Philip Race  wrote:
> 
> Alan,
> 
> I am not sure this is a known issue. We'll need a lot more details.

I figured you would. :-)


> What is your h+w and OS update ?

iMac 27 inch 2020 Radeon Pro 5500 XT 8 GB
11.3.1


> Is this all windows in an app or just the first one ?

Definitely not just the first one, but not all of them, either.


> Does it matter what the window content is ?

It might. The app is a test program that can create any of 30 different kinds 
of windows on demand.

So far, I’ve seen the problem in 11 kinds of windows but not in the others. No 
obvious pattern in the content.

The problem is most likely to happen the first time a given window is shown, 
but it can also happen on later instances of the same kind of window.
I just tried a new kind of window and it happened the first 3 times, but not 
the 4th, then about 50% of the time.
I tried creating a different window about 25 times, and it happened on #1, #4, 
and #25.

> Any app or some specific app ?

I’ve only tried a couple of apps and only this test program has shown the 
problem.



> -phil.
> 
> 
> On 4/29/21 7:18 PM, Alan Snyder wrote:
>> I am seeing some unusual behavior (in b20) that I do not see using OpenGL 
>> (or using JDK 16).
>> 
>> Sometimes when I open a new window, the window appears blank (except for the 
>> title bar) for about two seconds before the content appears.
>> 
>> This behavior is not consistent. Opening another instance of the same window 
>> might be fast or slow. It happens with a variety of window classes.
>> 
>> In JDK16 and using OpenGL, the content always appears immediately.
>> 
>> 
>> 
>> 
>> 
>>> On Apr 23, 2021, at 1:13 PM, Philip Race  wrote:
>>> 
>>> FYI to the wider community that may not subscribe to the client mailing 
>>> lists, nor appreciate too much cross-posting.
>>> 
>>> -phil.
>>> 
>>> 
>>>  Forwarded Message 
>>> Subject:Heads up : JDK 17 b19 through b22 will use Metal instead of 
>>> OpenGL for Java 2D rendering on macOS.
>>> Date:   Fri, 23 Apr 2021 13:10:46 -0700
>>> From:   Philip Race 
>>> To: 2d-dev@openjdk.java.net <2d-dev@openjdk.java.net>
>>> CC: lanai-...@openjdk.java.net, swing-...@openjdk.java.net 
>>> , awt-...@openjdk.java.net 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Heads up to anyone who is testing JDK 17 for running apps on macOS.
>>> Starting with build 19 [1], JDK 17 for macOS is *temporarily* switched from 
>>> using OpenGL
>>> to using Apple's Metal API for Java 2D rendering. This should be invisible 
>>> to applications.
>>> We expect to revert this temporary switch in JDK 17 build 23,meaning b22 
>>> will be the last build with Metal as default.
>>> 
>>> See JEP 382 [2] for more information about how Metal is used by JDK.
>>> 
>>> If you are running any kind of 2D / Swing/ AWT UI application on macOS, and 
>>> see any rendering related problems
>>> starting with JDK 17 b19, please do report them to us at either the usual 
>>> bug submission channel [3],
>>> or on the 2d-dev@openjdk.java.net OpenJDK mailing list [4]
>>> Please be ready to provide us with a test case and screen shots.
>>> 
>>> You may also set "-Dsun.java2d.opengl=true" to re-enable OpenGL - which  
>>> implicitly disables Metal -
>>> to confirm that any problem you see is a Metal related rendering glitch.
>>> 
>>> I will also forward this email to jdk-...@openjdk.java.net
>>> 
>>> -Phil.
>>> 
>>> [1] https://jdk.java.net/17/
>>> [2] https://openjdk.java.net/jeps/382 <https://openjdk.java.net/jeps/382>
>>> [3] https://bugreport.java.com/bugreport/
>>> [4] https://mail.openjdk.java.net/mailman/listinfo/2d-dev
>>> 
> 

Re: [OpenJDK 2D-Dev] Heads up : JDK 17 b19 through b22 will use Metal instead of OpenGL for Java 2D rendering on macOS.

[Alan Snyder]

I am seeing some unusual behavior (in b20) that I do not see using OpenGL (or 
using JDK 16).

Sometimes when I open a new window, the window appears blank (except for the 
title bar) for about two seconds before the content appears.

This behavior is not consistent. Opening another instance of the same window 
might be fast or slow. It happens with a variety of window classes.

In JDK16 and using OpenGL, the content always appears immediately.





> On Apr 23, 2021, at 1:13 PM, Philip Race  wrote:
> 
> FYI to the wider community that may not subscribe to the client mailing 
> lists, nor appreciate too much cross-posting.
> 
> -phil.
> 
> 
>  Forwarded Message 
> Subject:  Heads up : JDK 17 b19 through b22 will use Metal instead of 
> OpenGL for Java 2D rendering on macOS.
> Date: Fri, 23 Apr 2021 13:10:46 -0700
> From: Philip Race 
> To:   2d-dev@openjdk.java.net <2d-dev@openjdk.java.net>
> CC:   lanai-...@openjdk.java.net, swing-...@openjdk.java.net 
> , awt-...@openjdk.java.net 
> 
> 
> 
> 
> 
> Heads up to anyone who is testing JDK 17 for running apps on macOS.
> Starting with build 19 [1], JDK 17 for macOS is *temporarily* switched from 
> using OpenGL
> to using Apple's Metal API for Java 2D rendering. This should be invisible to 
> applications.
> We expect to revert this temporary switch in JDK 17 build 23,meaning b22 will 
> be the last build with Metal as default.
> 
> See JEP 382 [2] for more information about how Metal is used by JDK.
> 
> If you are running any kind of 2D / Swing/ AWT UI application on macOS, and 
> see any rendering related problems
> starting with JDK 17 b19, please do report them to us at either the usual bug 
> submission channel [3],
> or on the 2d-dev@openjdk.java.net OpenJDK mailing list [4]
> Please be ready to provide us with a test case and screen shots.
> 
> You may also set "-Dsun.java2d.opengl=true" to re-enable OpenGL - which  
> implicitly disables Metal -
> to confirm that any problem you see is a Metal related rendering glitch.
> 
> I will also forward this email to jdk-...@openjdk.java.net
> 
> -Phil.
> 
> [1] https://jdk.java.net/17/
> [2] https://openjdk.java.net/jeps/382 
> [3] https://bugreport.java.com/bugreport/
> [4] https://mail.openjdk.java.net/mailman/listinfo/2d-dev
> 

Re: [OpenJDK 2D-Dev] EA8 build of Project Lanai (Java 2D Metal rendering pipeline for macOS) is now posted

[Alan Snyder]

> On Dec 16, 2020, at 11:57 PM, Philip Race  wrote:
> 
> To anyone who has a mac still running 10.12 - we don't expect Metal to run 
> (it requires at least 10.13
> and maybe even later by the time it is final) but we would like confirmation 
> that nothing in Metal
> prevents OpenGL running on older releases.
> Note there is currently a hotspot build bug unrelated to Lanai that prevents 
> running on 10.10
> and maybe 10.11 but 10.12 will be a useful data point.

I tried running it on macOS 10.12 under Parallels Desktop.

If I don’t ask for metal, it seems to work.

If I ask for metal, it fails awkwardly:

Failed to load library. error (null)

Re: [OpenJDK 2D-Dev] [10] Review Request: 8187399 Different problems in the javadoc's links in java.desktop package

[Alan Snyder]

It’s also not English, which not using @code would make it look like...


> On Sep 15, 2017, at 12:34 PM, Phil Race  wrote:
> 
> 771  * @implNote Please note that for Mac OS, notifications
>  772  * are only sent if the Java app is a bundled application,
>  773  * with a {@code CFBundleDocumentTypes} array present in its
>  774  * Info.plist.
> 
> Should we be using {@code ..} for CFBundleDocumentTypes. It is not Java code,
> but this makes it look like it is.
> 

Re: [OpenJDK 2D-Dev] thoughts on multiresolution images

[Alan Snyder]

> On Mar 11, 2016, at 6:27 AM, Alexander Scherbatiy 
>  wrote:
> 
> 
> Hello Alan,
> 
> Thank you for the feedback.
> 
> On 04/03/16 19:59, Alan Snyder wrote:
>> I am writing to share some thoughts based on recent experience using 
>> multiresolution images. My experience was not entirely pleasant. I am using 
>> JDK 8, although I see no relevant differences in JDK 9.
>> 
>> One of the critical issues using multiresolution images is that the 
>> selection of a specific image is not made until the application attempts to 
>> draw the image. If the returned image is fully available at that time, then 
>> it is drawn with no problem. Otherwise, the image observer is called. 
>> Typically, this will call repaint() on a component.
>> 
>> There are two potential problems:
>> 
>> (1) If the component drawing the image is actually a cell renderer, then 
>> probably repaint() does nothing. The drawing will be incomplete and may not 
>> be fixed.
>   Does the same problem exist if ordinary non multi-resolution image is used? 
> Could you provide a code sample which illustrate the issue?


In theory, the same problem can arise with an ordinary image, but in practice 
it would be unlikely. Most cell renderers are based on JLabel, which takes 
icons, not images. Creating an icon from an image using ImageIcon solves the 
problem for most ordinary images because the constructor waits for the image to 
be available. Perhaps this is why it waits!

A test case could be developed using a custom asynchronous image that is not a 
ToolkitImage, since SunToolkit.checkImage() only understands ToolkitImages.


>> 
>> (2) Otherwise, if the resolution variant was created on the fly and not 
>> cached, then when the repainting occurs, a new resolution variant image will 
>> be created. Most likely, it will not be fully available, either, so the 
>> result is a possibly endless repaint loop.
>> 
>> I don't know of a solution to problem (1). It is not a new problem. However, 
>> what is new is that the common workaround of creating an ImageIcon may not 
>> work in this case, because only certain platform created multiresolution 
>> images are recognized by ImageIcon/MediaTracker/SunToolkit. In the general 
>> case, the component does not know which resolution variant is actually 
>> needed and thus is unable to wait for its full availability. The approach of 
>> waiting for all variants to be available is suboptimal and limiting (see 
>> below).
>> 
>> Problem (2) can be solved by caching. Given the importance of caching when 
>> arbitrary images might be in use, it is surprising that there is no public 
>> support for caching. The MultiResolutionCachedImage class is JDK internal, 
>> as is the supporting ImageCache class.
>> 
>> Another problem with multiresolution images is that anything that uses the 
>> getSource() method to obtain an ImageProducer will probably not do the right 
>> thing. An important example is using FilteredImageSource and an ImageFilter 
>> to create a derived image. There is no specific specification of what 
>> getSource() should return when invoked on a multiresolution image, but 
>> whatever it returns is a single-resolution image and therefore will not be 
>> the proper image in some circumstances.
>   The ImageProducer does not contain information about Image resolution 
> variants.
>   Where is a discussion about it:
> http://mail.openjdk.java.net/pipermail/2d-dev/2016-March/006448.html 
> <http://mail.openjdk.java.net/pipermail/2d-dev/2016-March/006448.html>
>> 
>> Perhaps getSource() on a multiresolution image should thrown an exception.
>  It could break existing applications which starts to use multi-resolution 
> images and calls getSource() on it. In the current case these images are 
> silently handled as ordinary images.

This issue is moot if multiresolution image producers are used.

In the current situation, an incorrect image may be produced - either the right 
size and fuzzy or the wrong size. Throwing an exception alerts the developer 
that his code needs to be changed. Otherwise, the problem may not be discovered 
until someone runs the program on an appropriate display and notices the 
problem.





>> 
>> There seems to be an assumption that a multiresolution image should contain 
>> a "base image" at 1x. I do not see any basis for making that assumption. It 
>> seems reasonable to me to create a multiresolution image with a single, 
>> higher resolution image. The effective result is a dynamically scaled image, 
>> where the scaling factor is determined at the last possible moment, so that 
>> no resolution 

Re: [OpenJDK 2D-Dev] Fwd: creating images using native code

[Alan Snyder]

I have filed an RFE <https://bugs.openjdk.java.net/browse/JDK-8133998>.

  Alan



> On Aug 18, 2015, at 9:58 AM, Sergey Bylokhov  
> wrote:
> 
> This is 2d related question (cc 2d-dev).
>> From: Alan Snyder < <mailto:javali...@cbfiddle.com>javali...@cbfiddle.com 
>> <mailto:javali...@cbfiddle.com>>
>> Subject:  creating images using native code
>> Date: August 11, 2015 at 5:36:55 PM PDT
>> To: awt-...@openjdk.java.net <mailto:awt-...@openjdk.java.net>
>> 
>> I am currently creating images with data provided by native code by 
>> capturing the data in an int array then using that array to construct a 
>> DataBufferInt which is used to create a WritableRaster which is used to 
>> create a BufferedImage.
>> 
>> That seems to work fine, except that looking at the code it appears that the 
>> buffer is considered untrackable, which sounds like a bad thing. Does that 
>> in fact prevent caching the image in a GPU, for example?
> Yes you are right, because in this case we cannot be sure when the data 
> inside the raster is changed, this can occur for example when we copy this 
> data from/to gpu.
> 
>> 
>> If so, is there a good way to create a cacheable image?
> The public way is to draw the image once again to another one which will be 
> cached instead.
> 
>> 
>> The solutions I have found so far all wind up processing the pixels one at a 
>> time at some point, as far as I can tell, which should not be necessary.
>> 
>> I’m wondering why there is no way to simply declare that the buffer will not 
>> change in the future.
>> 
>>  Alan
>> 
> 
> 
> 
> -- 
> Best regards, Sergey. 

Re: [OpenJDK 2D-Dev] Review request for 8029339 Custom MultiResolution image support on HiDPI displays

[Alan Snyder]

It seems you are making an assumption that the code that creates the image is 
somehow initiated from a paint method so that the display scale factor can be 
obtained from the graphics context and used to determine the required image 
resolution. That is not the scenario I am concerned about.

I am talking about what I would consider a common case: a program that was 
written before the concept of HiDPI displays existed and is being adapted to 
support HiDPI displays. In this case, because the image creation is potentially 
a long running process, the program is written to use a background thread to 
create the image. Once the image is created, it is stored somewhere that can be 
accessed by a painting method of a component. Prior to the image becoming 
available, the component will paint some default background or perhaps paint a 
default image.

I do not want to rearchitect the program to support HiDPI displays. Using the 
display scale information for the available displays, I could create a 
multiresolution image in the background thread that either contains multiple 
images for all available displays or contains one image at the maximum required 
resolution of the available displays. However, these solutions are non-optimal. 
Either images may be created that will never be used or image scaling may be 
used to reduce the resolution, which may result in a lower quality image.

What I would like to do is create a multiresolution image that tells my code 
what resolution image is needed and allows me to create that image on a 
background thread. In additional to solving the HiDPI problem optimally, this 
solution would support less common cases, such as displays added to the system 
at run time and graphics contexts that are scaled for some reason other than 
supporting a HiDPI display.

I am assuming this capability already exists to support toolkit images. It 
would be nice to make that capability available to applications.

  Alan




> On Jul 15, 2015, at 4:07 AM, Alexander Scherbatiy 
>  wrote:
> 
> On 7/14/2015 3:18 AM, Alan Snyder wrote:
>> I have a concern about how custom multiresolution images are supported based 
>> on a problem I ran into using the JDK 8 classes, which are similar.
>> 
>> The problem is that the selection of a variant image happens during 
>> painting. That is a very bad time to actually try to render an image, if one 
>> is in a situation where images of arbitrary resolution can be produced by 
>> rendering.
>> 
>> If I have code now that uses a background thread to create an image in this 
>> manner, how can I support HiDPI displays? Creating a multiresolution image 
>> defeats the purpose of using a background thread if it defers rendering 
>> until paint time. It seems that the best I can do is to guess what 
>> resolutions might be wanted and generate them all in advance or, 
>> alternatively, generate the highest resolution I expect to need, and have 
>> the graphics system scale it down for lower resolutions when requested. 
>> Neither of these alternatives is optimal.
>   JDK 9 has the fix that includes the scale factor in the 
> GraphicsConfiguration default transform: 
> http://hg.openjdk.java.net/jdk9/client/jdk/rev/661136704d07
> 
>   It is possible to iterate over all graphics devices, decide on which device 
> the image is supposed to be rendered and generate an image resolution 
> according to the scale factor from the device graphics configuration.
> 
>  Thanks,
>  Alexandr.
> 
>> 
>> Although the Image class does allow images to be produced in the background, 
>> that API is way too complex for use by an application, and the supporting 
>> code for Toolkit images is not accessible.
>> 
>> It would be nice to have a simple way to return an image that is produced in 
>> the background. Something like Future?
>> 
>>   Alan
>> 
> 

Re: [OpenJDK 2D-Dev] Review request for 8029339 Custom MultiResolution image support on HiDPI displays

[Alan Snyder]

I am commenting on your suggestion that I can determine which display the image 
will displayed on.



> On Jul 15, 2015, at 7:42 AM, Alexander Scherbatiy 
>  wrote:
> 
> On 7/15/2015 5:09 PM, Alan Snyder wrote:
>> 
>> It seems you are making an assumption that the code that creates the image 
>> is somehow initiated from a paint method so that the display scale factor 
>> can be obtained from the graphics context and used to determine the required 
>> image resolution. That is not the scenario I am concerned about.
>> 
>   The information about screen devices are available from the 
> GraphicsEnvironment.
>   Something like:
>  --
>public static void main(String[] args) {
>GraphicsEnvironment ge = 
> GraphicsEnvironment.getLocalGraphicsEnvironment();
>for(GraphicsDevice device: ge.getScreenDevices()){
>for(GraphicsConfiguration config: device.getConfigurations()){
>if(!config.getDefaultTransform().isIdentity()){
>System.out.println("Start high-resolution image loading");
>}
>}
>}
>}
>  --
> 
>  Thanks,
>  Alexandr.
>> 
>> 
>> I am talking about what I would consider a common case: a program that was 
>> written before the concept of HiDPI displays existed and is being adapted to 
>> support HiDPI displays. In this case, because the image creation is 
>> potentially a long running process, the program is written to use a 
>> background thread to create the image. Once the image is created, it is 
>> stored somewhere that can be accessed by a painting method of a component. 
>> Prior to the image becoming available, the component will paint some default 
>> background or perhaps paint a default image.
>> 
>> I do not want to rearchitect the program to support HiDPI displays. Using 
>> the display scale information for the available displays, I could create a 
>> multiresolution image in the background thread that either contains multiple 
>> images for all available displays or contains one image at the maximum 
>> required resolution of the available displays. However, these solutions are 
>> non-optimal. Either images may be created that will never be used or image 
>> scaling may be used to reduce the resolution, which may result in a lower 
>> quality image.
>> 
>> What I would like to do is create a multiresolution image that tells my code 
>> what resolution image is needed and allows me to create that image on a 
>> background thread. In additional to solving the HiDPI problem optimally, 
>> this solution would support less common cases, such as displays added to the 
>> system at run time and graphics contexts that are scaled for some reason 
>> other than supporting a HiDPI display.
>> 
>> I am assuming this capability already exists to support toolkit images. It 
>> would be nice to make that capability available to applications.
>> 
>>  Alan
>> 
>> 
>>> On Jul 15, 2015, at 4:07 AM, Alexander Scherbatiy 
>>> mailto:alexandr.scherba...@oracle.com> 
>>> <mailto:alexandr.scherba...@oracle.com 
>>> <mailto:alexandr.scherba...@oracle.com>>> wrote:
>>> 
>>> On 7/14/2015 3:18 AM, Alan Snyder wrote:
>>>> I have a concern about how custom multiresolution images are supported 
>>>> based on a problem I ran into using the JDK 8 classes, which are similar.
>>>> 
>>>> The problem is that the selection of a variant image happens during 
>>>> painting. That is a very bad time to actually try to render an image, if 
>>>> one is in a situation where images of arbitrary resolution can be produced 
>>>> by rendering.
>>>> 
>>>> If I have code now that uses a background thread to create an image in 
>>>> this manner, how can I support HiDPI displays? Creating a multiresolution 
>>>> image defeats the purpose of using a background thread if it defers 
>>>> rendering until paint time. It seems that the best I can do is to guess 
>>>> what resolutions might be wanted and generate them all in advance or, 
>>>> alternatively, generate the highest resolution I expect to need, and have 
>>>> the graphics system scale it down for lower resolutions when requested. 
>>>> Neither of these alternatives is optimal.
>>>  JDK 9 has the fix that includes the scale factor in the 
>>> GraphicsConfiguration default transform: 
>>> http://hg.openjdk.java.net/jdk9/client/jdk/rev/661136704d0

Re: [OpenJDK 2D-Dev] Review request for 8029339 Custom MultiResolution image support on HiDPI displays

[Alan Snyder]

I have a concern about how custom multiresolution images are supported based on 
a problem I ran into using the JDK 8 classes, which are similar.

The problem is that the selection of a variant image happens during painting. 
That is a very bad time to actually try to render an image, if one is in a 
situation where images of arbitrary resolution can be produced by rendering.

If I have code now that uses a background thread to create an image in this 
manner, how can I support HiDPI displays? Creating a multiresolution image 
defeats the purpose of using a background thread if it defers rendering until 
paint time. It seems that the best I can do is to guess what resolutions might 
be wanted and generate them all in advance or, alternatively, generate the 
highest resolution I expect to need, and have the graphics system scale it down 
for lower resolutions when requested. Neither of these alternatives is optimal.

Although the Image class does allow images to be produced in the background, 
that API is way too complex for use by an application, and the supporting code 
for Toolkit images is not accessible.

It would be nice to have a simple way to return an image that is produced in 
the background. Something like Future?

  Alan