Hello Alan,
Thank you for the feedback.
On 04/03/16 19:59, Alan Snyder wrote:
I am writing to share some thoughts based on recent experience using
multiresolution images. My experience was not entirely pleasant. I am using JDK
8, although I see no relevant differences in JDK 9.
One of the critical issues using multiresolution images is that the selection
of a specific image is not made until the application attempts to draw the
image. If the returned image is fully available at that time, then it is drawn
with no problem. Otherwise, the image observer is called. Typically, this will
call repaint() on a component.
There are two potential problems:
(1) If the component drawing the image is actually a cell renderer, then
probably repaint() does nothing. The drawing will be incomplete and may not be
fixed.
Does the same problem exist if ordinary non multi-resolution image
is used? Could you provide a code sample which illustrate the issue?
(2) Otherwise, if the resolution variant was created on the fly and not cached,
then when the repainting occurs, a new resolution variant image will be
created. Most likely, it will not be fully available, either, so the result is
a possibly endless repaint loop.
I don't know of a solution to problem (1). It is not a new problem. However,
what is new is that the common workaround of creating an ImageIcon may not work
in this case, because only certain platform created multiresolution images are
recognized by ImageIcon/MediaTracker/SunToolkit. In the general case, the
component does not know which resolution variant is actually needed and thus is
unable to wait for its full availability. The approach of waiting for all
variants to be available is suboptimal and limiting (see below).
Problem (2) can be solved by caching. Given the importance of caching when
arbitrary images might be in use, it is surprising that there is no public
support for caching. The MultiResolutionCachedImage class is JDK internal, as
is the supporting ImageCache class.
Another problem with multiresolution images is that anything that uses the
getSource() method to obtain an ImageProducer will probably not do the right
thing. An important example is using FilteredImageSource and an ImageFilter to
create a derived image. There is no specific specification of what getSource()
should return when invoked on a multiresolution image, but whatever it returns
is a single-resolution image and therefore will not be the proper image in some
circumstances.
The ImageProducer does not contain information about Image
resolution variants.
Where is a discussion about it:
http://mail.openjdk.java.net/pipermail/2d-dev/2016-March/006448.html
Perhaps getSource() on a multiresolution image should thrown an exception.
It could break existing applications which starts to use
multi-resolution images and calls getSource() on it. In the current case
these images are silently handled as ordinary images.
There seems to be an assumption that a multiresolution image should contain a "base
image" at 1x. I do not see any basis for making that assumption. It seems reasonable
to me to create a multiresolution image with a single, higher resolution image. The
effective result is a dynamically scaled image, where the scaling factor is determined at
the last possible moment, so that no resolution is lost unnecessarily. I also observe
that in the future, 1x representations will be less and less useful.
The base image is used to get size/properties/source/... from the
main image. It is possible to override them without the base image usage.
The base image width and height are really necessary for the
resolution variant size calculation. SunGraphics2D calls
mrImage.getResolutionVariant(scale * baseImageWidth, scale *
baseImageHeight) to obtain the high-resolution image.
MultiResolutionCachedImage takes base image width and height as a
parameter to avoid triggering base image creation.
I also question the rationale for the getResolutionVariants() method. This
method assumes that a MultiResolutionImage contains a fixed number of variants.
I do not see any reason to make that restriction. The resolution variants might
be created from a scalable description, such as vector graphics. Even if the
number of variants is fixed, if the image is served remotely, it might be very
expensive to obtain them all. A lazy approach to creating derived
multiresolution images is better.
This is mostly used for native image construction when it is passed
to the native system.
For example to set a high-resolution cursor it is only necessary to call
Toolkit.getDefaultToolkit().createCustomCursor(mrImage, ...).
Then the NSImage with given low and high resolution representation
are created on Mac OS X.
To work around some of these problems, I created my own API that includes a
method similar to the map() method of MultiResolutionCachedImage. It seems to
me that a method like this is needed, not just in MultiResolutionImage, but in
Image itself, so that applications can write code that works on Images in
general, including the MultiResolutionImage variety.
Previously [1], the following code was suggested as a way to create a filtered
MultiResolutionImage:
static Image applyFilter(Image image) {
// apply a filter to create ligtened image
}
static class LigtenedMultiresolutionImage extends
AbstractMultiResolutionImage {
private final Image baseImage;
public LigtenedMultiresolutionImage(Image baseImage) {
this.baseImage = baseImage;
}
@Override
public Image getResolutionVariant(float destImageWidth, float
destImageHeight) {
Image rvImage = ((MultiResolutionImage) baseImage).
getResolutionVariant(destImageWidth, destImageHeight);
return applyFilter(rvImage);
}
@Override
public List<Image> getResolutionVariants() {
List<Image> resolutionvariants = new LinkedList<>();
for (Image image : ((MultiResolutionImage) baseImage).
getResolutionVariants()) {
resolutionvariants.add(applyFilter(image));
}
return resolutionvariants;
}
@Override
protected Image getBaseImage() {
return applyFilter(baseImage);
}
}
I note that the lack of caching means that this code will be reliable only if
applyFilter() returns a fully available image.
My observation is one needs to know a lot more than one might expect to succeed
at using multiresolution images.
Will the public CachedMultiResolutionImage class and "Image map(Image
image, Function<Image, Image> mapper)" method help to solve most of the
described problems?
Could your create an enhancement on it: http://bugs.java.com ?
Thanks,
Alexandr.
[1] http://mail.openjdk.java.net/pipermail/2d-dev/2014-June/004638.html