> On Mar 11, 2016, at 6:27 AM, Alexander Scherbatiy > <alexandr.scherba...@oracle.com> wrote: > > > Hello Alan, > > Thank you for the feedback. > > On 04/03/16 19:59, Alan Snyder wrote: >> I am writing to share some thoughts based on recent experience using >> multiresolution images. My experience was not entirely pleasant. I am using >> JDK 8, although I see no relevant differences in JDK 9. >> >> One of the critical issues using multiresolution images is that the >> selection of a specific image is not made until the application attempts to >> draw the image. If the returned image is fully available at that time, then >> it is drawn with no problem. Otherwise, the image observer is called. >> Typically, this will call repaint() on a component. >> >> There are two potential problems: >> >> (1) If the component drawing the image is actually a cell renderer, then >> probably repaint() does nothing. The drawing will be incomplete and may not >> be fixed. > Does the same problem exist if ordinary non multi-resolution image is used? > Could you provide a code sample which illustrate the issue?
In theory, the same problem can arise with an ordinary image, but in practice it would be unlikely. Most cell renderers are based on JLabel, which takes icons, not images. Creating an icon from an image using ImageIcon solves the problem for most ordinary images because the constructor waits for the image to be available. Perhaps this is why it waits! A test case could be developed using a custom asynchronous image that is not a ToolkitImage, since SunToolkit.checkImage() only understands ToolkitImages. >> >> (2) Otherwise, if the resolution variant was created on the fly and not >> cached, then when the repainting occurs, a new resolution variant image will >> be created. Most likely, it will not be fully available, either, so the >> result is a possibly endless repaint loop. >> >> I don't know of a solution to problem (1). It is not a new problem. However, >> what is new is that the common workaround of creating an ImageIcon may not >> work in this case, because only certain platform created multiresolution >> images are recognized by ImageIcon/MediaTracker/SunToolkit. In the general >> case, the component does not know which resolution variant is actually >> needed and thus is unable to wait for its full availability. The approach of >> waiting for all variants to be available is suboptimal and limiting (see >> below). >> >> Problem (2) can be solved by caching. Given the importance of caching when >> arbitrary images might be in use, it is surprising that there is no public >> support for caching. The MultiResolutionCachedImage class is JDK internal, >> as is the supporting ImageCache class. >> >> Another problem with multiresolution images is that anything that uses the >> getSource() method to obtain an ImageProducer will probably not do the right >> thing. An important example is using FilteredImageSource and an ImageFilter >> to create a derived image. There is no specific specification of what >> getSource() should return when invoked on a multiresolution image, but >> whatever it returns is a single-resolution image and therefore will not be >> the proper image in some circumstances. > The ImageProducer does not contain information about Image resolution > variants. > Where is a discussion about it: > http://mail.openjdk.java.net/pipermail/2d-dev/2016-March/006448.html > <http://mail.openjdk.java.net/pipermail/2d-dev/2016-March/006448.html> >> >> Perhaps getSource() on a multiresolution image should thrown an exception. > It could break existing applications which starts to use multi-resolution > images and calls getSource() on it. In the current case these images are > silently handled as ordinary images. This issue is moot if multiresolution image producers are used. In the current situation, an incorrect image may be produced - either the right size and fuzzy or the wrong size. Throwing an exception alerts the developer that his code needs to be changed. Otherwise, the problem may not be discovered until someone runs the program on an appropriate display and notices the problem. >> >> There seems to be an assumption that a multiresolution image should contain >> a "base image" at 1x. I do not see any basis for making that assumption. It >> seems reasonable to me to create a multiresolution image with a single, >> higher resolution image. The effective result is a dynamically scaled image, >> where the scaling factor is determined at the last possible moment, so that >> no resolution is lost unnecessarily. I also observe that in the future, 1x >> representations will be less and less useful. > The base image is used to get size/properties/source/... from the main > image. It is possible to override them without the base image usage. > > The base image width and height are really necessary for the resolution > variant size calculation. SunGraphics2D calls > mrImage.getResolutionVariant(scale * baseImageWidth, scale * baseImageHeight) > to obtain the high-resolution image. > > MultiResolutionCachedImage takes base image width and height as a parameter > to avoid triggering base image creation. > I agree that every image needs to have a “layout size” and that a 1x base image may be used to get the layout size and to get properties. I do not believe that the base image needs to be 1x or that there needs to be a base image at all (even if it is constructed lazily). I disagree that a base image is appropriate for getSource(), per the above discussion. SunGraphics2D does not use the base image sizes. It calls getWidth() and getHeight(). I’m being picky here because only if you make the assumption of a 1x base image does your statement make sense. I think “layout size” is a better term for what you are calling the base image size. An image needs to have a size for layout. It does not need to have a base image. >> >> I also question the rationale for the getResolutionVariants() method. This >> method assumes that a MultiResolutionImage contains a fixed number of >> variants. I do not see any reason to make that restriction. The resolution >> variants might be created from a scalable description, such as vector >> graphics. Even if the number of variants is fixed, if the image is served >> remotely, it might be very expensive to obtain them all. A lazy approach to >> creating derived multiresolution images is better. > This is mostly used for native image construction when it is passed to the > native system. > > For example to set a high-resolution cursor it is only necessary to call > Toolkit.getDefaultToolkit().createCustomCursor(mrImage, ...). > > Then the NSImage with given low and high resolution representation are > created on Mac OS X. There are two options that would work for this usage without needing to enumerate image representations. The simplest option is to determine which two resolutions you need and ask for them directly using getResolutionVariant(). The other option, if you cannot determine which resolutions might be needed, is a lazy implementation. Admittedly, that would be more work. I would recommend either removing getResolutionVariants() from the MultiResolutionImage interface or making its implementation optional (e.g. return null). >> >> To work around some of these problems, I created my own API that includes a >> method similar to the map() method of MultiResolutionCachedImage. It seems >> to me that a method like this is needed, not just in MultiResolutionImage, >> but in Image itself, so that applications can write code that works on >> Images in general, including the MultiResolutionImage variety. >> >> Previously [1], the following code was suggested as a way to create a >> filtered MultiResolutionImage: >> >> static Image applyFilter(Image image) { >> // apply a filter to create ligtened image >> } >> >> static class LigtenedMultiresolutionImage extends >> AbstractMultiResolutionImage { >> >> private final Image baseImage; >> >> public LigtenedMultiresolutionImage(Image baseImage) { >> this.baseImage = baseImage; >> } >> >> @Override >> public Image getResolutionVariant(float destImageWidth, float >> destImageHeight) { >> Image rvImage = ((MultiResolutionImage) baseImage). >> getResolutionVariant(destImageWidth, destImageHeight); >> return applyFilter(rvImage); >> } >> >> @Override >> public List<Image> getResolutionVariants() { >> List<Image> resolutionvariants = new LinkedList<>(); >> for (Image image : ((MultiResolutionImage) baseImage). >> getResolutionVariants()) { >> resolutionvariants.add(applyFilter(image)); >> } >> return resolutionvariants; >> } >> >> @Override >> protected Image getBaseImage() { >> return applyFilter(baseImage); >> } >> } >> >> I note that the lack of caching means that this code will be reliable only >> if applyFilter() returns a fully available image. >> >> My observation is one needs to know a lot more than one might expect to >> succeed at using multiresolution images. > Will the public CachedMultiResolutionImage class and "Image map(Image image, > Function<Image, Image> mapper)" method help to solve most of the described > problems? > Could your create an enhancement on it: http://bugs.java.com > <http://bugs.java.com/> ? > I will consider this and submit an RFE if appropriate. > Thanks, > Alexandr. >> [1] http://mail.openjdk.java.net/pipermail/2d-dev/2014-June/004638.html >> <http://mail.openjdk.java.net/pipermail/2d-dev/2014-June/004638.html>
