On Wed, 31 May 2023 15:03:04 GMT, Alexander Zvegintsev <azveg...@openjdk.org> wrote:
>> Modern Linux systems often come with >> [Wayland](https://wayland.freedesktop.org/) by default. >> This comes with some difficulties, and one of them is the inability to get >> screenshots from the system. >> This is because we now use the [X Window System >> API](https://en.wikipedia.org/wiki/X_Window_System) to capture screenshots >> and it cannot access data outside the [XWayland >> server](https://wayland.freedesktop.org/xserver.html) >> >> But this functionality is a very important part of automated testing. >> >> >> At the moment there are two obvious solutions to this problem, and both use >> [xdg-desktop-portal](https://github.com/flatpak/xdg-desktop-portal): >> >> 1. [org.freedesktop.portal.Screenshot DBUS >> API](https://flatpak.github.io/xdg-desktop-portal/#gdbus-org.freedesktop.portal.Screenshot) >> It has several drawbacks though: >> + It saves a screenshot to disk, which must be read and deleted(may add some >> delays depending on the type of a disk drive). >> + There is no way to disable the visual "screen flash" after screenshot >> + It asks a user confirmation to save a screenshot. This confirmation can be >> saved on Gnome 43+. >> Since we would like Ubuntu 22.04 LTS which comes with Gnome 42 this option >> is not acceptable for us because it would require user confirmation for each >> screenshot. >> But we still can consider this option as a fallback. >> >> >> >> 2. >> [org.freedesktop.portal.ScreenCast](https://flatpak.github.io/xdg-desktop-portal/#gdbus-org.freedesktop.portal.ScreenCast) >> It typically used by applications that need to capture the contents of the >> user's screen or a specific window for the purpose of sharing, recording, or >> streaming. >> This might be a bit of overkill, but it avoids several of the problems >> mentioned in the Screenshot API. >> >> + implementation is more complicated comparing to Screenshot API >> + no intermediate file, screenshot data can be obtained from memory >> + Permission to make screenshots can be stored with >> [`restore_token`](https://flatpak.github.io/xdg-desktop-portal/#gdbus-method-org-freedesktop-portal-ScreenCast.SelectSources) >> >> >> So this PR adds the ability to take screenshots using the ScreenCast API. >> This functionality is currently disabled by default. >> >> This change also introduces some new behavior for the robot: >> A system window now appears asking for confirmation from the user to capture >> the screen. >> + The user can refuse the screen capture completely. In this case a security >> exception will be thrown. >> + The user can allow... > > Alexander Zvegintsev has updated the pull request incrementally with four > additional commits since the last revision: > > - improve retVal processing > - address token storage comments > - removing non-ascii > - EXCEPTION_CHECK_DESCRIBE_CLEAR -> EXCEPTION_CHECK_DESCRIBE I spoke too soon :-( I remembered about an odd result from last week and it seems that whenever I have a full screen window capture fails - at least on my system and I think one lab system. See test and output below from a fully patched Ubuntu 22.04.02 system import java.awt.Frame; import java.awt.GraphicsDevice; import java.awt.Rectangle; import java.awt.Robot; public class FSW { public static void main(String args[]) throws Exception { Robot robot = new Robot(); Frame frame = new Frame("FSW"); GraphicsDevice gd = frame.getGraphicsConfiguration().getDevice(); try { gd.setFullScreenWindow(frame); robot.delay(5000); robot.createScreenCapture(new Rectangle(50, 50, 10, 10)); } finally { gd.setFullScreenWindow(null); frame.dispose(); } } } % java FSW cropTo:163 Unexpected stride / 4: 0 srcW: 1920 and the capture fails every time. And again with debugging turned on ScreencastWatcher: started // getRGBPixels affectedScreenBounds [java.awt.Rectangle[x=0,y=0,width=1920,height=1200]] // getTokens exact matches 1. [Token: 41dc61fb-279b-4069-b408-30873451fb03 java.awt.Rectangle[x=0,y=0,width=1920,height=1200] ] // getTokens same sizes 2. [Token: 41dc61fb-279b-4069-b408-30873451fb03 java.awt.Rectangle[x=0,y=0,width=1920,height=1200] ] // storeToken old: |41dc61fb-279b-4069-b408-30873451fb03| new |41dc61fb-279b-4069-b408-30873451fb03| allowed bounds [0, 0, 1920, 1200] // Storing TokenItem: Token: 41dc61fb-279b-4069-b408-30873451fb03 java.awt.Rectangle[x=0,y=0,width=1920,height=1200] initXdgDesktopPortal:236 connection/sender name :1.525 / 1_525 checkVersion:190 ScreenCast protocol version 4 Java_sun_awt_screencast_ScreencastHelper_getRGBPixelsImpl:833 taking screenshot at x: 50 y 50 w 10 h 10 with token |41dc61fb-279b-4069-b408-30873451fb03| initXdgDesktopPortal:236 connection/sender name :1.525 / 1_525 checkVersion:190 ScreenCast protocol version 4 callbackScreenCastStart:611 available screen count 1 rebuildScreenData:87 ==== screenId#35 rebuildScreenData:132 ----------------------- rebuildScreenData:133 screenId#35 || bounds x 0 y 0 w 1920 h 1200 || capture area x 0 y 0 w 0 h 0 shouldCapture 0 rebuildScreenData:134 #---------------------# callbackScreenCastStart:617 rebuildScreenData result |0| callbackScreenCastStart:630 restore_token |41dc61fb-279b-4069-b408-30873451fb03| storeRestoreToken:653 saving token, old: |41dc61fb-279b-4069-b408-30873451fb03| > new: |41dc61fb-279b-4069-b408-30873451fb03| portalScreenCastStart:709 ScreenCastStartResult |0| getPipewireFd:886 portalScreenCastStart result |0| checkCanCaptureAllRequiredScreens:847 Found allowed screen bounds in affected screen bounds 0 0 1920 1200 getPipewireFd:900 --- portalScreenCastStart getPipewireFd:907 pwFd 17 doLoop:557 screenId#35[loc(0,0) size(1920x1200)] @@@ adding screen 0 checkScreen:472 screenId#35 || bounds x 0 y 0 w 1920 h 1200 || capture area x 50 y 50 w 10 h 10 shouldCapture 1 connectStream:373 @@@ using screen 0 connectStream:411 screenId#35 || bounds x 0 y 0 w 1920 h 1200 || capture area x 50 y 50 w 10 h 10 shouldCapture 1 startStream:355 screenId#35: stream connecting 0x7f04548e3600 onStreamStateChanged:303 screenId#35[loc(0,0) size(1920x1200)] state 0 (unconnected) -> 1 (connecting) err (null) onStreamStateChanged:303 screenId#35[loc(0,0) size(1920x1200)] state 1 (connecting) -> 2 (paused) err (null) onStreamParamChanged:197 screenId#35[loc(0,0) size(1920x1200)] param event id 4 onStreamParamChanged:218 screenId#35[loc(0,0) size(1920x1200)] stream format: Spa:Enum:VideoFormat:BGRx (8) 1920x1200 connectStream:424 screenId#35[loc(0,0) size(1920x1200)] frame size: 1920x1200 doLoop:563 screenId#35[loc(0,0) size(1920x1200)] @@@ screen processed 0 onStreamStateChanged:303 screenId#35[loc(0,0) size(1920x1200)] state 2 (paused) -> 3 (streaming) err (null) onStreamProcess:236 screenId#35[loc(0,0) size(1920x1200)] hasFormat 1 captureDataReady 0 shouldCapture 1 onStreamProcess:271 screenId#35 || bounds x 0 y 0 w 1920 h 1200 || capture area x 50 y 50 w 10 h 10 shouldCapture 1 onStreamProcess:272 screenId#35[loc(0,0) size(1920x1200)] got a frame of size 0 offset 0 stride 0 flags 0 FD 31 captureDataReady 0 onStreamProcess:292 screenId#35[loc(0,0) size(1920x1200)] data ready Java_sun_awt_screencast_ScreencastHelper_getRGBPixelsImpl:856 all data ready Java_sun_awt_screencast_ScreencastHelper_getRGBPixelsImpl:864 screenId#35[loc(0,0) size(1920x1200)] @@@ copying screen data 0, captureData 0x7f03d40042a0 || x 50 y 50 w 10 h 10 requested area || x 0 y 0 w 1920 h 1200 screen bound || x 50 y 50 w 10 h 10 in-screen coords capture area onStreamParamChanged:197 screenId#35[loc(0,0) size(1920x1200)] param event id 4 onStreamStateChanged:303 screenId#35[loc(0,0) size(1920x1200)] state 3 (streaming) -> 2 (paused) err (null) onStreamStateChanged:303 screenId#35[loc(0,0) size(1920x1200)] state 2 (paused) -> 0 (unconnected) err (null) doCleanup:109 STOPPING loop ------------- PR Comment: https://git.openjdk.org/jdk/pull/13803#issuecomment-1571048110
