On Wed, Jan 12, 2022 at 8:06 AM Ramin Halavati <[email protected]> wrote:

> Thank you very much Philip. I will write a 1-pager on this discussion for
> posteriority and will proceed with the inside blink approach.
>
> Just as the concluding question, I have a draft CL on it (
> crrev.com/c/3370381). Would you suggest that I take a (possibly
> simplified) copy of the DataTransfer::NodeImage
> in third_party/blink/renderer/modules/accessibility, or refactor the
> existing code in DataTransfer to fit the existing and the new use case. I
> prefer the latter as it is a very delicate and complex process. If you
> agree with that, do you have any suggestions on the refactoring or I would
> just go with it and ask you to take a look after?
>

If the accessibility and drag-image code are unlikely to diverge too much,
I think it makes sense to factor out the code that creates a PaintArtifact
for a Node out of DraggedNodeImageBuilder and into a helper. This helper
could include a comment about how the painting starts from the containing
stacking context.


>
> Best,
> Ramin
>
> On Tue, Jan 11, 2022 at 6:58 PM Philip Rogers <[email protected]> wrote:
>
>>
>>
>> On Tue, Jan 11, 2022 at 7:42 AM Ramin Halavati <[email protected]>
>> wrote:
>>
>>> Thank you very much Philip,
>>>
>>> About the concern that a repaint based approach (similar to NodeImage)
>>> will not have the backgrounds or other higher level effects, can we say
>>> that on the other hand (specially since this function is explicitly
>>> querying the image for a certain node) the full-page-paint+crop approach
>>> also suffers from the possibility that the node maybe be partially covered
>>> by another node, or being out of the screen's visible area (needing
>>> scroll), or even being covered by a dialog box?
>>>
>>
>> That's correct: the full-page-crop approach will include content on top
>> of the target content, will not include the target if it is scrolled
>> offscreen, etc.
>>
>>
>>>
>>> On Mon, Jan 10, 2022 at 8:29 PM Philip Rogers <[email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Jan 10, 2022 at 6:51 AM Ramin Halavati <[email protected]>
>>>> wrote:
>>>>
>>>>> Thank you Philip.
>>>>>
>>>>> Since this accessibility support function is called for all renderer
>>>>> types (in ChromeOS including Arc++ and Ash), there would be a better
>>>>> consistency if we have the snapshotting code in the renderer. Therefore I
>>>>> am trying to avoid using the browser side ui::GrabViewSnapshot (as in
>>>>> DevTools), unless the alternative approach proves to be too complicated or
>>>>> unnecessarily heavy.
>>>>>
>>>>> I tried using DataTransfer::NodeImage in this draft CL
>>>>> <http://crrev.com/c/3370381>, but my understanding is that since it
>>>>> triggers a repaint, we should make sure it is not in the middle of a
>>>>> lifecycle update to avoid conflicts. Is there a notification of a
>>>>> signal that tells when it's safe to request it?
>>>>>
>>>>
>>>> It is important that we do not introduce lifecycle violations but I
>>>> think that may already be guaranteed from when the accessibility code runs.
>>>> For example, AxObject::GetRelativeBounds cannot be called from within a
>>>> lifecycle update either. You can probably just synchronously call the paint
>>>> function you need.
>>>>
>>>>
>>>>> I assume it's the same for the SpoolSinglePage as it also redraws the
>>>>> image. And performance wise, can we say that as this approach triggers a
>>>>> repaint, it's heavier than the browser based snapshotting (using
>>>>> ui::GrabViewSnapshot), as the latter is using the already painted output
>>>>> and does not require a repaint?
>>>>>
>>>>
>>>> The paint code in NodeImage is really just a walk of layout
>>>> datastructures and isn't more expensive than things like layout, style, etc
>>>> (i.e., it's O(|layout|)). Doing this for each object would be wildly
>>>> inefficient though.
>>>>
>>>> Expanding on the thing about this being an ambiguous problem:
>>>> data:text/html,<body style="background: rebeccapurple; filter:
>>>> blur(1px)"><div style="height: 100px; width: 100px; border: 1px solid
>>>> white; color:white;" draggable=true>hello world</div>
>>>> The drag image for the div will only contain white and transparent
>>>> pixels, and will not be blurred. If you want the purple background too,
>>>> you'll want to do the full-page-paint+crop approach.
>>>>
>>>>
>>>>>
>>>>> Best,
>>>>> Ramin
>>>>>
>>>>> On Fri, Jan 7, 2022 at 11:06 PM Philip Rogers <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> This is a deceptively difficult problem in general. Our paint system
>>>>>> uses the CSS definition of paint order which starts painting at stacking
>>>>>> contexts and it is challenging to exclude non-stacked siblings of a
>>>>>> non-stacked target element. Some other complications are whether ancestor
>>>>>> effects/clips should be applied, and whether content behind the target
>>>>>> should be included.
>>>>>>
>>>>>> DataTransfer::NodeImage is an attempt at this. Another option is to
>>>>>> get the entire page painted (examples: devtools uses
>>>>>> PageHandler::CaptureScreenshot and printing uses SpoolSinglePage), then
>>>>>> crop it to the target element.
>>>>>>
>>>>>> On Fri, Jan 7, 2022 at 7:01 AM 'Ramin Halavati' via blink-dev <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am trying to find a way to get pixels for any general node in
>>>>>>> *third_party/blink/renderer/modules/accessibility. *We already have
>>>>>>> AXNodeObject::ImageDataUrl
>>>>>>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/modules/accessibility/ax_node_object.cc;drc=a3ae56ec162fcbf9ee400e9ff80f1ae572957fd3;bpv=1;bpt=1;l=2355?gsn=ImageDataUrl&gs=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dsrc%2Fthird_party%2Fblink%2Frenderer%2Fmodules%2Faccessibility%2Fax_node_object.cc%23rau1_dUeeJHt__0C5nqtyH5ouOMnhdEN55IRAHfVRMk&gs=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dsrc%2Fthird_party%2Fblink%2Frenderer%2Fmodules%2Faccessibility%2Fax_node_object.h%23z0USMj0KE_TnmxgTOdb4D-K1NmRUt1i1gyvtbSdYs5w>
>>>>>>>  function
>>>>>>> to return the bitmap of a certain node, but it is only supporting HTML
>>>>>>> element types that already include a picture (namely image, canvas, 
>>>>>>> video)
>>>>>>> and I am trying to expand to to any general element type.
>>>>>>>
>>>>>>> So far I've been looking for a way to get access to a painter or
>>>>>>> compositor, but I did not find any promising solution yet. Can you 
>>>>>>> point me
>>>>>>> in the right direction? Is it even expected at this abstraction level?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ramin
>>>>>>>
>>>>>>> P.S., Sorry for naive questions and please point me to the right
>>>>>>> email thread if this is not.
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "blink-dev" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAORodQi_LSUFXWxpfAKQKKVhRzK1UvGVz%2B4unQUN2qskFSSafw%40mail.gmail.com
>>>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAORodQi_LSUFXWxpfAKQKKVhRzK1UvGVz%2B4unQUN2qskFSSafw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to [email protected].
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJgFLLvctS5hZxY5SHVisrpgkfX0zL5C_Y%2BH%2Bbkhg%3D8TontLAQ%40mail.gmail.com.

Reply via email to