I filed an issue here to track the keydown/compositionstart ordering question: https://github.com/w3c/uievents/issues/360.
We’ve received the OT feedback over email so I don’t have a public link to share. Aside from specific bugs which we fixed, the feedback has been positive; Docs and Adobe were able to use EditContext to replace hidden contenteditable element hacks during composition. I’ll ask them to weigh in on whether this event ordering question has been a problem for them. -- Dan From: Ian Kilpatrick <ikilpatr...@chromium.org> Sent: Thursday, November 9, 2023 10:07 AM To: Rick Byers <rby...@chromium.org> Cc: Gregg Tavares <g...@chromium.org>; Daniel Bratell <bratel...@gmail.com>; Mike Taylor <miketa...@chromium.org>; Alex Russell <slightly...@chromium.org>; blink-dev <blink-dev@chromium.org>; Alex Keng <shih...@microsoft.com>; Anupam Snigdha <sni...@microsoft.com>; Koji Ishii <ko...@chromium.org>; Daniel Clark <dan...@microsoft.com> Subject: Re: [blink-dev] Intent to Ship: EditContext API You don't often get email from ikilpatr...@chromium.org<mailto:ikilpatr...@chromium.org>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Providing a little more context for the benefit of this API - the complexity that editors (Google Docs, Adobe tools, MS Office, etc) have to deal with for providing a good user experience is substantial and multiple engineer years worth of work to get close to correct. They typically have to create a hidden contenteditable element then trap + mutate all sorts of inputs to correctly (across different OSes, user locales, user input methods) to provide an experience inline with what the user expects - then render all that to an entirely different surface (canvas, svg, parallel dom, etc). Even with this amount of effort there are typically lots of issues/bugs<https://github.com/w3c/edit-context/blob/gh-pages/explainer.md#real-world-examples-of-text-input-issues-in-top-sites-and-frameworks> due to this approach. This API reduces the complexity this style of editors have to deal with substantially. I personally hope we'll see more high quality editors from smaller teams as a result of this API given for most web developers providing a good experience to most users is an insurmountable task. Ian On Thu, Nov 9, 2023 at 8:31 AM Rick Byers <rby...@chromium.org<mailto:rby...@chromium.org>> wrote: On Wed, Nov 8, 2023 at 5:56 PM Gregg Tavares <g...@chromium.org<mailto:g...@chromium.org>> wrote: Changing the event order seems like something you'd be opting into since this API has not shipped yet. Don't use the API, get the existing order, Do use the API, get the more useful order. Web compat is almost always more complicated than that. Many real websites are a collection of scripts from a variety of authors and often served by different origins (eg. think of RUM analytics providers). So it would be super weird and confusing if simply using a new API in one script caused DOM event order to change for all other scripts on the page (and what about 3p iframes?). I'd expect this to cause adoption-blocking issues for EditContext since anyone starting to use the API couldn't reason about how doing so might break unrelated scripts on the same page. More pragmatic, I think, would be a document policy a page can opt into which changes the event order, not coupled to any other API. That could potentially be a good thing to do independently. I'm skeptical this API actually fixes the real issues. The examples are incomplete and therefore unconvincing (to me) Maybe I can find a way into the docs trial and test. If I find issues will that change shipping vs not shipping? If it's going to ship regardless of what I find then there's no reason for me to check. Given that this feature is designed for major platforms like MS Office, Google Docs and Adobe tools, I think it's their feedback which I'd want to most rely on for a shipping decision. I certainly don't trust my own understanding of this API over that of experts working on products like that. Dan, do you have public OT feedback somewhere? The OT feedback link in the e-mail is just to the repo<https://github.com/w3c/edit-context/>. Or maybe we could take this off list and reach out to one of our partners to ask specifically how they feel about this concern? Regardless, we should be tracking the debate in detail on a github issue, and coming back to blink-dev with a summary when it's concluded. On Thu, Nov 9, 2023 at 5:15 AM Daniel Bratell <bratel...@gmail.com<mailto:bratel...@gmail.com>> wrote: LGTM4 (yes, a bonus LGTM since I felt ninja'd by Mike and I think this will be a nice addition to the web platform) /Daniel On 2023-11-08 20:42, Mike Taylor wrote: LGTM3 On 11/8/23 12:13 PM, Alex Russell wrote: +1 to the evidence from OT being persuasive. LGTM2 On Wednesday, November 8, 2023 at 8:56:24 AM UTC-8 Rick Byers wrote: It certainly sounds to me that there's additional work to be done here, but I agree with Daniel that changing event order is not something we can do lightly. Gregg, could you please file an issue<https://github.com/w3c/edit-context/issues> on the spec so the discussion can continue there? Given the positive feedback we have from major real-world deployments in OT and the 3+ years of design work that's gone into this API via the editing WG, I personally don't think this concern is sufficient to block shipping at this stage. Editing is going to continue to be an area to work on improving rationality and interop and shipping this API now seems like the right pragmatic next step to me. Since fundamentally fixing this issue is going to need breaking changes anyway, and EditContext is most likely to be eagerly adopted just by a few large applications, I'm reasonably confident that we can continue to improve over time as driven by real-world deployment experience. LGTM1 Rick On Wed, Nov 1, 2023 at 2:55 PM 'Daniel Clark' via blink-dev <blink-dev@chromium.org<mailto:blink-dev@chromium.org>> wrote: To the extent that firing keydown before compositionstart is a problem for apps, that’s an issue that equally affects contenteditable. Changing the order may be a good improvement, but it will be a breaking change for the web that should be carefully considered and hopefully done in tandem with Firefox and Webkit so that browsers can arrive at matching behavior. I think tying EditContext to this is not necessary, as EditContext aims to solve a variety of problems around receiving text input aside from how keydown specifically is handled. Since EditContext does not make any changes around the order of keydown relative to other events, shipping EditContext should not add any additional complications to changing that event order if the Editing Working group decides to go that route. The API is supported on Mac as well. If the sample you tried didn’t work it may be because we failed to update it in response to breaking changes. This sample is up-to-date and should serve to show simple text input and composition: https://magic-organic-yam.glitch.me/. Note that a lot of basic editing functionality isn’t implemented here, and the text formatting for compositions is with highlights rather than underlines -- this was built for testing the API, not real-world editing. As part of the Origin Trial, developers at Google Docs and Adobe included Mac in their testing and reported several Mac-specific issues that we’ve since fixed. If you’d like to try with the Docs implementation, let me know and I can ask about getting you opted in to that rollout. For reconversions, when a page is using EditContext the same UI will still be available to the user via the menu or hotkeys. The page keeps the platform informed about which text is selected by the user by calling EditContext.updateSelection()<https://w3c.github.io/edit-context/#dom-editcontext-updateselection>. When the user triggers a reconversion, Blink will send the page another TextUpdateEvent<https://w3c.github.io/edit-context/#dom-textupdateevent> indicating to it that it should change the text in the specified range to the new value. -- Dan From: Gregg Tavares <g...@chromium.org<mailto:g...@chromium.org>> Sent: Tuesday, October 31, 2023 5:19 PM To: Daniel Clark <dan...@microsoft.com<mailto:dan...@microsoft.com>> Cc: Gregg Tavares <g...@chromium.org<mailto:g...@chromium.org>>; blink-dev <blink-dev@chromium.org<mailto:blink-dev@chromium.org>>; Alex Keng <shih...@microsoft.com<mailto:shih...@microsoft.com>>; Anupam Snigdha <sni...@microsoft.com<mailto:sni...@microsoft.com>>; ko...@chromium.org<mailto:ko...@chromium.org> Subject: Re: [blink-dev] Intent to Ship: EditContext API You don't often get email from g...@chromium.org<mailto:g...@chromium.org>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> I don't think the keydown issue should be ignored until after shipping this API. I think it's essential for this API to actually function. Consider an app that responds to Control-Shift-J for say "jump to definition". keydown = key = 'Control' keydown = key = 'Shift' (CtrlKey = true) keydown = key = 'K' (CtrlKey = true, Shift = true) And the app would trigger it's jump-to-definition command But what should have happened is the IME switched to Japanese mode (on Mac) and nothing happened in the app. It seems like this API is supposed to solve these issues but if keydown happens first then it seems like it fails at it's actual goal. The same issue but for different keys exists because of the difference in Firefox vs Chrome where in Chrome the app will mistakenly respond to keys it shouldn't and in Firefox it won't since Firefox hides those keys as 'Process'. Other questions: (*) Are we sure this API matches all platforms? It appears to be only implemented on Windows so far. The concern being, without implementing on other platforms before shipping, we may run into design issues that require changes to the API. I'm not remotely an expert in IME APIs but I know in my own domain, GPUs, if we didn't actually implement across APIs we'd have that issue. I wanted to test some things but I don't currently have access to a windows device and the sample didn't seem to work on my Mac with the EditContext API enabled. (*) How is recoversion handled in the canvas example? In a normal text editing area the user can select a portion of the text and pick "reconvert" either from menus or via hotkeys On Wed, Nov 1, 2023 at 8:09 AM Daniel Clark <dan...@microsoft.com<mailto:dan...@microsoft.com>> wrote: EditContext is not meant to be an interchangeable replacement for <input type=”text”>, contenteditable, etc, and most sites that want to receive simple text input will want to continue using the existing set of editing features. The target user of EditContext is one that has already reimplemented a lot of the editing stack, such that the browser’s built-in editing functionality is more of a hindrance than a help – the typical case here is something like Google Docs (where the entire editor view is reimplemented in a <canvas>). EditContext replaces hacks that sites like these often have to resort to such as hidden contenteditable elements that are floated around the page to position the IME window. A site that just wants to receive text input without building out their own fully-featured editing experience can and should continue using the existing “batteries-included” tools like <textarea> or contenteditable. The keydown event coming before compositionstart seems to be consistent with the existing contenteditable behavior in both Chromium and Firefox. While EditContext changes how some editing-related events are fired, some of the existing orderings like this were left in place for consistency’s sake when there wasn’t a strong reason to change them. The keydownevent.key interop difference is a good one to note, but I think it should be resolved orthogonally to EditContext. Since that behavior difference is present for both EditContext and contenteditable, the ideal outcome would be to bring this behavior in line across browsers for all editable fields. It looks like there are some stale issues in the EditingWG in that area, e.g. this one<https://github.com/w3c/uievents/issues/75> from before Gecko started firing keydown/keyup events during composition; maybe this should be taken back up by the WG to try to drive further interoperability in the area. If we end up making a change there it would apply both to EditContext and to other types of editable fields. -- Dan From: Gregg Tavares <g...@chromium.org<mailto:g...@chromium.org>> Sent: Monday, October 30, 2023 10:19 PM To: Daniel Clark <dan...@microsoft.com<mailto:dan...@microsoft.com>> Cc: blink-dev <blink-dev@chromium.org<mailto:blink-dev@chromium.org>>; Alex Keng <shih...@microsoft.com<mailto:shih...@microsoft.com>>; Anupam Snigdha <sni...@microsoft.com<mailto:sni...@microsoft.com>>; ko...@chromium.org<mailto:ko...@chromium.org> Subject: Re: [blink-dev] Intent to Ship: EditContext API You don't often get email from g...@chromium.org<mailto:g...@chromium.org>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Not a decider but one that sees the IME on many sites that try to roll their own text input. This sounds like a "if you do all of these 30 things perfectly, then maybe your site will work with most IME issues but you won't know unless you get someone experienced with IME users to test for you" solution Vs. some other solution which is "do nothing and it just works". The current "do nothing and it just works" is, use <input type="text"> or <textarea> or contenteditable. Is this API just giving developers lots of rope to hang themselves? Also, how does it align with other browsers? For example the explainer shows a sequence of events Event EventTarget key code event.text keydown focused element 'S' compositionstart active EditContext textupdate active EditContext 'S' textformatupdate active EditContext keyup focused element 'S' keydown focused element 'U' textupdate active EditContext 'す' textformatupdate active EditContext keyup focused element 'U' keydown focused element 'Space' textupdate active EditContext '巣' textformatupdate active EditContext compositionend active EditContext keyup focused element 'Space' That seems non-intuitive to me. I get a keydown first, at which point my app reacts when it wasn't supposed to as the key was meant for the IME. Also, this table doesn't seem to match Firefox for example, pressing 's' when in Japanese input mode pops up the IME and in firefox it produces event.key = 'Process', not event.key = 's' which at least makes more sense since the input should be going to the IME, not the page. On Tue, Oct 31, 2023 at 2:52 AM 'Daniel Clark' via blink-dev <blink-dev@chromium.org<mailto:blink-dev@chromium.org>> wrote: Contact emails dan...@microsoft.com<mailto:dan...@microsoft.com>, sni...@microsoft.com<mailto:sni...@microsoft.com>, shih...@microsoft.com<mailto:shih...@microsoft.com> Explainer https://github.com/w3c/edit-context/blob/gh-pages/explainer.md Specification https://w3c.github.io/edit-context Design docs https://github.com/w3c/edit-context/blob/gh-pages/dev-design.md Summary The EditContext API simplifies the process of integrating a web app with advanced text input methods such as IME Compositions and speech recognition, and unlocks new capabilities for web-based editors. Blink component Blink>Editing<https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EEditing> Search tags editing<https://chromestatus.com/features#tags:editing>, contenteditable<https://chromestatus.com/features#tags:contenteditable>, input<https://chromestatus.com/features#tags:input>, rawinput<https://chromestatus.com/features#tags:rawinput>, ime<https://chromestatus.com/features#tags:ime> TAG review Completed (Resolution: satisifed) at https://github.com/w3ctag/design-reviews/issues/416 TAG review status Issues addressed Chromium Trial Name EditContext Link to origin trial feedback summary https://github.com/w3c/edit-context/ Origin Trial documentation link https://github.com/w3c/edit-context/blob/gh-pages/explainer.md In the Origin Trial the Google Docs team used EditContext to receive IME input and position the IME window for Docs, replacing the current approach of manually positioning a hidden contenteditable element over the document when composing text. The new EditContext approach is more performant and supports a wider range of IME interactions. We received similar feedback from Adobe, who are also using EditContext to replace a hidden text input element for triggering the IME. Risks Interoperability and Compatibility There are no known interop or compat risks. Gecko: Under consideration (https://github.com/mozilla/standards-positions/issues/199) WebKit: No signal (https://github.com/WebKit/standards-positions/issues/243) Web developers: Strongly positive Positive feedback from Word online, Adobe and Figma, Google Docs Other signals: Ergonomics None. Activation Developers interested in this feature will typically have their own polyfill for text input using hidden textarea or contenteditable elements. Feature detecting and using new API to avoid side effects of previous approaches is intended to be easily adoptable. Security No particular security risks. See https://github.com/w3c/edit-context/blob/gh-pages/security-privacy.md. WebView application risks Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications? None. Debuggability Existing DevTools features should be sufficient for debugging EditContext. Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? Yes. This is a core web platform feature that is not limited to any particular underlying platform. Is this feature fully tested by web-platform-tests<https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md>? Yes. Tests are available at https://wpt.fyi/results/editing/edit-context?label=experimental&label=master&aligned Note that some composition scenarios are not yet testable in WPT due to a dependency on content_shell-only test APIs. Work is underway to add functionality for mocking IME input in WPTs such that these tests can be moved to WPT. Flag name on chrome://flags edit-context Finch feature name EditContext Requires code in //chrome? False Tracking bug https://bugs.chromium.org/p/chromium/issues/detail?id=999184 Measurement The UseCounter WebFeature::kEditContext tracks instantiation of EditContext. Availability expectation We expect other browser vendors to be interested in implementing this feature, though we cannot comment on specific timelines. Adoption expectation Feature will be used by Google Docs upon launch in Chrome. Adoption plan We are already working with the Docs team as a partner in the feature's Origin Trial, where they have implemented composition using EditContext. Non-OSS dependencies Does the feature depend on any code or APIs outside the Chromium open source repository and its open-source dependencies to function? None. Estimated milestones Shipping on desktop 121 OriginTrial desktop last 120 OriginTrial desktop first 116 OriginTrial Android last 120 OriginTrial Android first 116 OriginTrial webView last 120 OriginTrial webView first 116 Anticipated spec changes Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way). Open spec issues can be found here: https://github.com/w3c/edit-context/issues We expect these issues to be resolved in a forward-compatible way and/or to only affect rare corner-cases. Many of these discuss potential additions to the feature that will be considered based on ongoing developer feedback as EditContext is adopted more widely. Link to entry on the Chrome Platform Status https://chromestatus.com/feature/5041440373604352 Links to previous Intent discussions Intent to Implement: https://groups.google.com/u/1/a/chromium.org/g/blink-dev/c/OHqvPx9mFww/m/1za_qdEHDwAJ Intent to Experiment: https://groups.google.com/u/1/a/chromium.org/g/blink-dev/c/QZQrESwcK3o/m/k3pfYBcRBAAJ -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org<mailto:blink-dev+unsubscr...@chromium.org>. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/MW4PR00MB1654118011BA087BA5058967C5A1A%40MW4PR00MB1654.namprd00.prod.outlook.com<https://groups.google.com/a/chromium.org/d/msgid/blink-dev/MW4PR00MB1654118011BA087BA5058967C5A1A%40MW4PR00MB1654.namprd00.prod.outlook.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org<mailto:blink-dev+unsubscr...@chromium.org>. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/PH7PR00MB1637386167B3A830D5D06D0AC5A7A%40PH7PR00MB1637.namprd00.prod.outlook.com<https://groups.google.com/a/chromium.org/d/msgid/blink-dev/PH7PR00MB1637386167B3A830D5D06D0AC5A7A%40PH7PR00MB1637.namprd00.prod.outlook.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org<mailto:blink-dev+unsubscr...@chromium.org>. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6707471d-651b-482d-9047-a5fb0f493680n%40chromium.org<https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6707471d-651b-482d-9047-a5fb0f493680n%40chromium.org?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org<mailto:blink-dev+unsubscr...@chromium.org>. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/4c6eed01-7341-41d9-87ee-fae9876a4752%40chromium.org<https://groups.google.com/a/chromium.org/d/msgid/blink-dev/4c6eed01-7341-41d9-87ee-fae9876a4752%40chromium.org?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org<mailto:blink-dev+unsubscr...@chromium.org>. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY_pZEbTFxFCSMO_zUJ8%2B3mrGT5zH%2B-wg1kNF-VjBS5BHA%40mail.gmail.com<https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY_pZEbTFxFCSMO_zUJ8%2B3mrGT5zH%2B-wg1kNF-VjBS5BHA%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/SA1PR00MB16441CCE6EA0FC6BA3213AE1C5AFA%40SA1PR00MB1644.namprd00.prod.outlook.com.
