On Tue, Mar 25, 2025 at 12:10 PM Chromestatus < ad...@cr-status.appspotmail.com> wrote:
> Contact emails dalecur...@chromium.org > > Explainer None > > Specification https://www.w3.org/TR/webcodecs > > Summary > > Introduces "rotation: int" and "flip: bool" values to various video > related interfaces in WebCodecs so that developers can work with frame > sources that have orientation (E.g., Android cameras, certain media). The > VideoFrame interface grows the ability to create VideoFrames with arbitrary > rotation and flip as well as accessors for this information on the > VideoFrame object. The VideoDecoderConfig object gains rotation and flip > fields that are emitted on decoded VideoFrame objects automatically. The > VideoEncoder class gains mechanisms for passing rotation and flip > information from encode() to the VideoDecoderConfig emitted as part of > EncodedVideoChunkMetadata. If encode() is called with frames with different > orientations a non-fatal exception will be thrown. configure() may be used > to reset the allowed orientation. > Back in the day I implemented the CSS image-orientation property, which is conceptually addressing the same kind of thing in the image case. There were privacy concerns about exposing image metadata content through the orientation property, leading to the requirement that cross-origin content always render using the image's EXIF orientation. This prevents pages from applying the image-orientation property, detecting a change in rendering, and hence inferring the metadata. As I understand the spec link and this intent, it seems like existing behavior is to always use the encoded video orientation, and this feature allows sites to modify the orientation. If that's so, could it expose information about users' cameras via video length/width, thus leading to additional fingerprinting surface? Did this come up in the spec discussions? Stephen. > > > Blink component Blink>Media>WebCodecs > <https://issues.chromium.org/issues?q=customfield1222907:%22Blink%3EMedia%3EWebCodecs%22> > > Search tags webcodecs <http:///features#tags:webcodecs>, videoframe > <http:///features#tags:videoframe>, orientation > <http:///features#tags:orientation>, videodecoder > <http:///features#tags:videodecoder>, videoencoder > <http:///features#tags:videoencoder> > > TAG review None > > TAG review status Not applicable > > Risks > > > Interoperability and Compatibility > > Low, feature designed in conjunction with Firefox and the Media WG. > > > *Gecko*: Positive (https://bugzilla.mozilla.org/show_bug.cgi?id=1955904) > Spec co-editor, approved all spec changes, implementation planned. > > *WebKit*: No signal ( > https://github.com/WebKit/standards-positions/issues/475) Participated in > some of the standards discussions around this feature. > > *Web developers*: Positive (https://github.com/w3c/webcodecs/issues/351) > Active requests from Zoom and others developers (see issue) who are working > with cameras with orientation metadata. > > *Other signals*: > > Ergonomics > > No risk. Small incremental change to WebCodecs API. > > > Activation > > No risk. Small incremental change to WebCodecs API. MDN documentation will > be updated. > > > Security > > No risk. No new control surfaces. Video elements already have this > functionality. > > > WebView application risks > > Does this intent deprecate or change behavior of existing APIs, such that > it has potentially high risk for Android WebView-based applications? > > Some cameras and media will immediately begin exposing orientation > information. In uncommon cases this orientation may change. If orientation > changes during encoding through VideoEncoder frames with different > orientation than the first are currently encoded with incorrect size and > orientation. However the operation completes without a hard error. During > discussions in the media working group it was decided that these cases > should instead throw a non-fatal exception during encode(). There is > already precedent for exception handling during encode() so authors should > already be handling this case. > > > Debuggability > > Rotation information shows up in media configs emitted in media dev tools. > > > Will this feature be supported on all six Blink platforms (Windows, Mac, > Linux, ChromeOS, Android, and Android WebView)? Yes > > Mostly just exposes existing under the hood information used by video > elements, so is already supported everywhere. > > > Is this feature fully tested by web-platform-tests > <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> > ? Yes > > > https://wpt.fyi/results/webcodecs?label=master&label=experimental&aligned&q=orientation > > > Flag name on about://flags WebCodecsOrientation > > Finch feature name WebCodecsOrientation > > Requires code in //chrome? False > > Tracking bug https://crbug.com/40243431 > > Measurement WebCodecs has use counters, but no specific use counter has > been added for the rotation and flip usage. > > Availability expectation Available on all platforms except for iOS. > > Adoption expectation Should be adopted by sites consuming camera feeds > with WebCodecs. > > Estimated milestones > Shipping on desktop 137 > DevTrial on desktop 136 > Shipping on Android 137 > DevTrial on Android 136 > Shipping on WebView 137 > > Anticipated spec changes > > Open questions about a feature may be a source of future web compat or > interop issues. Please list open issues (e.g. links to known github issues > in the project for the feature specification) whose resolution may > introduce web compat/interop risk (e.g., changing to naming or structure of > the API in a non-backward-compatible way). > Spec changes have already been landed after review with Firefox and the > Media Working Group: * https://github.com/w3c/webcodecs/pull/884 * > https://github.com/w3c/webcodecs/pull/874 * > https://github.com/w3c/webcodecs/pull/840 * > https://github.com/w3c/webcodecs/pull/559 > > Link to entry on the Chrome Platform Status > https://chromestatus.com/feature/5128855343595520?gate=5187346531876864 > > This intent message was generated by Chrome Platform Status > <https://chromestatus.com>. > > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+unsubscr...@chromium.org. > To view this discussion visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/67e2d562.170a0220.e1a1e.07a6.GAE%40google.com > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/67e2d562.170a0220.e1a1e.07a6.GAE%40google.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGsbWzQU9%3D2J0s8fN%3DoFOzmvoH-uMk5coAM%3DpfH2hPBhtTnGwA%40mail.gmail.com.
