Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 62987e20ab97418bd379d1fddeab9a181c064e3c
https://github.com/WebKit/WebKit/commit/62987e20ab97418bd379d1fddeab9a181c064e3c
Author: Jean-Yves Avenard <[email protected]>
Date: 2026-05-06 (Wed, 06 May 2026)
Changed paths:
M Source/WebCore/platform/graphics/TrackBuffer.cpp
M Source/WebCore/platform/graphics/TrackBuffer.h
M Source/WebCore/platform/graphics/avfoundation/AudioVideoRendererAVFObjC.mm
M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h
M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm
Log Message:
-----------
[MSE][Cocoa] Silence spurious UpcomingPTSExpectation warnings from
AVSampleBufferVideoRenderer on B-frame content
https://bugs.webkit.org/show_bug.cgi?id=314077
rdar://176260662
Reviewed by Jer Noble.
AVSampleBufferVideoRenderer's _enqueueSingleSampleBuffer: compares each
incoming sample's PTS against a floor that WebKit publishes via
-[AVSampleBufferDisplayLayer expectMinimumUpcomingSampleBufferPresentationTime:]
and logs "UpcomingPTSExpectation is enabled, but enqueuePTS:X is smaller
than expectedMinimumUpcomingPTS:Y" whenever a later sample arrives with
a lower PTS. The log is informational only — samples are still accepted
— but on B-frame streams the message fires every frame pair: a 5-minute
Twitch capture produced 6197 lines, burying other signal during stall
investigations.
The floor came from TrackBuffer::minimumEnqueuedPresentationTime(),
recomputed by TrackBuffer::updateMinimumUpcomingPresentationTime() as a
sliding-window minimum over m_decodeQueue. When the parser feeds samples
one at a time (typical Twitch behaviour, and the common case during
post-changeType recovery) the decode queue is near-empty after each pop
and the scan can only see the current sample's own PTS. In decode order
that's a P-frame at the top of a GOP triplet, with two B-frames at lower
PTS arriving next — and the floor we publish is therefore always too
high by the B-frame reorder distance.
Two coordinated changes:
1. TrackBuffer now tracks the max observed reorder depth. On every
addSample() we count samples in decode order since the most recent
new-max PTS; the running maximum is stored in
m_maxObservedReorderDepth, seeded to a conservative default of 3 to
cover typical IPBB/IPBBB patterns and growing on deeper reorder.
updateMinimumUpcomingPresentationTime() refuses to publish a value
until the queue contains at least m_maxObservedReorderDepth + 1
samples past the head when m_hasOutOfOrderFrames is set — that's the
point at which the sliding-window scan has actually seen every
B-frame that could come before the head's PTS. A new
setInitialReorderDepth(size_t) hook is exposed so a future change
can seed the depth from the codec config (H.264 max_num_reorder_frames
/ H.265 sps_max_num_reorder_pics); today it is unused and the
mechanism is purely empirical.
2. Propagate the "no lookahead available" state cleanly down to the
renderer instead of masking it. AudioVideoRendererAVFObjC::enqueueSample()
no longer falls back to sample->presentationTime() when the caller
passes std::nullopt; VideoMediaSampleRenderer::enqueueSample() now
takes std::optional<MediaTime> and stores the optional in
m_compressedSampleQueue. decodeNextSampleIfNeeded() only calls
-expectMinimumUpcomingSampleBufferPresentationTime: when the stored
optional holds a value. Without a lookahead it falls through and
keeps decoding, which was the previous unintended behaviour anyway
(the floor we were publishing was wrong).
Net effect: zero warnings in steady-state Twitch playback, and none
during ad/content transitions once the first handful of samples have
established the reorder depth.
No tests added; manual verification via
log stream --predicate='(process="Safari" or
process="com.apple.WebKit.GPU.Development")'
with a Twitch session shows the "UpcomingPTSExpectation is enabled"
message count drop from ~6000 / 5 min to near zero.
* Source/WebCore/platform/graphics/TrackBuffer.h:
(WebCore::TrackBuffer::setInitialReorderDepth):
Add setInitialReorderDepth() and the three new members backing the
reorder-depth observation.
* Source/WebCore/platform/graphics/TrackBuffer.cpp:
(WebCore::TrackBuffer::addSample):
Track the running reorder depth alongside m_hasOutOfOrderFrames.
(WebCore::TrackBuffer::updateMinimumUpcomingPresentationTime):
Gate publication on having at least m_maxObservedReorderDepth + 1
samples in the queue when m_hasOutOfOrderFrames is set.
(WebCore::TrackBuffer::clearDecodeQueue):
Reset the observation counters but keep the learned depth.
* Source/WebCore/platform/graphics/avfoundation/AudioVideoRendererAVFObjC.mm:
(WebCore::AudioVideoRendererAVFObjC::enqueueSample):
Pass the std::optional<MediaTime> through instead of substituting the
sample's own PTS.
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h:
Change enqueueSample() signature and m_compressedSampleQueue element
type to carry std::optional<MediaTime>.
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm:
(WebCore::VideoMediaSampleRenderer::enqueueSample):
Accept std::optional<MediaTime>.
(WebCore::VideoMediaSampleRenderer::decodeNextSampleIfNeeded):
Only publish a floor when the stored optional is present; fall through
and decode otherwise.
Canonical link: https://commits.webkit.org/312763@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
