- Revision
- 258977
- Author
- you...@apple.com
- Date
- 2020-03-25 06:44:17 -0700 (Wed, 25 Mar 2020)
Log Message
Audio fails to capture stream in WebRTC if AudioSession gets interrupted
https://bugs.webkit.org/show_bug.cgi?id=208516
<rdar://problem/60020467>
Reviewed by Eric Carlson.
In case of page going to hidden, continue calling each capture factory to mute the corresponding sources if needed.
In case of page being visible again, reset all tracks according page muted state. This allows restarting tracks that have been
muted while page was hidden (video tracks or suspended audio tracks).
Since tracks can go to muted when visibility changes, we no longer return early when setting the muted state of a page to the same value.
Instead we apply it which ensures we comply with what UIProcess wants.
We start removing the concept of a RealtimeMediaSource be interrupted. Instead we use muting of sources.
This allows UIProcess or the page to override any muted state, for instance if page goes in foreground again.
We update the AudioSharedUnit to allow restarting capture even if suspended.
This ensures that we are able to restart capturing even if we do not receive the audio session end of interruption.
Also, this notification sometimes takes a long time to happen and we do not want to wait for it when user is interacting with the page.
A future refactoring will further remove RealtimeMediaSource interrupted-related code.
Manually tested.
* dom/Document.cpp:
(WebCore::Document::visibilityStateChanged):
* page/Page.cpp:
(WebCore::Page::setMuted):
* platform/audio/PlatformMediaSessionManager.h:
(WebCore::PlatformMediaSessionManager::isInterrupted const):
* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::setInterrupted):
(WebCore::RealtimeMediaSource::setMuted):
* platform/mediastream/mac/BaseAudioSharedUnit.cpp:
(WebCore::BaseAudioSharedUnit::startProducingData):
(WebCore::BaseAudioSharedUnit::resume):
(WebCore::BaseAudioSharedUnit::suspend):
Modified Paths
Diff
Modified: trunk/Source/WebCore/ChangeLog (258976 => 258977)
--- trunk/Source/WebCore/ChangeLog 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/ChangeLog 2020-03-25 13:44:17 UTC (rev 258977)
@@ -1,3 +1,42 @@
+2020-03-25 Youenn Fablet <you...@apple.com>
+
+ Audio fails to capture stream in WebRTC if AudioSession gets interrupted
+ https://bugs.webkit.org/show_bug.cgi?id=208516
+ <rdar://problem/60020467>
+
+ Reviewed by Eric Carlson.
+
+ In case of page going to hidden, continue calling each capture factory to mute the corresponding sources if needed.
+ In case of page being visible again, reset all tracks according page muted state. This allows restarting tracks that have been
+ muted while page was hidden (video tracks or suspended audio tracks).
+
+ Since tracks can go to muted when visibility changes, we no longer return early when setting the muted state of a page to the same value.
+ Instead we apply it which ensures we comply with what UIProcess wants.
+
+ We start removing the concept of a RealtimeMediaSource be interrupted. Instead we use muting of sources.
+ This allows UIProcess or the page to override any muted state, for instance if page goes in foreground again.
+
+ We update the AudioSharedUnit to allow restarting capture even if suspended.
+ This ensures that we are able to restart capturing even if we do not receive the audio session end of interruption.
+ Also, this notification sometimes takes a long time to happen and we do not want to wait for it when user is interacting with the page.
+ A future refactoring will further remove RealtimeMediaSource interrupted-related code.
+
+ Manually tested.
+
+ * dom/Document.cpp:
+ (WebCore::Document::visibilityStateChanged):
+ * page/Page.cpp:
+ (WebCore::Page::setMuted):
+ * platform/audio/PlatformMediaSessionManager.h:
+ (WebCore::PlatformMediaSessionManager::isInterrupted const):
+ * platform/mediastream/RealtimeMediaSource.cpp:
+ (WebCore::RealtimeMediaSource::setInterrupted):
+ (WebCore::RealtimeMediaSource::setMuted):
+ * platform/mediastream/mac/BaseAudioSharedUnit.cpp:
+ (WebCore::BaseAudioSharedUnit::startProducingData):
+ (WebCore::BaseAudioSharedUnit::resume):
+ (WebCore::BaseAudioSharedUnit::suspend):
+
2020-03-25 Charlie Turner <ctur...@igalia.com>
[GStreamer] Remove noisy warning about broken LC support
Modified: trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp (258976 => 258977)
--- trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp 2020-03-25 13:44:17 UTC (rev 258977)
@@ -467,13 +467,43 @@
return state;
}
+#if PLATFORM(IOS_FAMILY)
+static MediaStreamTrack* findActiveCaptureTrackForDocument(Document& document, RealtimeMediaSource* activeSource, RealtimeMediaSource::Type type)
+{
+ MediaStreamTrack* selectedTrack = nullptr;
+ for (auto* captureTrack : allCaptureTracks()) {
+ if (captureTrack->document() != &document || captureTrack->ended())
+ continue;
+
+ if (&captureTrack->source() == activeSource)
+ return captureTrack;
+
+ // If the document has a live capture track, which is not the active one, we pick the first one.
+ // FIXME: We should probably store per page active audio/video capture tracks.
+ if (!selectedTrack && captureTrack->privateTrack().type() == type)
+ selectedTrack = captureTrack;
+ }
+ return selectedTrack;
+}
+#endif
+
void MediaStreamTrack::updateCaptureAccordingToMutedState(Document& document)
{
+#if PLATFORM(IOS_FAMILY)
+ auto* activeAudioSource = RealtimeMediaSourceCenter::singleton().audioCaptureFactory().activeSource();
+ if (auto* audioCaptureTrack = findActiveCaptureTrackForDocument(document, activeAudioSource, RealtimeMediaSource::Type::Audio))
+ audioCaptureTrack->setMuted(document.page()->mutedState());
+
+ auto* activeVideoSource = RealtimeMediaSourceCenter::singleton().videoCaptureFactory().activeSource();
+ if (auto* videoCaptureTrack = findActiveCaptureTrackForDocument(document, activeVideoSource, RealtimeMediaSource::Type::Video))
+ videoCaptureTrack->setMuted(document.page()->mutedState());
+#else
for (auto* captureTrack : allCaptureTracks()) {
if (captureTrack->document() != &document || captureTrack->ended())
continue;
captureTrack->setMuted(document.page()->mutedState());
}
+#endif
}
void MediaStreamTrack::endCapture(Document& document)
Modified: trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h (258976 => 258977)
--- trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h 2020-03-25 13:44:17 UTC (rev 258977)
@@ -154,6 +154,8 @@
void setIdForTesting(String&& id) { m_private->setIdForTesting(WTFMove(id)); }
+ Document* document() const;
+
#if !RELEASE_LOG_DISABLED
const Logger& logger() const final { return m_private->logger(); }
const void* logIdentifier() const final { return m_private->logIdentifier(); }
@@ -171,8 +173,6 @@
void configureTrackRendering();
- Document* document() const;
-
// ActiveDOMObject API.
void stop() final { stopTrack(); }
const char* activeDOMObjectName() const override;
Modified: trunk/Source/WebCore/dom/Document.cpp (258976 => 258977)
--- trunk/Source/WebCore/dom/Document.cpp 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/dom/Document.cpp 2020-03-25 13:44:17 UTC (rev 258977)
@@ -1744,9 +1744,13 @@
for (auto* client : m_visibilityStateCallbackClients)
client->visibilityStateChanged();
-#if ENABLE(MEDIA_STREAM)
- if (auto* page = this->page())
- RealtimeMediaSourceCenter::singleton().setCapturePageState(hidden(), page->isMediaCaptureMuted());
+#if PLATFORM(IOS_FAMILY)
+ if (hidden()) {
+ RealtimeMediaSourceCenter::singleton().setCapturePageState(hidden(), page()->isMediaCaptureMuted());
+ return;
+ }
+ if (!PlatformMediaSessionManager::sharedManager().isInterrupted())
+ MediaStreamTrack::updateCaptureAccordingToMutedState(*this);
#endif
}
Modified: trunk/Source/WebCore/page/Page.cpp (258976 => 258977)
--- trunk/Source/WebCore/page/Page.cpp 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/page/Page.cpp 2020-03-25 13:44:17 UTC (rev 258977)
@@ -1781,9 +1781,6 @@
void Page::setMuted(MediaProducer::MutedStateFlags muted)
{
- if (m_mutedState == muted)
- return;
-
m_mutedState = muted;
forEachDocument([] (Document& document) {
Modified: trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h (258976 => 258977)
--- trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h 2020-03-25 13:44:17 UTC (rev 258977)
@@ -140,6 +140,8 @@
WEBCORE_EXPORT void processDidReceiveRemoteControlCommand(PlatformMediaSession::RemoteControlCommandType, const PlatformMediaSession::RemoteCommandArgument*);
+ bool isInterrupted() const { return m_interrupted; }
+
protected:
friend class PlatformMediaSession;
PlatformMediaSessionManager();
Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp (258976 => 258977)
--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp 2020-03-25 13:44:17 UTC (rev 258977)
@@ -76,25 +76,12 @@
void RealtimeMediaSource::setInterrupted(bool interrupted, bool pageMuted)
{
- if (interrupted == m_interrupted)
- return;
-
ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, interrupted, ", page muted : ", pageMuted);
-
- m_interrupted = interrupted;
- if (!interrupted && pageMuted)
- return;
-
setMuted(interrupted);
}
void RealtimeMediaSource::setMuted(bool muted)
{
- if (!muted && interrupted()) {
- ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, "ignoring unmute because of interruption");
- return;
- }
-
ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, muted);
// Changed m_muted before calling start/stop so muted() will reflect the correct state.
Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h (258976 => 258977)
--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h 2020-03-25 13:44:17 UTC (rev 258977)
@@ -45,7 +45,6 @@
virtual void setActiveSource(RealtimeMediaSource&);
void unsetActiveSource(RealtimeMediaSource&);
-protected:
RealtimeMediaSource* activeSource() { return m_activeSource; }
private:
Modified: trunk/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp (258976 => 258977)
--- trunk/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp 2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp 2020-03-25 13:44:17 UTC (rev 258977)
@@ -74,6 +74,9 @@
{
ASSERT(isMainThread());
+ if (m_suspended)
+ resume();
+
if (++m_producingCount != 1)
return;
@@ -80,11 +83,6 @@
if (isProducingData())
return;
- if (m_suspended) {
- RELEASE_LOG_INFO(WebRTC, "BaseAudioSharedUnit::startProducingData - exiting early as suspended");
- return;
- }
-
if (hasAudioUnit()) {
cleanupAudioUnit();
ASSERT(!hasAudioUnit());
@@ -159,7 +157,9 @@
OSStatus BaseAudioSharedUnit::resume()
{
ASSERT(isMainThread());
- ASSERT(m_suspended);
+ if (!m_suspended)
+ return 0;
+
ASSERT(!isProducingData());
RELEASE_LOG_INFO(WebRTC, "BaseAudioSharedUnit::resume");
@@ -180,7 +180,7 @@
}
forEachClient([](auto& client) {
- client.notifyMutedChange(false);
+ client.setMuted(false);
});
return 0;
@@ -196,7 +196,7 @@
stopInternal();
forEachClient([](auto& client) {
- client.notifyMutedChange(true);
+ client.setMuted(true);
});
return 0;
