[webkit-changes] [258977] trunk/Source/WebCore

Wed, 25 Mar 2020 06:45:22 -0700

Title: [258977] trunk/Source/WebCore
Revision
258977
Author
you...@apple.com
Date
2020-03-25 06:44:17 -0700 (Wed, 25 Mar 2020)

Log Message

Audio fails to capture stream in WebRTC if AudioSession gets interrupted
https://bugs.webkit.org/show_bug.cgi?id=208516
<rdar://problem/60020467>

Reviewed by Eric Carlson.

In case of page going to hidden, continue calling each capture factory to mute the corresponding sources if needed.
In case of page being visible again, reset all tracks according page muted state. This allows restarting tracks that have been
muted while page was hidden (video tracks or suspended audio tracks).

Since tracks can go to muted when visibility changes, we no longer return early when setting the muted state of a page to the same value.
Instead we apply it which ensures we comply with what UIProcess wants.

We start removing the concept of a RealtimeMediaSource be interrupted. Instead we use muting of sources.
This allows UIProcess or the page to override any muted state, for instance if page goes in foreground again.

We update the AudioSharedUnit to allow restarting capture even if suspended.
This ensures that we are able to restart capturing even if we do not receive the audio session end of interruption.
Also, this notification sometimes takes a long time to happen and we do not want to wait for it when user is interacting with the page.
A future refactoring will further remove RealtimeMediaSource interrupted-related code.

Manually tested.

* dom/Document.cpp:
(WebCore::Document::visibilityStateChanged):
* page/Page.cpp:
(WebCore::Page::setMuted):
* platform/audio/PlatformMediaSessionManager.h:
(WebCore::PlatformMediaSessionManager::isInterrupted const):
* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::setInterrupted):
(WebCore::RealtimeMediaSource::setMuted):
* platform/mediastream/mac/BaseAudioSharedUnit.cpp:
(WebCore::BaseAudioSharedUnit::startProducingData):
(WebCore::BaseAudioSharedUnit::resume):
(WebCore::BaseAudioSharedUnit::suspend):

Modified Paths

Diff

Modified: trunk/Source/WebCore/ChangeLog (258976 => 258977)


--- trunk/Source/WebCore/ChangeLog	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/ChangeLog	2020-03-25 13:44:17 UTC (rev 258977)
@@ -1,3 +1,42 @@
+2020-03-25  Youenn Fablet  <you...@apple.com>
+
+        Audio fails to capture stream in WebRTC if AudioSession gets interrupted
+        https://bugs.webkit.org/show_bug.cgi?id=208516
+        <rdar://problem/60020467>
+
+        Reviewed by Eric Carlson.
+
+        In case of page going to hidden, continue calling each capture factory to mute the corresponding sources if needed.
+        In case of page being visible again, reset all tracks according page muted state. This allows restarting tracks that have been
+        muted while page was hidden (video tracks or suspended audio tracks).
+
+        Since tracks can go to muted when visibility changes, we no longer return early when setting the muted state of a page to the same value.
+        Instead we apply it which ensures we comply with what UIProcess wants.
+
+        We start removing the concept of a RealtimeMediaSource be interrupted. Instead we use muting of sources.
+        This allows UIProcess or the page to override any muted state, for instance if page goes in foreground again.
+
+        We update the AudioSharedUnit to allow restarting capture even if suspended.
+        This ensures that we are able to restart capturing even if we do not receive the audio session end of interruption.
+        Also, this notification sometimes takes a long time to happen and we do not want to wait for it when user is interacting with the page.
+        A future refactoring will further remove RealtimeMediaSource interrupted-related code.
+
+        Manually tested.
+
+        * dom/Document.cpp:
+        (WebCore::Document::visibilityStateChanged):
+        * page/Page.cpp:
+        (WebCore::Page::setMuted):
+        * platform/audio/PlatformMediaSessionManager.h:
+        (WebCore::PlatformMediaSessionManager::isInterrupted const):
+        * platform/mediastream/RealtimeMediaSource.cpp:
+        (WebCore::RealtimeMediaSource::setInterrupted):
+        (WebCore::RealtimeMediaSource::setMuted):
+        * platform/mediastream/mac/BaseAudioSharedUnit.cpp:
+        (WebCore::BaseAudioSharedUnit::startProducingData):
+        (WebCore::BaseAudioSharedUnit::resume):
+        (WebCore::BaseAudioSharedUnit::suspend):
+
 2020-03-25  Charlie Turner  <ctur...@igalia.com>
 
         [GStreamer] Remove noisy warning about broken LC support

Modified: trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp (258976 => 258977)


--- trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp	2020-03-25 13:44:17 UTC (rev 258977)
@@ -467,13 +467,43 @@
     return state;
 }
 
+#if PLATFORM(IOS_FAMILY)
+static MediaStreamTrack* findActiveCaptureTrackForDocument(Document& document, RealtimeMediaSource* activeSource, RealtimeMediaSource::Type type)
+{
+    MediaStreamTrack* selectedTrack = nullptr;
+    for (auto* captureTrack : allCaptureTracks()) {
+        if (captureTrack->document() != &document || captureTrack->ended())
+            continue;
+
+        if (&captureTrack->source() == activeSource)
+            return captureTrack;
+
+        // If the document has a live capture track, which is not the active one, we pick the first one.
+        // FIXME: We should probably store per page active audio/video capture tracks.
+        if (!selectedTrack && captureTrack->privateTrack().type() == type)
+            selectedTrack = captureTrack;
+    }
+    return selectedTrack;
+}
+#endif
+
 void MediaStreamTrack::updateCaptureAccordingToMutedState(Document& document)
 {
+#if PLATFORM(IOS_FAMILY)
+    auto* activeAudioSource = RealtimeMediaSourceCenter::singleton().audioCaptureFactory().activeSource();
+    if (auto* audioCaptureTrack = findActiveCaptureTrackForDocument(document, activeAudioSource, RealtimeMediaSource::Type::Audio))
+        audioCaptureTrack->setMuted(document.page()->mutedState());
+
+    auto* activeVideoSource = RealtimeMediaSourceCenter::singleton().videoCaptureFactory().activeSource();
+    if (auto* videoCaptureTrack = findActiveCaptureTrackForDocument(document, activeVideoSource, RealtimeMediaSource::Type::Video))
+        videoCaptureTrack->setMuted(document.page()->mutedState());
+#else
     for (auto* captureTrack : allCaptureTracks()) {
         if (captureTrack->document() != &document || captureTrack->ended())
             continue;
         captureTrack->setMuted(document.page()->mutedState());
     }
+#endif
 }
 
 void MediaStreamTrack::endCapture(Document& document)

Modified: trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h (258976 => 258977)


--- trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/Modules/mediastream/MediaStreamTrack.h	2020-03-25 13:44:17 UTC (rev 258977)
@@ -154,6 +154,8 @@
 
     void setIdForTesting(String&& id) { m_private->setIdForTesting(WTFMove(id)); }
 
+    Document* document() const;
+
 #if !RELEASE_LOG_DISABLED
     const Logger& logger() const final { return m_private->logger(); }
     const void* logIdentifier() const final { return m_private->logIdentifier(); }
@@ -171,8 +173,6 @@
 
     void configureTrackRendering();
 
-    Document* document() const;
-
     // ActiveDOMObject API.
     void stop() final { stopTrack(); }
     const char* activeDOMObjectName() const override;

Modified: trunk/Source/WebCore/dom/Document.cpp (258976 => 258977)


--- trunk/Source/WebCore/dom/Document.cpp	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/dom/Document.cpp	2020-03-25 13:44:17 UTC (rev 258977)
@@ -1744,9 +1744,13 @@
     for (auto* client : m_visibilityStateCallbackClients)
         client->visibilityStateChanged();
 
-#if ENABLE(MEDIA_STREAM)
-    if (auto* page = this->page())
-        RealtimeMediaSourceCenter::singleton().setCapturePageState(hidden(), page->isMediaCaptureMuted());
+#if PLATFORM(IOS_FAMILY)
+    if (hidden()) {
+        RealtimeMediaSourceCenter::singleton().setCapturePageState(hidden(), page()->isMediaCaptureMuted());
+        return;
+    }
+    if (!PlatformMediaSessionManager::sharedManager().isInterrupted())
+        MediaStreamTrack::updateCaptureAccordingToMutedState(*this);
 #endif
 }

Modified: trunk/Source/WebCore/page/Page.cpp (258976 => 258977)


--- trunk/Source/WebCore/page/Page.cpp	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/page/Page.cpp	2020-03-25 13:44:17 UTC (rev 258977)
@@ -1781,9 +1781,6 @@
 
 void Page::setMuted(MediaProducer::MutedStateFlags muted)
 {
-    if (m_mutedState == muted)
-        return;
-
     m_mutedState = muted;
 
     forEachDocument([] (Document& document) {

Modified: trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h (258976 => 258977)


--- trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/platform/audio/PlatformMediaSessionManager.h	2020-03-25 13:44:17 UTC (rev 258977)
@@ -140,6 +140,8 @@
 
     WEBCORE_EXPORT void processDidReceiveRemoteControlCommand(PlatformMediaSession::RemoteControlCommandType, const PlatformMediaSession::RemoteCommandArgument*);
 
+    bool isInterrupted() const { return m_interrupted; }
+
 protected:
     friend class PlatformMediaSession;
     PlatformMediaSessionManager();

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp (258976 => 258977)


--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp	2020-03-25 13:44:17 UTC (rev 258977)
@@ -76,25 +76,12 @@
 
 void RealtimeMediaSource::setInterrupted(bool interrupted, bool pageMuted)
 {
-    if (interrupted == m_interrupted)
-        return;
-
     ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, interrupted, ", page muted : ", pageMuted);
-
-    m_interrupted = interrupted;
-    if (!interrupted && pageMuted)
-        return;
-
     setMuted(interrupted);
 }
 
 void RealtimeMediaSource::setMuted(bool muted)
 {
-    if (!muted && interrupted()) {
-        ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, "ignoring unmute because of interruption");
-        return;
-    }
-
     ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, muted);
 
     // Changed m_muted before calling start/stop so muted() will reflect the correct state.

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h (258976 => 258977)


--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h	2020-03-25 13:44:17 UTC (rev 258977)
@@ -45,7 +45,6 @@
     virtual void setActiveSource(RealtimeMediaSource&);
     void unsetActiveSource(RealtimeMediaSource&);
 
-protected:
     RealtimeMediaSource* activeSource() { return m_activeSource; }
 
 private:

Modified: trunk/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp (258976 => 258977)


--- trunk/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp	2020-03-25 11:11:47 UTC (rev 258976)
+++ trunk/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp	2020-03-25 13:44:17 UTC (rev 258977)
@@ -74,6 +74,9 @@
 {
     ASSERT(isMainThread());
 
+    if (m_suspended)
+        resume();
+
     if (++m_producingCount != 1)
         return;
 
@@ -80,11 +83,6 @@
     if (isProducingData())
         return;
 
-    if (m_suspended) {
-        RELEASE_LOG_INFO(WebRTC, "BaseAudioSharedUnit::startProducingData - exiting early as suspended");
-        return;
-    }
-
     if (hasAudioUnit()) {
         cleanupAudioUnit();
         ASSERT(!hasAudioUnit());
@@ -159,7 +157,9 @@
 OSStatus BaseAudioSharedUnit::resume()
 {
     ASSERT(isMainThread());
-    ASSERT(m_suspended);
+    if (!m_suspended)
+        return 0;
+
     ASSERT(!isProducingData());
 
     RELEASE_LOG_INFO(WebRTC, "BaseAudioSharedUnit::resume");
@@ -180,7 +180,7 @@
     }
 
     forEachClient([](auto& client) {
-        client.notifyMutedChange(false);
+        client.setMuted(false);
     });
 
     return 0;
@@ -196,7 +196,7 @@
     stopInternal();
 
     forEachClient([](auto& client) {
-        client.notifyMutedChange(true);
+        client.setMuted(true);
     });
 
     return 0;

_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to