Title: [198741] trunk
Revision
198741
Author
cdu...@apple.com
Date
2016-03-28 09:04:33 -0700 (Mon, 28 Mar 2016)

Log Message

Disk cache speculative validation requests do not have the 'Referer' HTTP header set
https://bugs.webkit.org/show_bug.cgi?id=155890
<rdar://problem/25279962>

Reviewed by Antti Koivisto.

Source/WebCore:

Export a couple more symbols so we can use them from WebKit2.

* platform/network/HTTPHeaderMap.h:
* platform/network/ResourceRequestBase.h:

Source/WebKit2:

Disk cache speculative validation requests did not have the 'Referer'
HTTP header set. This was breaking some streaming sites (such as
twitch.tv).

We now store all the original request's HTTP headers in the disk cache
so we can re-use them for the speculative validation requests. When the
actual request comes later on, we also make sure that the HTTP headers
used in the speculative validation request match the ones from the
effective request. If they don't we don't use the speculatively
validated resource as the server may return a different resource in
such case.

* NetworkProcess/cache/NetworkCache.cpp:
(WebKit::NetworkCache::Cache::retrieve):
Pass the effective ResourceRequest to the NetworkCacheSpeculativeLoadManager
so it can check that the speculative validation request's HTTP headers
match the effective request's headers.

* NetworkProcess/cache/NetworkCacheSpeculativeLoad.h:
* NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.cpp:
(WebKit::NetworkCache::constructRevalidationRequest):
(WebKit::NetworkCache::SpeculativeLoadManager::PreloadedEntry::PreloadedEntry):
(WebKit::NetworkCache::SpeculativeLoadManager::PreloadedEntry::revalidationRequest):
(WebKit::NetworkCache::SpeculativeLoadManager::PreloadedEntry::wasRevalidated):
We now have a member in PreloadedEntry to keep the request used for validation, if
speculative validation was done. This is so that we can compare its HTTP headers
with the ones of the effective request later on.

(WebKit::NetworkCache::dumpHTTPHeadersDiff):
Debug function that prints which HTTP headers did not match in the speculative
validation request and in the effective request.

(WebKit::NetworkCache::requestsHeadersMatch):
New utility function to check that the speculative validation request's HTTP
headers match the ones of the effective requests. Note that we allow for
headers related to validation to differ.

(WebKit::NetworkCache::SpeculativeLoadManager::canUsePreloadedEntry):
(WebKit::NetworkCache::SpeculativeLoadManager::canUsePendingPreload):
(WebKit::NetworkCache::SpeculativeLoadManager::retrieve):
Check that the speculative validation request's HTTP headers match the
ones of the effective request. If they don't then do not use the
speculatively validated resource.

(WebKit::NetworkCache::SpeculativeLoadManager::addPreloadedEntry):
(WebKit::NetworkCache::SpeculativeLoadManager::revalidateEntry):
(WebKit::NetworkCache::SpeculativeLoadManager::preloadEntry):
(WebKit::NetworkCache::SpeculativeLoadManager::startSpeculativeRevalidation):
* NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.h:

* NetworkProcess/cache/NetworkCacheSubresourcesEntry.cpp:
(WebKit::NetworkCache::SubresourceInfo::encode):
(WebKit::NetworkCache::SubresourceInfo::decode):
(WebKit::NetworkCache::SubresourcesEntry::encodeAsStorageRecord):
(WebKit::NetworkCache::SubresourcesEntry::decodeStorageRecord):
* NetworkProcess/cache/NetworkCacheSubresourcesEntry.h:
(WebKit::NetworkCache::SubresourceInfo::SubresourceInfo):
- Keep all the request's headers in SubresourceInfo instead of just the
  'User-Agent' one.
- Drop the custom copy constructor / assignment operator for
  SubresourceInfo that were making isolated copies on the members.
  We technically don't need to use SubresourceInfo objects in other
  threads than the main one. In SpeculativeLoadManager::preloadEntry(),
  we now make capture a SubresourceInfo* in the lambda to avoid calling
  the copy constructor. We also make sure that the object gets destroyed
  in the lambda function, which is executed in the main thread.

LayoutTests:

Update existing layout test to make sure that speculative validation
requests have their HTTP 'Referer' header set.

* http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt:
* http/tests/cache/disk-cache/speculative-validation/validation-request.html:

Modified Paths

Diff

Modified: trunk/LayoutTests/ChangeLog (198740 => 198741)


--- trunk/LayoutTests/ChangeLog	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/LayoutTests/ChangeLog	2016-03-28 16:04:33 UTC (rev 198741)
@@ -1,3 +1,17 @@
+2016-03-28  Chris Dumez  <cdu...@apple.com>
+
+        Disk cache speculative validation requests do not have the 'Referer' HTTP header set
+        https://bugs.webkit.org/show_bug.cgi?id=155890
+        <rdar://problem/25279962>
+
+        Reviewed by Antti Koivisto.
+
+        Update existing layout test to make sure that speculative validation
+        requests have their HTTP 'Referer' header set.
+
+        * http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt:
+        * http/tests/cache/disk-cache/speculative-validation/validation-request.html:
+
 2016-03-28  Gyuyoung Kim  <gyuyoung....@webkit.org>
 
         [EFL] Mark failing indexeddb tests to failure

Modified: trunk/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt (198740 => 198741)


--- trunk/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request-expected.txt	2016-03-28 16:04:33 UTC (rev 198741)
@@ -4,7 +4,12 @@
 
 
 PASS validationRequestHeader('If-None-Match') is "123456789"
-PASS validationRequestHeader('User-Agent') is initialUserAgent
+PASS validationRequestHeader('Accept') is initialHeaderValues['Accept']
+PASS validationRequestHeader('Accept-Encoding') is initialHeaderValues['Accept-Encoding']
+PASS validationRequestHeader('Accept-Language') is initialHeaderValues['Accept-Language']
+PASS validationRequestHeader('Connection') is initialHeaderValues['Connection']
+PASS validationRequestHeader('Referer') is initialHeaderValues['Referer']
+PASS validationRequestHeader('User-Agent') is initialHeaderValues['User-Agent']
 PASS successfullyParsed is true
 
 TEST COMPLETE

Modified: trunk/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request.html (198740 => 198741)


--- trunk/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request.html	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/LayoutTests/http/tests/cache/disk-cache/speculative-validation/validation-request.html	2016-03-28 16:04:33 UTC (rev 198741)
@@ -5,6 +5,9 @@
 
 state = "warmup";
 
+var headersToCheck = ["Accept", "Accept-Encoding", "Accept-Language", "Connection", "Referer", "User-Agent"];
+var headerToCheck;
+
 function validationRequestHeader(headerName)
 {
     return frames[0].allRequestHeaders[headerName];
@@ -14,7 +17,11 @@
 function frameLoaded()
 {
     if (state == "warmup") {
-        initialUserAgent = validationRequestHeader('User-Agent');
+        initialHeaderValues = [];
+        for (var i = 0; i < headersToCheck.length; i++) {
+            headerToCheck = headersToCheck[i];
+            initialHeaderValues[headerToCheck] = validationRequestHeader(headerToCheck);
+        }
         // Navigate frame to 'about:blank' to flush to subresource loads metadata to disk.
         state = "flushingMetadata";
         document.getElementById("testFrame").src = ""
@@ -30,7 +37,11 @@
     if (state == "speculativeRevalidation") {
         // Validate the HTTP headers of the speculative validation request.
         shouldBeEqualToString("validationRequestHeader('If-None-Match')", "123456789");
-        shouldBe("validationRequestHeader('User-Agent')", "initialUserAgent");
+
+        for (var i = 0; i < headersToCheck.length; i++) {
+            headerToCheck = headersToCheck[i];
+            shouldBe("validationRequestHeader('" + headerToCheck + "')", "initialHeaderValues['" + headerToCheck + "']");
+        }
         finishJSTest();
         return;
     }

Modified: trunk/Source/WebCore/ChangeLog (198740 => 198741)


--- trunk/Source/WebCore/ChangeLog	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebCore/ChangeLog	2016-03-28 16:04:33 UTC (rev 198741)
@@ -1,3 +1,16 @@
+2016-03-28  Chris Dumez  <cdu...@apple.com>
+
+        Disk cache speculative validation requests do not have the 'Referer' HTTP header set
+        https://bugs.webkit.org/show_bug.cgi?id=155890
+        <rdar://problem/25279962>
+
+        Reviewed by Antti Koivisto.
+
+        Export a couple more symbols so we can use them from WebKit2.
+
+        * platform/network/HTTPHeaderMap.h:
+        * platform/network/ResourceRequestBase.h:
+
 2016-03-24  Timothy Hatcher  <timo...@apple.com>
 
         Web Automation: Add commands to compute layout of an element

Modified: trunk/Source/WebCore/platform/network/HTTPHeaderMap.h (198740 => 198741)


--- trunk/Source/WebCore/platform/network/HTTPHeaderMap.h	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebCore/platform/network/HTTPHeaderMap.h	2016-03-28 16:04:33 UTC (rev 198741)
@@ -139,7 +139,7 @@
     WEBCORE_EXPORT String get(const String& name) const;
     WEBCORE_EXPORT void set(const String& name, const String& value);
     void add(const String& name, const String& value);
-    bool contains(const String&) const;
+    WEBCORE_EXPORT bool contains(const String&) const;
     bool remove(const String&);
 
     WEBCORE_EXPORT String get(HTTPHeaderName) const;

Modified: trunk/Source/WebCore/platform/network/ResourceRequestBase.h (198740 => 198741)


--- trunk/Source/WebCore/platform/network/ResourceRequestBase.h	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebCore/platform/network/ResourceRequestBase.h	2016-03-28 16:04:33 UTC (rev 198741)
@@ -130,7 +130,7 @@
         WEBCORE_EXPORT void setPriority(ResourceLoadPriority);
 
         WEBCORE_EXPORT bool isConditional() const;
-        void makeUnconditional();
+        WEBCORE_EXPORT void makeUnconditional();
 
         // Whether the associated ResourceHandleClient needs to be notified of
         // upload progress made for that resource.

Modified: trunk/Source/WebKit2/ChangeLog (198740 => 198741)


--- trunk/Source/WebKit2/ChangeLog	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebKit2/ChangeLog	2016-03-28 16:04:33 UTC (rev 198741)
@@ -1,3 +1,78 @@
+2016-03-28  Chris Dumez  <cdu...@apple.com>
+
+        Disk cache speculative validation requests do not have the 'Referer' HTTP header set
+        https://bugs.webkit.org/show_bug.cgi?id=155890
+        <rdar://problem/25279962>
+
+        Reviewed by Antti Koivisto.
+
+        Disk cache speculative validation requests did not have the 'Referer'
+        HTTP header set. This was breaking some streaming sites (such as
+        twitch.tv).
+
+        We now store all the original request's HTTP headers in the disk cache
+        so we can re-use them for the speculative validation requests. When the
+        actual request comes later on, we also make sure that the HTTP headers
+        used in the speculative validation request match the ones from the
+        effective request. If they don't we don't use the speculatively
+        validated resource as the server may return a different resource in
+        such case.
+
+        * NetworkProcess/cache/NetworkCache.cpp:
+        (WebKit::NetworkCache::Cache::retrieve):
+        Pass the effective ResourceRequest to the NetworkCacheSpeculativeLoadManager
+        so it can check that the speculative validation request's HTTP headers
+        match the effective request's headers.
+
+        * NetworkProcess/cache/NetworkCacheSpeculativeLoad.h:
+        * NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.cpp:
+        (WebKit::NetworkCache::constructRevalidationRequest):
+        (WebKit::NetworkCache::SpeculativeLoadManager::PreloadedEntry::PreloadedEntry):
+        (WebKit::NetworkCache::SpeculativeLoadManager::PreloadedEntry::revalidationRequest):
+        (WebKit::NetworkCache::SpeculativeLoadManager::PreloadedEntry::wasRevalidated):
+        We now have a member in PreloadedEntry to keep the request used for validation, if
+        speculative validation was done. This is so that we can compare its HTTP headers
+        with the ones of the effective request later on.
+
+        (WebKit::NetworkCache::dumpHTTPHeadersDiff):
+        Debug function that prints which HTTP headers did not match in the speculative
+        validation request and in the effective request.
+
+        (WebKit::NetworkCache::requestsHeadersMatch):
+        New utility function to check that the speculative validation request's HTTP
+        headers match the ones of the effective requests. Note that we allow for
+        headers related to validation to differ.
+
+        (WebKit::NetworkCache::SpeculativeLoadManager::canUsePreloadedEntry):
+        (WebKit::NetworkCache::SpeculativeLoadManager::canUsePendingPreload):
+        (WebKit::NetworkCache::SpeculativeLoadManager::retrieve):
+        Check that the speculative validation request's HTTP headers match the
+        ones of the effective request. If they don't then do not use the
+        speculatively validated resource.
+
+        (WebKit::NetworkCache::SpeculativeLoadManager::addPreloadedEntry):
+        (WebKit::NetworkCache::SpeculativeLoadManager::revalidateEntry):
+        (WebKit::NetworkCache::SpeculativeLoadManager::preloadEntry):
+        (WebKit::NetworkCache::SpeculativeLoadManager::startSpeculativeRevalidation):
+        * NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.h:
+
+        * NetworkProcess/cache/NetworkCacheSubresourcesEntry.cpp:
+        (WebKit::NetworkCache::SubresourceInfo::encode):
+        (WebKit::NetworkCache::SubresourceInfo::decode):
+        (WebKit::NetworkCache::SubresourcesEntry::encodeAsStorageRecord):
+        (WebKit::NetworkCache::SubresourcesEntry::decodeStorageRecord):
+        * NetworkProcess/cache/NetworkCacheSubresourcesEntry.h:
+        (WebKit::NetworkCache::SubresourceInfo::SubresourceInfo):
+        - Keep all the request's headers in SubresourceInfo instead of just the
+          'User-Agent' one.
+        - Drop the custom copy constructor / assignment operator for
+          SubresourceInfo that were making isolated copies on the members.
+          We technically don't need to use SubresourceInfo objects in other
+          threads than the main one. In SpeculativeLoadManager::preloadEntry(),
+          we now make capture a SubresourceInfo* in the lambda to avoid calling
+          the copy constructor. We also make sure that the object gets destroyed
+          in the lambda function, which is executed in the main thread.
+
 2016-03-25  Timothy Hatcher  <timo...@apple.com>
 
         Web Automation: Add Automation protocol commands to handle _javascript_ dialogs

Modified: trunk/Source/WebKit2/NetworkProcess/cache/NetworkCache.cpp (198740 => 198741)


--- trunk/Source/WebKit2/NetworkProcess/cache/NetworkCache.cpp	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebKit2/NetworkProcess/cache/NetworkCache.cpp	2016-03-28 16:04:33 UTC (rev 198741)
@@ -368,7 +368,7 @@
     }
 
 #if ENABLE(NETWORK_CACHE_SPECULATIVE_REVALIDATION)
-    if (m_speculativeLoadManager && m_speculativeLoadManager->retrieve(frameID, storageKey, [request, completionHandler](std::unique_ptr<Entry> entry) {
+    if (m_speculativeLoadManager && m_speculativeLoadManager->retrieve(frameID, storageKey, request, [request, completionHandler](std::unique_ptr<Entry> entry) {
         if (entry && verifyVaryingRequestHeaders(entry->varyingRequestHeaders(), request))
             completionHandler(WTFMove(entry));
         else

Modified: trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoad.h (198740 => 198741)


--- trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoad.h	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoad.h	2016-03-28 16:04:33 UTC (rev 198741)
@@ -49,6 +49,8 @@
 
     virtual ~SpeculativeLoad();
 
+    const WebCore::ResourceRequest& originalRequest() const { return m_originalRequest; }
+
 private:
     // NetworkLoadClient.
     void didSendData(unsigned long long bytesSent, unsigned long long totalBytesToBeSent) override { }

Modified: trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.cpp (198740 => 198741)


--- trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.cpp	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.cpp	2016-03-28 16:04:33 UTC (rev 198741)
@@ -84,24 +84,25 @@
     return Key(resourceKey.partition(), subresourcesType(), resourceKey.range(), resourceKey.identifier());
 }
 
-static inline ResourceRequest constructRevalidationRequest(const Entry& entry, const SubresourceInfo& subResourceInfo)
+static inline std::unique_ptr<ResourceRequest> constructRevalidationRequest(const Entry& entry, const SubresourceInfo& subResourceInfo)
 {
-    ResourceRequest revalidationRequest(entry.key().identifier());
-    revalidationRequest.setHTTPUserAgent(subResourceInfo.httpUserAgent);
-    revalidationRequest.setFirstPartyForCookies(subResourceInfo.firstPartyForCookies);
+    auto revalidationRequest = std::make_unique<ResourceRequest>(entry.key().identifier());
+    revalidationRequest->setHTTPHeaderFields(subResourceInfo.requestHeaders);
+    revalidationRequest->setFirstPartyForCookies(subResourceInfo.firstPartyForCookies);
 #if ENABLE(CACHE_PARTITIONING)
     if (entry.key().hasPartition())
-        revalidationRequest.setCachePartition(entry.key().partition());
+        revalidationRequest->setCachePartition(entry.key().partition());
 #endif
     ASSERT_WITH_MESSAGE(entry.key().range().isEmpty(), "range is not supported");
 
+    revalidationRequest->makeUnconditional();
     String eTag = entry.response().httpHeaderField(HTTPHeaderName::ETag);
     if (!eTag.isEmpty())
-        revalidationRequest.setHTTPHeaderField(HTTPHeaderName::IfNoneMatch, eTag);
+        revalidationRequest->setHTTPHeaderField(HTTPHeaderName::IfNoneMatch, eTag);
 
     String lastModified = entry.response().httpHeaderField(HTTPHeaderName::LastModified);
     if (!lastModified.isEmpty())
-        revalidationRequest.setHTTPHeaderField(HTTPHeaderName::IfModifiedSince, lastModified);
+        revalidationRequest->setHTTPHeaderField(HTTPHeaderName::IfModifiedSince, lastModified);
 
     return revalidationRequest;
 }
@@ -132,10 +133,10 @@
 class SpeculativeLoadManager::PreloadedEntry : private ExpiringEntry {
     WTF_MAKE_FAST_ALLOCATED;
 public:
-    PreloadedEntry(std::unique_ptr<Entry> entry, WasRevalidated wasRevalidated, std::function<void()>&& lifetimeReachedHandler)
+    PreloadedEntry(std::unique_ptr<Entry> entry, std::unique_ptr<ResourceRequest> speculativeValidationRequest, std::function<void()>&& lifetimeReachedHandler)
         : ExpiringEntry(WTFMove(lifetimeReachedHandler))
         , m_entry(WTFMove(entry))
-        , m_wasRevalidated(wasRevalidated == WasRevalidated::Yes)
+        , m_speculativeValidationRequest(WTFMove(speculativeValidationRequest))
     { }
 
     std::unique_ptr<Entry> takeCacheEntry()
@@ -144,11 +145,12 @@
         return WTFMove(m_entry);
     }
 
-    bool wasRevalidated() const { return m_wasRevalidated; }
+    ResourceRequest* revalidationRequest() const { return m_speculativeValidationRequest.get(); }
+    bool wasRevalidated() const { return !!m_speculativeValidationRequest; }
 
 private:
     std::unique_ptr<Entry> m_entry;
-    bool m_wasRevalidated;
+    std::unique_ptr<ResourceRequest> m_speculativeValidationRequest;
 };
 
 class SpeculativeLoadManager::PendingFrameLoad : public RefCounted<PendingFrameLoad> {
@@ -246,22 +248,89 @@
 {
 }
 
-bool SpeculativeLoadManager::retrieve(const GlobalFrameID& frameID, const Key& storageKey, const RetrieveCompletionHandler& completionHandler)
+#if !LOG_DISABLED
+
+static void dumpHTTPHeadersDiff(const HTTPHeaderMap& headersA, const HTTPHeaderMap& headersB)
 {
+    auto aEnd = headersA.end();
+    for (auto it = headersA.begin(); it != aEnd; ++it) {
+        String valueB = headersB.get(it->key);
+        if (valueB.isNull())
+            LOG(NetworkCacheSpeculativePreloading, "* '%s' HTTP header is only in first request (value: %s)", it->key.utf8().data(), it->value.utf8().data());
+        else if (it->value != valueB)
+            LOG(NetworkCacheSpeculativePreloading, "* '%s' HTTP header differs in both requests: %s != %s", it->key.utf8().data(), it->value.utf8().data(), valueB.utf8().data());
+    }
+    auto bEnd = headersB.end();
+    for (auto it = headersB.begin(); it != bEnd; ++it) {
+        if (!headersA.contains(it->key))
+            LOG(NetworkCacheSpeculativePreloading, "* '%s' HTTP header is only in second request (value: %s)", it->key.utf8().data(), it->value.utf8().data());
+    }
+}
+
+#endif
+
+static bool requestsHeadersMatch(const ResourceRequest& a, const ResourceRequest& b)
+{
+    static const HTTPHeaderName headersAllowedToMismatch[] = {
+        HTTPHeaderName::IfMatch,
+        HTTPHeaderName::IfModifiedSince,
+        HTTPHeaderName::IfNoneMatch,
+        HTTPHeaderName::IfRange,
+        HTTPHeaderName::IfUnmodifiedSince,
+        HTTPHeaderName::CacheControl
+    };
+
+    HTTPHeaderMap headersA = a.httpHeaderFields();
+    HTTPHeaderMap headersB = b.httpHeaderFields();
+    for (auto headerName : headersAllowedToMismatch) {
+        headersA.remove(headerName);
+        headersB.remove(headerName);
+    }
+
+    if (headersA != headersB) {
+        LOG(NetworkCacheSpeculativePreloading, "Cannot reuse speculatively validated entry because HTTP headers used for validation do not match");
+#if !LOG_DISABLED
+        dumpHTTPHeadersDiff(headersA, headersB);
+#endif
+        return false;
+    }
+    return true;
+}
+
+bool SpeculativeLoadManager::canUsePreloadedEntry(const PreloadedEntry& entry, const ResourceRequest& actualRequest)
+{
+    if (!entry.wasRevalidated())
+        return true;
+
+    ASSERT(entry.revalidationRequest());
+    return requestsHeadersMatch(*entry.revalidationRequest(), actualRequest);
+}
+
+bool SpeculativeLoadManager::canUsePendingPreload(const SpeculativeLoad& load, const ResourceRequest& actualRequest)
+{
+    return requestsHeadersMatch(load.originalRequest(), actualRequest);
+}
+
+bool SpeculativeLoadManager::retrieve(const GlobalFrameID& frameID, const Key& storageKey, const WebCore::ResourceRequest& request, const RetrieveCompletionHandler& completionHandler)
+{
     // Check already preloaded entries.
     if (auto preloadedEntry = m_preloadedEntries.take(storageKey)) {
+        if (!canUsePreloadedEntry(*preloadedEntry, request)) {
+            LOG(NetworkCacheSpeculativePreloading, "(NetworkProcess) Retrieval: Could not use preloaded entry to satisfy request for '%s' due to HTTP headers mismatch:", storageKey.identifier().utf8().data());
+            logSpeculativeLoadingDiagnosticMessage(frameID, preloadedEntry->wasRevalidated() ? DiagnosticLoggingKeys::wastedSpeculativeWarmupWithRevalidationKey() : DiagnosticLoggingKeys::wastedSpeculativeWarmupWithoutRevalidationKey());
+            return false;
+        }
+
         LOG(NetworkCacheSpeculativePreloading, "(NetworkProcess) Retrieval: Using preloaded entry to satisfy request for '%s':", storageKey.identifier().utf8().data());
-        if (preloadedEntry->wasRevalidated())
-            logSpeculativeLoadingDiagnosticMessage(frameID, DiagnosticLoggingKeys::successfulSpeculativeWarmupWithRevalidationKey());
-        else
-            logSpeculativeLoadingDiagnosticMessage(frameID, DiagnosticLoggingKeys::successfulSpeculativeWarmupWithoutRevalidationKey());
+        logSpeculativeLoadingDiagnosticMessage(frameID, preloadedEntry->wasRevalidated() ? DiagnosticLoggingKeys::successfulSpeculativeWarmupWithRevalidationKey() : DiagnosticLoggingKeys::successfulSpeculativeWarmupWithoutRevalidationKey());
 
         completionHandler(preloadedEntry->takeCacheEntry());
         return true;
     }
 
     // Check pending speculative revalidations.
-    if (!m_pendingPreloads.contains(storageKey)) {
+    auto* pendingPreload = m_pendingPreloads.get(storageKey);
+    if (!pendingPreload) {
         if (m_notPreloadedEntries.remove(storageKey))
             logSpeculativeLoadingDiagnosticMessage(frameID, DiagnosticLoggingKeys::entryWronglyNotWarmedUpKey());
         else
@@ -270,6 +339,12 @@
         return false;
     }
 
+    if (!canUsePendingPreload(*pendingPreload, request)) {
+        LOG(NetworkCacheSpeculativePreloading, "(NetworkProcess) Retrieval: revalidation already in progress for '%s' but unusable due to HTTP headers mismatch:", storageKey.identifier().utf8().data());
+        logSpeculativeLoadingDiagnosticMessage(frameID, DiagnosticLoggingKeys::wastedSpeculativeWarmupWithRevalidationKey());
+        return false;
+    }
+
     LOG(NetworkCacheSpeculativePreloading, "(NetworkProcess) Retrieval: revalidation already in progress for '%s':", storageKey.identifier().utf8().data());
 
     // FIXME: This breaks incremental loading when the revalidation is not successful.
@@ -316,12 +391,12 @@
         pendingFrameLoad->registerSubresourceLoad(request, resourceKey);
 }
 
-void SpeculativeLoadManager::addPreloadedEntry(std::unique_ptr<Entry> entry, const GlobalFrameID& frameID, WasRevalidated wasRevalidated)
+void SpeculativeLoadManager::addPreloadedEntry(std::unique_ptr<Entry> entry, const GlobalFrameID& frameID, std::unique_ptr<ResourceRequest> revalidationRequest)
 {
     ASSERT(entry);
     ASSERT(!entry->needsValidation());
     auto key = entry->key();
-    m_preloadedEntries.add(key, std::make_unique<PreloadedEntry>(WTFMove(entry), wasRevalidated, [this, key, frameID] {
+    m_preloadedEntries.add(key, std::make_unique<PreloadedEntry>(WTFMove(entry), WTFMove(revalidationRequest), [this, key, frameID] {
         auto preloadedEntry = m_preloadedEntries.take(key);
         ASSERT(preloadedEntry);
         if (preloadedEntry->wasRevalidated())
@@ -381,13 +456,15 @@
     if (!key.range().isEmpty())
         return;
 
-    ResourceRequest revalidationRequest = constructRevalidationRequest(*entry, subresourceInfo);
+    ResourceRequest* revalidationRequestPtr = constructRevalidationRequest(*entry, subresourceInfo).release();
 
     LOG(NetworkCacheSpeculativePreloading, "(NetworkProcess) Speculatively revalidating '%s':", key.identifier().utf8().data());
-    auto revalidator = std::make_unique<SpeculativeLoad>(frameID, revalidationRequest, WTFMove(entry), [this, key, frameID](std::unique_ptr<Entry> revalidatedEntry) {
+
+    auto revalidator = std::make_unique<SpeculativeLoad>(frameID, *revalidationRequestPtr, WTFMove(entry), [this, key, revalidationRequestPtr, frameID](std::unique_ptr<Entry> revalidatedEntry) {
         ASSERT(!revalidatedEntry || !revalidatedEntry->needsValidation());
         ASSERT(!revalidatedEntry || revalidatedEntry->key() == key);
 
+        auto revalidationRequest = std::unique_ptr<ResourceRequest>(revalidationRequestPtr);
         auto protectRevalidator = m_pendingPreloads.take(key);
         LOG(NetworkCacheSpeculativePreloading, "(NetworkProcess) Speculative revalidation completed for '%s':", key.identifier().utf8().data());
 
@@ -398,7 +475,7 @@
         }
 
         if (revalidatedEntry)
-            addPreloadedEntry(WTFMove(revalidatedEntry), frameID, WasRevalidated::Yes);
+            addPreloadedEntry(WTFMove(revalidatedEntry), frameID, WTFMove(revalidationRequest));
     });
     m_pendingPreloads.add(key, WTFMove(revalidator));
 }
@@ -406,7 +483,9 @@
 void SpeculativeLoadManager::preloadEntry(const Key& key, const SubresourceInfo& subResourceInfo, const GlobalFrameID& frameID)
 {
     m_pendingPreloads.add(key, nullptr);
-    retrieveEntryFromStorage(key, [this, key, subResourceInfo, frameID](std::unique_ptr<Entry> entry) {
+    auto* subResourceInfoPtr = new SubresourceInfo(subResourceInfo);
+    retrieveEntryFromStorage(key, [this, key, subResourceInfoPtr, frameID](std::unique_ptr<Entry> entry) {
+        auto subResourceInfo = std::unique_ptr<SubresourceInfo>(subResourceInfoPtr);
         m_pendingPreloads.remove(key);
 
         if (satisfyPendingRequests(key, entry.get())) {
@@ -419,17 +498,17 @@
             return;
 
         if (entry->needsValidation())
-            revalidateEntry(WTFMove(entry), subResourceInfo, frameID);
+            revalidateEntry(WTFMove(entry), *subResourceInfo, frameID);
         else
-            addPreloadedEntry(WTFMove(entry), frameID, WasRevalidated::No);
+            addPreloadedEntry(WTFMove(entry), frameID);
     });
 }
 
 void SpeculativeLoadManager::startSpeculativeRevalidation(const GlobalFrameID& frameID, SubresourcesEntry& entry)
 {
     for (auto& subresourcePair : entry.subresources()) {
-        auto key = subresourcePair.key;
-        auto subresourceInfo = subresourcePair.value;
+        auto& key = subresourcePair.key;
+        auto& subresourceInfo = subresourcePair.value;
         if (!subresourceInfo.isTransient)
             preloadEntry(key, subresourceInfo, frameID);
         else {

Modified: trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.h (198740 => 198741)


--- trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.h	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSpeculativeLoadManager.h	2016-03-28 16:04:33 UTC (rev 198741)
@@ -51,11 +51,12 @@
     void registerLoad(const GlobalFrameID&, const WebCore::ResourceRequest&, const Key& resourceKey);
 
     typedef std::function<void (std::unique_ptr<Entry>)> RetrieveCompletionHandler;
-    bool retrieve(const GlobalFrameID&, const Key& storageKey, const RetrieveCompletionHandler&);
+    bool retrieve(const GlobalFrameID&, const Key& storageKey, const WebCore::ResourceRequest&, const RetrieveCompletionHandler&);
 
 private:
-    enum class WasRevalidated { No, Yes };
-    void addPreloadedEntry(std::unique_ptr<Entry>, const GlobalFrameID&, WasRevalidated);
+    class PreloadedEntry;
+
+    void addPreloadedEntry(std::unique_ptr<Entry>, const GlobalFrameID&, std::unique_ptr<WebCore::ResourceRequest> revalidationRequest = nullptr);
     void preloadEntry(const Key&, const SubresourceInfo&, const GlobalFrameID&);
     void retrieveEntryFromStorage(const Key&, const RetrieveCompletionHandler&);
     void revalidateEntry(std::unique_ptr<Entry>, const SubresourceInfo&, const GlobalFrameID&);
@@ -63,6 +64,9 @@
     void retrieveSubresourcesEntry(const Key& storageKey, std::function<void (std::unique_ptr<SubresourcesEntry>)>);
     void startSpeculativeRevalidation(const GlobalFrameID&, SubresourcesEntry&);
 
+    static bool canUsePreloadedEntry(const PreloadedEntry&, const WebCore::ResourceRequest& actualRequest);
+    static bool canUsePendingPreload(const SpeculativeLoad&, const WebCore::ResourceRequest& actualRequest);
+
     Storage& m_storage;
 
     class PendingFrameLoad;
@@ -71,7 +75,6 @@
     HashMap<Key, std::unique_ptr<SpeculativeLoad>> m_pendingPreloads;
     HashMap<Key, std::unique_ptr<Vector<RetrieveCompletionHandler>>> m_pendingRetrieveRequests;
 
-    class PreloadedEntry;
     HashMap<Key, std::unique_ptr<PreloadedEntry>> m_preloadedEntries;
 
     class ExpiringEntry;

Modified: trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSubresourcesEntry.cpp (198740 => 198741)


--- trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSubresourcesEntry.cpp	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSubresourcesEntry.cpp	2016-03-28 16:04:33 UTC (rev 198741)
@@ -40,7 +40,7 @@
 {
     encoder << firstPartyForCookies;
     encoder << isTransient;
-    encoder << httpUserAgent;
+    requestHeaders.encode(encoder);
 }
 
 bool SubresourceInfo::decode(Decoder& decoder, SubresourceInfo& info)
@@ -49,26 +49,11 @@
         return false;
     if (!decoder.decode(info.isTransient))
         return false;
-    if (!decoder.decode(info.httpUserAgent))
+    if (!WebCore::HTTPHeaderMap::decode(decoder, info.requestHeaders))
         return false;
     return true;
 }
 
-SubresourceInfo::SubresourceInfo(const SubresourceInfo& subresourceInfo)
-    : firstPartyForCookies(subresourceInfo.firstPartyForCookies.isolatedCopy())
-    , httpUserAgent(subresourceInfo.httpUserAgent.isolatedCopy())
-    , isTransient(subresourceInfo.isTransient)
-{
-}
-
-SubresourceInfo& SubresourceInfo::operator=(const SubresourceInfo& other)
-{
-    firstPartyForCookies = other.firstPartyForCookies.isolatedCopy();
-    httpUserAgent = other.httpUserAgent.isolatedCopy();
-    isTransient = other.isTransient;
-    return *this;
-}
-
 Storage::Record SubresourcesEntry::encodeAsStorageRecord() const
 {
     Encoder encoder;

Modified: trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSubresourcesEntry.h (198740 => 198741)


--- trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSubresourcesEntry.h	2016-03-28 15:57:49 UTC (rev 198740)
+++ trunk/Source/WebKit2/NetworkProcess/cache/NetworkCacheSubresourcesEntry.h	2016-03-28 16:04:33 UTC (rev 198741)
@@ -39,23 +39,21 @@
 namespace NetworkCache {
 
 struct SubresourceInfo {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+
     void encode(Encoder&) const;
     static bool decode(Decoder&, SubresourceInfo&);
 
     SubresourceInfo() = default;
     SubresourceInfo(const WebCore::ResourceRequest& request, bool isTransient = false)
         : firstPartyForCookies(request.firstPartyForCookies())
-        , httpUserAgent(request.httpUserAgent())
+        , requestHeaders(request.httpHeaderFields())
         , isTransient(isTransient)
     { }
 
-    SubresourceInfo(const SubresourceInfo&);
-    SubresourceInfo(SubresourceInfo&&) = default;
-    SubresourceInfo& operator=(const SubresourceInfo&);
-    SubresourceInfo& operator=(SubresourceInfo&&) = default;
-
     WebCore::URL firstPartyForCookies;
-    String httpUserAgent;
+    WebCore::HTTPHeaderMap requestHeaders;
     bool isTransient { false };
 };
 
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to