Title: [219312] trunk/Source/WebKit2
Revision
219312
Author
cdu...@apple.com
Date
2017-07-10 16:00:42 -0700 (Mon, 10 Jul 2017)

Log Message

Merge ResourceLoadStatisticsStore into WebResourceLoadStatisticsStore
https://bugs.webkit.org/show_bug.cgi?id=174203

Reviewed by Brent Fulgham.

Merge ResourceLoadStatisticsStore into WebResourceLoadStatisticsStore. The 2 classes
have a similar purpose and there is no clean separation between the 2. It makes more
sense to have a single store class for resource load statistics.

If we want to simplify the WebResourceLoadStatisticsStore class, I think it'd make
more sense to split the file system I/O code out. This code adds quite a bit of
complexity.

* CMakeLists.txt:
* UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
(-[WKWebsiteDataStore _resourceLoadStatisticsUpdateCookiePartitioning]):
(-[WKWebsiteDataStore _resourceLoadStatisticsSetShouldPartitionCookies:forHost:]):
(-[WKWebsiteDataStore _resourceLoadStatisticsClearInMemoryAndPersistentStore]):
(-[WKWebsiteDataStore _resourceLoadStatisticsClearInMemoryAndPersistentStoreModifiedSinceHours:]):
(-[WKWebsiteDataStore _resourceLoadStatisticsResetToConsistentState]):
* UIProcess/Storage/ResourceLoadStatisticsStore.cpp: Removed.
* UIProcess/Storage/ResourceLoadStatisticsStore.h: Removed.
* UIProcess/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
(WebKit::WebResourceLoadStatisticsStore::removeDataRecords):
(WebKit::WebResourceLoadStatisticsStore::processStatisticsAndDataRecords):
(WebKit::WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated):
(WebKit::WebResourceLoadStatisticsStore::grandfatherExistingWebsiteData):
(WebKit::WebResourceLoadStatisticsStore::readDataFromDiskIfNeeded):
(WebKit::WebResourceLoadStatisticsStore::refreshFromDisk):
(WebKit::WebResourceLoadStatisticsStore::writeStoreToDisk):
(WebKit::WebResourceLoadStatisticsStore::startMonitoringStatisticsStorage):
(WebKit::WebResourceLoadStatisticsStore::performDailyTasks):
(WebKit::WebResourceLoadStatisticsStore::submitTelemetry):
(WebKit::WebResourceLoadStatisticsStore::logUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::clearUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::hasHadUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::setPrevalentResource):
(WebKit::WebResourceLoadStatisticsStore::isPrevalentResource):
(WebKit::WebResourceLoadStatisticsStore::clearPrevalentResource):
(WebKit::WebResourceLoadStatisticsStore::setGrandfathered):
(WebKit::WebResourceLoadStatisticsStore::isGrandfathered):
(WebKit::WebResourceLoadStatisticsStore::setSubframeUnderTopFrameOrigin):
(WebKit::WebResourceLoadStatisticsStore::setSubresourceUnderTopFrameOrigin):
(WebKit::WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectTo):
(WebKit::WebResourceLoadStatisticsStore::scheduleCookiePartitioningUpdate):
(WebKit::WebResourceLoadStatisticsStore::scheduleCookiePartitioningUpdateForDomains):
(WebKit::WebResourceLoadStatisticsStore::scheduleClearInMemory):
(WebKit::WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent):
(WebKit::WebResourceLoadStatisticsStore::setTimeToLiveUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::setTimeToLiveCookiePartitionFree):
(WebKit::WebResourceLoadStatisticsStore::setGrandfatheringTime):
(WebKit::WebResourceLoadStatisticsStore::ensureResourceStatisticsForPrimaryDomain):
(WebKit::WebResourceLoadStatisticsStore::createEncoderFromData):
(WebKit::WebResourceLoadStatisticsStore::populateFromDecoder):
(WebKit::WebResourceLoadStatisticsStore::clearInMemory):
(WebKit::WebResourceLoadStatisticsStore::mergeStatistics):
(WebKit::WebResourceLoadStatisticsStore::shouldPartitionCookies):
(WebKit::WebResourceLoadStatisticsStore::updateCookiePartitioning):
(WebKit::WebResourceLoadStatisticsStore::updateCookiePartitioningForDomains):
(WebKit::WebResourceLoadStatisticsStore::processStatistics):
(WebKit::WebResourceLoadStatisticsStore::hasHadUnexpiredRecentUserInteraction):
(WebKit::WebResourceLoadStatisticsStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor):
(WebKit::WebResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary):
(WebKit::WebResourceLoadStatisticsStore::hasStatisticsExpired):
* UIProcess/WebResourceLoadStatisticsStore.h:
* UIProcess/WebResourceLoadStatisticsTelemetry.cpp:
(WebKit::sortedPrevalentResourceTelemetry):
(WebKit::WebResourceLoadStatisticsTelemetry::calculateAndSubmit):
* UIProcess/WebResourceLoadStatisticsTelemetry.h:
* UIProcess/WebsiteData/WebsiteDataStore.cpp:
(WebKit::WebsiteDataStore::removeData):
* WebKit2.xcodeproj/project.pbxproj:

Modified Paths

Removed Paths

Diff

Modified: trunk/Source/WebKit2/CMakeLists.txt (219311 => 219312)


--- trunk/Source/WebKit2/CMakeLists.txt	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/CMakeLists.txt	2017-07-10 23:00:42 UTC (rev 219312)
@@ -434,7 +434,6 @@
 
     UIProcess/Storage/LocalStorageDatabase.cpp
     UIProcess/Storage/LocalStorageDatabaseTracker.cpp
-    UIProcess/Storage/ResourceLoadStatisticsStore.cpp
 
     UIProcess/UserContent/WebScriptMessageHandler.cpp
     UIProcess/UserContent/WebUserContentControllerProxy.cpp

Modified: trunk/Source/WebKit2/ChangeLog (219311 => 219312)


--- trunk/Source/WebKit2/ChangeLog	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/ChangeLog	2017-07-10 23:00:42 UTC (rev 219312)
@@ -1,3 +1,79 @@
+2017-07-10  Chris Dumez  <cdu...@apple.com>
+
+        Merge ResourceLoadStatisticsStore into WebResourceLoadStatisticsStore
+        https://bugs.webkit.org/show_bug.cgi?id=174203
+
+        Reviewed by Brent Fulgham.
+
+        Merge ResourceLoadStatisticsStore into WebResourceLoadStatisticsStore. The 2 classes
+        have a similar purpose and there is no clean separation between the 2. It makes more
+        sense to have a single store class for resource load statistics.
+
+        If we want to simplify the WebResourceLoadStatisticsStore class, I think it'd make
+        more sense to split the file system I/O code out. This code adds quite a bit of
+        complexity.
+
+        * CMakeLists.txt:
+        * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+        (-[WKWebsiteDataStore _resourceLoadStatisticsUpdateCookiePartitioning]):
+        (-[WKWebsiteDataStore _resourceLoadStatisticsSetShouldPartitionCookies:forHost:]):
+        (-[WKWebsiteDataStore _resourceLoadStatisticsClearInMemoryAndPersistentStore]):
+        (-[WKWebsiteDataStore _resourceLoadStatisticsClearInMemoryAndPersistentStoreModifiedSinceHours:]):
+        (-[WKWebsiteDataStore _resourceLoadStatisticsResetToConsistentState]):
+        * UIProcess/Storage/ResourceLoadStatisticsStore.cpp: Removed.
+        * UIProcess/Storage/ResourceLoadStatisticsStore.h: Removed.
+        * UIProcess/WebResourceLoadStatisticsStore.cpp:
+        (WebKit::WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore):
+        (WebKit::WebResourceLoadStatisticsStore::removeDataRecords):
+        (WebKit::WebResourceLoadStatisticsStore::processStatisticsAndDataRecords):
+        (WebKit::WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated):
+        (WebKit::WebResourceLoadStatisticsStore::grandfatherExistingWebsiteData):
+        (WebKit::WebResourceLoadStatisticsStore::readDataFromDiskIfNeeded):
+        (WebKit::WebResourceLoadStatisticsStore::refreshFromDisk):
+        (WebKit::WebResourceLoadStatisticsStore::writeStoreToDisk):
+        (WebKit::WebResourceLoadStatisticsStore::startMonitoringStatisticsStorage):
+        (WebKit::WebResourceLoadStatisticsStore::performDailyTasks):
+        (WebKit::WebResourceLoadStatisticsStore::submitTelemetry):
+        (WebKit::WebResourceLoadStatisticsStore::logUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::clearUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::hasHadUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::setPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsStore::isPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsStore::clearPrevalentResource):
+        (WebKit::WebResourceLoadStatisticsStore::setGrandfathered):
+        (WebKit::WebResourceLoadStatisticsStore::isGrandfathered):
+        (WebKit::WebResourceLoadStatisticsStore::setSubframeUnderTopFrameOrigin):
+        (WebKit::WebResourceLoadStatisticsStore::setSubresourceUnderTopFrameOrigin):
+        (WebKit::WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectTo):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleCookiePartitioningUpdate):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleCookiePartitioningUpdateForDomains):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleClearInMemory):
+        (WebKit::WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent):
+        (WebKit::WebResourceLoadStatisticsStore::setTimeToLiveUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::setTimeToLiveCookiePartitionFree):
+        (WebKit::WebResourceLoadStatisticsStore::setGrandfatheringTime):
+        (WebKit::WebResourceLoadStatisticsStore::ensureResourceStatisticsForPrimaryDomain):
+        (WebKit::WebResourceLoadStatisticsStore::createEncoderFromData):
+        (WebKit::WebResourceLoadStatisticsStore::populateFromDecoder):
+        (WebKit::WebResourceLoadStatisticsStore::clearInMemory):
+        (WebKit::WebResourceLoadStatisticsStore::mergeStatistics):
+        (WebKit::WebResourceLoadStatisticsStore::shouldPartitionCookies):
+        (WebKit::WebResourceLoadStatisticsStore::updateCookiePartitioning):
+        (WebKit::WebResourceLoadStatisticsStore::updateCookiePartitioningForDomains):
+        (WebKit::WebResourceLoadStatisticsStore::processStatistics):
+        (WebKit::WebResourceLoadStatisticsStore::hasHadUnexpiredRecentUserInteraction):
+        (WebKit::WebResourceLoadStatisticsStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor):
+        (WebKit::WebResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary):
+        (WebKit::WebResourceLoadStatisticsStore::hasStatisticsExpired):
+        * UIProcess/WebResourceLoadStatisticsStore.h:
+        * UIProcess/WebResourceLoadStatisticsTelemetry.cpp:
+        (WebKit::sortedPrevalentResourceTelemetry):
+        (WebKit::WebResourceLoadStatisticsTelemetry::calculateAndSubmit):
+        * UIProcess/WebResourceLoadStatisticsTelemetry.h:
+        * UIProcess/WebsiteData/WebsiteDataStore.cpp:
+        (WebKit::WebsiteDataStore::removeData):
+        * WebKit2.xcodeproj/project.pbxproj:
+
 2017-07-10  Wenson Hsieh  <wenson_hs...@apple.com>
 
         [WK2] Ignore touch events that interrupt platform-driven momentum scrolling

Modified: trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebsiteDataStore.mm (219311 => 219312)


--- trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebsiteDataStore.mm	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebsiteDataStore.mm	2017-07-10 23:00:42 UTC (rev 219312)
@@ -360,7 +360,7 @@
     if (!store)
         return;
 
-    store->updateCookiePartitioning();
+    store->scheduleCookiePartitioningUpdate();
 }
 
 - (void)_resourceLoadStatisticsSetShouldPartitionCookies:(BOOL)value forHost:(NSString *)host
@@ -370,9 +370,9 @@
         return;
 
     if (value)
-        store->updateCookiePartitioningForDomains({ }, { host }, WebKit::ShouldClearFirst::No);
+        store->scheduleCookiePartitioningUpdateForDomains({ }, { host }, WebKit::ShouldClearFirst::No);
     else
-        store->updateCookiePartitioningForDomains({ host }, { }, WebKit::ShouldClearFirst::No);
+        store->scheduleCookiePartitioningUpdateForDomains({ host }, { }, WebKit::ShouldClearFirst::No);
 }
 
 - (void)_resourceLoadStatisticsSubmitTelemetry
@@ -422,7 +422,7 @@
     if (!store)
         return;
 
-    store->clearInMemoryAndPersistent();
+    store->scheduleClearInMemoryAndPersistent();
 }
 
 - (void)_resourceLoadStatisticsClearInMemoryAndPersistentStoreModifiedSinceHours:(unsigned)hours
@@ -431,7 +431,7 @@
     if (!store)
         return;
 
-    store->clearInMemoryAndPersistent(std::chrono::system_clock::now() - std::chrono::hours(hours));
+    store->scheduleClearInMemoryAndPersistent(std::chrono::system_clock::now() - std::chrono::hours(hours));
 }
 
 - (void)_resourceLoadStatisticsResetToConsistentState
@@ -448,7 +448,7 @@
     store->setNotifyPagesWhenDataRecordsWereScanned(false);
     WebKit::WebResourceLoadStatisticsTelemetry::setNotifyPagesWhenTelemetryWasCaptured(false);
     store->setShouldClassifyResourcesBeforeDataRecordsRemoval(true);
-    store->clearInMemory();
+    store->scheduleClearInMemory();
 }
 
 @end

Deleted: trunk/Source/WebKit2/UIProcess/Storage/ResourceLoadStatisticsStore.cpp (219311 => 219312)


--- trunk/Source/WebKit2/UIProcess/Storage/ResourceLoadStatisticsStore.cpp	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/UIProcess/Storage/ResourceLoadStatisticsStore.cpp	2017-07-10 23:00:42 UTC (rev 219312)
@@ -1,351 +0,0 @@
-/*
- * Copyright (C) 2016-2017 Apple Inc. All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
- * THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "config.h"
-#include "ResourceLoadStatisticsStore.h"
-
-#include "WebsiteDataStore.h"
-#include <WebCore/KeyedCoding.h>
-#include <WebCore/ResourceLoadStatistics.h>
-#include <wtf/CrossThreadCopier.h>
-#include <wtf/RunLoop.h>
-
-namespace WebKit {
-
-using namespace WebCore;
-
-const unsigned operatingDatesWindow { 30 };
-const unsigned statisticsModelVersion { 6 };
-
-Ref<ResourceLoadStatisticsStore> ResourceLoadStatisticsStore::create(UpdateCookiePartitioningForDomainsHandler&& updateCookiePartitioningForDomainsHandler)
-{
-    return adoptRef(*new ResourceLoadStatisticsStore(WTFMove(updateCookiePartitioningForDomainsHandler)));
-}
-
-ResourceLoadStatisticsStore::ResourceLoadStatisticsStore(UpdateCookiePartitioningForDomainsHandler&& updateCookiePartitioningForDomainsHandler)
-    : m_updateCookiePartitioningForDomainsHandler(WTFMove(updateCookiePartitioningForDomainsHandler))
-{
-}
-
-bool ResourceLoadStatisticsStore::isPrevalentResource(const String& primaryDomain) const
-{
-    ASSERT(!RunLoop::isMain());
-    auto mapEntry = m_resourceStatisticsMap.find(primaryDomain);
-    if (mapEntry == m_resourceStatisticsMap.end())
-        return false;
-
-    return mapEntry->value.isPrevalentResource;
-}
-
-bool ResourceLoadStatisticsStore::isGrandFathered(const String& primaryDomain) const
-{
-    ASSERT(!RunLoop::isMain());
-    auto mapEntry = m_resourceStatisticsMap.find(primaryDomain);
-    if (mapEntry == m_resourceStatisticsMap.end())
-        return false;
-
-    return mapEntry->value.grandfathered;
-}
-
-bool ResourceLoadStatisticsStore::hasHadRecentUserInteraction(const String& primaryDomain)
-{
-    auto mapEntry = m_resourceStatisticsMap.find(primaryDomain);
-    if (mapEntry == m_resourceStatisticsMap.end())
-        return false;
-
-    return hasHadUnexpiredRecentUserInteraction(mapEntry->value);
-}
-    
-ResourceLoadStatistics& ResourceLoadStatisticsStore::ensureResourceStatisticsForPrimaryDomain(const String& primaryDomain)
-{
-    ASSERT(!RunLoop::isMain());
-    return m_resourceStatisticsMap.ensure(primaryDomain, [&primaryDomain] {
-        return ResourceLoadStatistics(primaryDomain);
-    }).iterator->value;
-}
-
-std::unique_ptr<KeyedEncoder> ResourceLoadStatisticsStore::createEncoderFromData() const
-{
-    ASSERT(!RunLoop::isMain());
-    auto encoder = KeyedEncoder::encoder();
-    encoder->encodeUInt32("version", statisticsModelVersion);
-    encoder->encodeDouble("endOfGrandfatheringTimestamp", m_endOfGrandfatheringTimestamp.secondsSinceEpoch().value());
-    
-    encoder->encodeObjects("browsingStatistics", m_resourceStatisticsMap.begin(), m_resourceStatisticsMap.end(), [](KeyedEncoder& encoderInner, const auto& origin) {
-        origin.value.encode(encoderInner);
-    });
-
-    encoder->encodeObjects("operatingDates", m_operatingDates.begin(), m_operatingDates.end(), [](KeyedEncoder& encoderInner, WallTime date) {
-        encoderInner.encodeDouble("date", date.secondsSinceEpoch().value());
-    });
-    
-    return encoder;
-}
-
-void ResourceLoadStatisticsStore::populateFromDecoder(KeyedDecoder& decoder)
-{
-    ASSERT(!RunLoop::isMain());
-    if (!m_resourceStatisticsMap.isEmpty())
-        return;
-
-    unsigned versionOnDisk;
-    if (!decoder.decodeUInt32("version", versionOnDisk))
-        return;
-
-    if (versionOnDisk != statisticsModelVersion)
-        return;
-
-    double endOfGrandfatheringTimestamp;
-    if (decoder.decodeDouble("endOfGrandfatheringTimestamp", endOfGrandfatheringTimestamp))
-        m_endOfGrandfatheringTimestamp = WallTime::fromRawSeconds(endOfGrandfatheringTimestamp);
-    else
-        m_endOfGrandfatheringTimestamp = { };
-
-    Vector<ResourceLoadStatistics> loadedStatistics;
-    bool succeeded = decoder.decodeObjects("browsingStatistics", loadedStatistics, [](KeyedDecoder& decoderInner, ResourceLoadStatistics& statistics) {
-        return statistics.decode(decoderInner);
-    });
-
-    if (!succeeded)
-        return;
-
-    Vector<String> prevalentResourceDomainsWithoutUserInteraction;
-    prevalentResourceDomainsWithoutUserInteraction.reserveInitialCapacity(loadedStatistics.size());
-    for (auto& statistics : loadedStatistics) {
-        if (statistics.isPrevalentResource && !statistics.hadUserInteraction) {
-            prevalentResourceDomainsWithoutUserInteraction.uncheckedAppend(statistics.highLevelDomain);
-            statistics.isMarkedForCookiePartitioning = true;
-        }
-        m_resourceStatisticsMap.add(statistics.highLevelDomain, WTFMove(statistics));
-    }
-
-    succeeded = decoder.decodeObjects("operatingDates", m_operatingDates, [](KeyedDecoder& decoder, WallTime& wallTime) {
-        double value;
-        if (!decoder.decodeDouble("date", value))
-            return false;
-        
-        wallTime = WallTime::fromRawSeconds(value);
-        return true;
-    });
-
-    if (!succeeded)
-        return;
-    
-    updateCookiePartitioningForDomains({ }, prevalentResourceDomainsWithoutUserInteraction, ShouldClearFirst::Yes);
-}
-
-void ResourceLoadStatisticsStore::clearInMemory()
-{
-    ASSERT(!RunLoop::isMain());
-    m_resourceStatisticsMap.clear();
-    m_operatingDates.clear();
-
-    updateCookiePartitioningForDomains({ }, { }, ShouldClearFirst::Yes);
-}
-
-void ResourceLoadStatisticsStore::mergeStatistics(Vector<ResourceLoadStatistics>&& statistics)
-{
-    ASSERT(!RunLoop::isMain());
-    for (auto& statistic : statistics) {
-        auto result = m_resourceStatisticsMap.ensure(statistic.highLevelDomain, [&statistic] {
-            return WTFMove(statistic);
-        });
-        if (!result.isNewEntry)
-            result.iterator->value.merge(statistic);
-    }
-}
-
-inline bool ResourceLoadStatisticsStore::shouldPartitionCookies(const ResourceLoadStatistics& statistic) const
-{
-    return statistic.isPrevalentResource && (!statistic.hadUserInteraction || WallTime::now() > statistic.mostRecentUserInteractionTime + m_timeToLiveCookiePartitionFree);
-}
-
-void ResourceLoadStatisticsStore::updateCookiePartitioning()
-{
-    ASSERT(!RunLoop::isMain());
-
-    Vector<String> domainsToRemove;
-    Vector<String> domainsToAdd;
-    for (auto& resourceStatistic : m_resourceStatisticsMap.values()) {
-        bool shouldPartition = shouldPartitionCookies(resourceStatistic);
-        if (resourceStatistic.isMarkedForCookiePartitioning && !shouldPartition) {
-            resourceStatistic.isMarkedForCookiePartitioning = false;
-            domainsToRemove.append(resourceStatistic.highLevelDomain);
-        } else if (!resourceStatistic.isMarkedForCookiePartitioning && shouldPartition) {
-            resourceStatistic.isMarkedForCookiePartitioning = true;
-            domainsToAdd.append(resourceStatistic.highLevelDomain);
-        }
-    }
-
-    if (domainsToRemove.isEmpty() && domainsToAdd.isEmpty())
-        return;
-
-    RunLoop::main().dispatch([this, protectedThis = makeRef(*this), domainsToRemove = CrossThreadCopier<Vector<String>>::copy(domainsToRemove), domainsToAdd = CrossThreadCopier<Vector<String>>::copy(domainsToAdd)] () {
-        m_updateCookiePartitioningForDomainsHandler(domainsToRemove, domainsToAdd, ShouldClearFirst::No);
-    });
-}
-
-void ResourceLoadStatisticsStore::updateCookiePartitioningForDomains(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst shouldClearFirst)
-{
-    ASSERT(!RunLoop::isMain());
-    if (domainsToRemove.isEmpty() && domainsToAdd.isEmpty())
-        return;
-    
-    RunLoop::main().dispatch([this, shouldClearFirst, protectedThis = makeRef(*this), domainsToRemove = CrossThreadCopier<Vector<String>>::copy(domainsToRemove), domainsToAdd = CrossThreadCopier<Vector<String>>::copy(domainsToAdd)] () {
-        m_updateCookiePartitioningForDomainsHandler(domainsToRemove, domainsToAdd, shouldClearFirst);
-    });
-    
-    if (shouldClearFirst == ShouldClearFirst::Yes) {
-        for (auto& resourceStatistic : m_resourceStatisticsMap.values())
-            resourceStatistic.isMarkedForCookiePartitioning = false;
-    } else {
-        for (auto& domain : domainsToRemove)
-            ensureResourceStatisticsForPrimaryDomain(domain).isMarkedForCookiePartitioning = false;
-    }
-
-    for (auto& domain : domainsToAdd)
-        ensureResourceStatisticsForPrimaryDomain(domain).isMarkedForCookiePartitioning = true;
-}
-
-void ResourceLoadStatisticsStore::setTimeToLiveUserInteraction(std::optional<Seconds> seconds)
-{
-    ASSERT(!seconds || seconds.value() >= 0_s);
-    m_timeToLiveUserInteraction = seconds;
-}
-
-void ResourceLoadStatisticsStore::setTimeToLiveCookiePartitionFree(Seconds seconds)
-{
-    ASSERT(seconds >= 0_s);
-    m_timeToLiveCookiePartitionFree = seconds;
-}
-
-void ResourceLoadStatisticsStore::setGrandfatheringTime(Seconds seconds)
-{
-    ASSERT(seconds >= 0_s);
-    m_grandfatheringTime = seconds;
-}
-
-void ResourceLoadStatisticsStore::processStatistics(const WTF::Function<void(ResourceLoadStatistics&)>& processFunction)
-{
-    ASSERT(!RunLoop::isMain());
-    for (auto& resourceStatistic : m_resourceStatisticsMap.values())
-        processFunction(resourceStatistic);
-}
-
-void ResourceLoadStatisticsStore::processStatistics(const WTF::Function<void (const ResourceLoadStatistics&)>& processFunction) const
-{
-    ASSERT(!RunLoop::isMain());
-    for (auto& resourceStatistic : m_resourceStatisticsMap.values())
-        processFunction(resourceStatistic);
-}
-
-bool ResourceLoadStatisticsStore::hasHadUnexpiredRecentUserInteraction(ResourceLoadStatistics& resourceStatistic) const
-{
-    if (!resourceStatistic.hadUserInteraction)
-        return false;
-
-    if (hasStatisticsExpired(resourceStatistic)) {
-        // Drop privacy sensitive data because we no longer need it.
-        // Set timestamp to 0 so that statistics merge will know
-        // it has been reset as opposed to its default -1.
-        resourceStatistic.mostRecentUserInteractionTime = { };
-        resourceStatistic.hadUserInteraction = false;
-
-        return false;
-    }
-
-    return true;
-}
-
-Vector<String> ResourceLoadStatisticsStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor()
-{
-    ASSERT(!RunLoop::isMain());
-
-    bool shouldCheckForGrandfathering = m_endOfGrandfatheringTimestamp > WallTime::now();
-    bool shouldClearGrandfathering = !shouldCheckForGrandfathering && m_endOfGrandfatheringTimestamp;
-
-    if (shouldClearGrandfathering)
-        m_endOfGrandfatheringTimestamp = { };
-
-    Vector<String> prevalentResources;
-    for (auto& statistic : m_resourceStatisticsMap.values()) {
-        if (statistic.isPrevalentResource && !hasHadUnexpiredRecentUserInteraction(statistic) && (!shouldCheckForGrandfathering || !statistic.grandfathered))
-            prevalentResources.append(statistic.highLevelDomain);
-
-        if (shouldClearGrandfathering && statistic.grandfathered)
-            statistic.grandfathered = false;
-    }
-
-    return prevalentResources;
-}
-
-void ResourceLoadStatisticsStore::updateStatisticsForRemovedDataRecords(const HashSet<String>& prevalentResourceDomains)
-{
-    ASSERT(!RunLoop::isMain());
-    for (auto& prevalentResourceDomain : prevalentResourceDomains) {
-        auto& statistic = ensureResourceStatisticsForPrimaryDomain(prevalentResourceDomain);
-        ++statistic.dataRecordsRemoved;
-    }
-}
-
-void ResourceLoadStatisticsStore::handleFreshStartWithEmptyOrNoStore(HashSet<String>&& topPrivatelyControlledDomainsToGrandfather)
-{
-    ASSERT(!RunLoop::isMain());
-    for (auto& topPrivatelyControlledDomain : topPrivatelyControlledDomainsToGrandfather) {
-        auto& statistic = ensureResourceStatisticsForPrimaryDomain(topPrivatelyControlledDomain);
-        statistic.grandfathered = true;
-    }
-    m_endOfGrandfatheringTimestamp = WallTime::now() + m_grandfatheringTime;
-}
-
-void ResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary()
-{
-    if (!m_operatingDates.isEmpty() && (WallTime::now() - m_operatingDates.last() < 24_h))
-        return;
-
-    while (m_operatingDates.size() >= operatingDatesWindow)
-        m_operatingDates.removeFirst();
-
-    m_operatingDates.append(WallTime::now());
-}
-    
-bool ResourceLoadStatisticsStore::hasStatisticsExpired(const ResourceLoadStatistics& resourceStatistic) const
-{
-    if (m_operatingDates.size() >= operatingDatesWindow) {
-        if (resourceStatistic.mostRecentUserInteractionTime < m_operatingDates.first())
-            return true;
-    }
-
-    // If we don't meet the real criteria for an expired statistic, check the user
-    // setting for a tighter restriction (mainly for testing).
-    if (m_timeToLiveUserInteraction) {
-        if (WallTime::now() > resourceStatistic.mostRecentUserInteractionTime + m_timeToLiveUserInteraction.value())
-            return true;
-    }
-    
-    return false;
-}
-    
-}

Deleted: trunk/Source/WebKit2/UIProcess/Storage/ResourceLoadStatisticsStore.h (219311 => 219312)


--- trunk/Source/WebKit2/UIProcess/Storage/ResourceLoadStatisticsStore.h	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/UIProcess/Storage/ResourceLoadStatisticsStore.h	2017-07-10 23:00:42 UTC (rev 219312)
@@ -1,101 +0,0 @@
-/*
- * Copyright (C) 2016-2017 Apple Inc.  All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
- * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
- * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
- * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
- * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
- * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
- * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#pragma once
-
-#include <wtf/Deque.h>
-#include <wtf/Function.h>
-#include <wtf/HashMap.h>
-#include <wtf/HashSet.h>
-#include <wtf/MonotonicTime.h>
-#include <wtf/WallTime.h>
-#include <wtf/text/WTFString.h>
-
-namespace WebCore {
-class KeyedDecoder;
-class KeyedEncoder;
-struct ResourceLoadStatistics;
-}
-
-namespace WebKit {
-
-enum class ShouldClearFirst;
-
-// FIXME: We should probably consider merging this with WebResourceLoadStatisticsStore.
-class ResourceLoadStatisticsStore : public ThreadSafeRefCounted<ResourceLoadStatisticsStore> {
-public:
-    using UpdateCookiePartitioningForDomainsHandler = WTF::Function<void(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst)>;
-    static Ref<ResourceLoadStatisticsStore> create(UpdateCookiePartitioningForDomainsHandler&&);
-
-    std::unique_ptr<WebCore::KeyedEncoder> createEncoderFromData() const;
-    void populateFromDecoder(WebCore::KeyedDecoder&);
-
-    bool isEmpty() const { return m_resourceStatisticsMap.isEmpty(); }
-    void clearInMemory();
-
-    WebCore::ResourceLoadStatistics& ensureResourceStatisticsForPrimaryDomain(const String&);
-
-    bool isPrevalentResource(const String&) const;
-    bool isGrandFathered(const String&) const;
-    bool hasHadRecentUserInteraction(const String&);
-    
-    void mergeStatistics(Vector<WebCore::ResourceLoadStatistics>&&);
-
-    void setTimeToLiveUserInteraction(std::optional<Seconds>);
-    void setTimeToLiveCookiePartitionFree(Seconds);
-    void setGrandfatheringTime(Seconds);
-
-    void updateCookiePartitioning();
-    void updateCookiePartitioningForDomains(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst);
-
-    void processStatistics(const WTF::Function<void (WebCore::ResourceLoadStatistics&)>&);
-    void processStatistics(const WTF::Function<void (const WebCore::ResourceLoadStatistics&)>&) const;
-
-    Vector<String> topPrivatelyControlledDomainsToRemoveWebsiteDataFor();
-    void updateStatisticsForRemovedDataRecords(const HashSet<String>& prevalentResourceDomains);
-
-    void handleFreshStartWithEmptyOrNoStore(HashSet<String>&& topPrivatelyControlledDomainsToGrandfather);
-    void includeTodayAsOperatingDateIfNecessary();
-
-private:
-    explicit ResourceLoadStatisticsStore(UpdateCookiePartitioningForDomainsHandler&&);
-
-    bool shouldPartitionCookies(const WebCore::ResourceLoadStatistics&) const;
-    bool hasStatisticsExpired(const WebCore::ResourceLoadStatistics&) const;
-    bool hasHadUnexpiredRecentUserInteraction(WebCore::ResourceLoadStatistics&) const;
-
-    HashMap<String, WebCore::ResourceLoadStatistics> m_resourceStatisticsMap;
-    Deque<WTF::WallTime> m_operatingDates;
-
-    UpdateCookiePartitioningForDomainsHandler m_updateCookiePartitioningForDomainsHandler;
-
-    std::optional<Seconds> m_timeToLiveUserInteraction;
-    Seconds m_timeToLiveCookiePartitionFree { 24_h };
-    Seconds m_grandfatheringTime { 1_h };
-
-    WallTime m_endOfGrandfatheringTimestamp;
-};
-    
-} // namespace WebKit

Modified: trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.cpp (219311 => 219312)


--- trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.cpp	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.cpp	2017-07-10 23:00:42 UTC (rev 219312)
@@ -27,7 +27,6 @@
 #include "WebResourceLoadStatisticsStore.h"
 
 #include "Logging.h"
-#include "ResourceLoadStatisticsStore.h"
 #include "WebProcessMessages.h"
 #include "WebProcessProxy.h"
 #include "WebResourceLoadStatisticsStoreMessages.h"
@@ -50,6 +49,8 @@
 namespace WebKit {
 
 constexpr Seconds minimumStatisticsFileWriteInterval { 5_min };
+constexpr unsigned operatingDatesWindow { 30 };
+constexpr unsigned statisticsModelVersion { 6 };
 
 template<typename T> static inline String primaryDomain(const T& value)
 {
@@ -81,8 +82,8 @@
 }
 
 WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(const String& resourceLoadStatisticsDirectory, UpdateCookiePartitioningForDomainsHandler&& updateCookiePartitioningForDomainsHandler)
-    : m_resourceLoadStatisticsStore(ResourceLoadStatisticsStore::create(WTFMove(updateCookiePartitioningForDomainsHandler)))
-    , m_statisticsQueue(WorkQueue::create("WebResourceLoadStatisticsStore Process Data Queue", WorkQueue::Type::Serial, WorkQueue::QOS::Utility))
+    : m_statisticsQueue(WorkQueue::create("WebResourceLoadStatisticsStore Process Data Queue", WorkQueue::Type::Serial, WorkQueue::QOS::Utility))
+    , m_updateCookiePartitioningForDomainsHandler(WTFMove(updateCookiePartitioningForDomainsHandler))
     , m_statisticsStoragePath(resourceLoadStatisticsDirectory)
     , m_dailyTasksTimer(RunLoop::main(), this, &WebResourceLoadStatisticsStore::performDailyTasks)
 {
@@ -98,7 +99,7 @@
     });
     m_statisticsQueue->dispatchAfter(5_s, [this, protectedThis = makeRef(*this)] {
         if (m_shouldSubmitTelemetry)
-            WebResourceLoadStatisticsTelemetry::calculateAndSubmit(coreStore());
+            WebResourceLoadStatisticsTelemetry::calculateAndSubmit(*this);
     });
 
     m_dailyTasksTimer.startRepeating(24_h);
@@ -108,12 +109,6 @@
 {
 }
     
-void WebResourceLoadStatisticsStore::classifyResource(ResourceLoadStatistics& resourceStatistic)
-{
-    if (!resourceStatistic.isPrevalentResource && m_resourceLoadStatisticsClassifier.hasPrevalentResourceCharacteristics(resourceStatistic))
-        resourceStatistic.isPrevalentResource = true;
-}
-    
 void WebResourceLoadStatisticsStore::removeDataRecords()
 {
     ASSERT(!RunLoop::isMain());
@@ -121,7 +116,7 @@
     if (!shouldRemoveDataRecords())
         return;
 
-    auto prevalentResourceDomains = coreStore().topPrivatelyControlledDomainsToRemoveWebsiteDataFor();
+    auto prevalentResourceDomains = topPrivatelyControlledDomainsToRemoveWebsiteDataFor();
     if (prevalentResourceDomains.isEmpty())
         return;
     
@@ -130,7 +125,10 @@
     RunLoop::main().dispatch([prevalentResourceDomains = CrossThreadCopier<Vector<String>>::copy(prevalentResourceDomains), this, protectedThis = makeRef(*this)] () mutable {
         WebProcessProxy::deleteWebsiteDataForTopPrivatelyControlledDomainsInAllPersistentDataStores(dataTypesToRemove(), WTFMove(prevalentResourceDomains), m_shouldNotifyPagesWhenDataRecordsWereScanned, [this, protectedThis = WTFMove(protectedThis)](const HashSet<String>& domainsWithDeletedWebsiteData) mutable {
             m_statisticsQueue->dispatch([this, protectedThis = WTFMove(protectedThis), topDomains = CrossThreadCopier<HashSet<String>>::copy(domainsWithDeletedWebsiteData)] () mutable {
-                coreStore().updateStatisticsForRemovedDataRecords(topDomains);
+                for (auto& prevalentResourceDomain : topDomains) {
+                    auto& statistic = ensureResourceStatisticsForPrimaryDomain(prevalentResourceDomain);
+                    ++statistic.dataRecordsRemoved;
+                }
                 setDataRecordsBeingRemoved(false);
             });
         });
@@ -141,9 +139,10 @@
 {
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this)] () {
         if (m_shouldClassifyResourcesBeforeDataRecordsRemoval) {
-            coreStore().processStatistics([this] (ResourceLoadStatistics& resourceStatistic) {
-                classifyResource(resourceStatistic);
-            });
+            for (auto& resourceStatistic : m_resourceStatisticsMap.values()) {
+                if (!resourceStatistic.isPrevalentResource && m_resourceLoadStatisticsClassifier.hasPrevalentResourceCharacteristics(resourceStatistic))
+                    resourceStatistic.isPrevalentResource = true;
+            }
         }
         removeDataRecords();
         
@@ -161,9 +160,9 @@
 {
     ASSERT(!RunLoop::isMain());
 
-    coreStore().mergeStatistics(WTFMove(origins));
+    mergeStatistics(WTFMove(origins));
     // Fire before processing statistics to propagate user interaction as fast as possible to the network process.
-    coreStore().updateCookiePartitioning();
+    updateCookiePartitioning();
     processStatisticsAndDataRecords();
 }
 
@@ -174,7 +173,11 @@
     RunLoop::main().dispatch([this, protectedThis = makeRef(*this)] () mutable {
         WebProcessProxy::topPrivatelyControlledDomainsWithWebsiteData(dataTypesToRemove(), m_shouldNotifyPagesWhenDataRecordsWereScanned, [this, protectedThis = WTFMove(protectedThis)] (HashSet<String>&& topPrivatelyControlledDomainsWithWebsiteData) mutable {
             m_statisticsQueue->dispatch([this, protectedThis = WTFMove(protectedThis), topDomains = CrossThreadCopier<HashSet<String>>::copy(topPrivatelyControlledDomainsWithWebsiteData)] () mutable {
-                coreStore().handleFreshStartWithEmptyOrNoStore(WTFMove(topDomains));
+                for (auto& topPrivatelyControlledDomain : topDomains) {
+                    auto& statistic = ensureResourceStatisticsForPrimaryDomain(topPrivatelyControlledDomain);
+                    statistic.grandfathered = true;
+                }
+                m_endOfGrandfatheringTimestamp = WallTime::now() + m_grandfatheringTime;
             });
         });
     });
@@ -218,15 +221,15 @@
         return;
     }
 
-    coreStore().clearInMemory();
-    coreStore().populateFromDecoder(*decoder);
+    clearInMemory();
+    populateFromDecoder(*decoder);
 
     m_lastStatisticsFileSyncTime = readTime;
 
-    if (coreStore().isEmpty())
+    if (m_resourceStatisticsMap.isEmpty())
         grandfatherExistingWebsiteData();
 
-    coreStore().includeTodayAsOperatingDateIfNecessary();
+    includeTodayAsOperatingDateIfNecessary();
 }
     
 void WebResourceLoadStatisticsStore::refreshFromDisk()
@@ -249,7 +252,7 @@
     if (!decoder)
         return;
 
-    coreStore().populateFromDecoder(*decoder);
+    populateFromDecoder(*decoder);
     m_lastStatisticsFileSyncTime = readTime;
 }
     
@@ -299,11 +302,27 @@
 
     syncWithExistingStatisticsStorageIfNeeded();
 
-    auto encoder = coreStore().createEncoderFromData();
+    auto encoder = createEncoderFromData();
+    RefPtr<SharedBuffer> rawData = encoder->finishEncoding();
+    if (!rawData)
+        return;
 
-    String resourceLog = resourceLogFilePath();
-    writeEncoderToDisk(*encoder, resourceLog);
+    auto statisticsStoragePath = this->statisticsStoragePath();
+    if (!statisticsStoragePath.isEmpty()) {
+        makeAllDirectories(statisticsStoragePath);
+        platformExcludeFromBackup();
+    }
 
+    auto handle = openAndLockFile(resourceLogFilePath(), OpenForWrite);
+    if (handle == invalidPlatformFileHandle)
+        return;
+
+    int64_t writtenBytes = writeToFile(handle, rawData->data(), rawData->size());
+    unlockAndCloseFile(handle);
+
+    if (writtenBytes != static_cast<int64_t>(rawData->size()))
+        RELEASE_LOG_ERROR(ResourceLoadStatistics, "WebResourceLoadStatisticsStore: We only wrote %d out of %d bytes to disk", static_cast<unsigned>(writtenBytes), rawData->size());
+
     m_lastStatisticsFileSyncTime = WallTime::now();
     m_lastStatisticsWriteTime = MonotonicTime::now();
 
@@ -330,31 +349,6 @@
     writeStoreToDisk();
 }
 
-void WebResourceLoadStatisticsStore::writeEncoderToDisk(KeyedEncoder& encoder, const String& path) const
-{
-    ASSERT(!RunLoop::isMain());
-    
-    RefPtr<SharedBuffer> rawData = encoder.finishEncoding();
-    if (!rawData)
-        return;
-
-    auto statisticsStoragePath = this->statisticsStoragePath();
-    if (!statisticsStoragePath.isEmpty()) {
-        makeAllDirectories(statisticsStoragePath);
-        platformExcludeFromBackup();
-    }
-
-    auto handle = openAndLockFile(path, OpenForWrite);
-    if (handle == invalidPlatformFileHandle)
-        return;
-    
-    int64_t writtenBytes = writeToFile(handle, rawData->data(), rawData->size());
-    unlockAndCloseFile(handle);
-
-    if (writtenBytes != static_cast<int64_t>(rawData->size()))
-        RELEASE_LOG_ERROR(ResourceLoadStatistics, "WebResourceLoadStatisticsStore: We only wrote %d out of %d bytes to disk", static_cast<unsigned>(writtenBytes), rawData->size());
-}
-
 void WebResourceLoadStatisticsStore::deleteStoreFromDisk()
 {
     ASSERT(!RunLoop::isMain());
@@ -385,7 +379,7 @@
             refreshFromDisk();
             break;
         case FileMonitor::FileChangeType::Removal:
-            coreStore().clearInMemory();
+            clearInMemory();
             m_statisticsStorageMonitor = nullptr;
             break;
         }
@@ -451,7 +445,7 @@
 {
     ASSERT(RunLoop::isMain());
 
-    coreStore().includeTodayAsOperatingDateIfNecessary();
+    includeTodayAsOperatingDateIfNecessary();
     if (m_shouldSubmitTelemetry)
         submitTelemetry();
 }
@@ -460,7 +454,7 @@
 {
     ASSERT(RunLoop::isMain());
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this)] {
-        WebResourceLoadStatisticsTelemetry::calculateAndSubmit(coreStore());
+        WebResourceLoadStatisticsTelemetry::calculateAndSubmit(*this);
     });
 }
 
@@ -470,11 +464,11 @@
         return;
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), primaryDomainString = primaryDomain(url).isolatedCopy()] {
-        auto& statistics = coreStore().ensureResourceStatisticsForPrimaryDomain(primaryDomainString);
+        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomainString);
         statistics.hadUserInteraction = true;
         statistics.mostRecentUserInteractionTime = WallTime::now();
 
-        coreStore().updateCookiePartitioningForDomains({ primaryDomainString }, { }, ShouldClearFirst::No);
+        updateCookiePartitioningForDomains({ primaryDomainString }, { }, ShouldClearFirst::No);
     });
 }
 
@@ -484,7 +478,7 @@
         return;
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), url = "" {
-        auto& statistics = coreStore().ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
         statistics.hadUserInteraction = false;
         statistics.mostRecentUserInteractionTime = { };
     });
@@ -498,7 +492,8 @@
     }
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), url = "" completionHandler = WTFMove(completionHandler)] () mutable {
-        bool hadUserInteraction = coreStore().hasHadRecentUserInteraction(primaryDomain(url));
+        auto mapEntry = m_resourceStatisticsMap.find(primaryDomain(url));
+        bool hadUserInteraction = mapEntry == m_resourceStatisticsMap.end() ? false: hasHadUnexpiredRecentUserInteraction(mapEntry->value);
         RunLoop::main().dispatch([hadUserInteraction, completionHandler = WTFMove(completionHandler)] {
             completionHandler(hadUserInteraction);
         });
@@ -511,7 +506,7 @@
         return;
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), url = "" {
-        auto& statistics = coreStore().ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
         statistics.isPrevalentResource = true;
     });
 }
@@ -524,9 +519,10 @@
     }
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), url = "" completionHandler = WTFMove(completionHandler)] () mutable {
-        bool prevalentResource = coreStore().isPrevalentResource(primaryDomain(url));
-        RunLoop::main().dispatch([prevalentResource, completionHandler = WTFMove(completionHandler)] {
-            completionHandler(prevalentResource);
+        auto mapEntry = m_resourceStatisticsMap.find(primaryDomain(url));
+        bool isPrevalentResource = mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.isPrevalentResource;
+        RunLoop::main().dispatch([isPrevalentResource, completionHandler = WTFMove(completionHandler)] {
+            completionHandler(isPrevalentResource);
         });
     });
 }
@@ -537,7 +533,7 @@
         return;
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), url = "" {
-        auto& statistics = coreStore().ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
         statistics.isPrevalentResource = false;
     });
 }
@@ -548,7 +544,7 @@
         return;
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), url = "" value] {
-        auto& statistics = coreStore().ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
+        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain(url));
         statistics.grandfathered = value;
     });
 }
@@ -561,9 +557,10 @@
     }
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), completionHandler = WTFMove(completionHandler), url = "" () mutable {
-        bool grandFathered = coreStore().isGrandFathered(primaryDomain(url));
-        RunLoop::main().dispatch([grandFathered, completionHandler = WTFMove(completionHandler)] {
-            completionHandler(grandFathered);
+        auto mapEntry = m_resourceStatisticsMap.find(primaryDomain(url));
+        bool isGrandFathered = mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.grandfathered;
+        RunLoop::main().dispatch([isGrandFathered, completionHandler = WTFMove(completionHandler)] {
+            completionHandler(isGrandFathered);
         });
     });
 }
@@ -574,7 +571,7 @@
         return;
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), primaryTopFrameDomainString = primaryDomain(topFrame).isolatedCopy(), primarySubFrameDomainString = primaryDomain(subframe).isolatedCopy()] {
-        auto& statistics = coreStore().ensureResourceStatisticsForPrimaryDomain(primarySubFrameDomainString);
+        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubFrameDomainString);
         statistics.subframeUnderTopFrameOrigins.add(primaryTopFrameDomainString);
     });
 }
@@ -585,7 +582,7 @@
         return;
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), primaryTopFrameDomainString = primaryDomain(topFrame).isolatedCopy(), primarySubresourceDomainString = primaryDomain(subresource).isolatedCopy()] {
-        auto& statistics = coreStore().ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomainString);
+        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomainString);
         statistics.subresourceUnderTopFrameOrigins.add(primaryTopFrameDomainString);
     });
 }
@@ -596,63 +593,65 @@
         return;
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), primaryRedirectDomainString = primaryDomain(hostNameRedirectedTo).isolatedCopy(), primarySubresourceDomainString = primaryDomain(subresource).isolatedCopy()] {
-        auto& statistics = coreStore().ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomainString);
+        auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomainString);
         statistics.subresourceUniqueRedirectsTo.add(primaryRedirectDomainString);
     });
 }
 
-void WebResourceLoadStatisticsStore::updateCookiePartitioning()
+void WebResourceLoadStatisticsStore::scheduleCookiePartitioningUpdate()
 {
     // Helper function used by testing system. Should only be called from the main thread.
     ASSERT(RunLoop::isMain());
 
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this)] {
-        coreStore().updateCookiePartitioning();
+        updateCookiePartitioning();
     });
 }
 
-void WebResourceLoadStatisticsStore::updateCookiePartitioningForDomains(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst shouldClearFirst)
+void WebResourceLoadStatisticsStore::scheduleCookiePartitioningUpdateForDomains(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst shouldClearFirst)
 {
     // Helper function used by testing system. Should only be called from the main thread.
     ASSERT(RunLoop::isMain());
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this), domainsToRemove = CrossThreadCopier<Vector<String>>::copy(domainsToRemove), domainsToAdd = CrossThreadCopier<Vector<String>>::copy(domainsToAdd), shouldClearFirst] {
-        coreStore().updateCookiePartitioningForDomains(domainsToRemove, domainsToAdd, shouldClearFirst);
+        updateCookiePartitioningForDomains(domainsToRemove, domainsToAdd, shouldClearFirst);
     });
 }
 
-void WebResourceLoadStatisticsStore::clearInMemory()
+void WebResourceLoadStatisticsStore::scheduleClearInMemory()
 {
     ASSERT(RunLoop::isMain());
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this)] {
-        coreStore().clearInMemory();
+        clearInMemory();
     });
 }
 
-void WebResourceLoadStatisticsStore::clearInMemoryAndPersistent()
+void WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent()
 {
     ASSERT(RunLoop::isMain());
     m_statisticsQueue->dispatch([this, protectedThis = makeRef(*this)] {
-        coreStore().clearInMemory();
+        clearInMemory();
         deleteStoreFromDisk();
         grandfatherExistingWebsiteData();
     });
 }
 
-void WebResourceLoadStatisticsStore::clearInMemoryAndPersistent(std::chrono::system_clock::time_point modifiedSince)
+void WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent(std::chrono::system_clock::time_point modifiedSince)
 {
     // For now, be conservative and clear everything regardless of modifiedSince.
     UNUSED_PARAM(modifiedSince);
-    clearInMemoryAndPersistent();
+    scheduleClearInMemoryAndPersistent();
 }
 
 void WebResourceLoadStatisticsStore::setTimeToLiveUserInteraction(std::optional<Seconds> seconds)
 {
-    coreStore().setTimeToLiveUserInteraction(seconds);
+    ASSERT(!seconds || seconds.value() >= 0_s);
+    m_timeToLiveUserInteraction = seconds;
 }
 
 void WebResourceLoadStatisticsStore::setTimeToLiveCookiePartitionFree(Seconds seconds)
 {
-    coreStore().setTimeToLiveCookiePartitionFree(seconds);
+    ASSERT(seconds >= 0_s);
+    m_timeToLiveCookiePartitionFree = seconds;
 }
 
 void WebResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval(Seconds seconds)
@@ -663,7 +662,8 @@
 
 void WebResourceLoadStatisticsStore::setGrandfatheringTime(Seconds seconds)
 {
-    coreStore().setGrandfatheringTime(seconds);
+    ASSERT(seconds >= 0_s);
+    m_grandfatheringTime = seconds;
 }
 
 bool WebResourceLoadStatisticsStore::shouldRemoveDataRecords() const
@@ -682,5 +682,226 @@
     if (m_dataRecordsBeingRemoved)
         m_lastTimeDataRecordsWereRemoved = MonotonicTime::now();
 }
+
+ResourceLoadStatistics& WebResourceLoadStatisticsStore::ensureResourceStatisticsForPrimaryDomain(const String& primaryDomain)
+{
+    ASSERT(!RunLoop::isMain());
+    return m_resourceStatisticsMap.ensure(primaryDomain, [&primaryDomain] {
+        return ResourceLoadStatistics(primaryDomain);
+    }).iterator->value;
+}
+
+std::unique_ptr<KeyedEncoder> WebResourceLoadStatisticsStore::createEncoderFromData() const
+{
+    ASSERT(!RunLoop::isMain());
+    auto encoder = KeyedEncoder::encoder();
+    encoder->encodeUInt32("version", statisticsModelVersion);
+    encoder->encodeDouble("endOfGrandfatheringTimestamp", m_endOfGrandfatheringTimestamp.secondsSinceEpoch().value());
+
+    encoder->encodeObjects("browsingStatistics", m_resourceStatisticsMap.begin(), m_resourceStatisticsMap.end(), [](KeyedEncoder& encoderInner, const auto& origin) {
+        origin.value.encode(encoderInner);
+    });
+
+    encoder->encodeObjects("operatingDates", m_operatingDates.begin(), m_operatingDates.end(), [](KeyedEncoder& encoderInner, WallTime date) {
+        encoderInner.encodeDouble("date", date.secondsSinceEpoch().value());
+    });
+
+    return encoder;
+}
+
+void WebResourceLoadStatisticsStore::populateFromDecoder(KeyedDecoder& decoder)
+{
+    ASSERT(!RunLoop::isMain());
+    if (!m_resourceStatisticsMap.isEmpty())
+        return;
+
+    unsigned versionOnDisk;
+    if (!decoder.decodeUInt32("version", versionOnDisk))
+        return;
+
+    if (versionOnDisk != statisticsModelVersion)
+        return;
+
+    double endOfGrandfatheringTimestamp;
+    if (decoder.decodeDouble("endOfGrandfatheringTimestamp", endOfGrandfatheringTimestamp))
+        m_endOfGrandfatheringTimestamp = WallTime::fromRawSeconds(endOfGrandfatheringTimestamp);
+    else
+        m_endOfGrandfatheringTimestamp = { };
+
+    Vector<ResourceLoadStatistics> loadedStatistics;
+    bool succeeded = decoder.decodeObjects("browsingStatistics", loadedStatistics, [](KeyedDecoder& decoderInner, ResourceLoadStatistics& statistics) {
+        return statistics.decode(decoderInner);
+    });
+
+    if (!succeeded)
+        return;
+
+    Vector<String> prevalentResourceDomainsWithoutUserInteraction;
+    prevalentResourceDomainsWithoutUserInteraction.reserveInitialCapacity(loadedStatistics.size());
+    for (auto& statistics : loadedStatistics) {
+        if (statistics.isPrevalentResource && !statistics.hadUserInteraction) {
+            prevalentResourceDomainsWithoutUserInteraction.uncheckedAppend(statistics.highLevelDomain);
+            statistics.isMarkedForCookiePartitioning = true;
+        }
+        m_resourceStatisticsMap.add(statistics.highLevelDomain, WTFMove(statistics));
+    }
+
+    succeeded = decoder.decodeObjects("operatingDates", m_operatingDates, [](KeyedDecoder& decoder, WallTime& wallTime) {
+        double value;
+        if (!decoder.decodeDouble("date", value))
+            return false;
+
+        wallTime = WallTime::fromRawSeconds(value);
+        return true;
+    });
+
+    if (!succeeded)
+        return;
+
+    updateCookiePartitioningForDomains({ }, prevalentResourceDomainsWithoutUserInteraction, ShouldClearFirst::Yes);
+}
+
+void WebResourceLoadStatisticsStore::clearInMemory()
+{
+    ASSERT(!RunLoop::isMain());
+    m_resourceStatisticsMap.clear();
+    m_operatingDates.clear();
+
+    updateCookiePartitioningForDomains({ }, { }, ShouldClearFirst::Yes);
+}
+
+void WebResourceLoadStatisticsStore::mergeStatistics(Vector<ResourceLoadStatistics>&& statistics)
+{
+    ASSERT(!RunLoop::isMain());
+    for (auto& statistic : statistics) {
+        auto result = m_resourceStatisticsMap.ensure(statistic.highLevelDomain, [&statistic] {
+            return WTFMove(statistic);
+        });
+        if (!result.isNewEntry)
+            result.iterator->value.merge(statistic);
+    }
+}
+
+inline bool WebResourceLoadStatisticsStore::shouldPartitionCookies(const ResourceLoadStatistics& statistic) const
+{
+    return statistic.isPrevalentResource && (!statistic.hadUserInteraction || WallTime::now() > statistic.mostRecentUserInteractionTime + m_timeToLiveCookiePartitionFree);
+}
+
+void WebResourceLoadStatisticsStore::updateCookiePartitioning()
+{
+    ASSERT(!RunLoop::isMain());
+
+    Vector<String> domainsToRemove;
+    Vector<String> domainsToAdd;
+    for (auto& resourceStatistic : m_resourceStatisticsMap.values()) {
+        bool shouldPartition = shouldPartitionCookies(resourceStatistic);
+        if (resourceStatistic.isMarkedForCookiePartitioning && !shouldPartition) {
+            resourceStatistic.isMarkedForCookiePartitioning = false;
+            domainsToRemove.append(resourceStatistic.highLevelDomain);
+        } else if (!resourceStatistic.isMarkedForCookiePartitioning && shouldPartition) {
+            resourceStatistic.isMarkedForCookiePartitioning = true;
+            domainsToAdd.append(resourceStatistic.highLevelDomain);
+        }
+    }
+
+    if (domainsToRemove.isEmpty() && domainsToAdd.isEmpty())
+        return;
+
+    RunLoop::main().dispatch([this, protectedThis = makeRef(*this), domainsToRemove = CrossThreadCopier<Vector<String>>::copy(domainsToRemove), domainsToAdd = CrossThreadCopier<Vector<String>>::copy(domainsToAdd)] () {
+        m_updateCookiePartitioningForDomainsHandler(domainsToRemove, domainsToAdd, ShouldClearFirst::No);
+    });
+}
+
+void WebResourceLoadStatisticsStore::updateCookiePartitioningForDomains(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst shouldClearFirst)
+{
+    ASSERT(!RunLoop::isMain());
+    if (domainsToRemove.isEmpty() && domainsToAdd.isEmpty())
+        return;
+
+    RunLoop::main().dispatch([this, shouldClearFirst, protectedThis = makeRef(*this), domainsToRemove = CrossThreadCopier<Vector<String>>::copy(domainsToRemove), domainsToAdd = CrossThreadCopier<Vector<String>>::copy(domainsToAdd)] () {
+        m_updateCookiePartitioningForDomainsHandler(domainsToRemove, domainsToAdd, shouldClearFirst);
+    });
+
+    if (shouldClearFirst == ShouldClearFirst::Yes) {
+        for (auto& resourceStatistic : m_resourceStatisticsMap.values())
+            resourceStatistic.isMarkedForCookiePartitioning = false;
+    } else {
+        for (auto& domain : domainsToRemove)
+            ensureResourceStatisticsForPrimaryDomain(domain).isMarkedForCookiePartitioning = false;
+    }
+
+    for (auto& domain : domainsToAdd)
+        ensureResourceStatisticsForPrimaryDomain(domain).isMarkedForCookiePartitioning = true;
+}
+
+void WebResourceLoadStatisticsStore::processStatistics(const WTF::Function<void (const ResourceLoadStatistics&)>& processFunction) const
+{
+    ASSERT(!RunLoop::isMain());
+    for (auto& resourceStatistic : m_resourceStatisticsMap.values())
+        processFunction(resourceStatistic);
+}
+
+bool WebResourceLoadStatisticsStore::hasHadUnexpiredRecentUserInteraction(ResourceLoadStatistics& resourceStatistic) const
+{
+    if (resourceStatistic.hadUserInteraction && hasStatisticsExpired(resourceStatistic)) {
+        // Drop privacy sensitive data because we no longer need it.
+        // Set timestamp to 0 so that statistics merge will know
+        // it has been reset as opposed to its default -1.
+        resourceStatistic.mostRecentUserInteractionTime = { };
+        resourceStatistic.hadUserInteraction = false;
+    }
+
+    return resourceStatistic.hadUserInteraction;
+}
+
+Vector<String> WebResourceLoadStatisticsStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor()
+{
+    ASSERT(!RunLoop::isMain());
+
+    bool shouldCheckForGrandfathering = m_endOfGrandfatheringTimestamp > WallTime::now();
+    bool shouldClearGrandfathering = !shouldCheckForGrandfathering && m_endOfGrandfatheringTimestamp;
+
+    if (shouldClearGrandfathering)
+        m_endOfGrandfatheringTimestamp = { };
+
+    Vector<String> prevalentResources;
+    for (auto& statistic : m_resourceStatisticsMap.values()) {
+        if (statistic.isPrevalentResource && !hasHadUnexpiredRecentUserInteraction(statistic) && (!shouldCheckForGrandfathering || !statistic.grandfathered))
+            prevalentResources.append(statistic.highLevelDomain);
+
+        if (shouldClearGrandfathering && statistic.grandfathered)
+            statistic.grandfathered = false;
+    }
+
+    return prevalentResources;
+}
+
+void WebResourceLoadStatisticsStore::includeTodayAsOperatingDateIfNecessary()
+{
+    if (!m_operatingDates.isEmpty() && (WallTime::now() - m_operatingDates.last() < 24_h))
+        return;
+
+    while (m_operatingDates.size() >= operatingDatesWindow)
+        m_operatingDates.removeFirst();
+
+    m_operatingDates.append(WallTime::now());
+}
+
+bool WebResourceLoadStatisticsStore::hasStatisticsExpired(const ResourceLoadStatistics& resourceStatistic) const
+{
+    if (m_operatingDates.size() >= operatingDatesWindow) {
+        if (resourceStatistic.mostRecentUserInteractionTime < m_operatingDates.first())
+            return true;
+    }
+
+    // If we don't meet the real criteria for an expired statistic, check the user
+    // setting for a tighter restriction (mainly for testing).
+    if (m_timeToLiveUserInteraction) {
+        if (WallTime::now() > resourceStatistic.mostRecentUserInteractionTime + m_timeToLiveUserInteraction.value())
+            return true;
+    }
+
+    return false;
+}
     
 } // namespace WebKit

Modified: trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.h (219311 => 219312)


--- trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.h	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsStore.h	2017-07-10 23:00:42 UTC (rev 219312)
@@ -51,7 +51,6 @@
 
 namespace WebKit {
 
-class ResourceLoadStatisticsStore;
 class WebProcessProxy;
 
 enum class ShouldClearFirst;
@@ -88,14 +87,14 @@
     void setSubframeUnderTopFrameOrigin(const WebCore::URL& subframe, const WebCore::URL& topFrame);
     void setSubresourceUnderTopFrameOrigin(const WebCore::URL& subresource, const WebCore::URL& topFrame);
     void setSubresourceUniqueRedirectTo(const WebCore::URL& subresource, const WebCore::URL& hostNameRedirectedTo);
-    void updateCookiePartitioning();
-    void updateCookiePartitioningForDomains(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst);
+    void scheduleCookiePartitioningUpdate();
+    void scheduleCookiePartitioningUpdateForDomains(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst);
     void processStatisticsAndDataRecords();
     void submitTelemetry();
 
-    void clearInMemory();
-    void clearInMemoryAndPersistent();
-    void clearInMemoryAndPersistent(std::chrono::system_clock::time_point modifiedSince);
+    void scheduleClearInMemory();
+    void scheduleClearInMemoryAndPersistent();
+    void scheduleClearInMemoryAndPersistent(std::chrono::system_clock::time_point modifiedSince);
 
     void setTimeToLiveUserInteraction(std::optional<Seconds>);
     void setTimeToLiveCookiePartitionFree(Seconds);
@@ -102,15 +101,13 @@
     void setMinimumTimeBetweenDataRecordsRemoval(Seconds);
     void setGrandfatheringTime(Seconds);
 
+    void processStatistics(const WTF::Function<void (const WebCore::ResourceLoadStatistics&)>&) const;
+
 private:
     WebResourceLoadStatisticsStore(const String&, UpdateCookiePartitioningForDomainsHandler&&);
 
-    ResourceLoadStatisticsStore& coreStore() { return m_resourceLoadStatisticsStore.get(); }
-    const ResourceLoadStatisticsStore& coreStore() const { return m_resourceLoadStatisticsStore.get(); }
-
     void readDataFromDiskIfNeeded();
 
-    void classifyResource(WebCore::ResourceLoadStatistics&);
     void removeDataRecords();
     void startMonitoringStatisticsStorage();
     void stopMonitoringStatisticsStorage();
@@ -125,7 +122,6 @@
 
     void writeStoreToDisk();
     void scheduleOrWriteStoreToDisk();
-    void writeEncoderToDisk(WebCore::KeyedEncoder&, const String& path) const;
     std::unique_ptr<WebCore::KeyedDecoder> createDecoderFromDisk(const String& path) const;
     WallTime statisticsFileModificationTime(const String& label) const;
     void platformExcludeFromBackup() const;
@@ -137,11 +133,24 @@
     bool shouldRemoveDataRecords() const;
     void setDataRecordsBeingRemoved(bool);
 
+    bool shouldPartitionCookies(const WebCore::ResourceLoadStatistics&) const;
+    bool hasStatisticsExpired(const WebCore::ResourceLoadStatistics&) const;
+    bool hasHadUnexpiredRecentUserInteraction(WebCore::ResourceLoadStatistics&) const;
+    void includeTodayAsOperatingDateIfNecessary();
+    Vector<String> topPrivatelyControlledDomainsToRemoveWebsiteDataFor();
+    void updateCookiePartitioning();
+    void updateCookiePartitioningForDomains(const Vector<String>& domainsToRemove, const Vector<String>& domainsToAdd, ShouldClearFirst);
+    void mergeStatistics(Vector<WebCore::ResourceLoadStatistics>&&);
+    WebCore::ResourceLoadStatistics& ensureResourceStatisticsForPrimaryDomain(const String&);
+    std::unique_ptr<WebCore::KeyedEncoder> createEncoderFromData() const;
+    void populateFromDecoder(WebCore::KeyedDecoder&);
+    void clearInMemory();
+
 #if PLATFORM(COCOA)
     void registerUserDefaultsIfNeeded();
 #endif
 
-    Ref<ResourceLoadStatisticsStore> m_resourceLoadStatisticsStore;
+    HashMap<String, WebCore::ResourceLoadStatistics> m_resourceStatisticsMap;
 #if HAVE(CORE_PREDICTION)
     ResourceLoadStatisticsClassifierCocoa m_resourceLoadStatisticsClassifier;
 #else
@@ -149,6 +158,11 @@
 #endif
     Ref<WTF::WorkQueue> m_statisticsQueue;
     std::unique_ptr<WebCore::FileMonitor> m_statisticsStorageMonitor;
+    Deque<WTF::WallTime> m_operatingDates;
+
+    UpdateCookiePartitioningForDomainsHandler m_updateCookiePartitioningForDomainsHandler;
+
+    WallTime m_endOfGrandfatheringTimestamp;
     const String m_statisticsStoragePath;
     WallTime m_lastStatisticsFileSyncTime;
     MonotonicTime m_lastStatisticsWriteTime;
@@ -155,6 +169,9 @@
     RunLoop::Timer<WebResourceLoadStatisticsStore> m_dailyTasksTimer;
     MonotonicTime m_lastTimeDataRecordsWereRemoved;
     Seconds m_minimumTimeBetweenDataRecordsRemoval { 1_h };
+    std::optional<Seconds> m_timeToLiveUserInteraction;
+    Seconds m_timeToLiveCookiePartitionFree { 24_h };
+    Seconds m_grandfatheringTime { 1_h };
     bool m_dataRecordsBeingRemoved { false };
     bool m_didScheduleWrite { false };
     bool m_shouldNotifyPagesWhenDataRecordsWereScanned { false };

Modified: trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsTelemetry.cpp (219311 => 219312)


--- trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsTelemetry.cpp	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsTelemetry.cpp	2017-07-10 23:00:42 UTC (rev 219312)
@@ -26,9 +26,9 @@
 #include "config.h"
 #include "WebResourceLoadStatisticsTelemetry.h"
 
-#include "ResourceLoadStatisticsStore.h"
 #include "WebProcessPool.h"
 #include "WebProcessProxy.h"
+#include "WebResourceLoadStatisticsStore.h"
 #include <WebCore/DiagnosticLoggingKeys.h>
 #include <WebCore/ResourceLoadStatistics.h>
 #include <wtf/MainThread.h>
@@ -52,7 +52,7 @@
     unsigned subresourceUniqueRedirectsTo;
 };
 
-static Vector<PrevalentResourceTelemetry> sortedPrevalentResourceTelemetry(const ResourceLoadStatisticsStore& store)
+static Vector<PrevalentResourceTelemetry> sortedPrevalentResourceTelemetry(const WebResourceLoadStatisticsStore& store)
 {
     ASSERT(!RunLoop::isMain());
     Vector<PrevalentResourceTelemetry> sorted;
@@ -224,7 +224,7 @@
     notifyPages(sortedPrevalentResources.size(), totalNumberOfPrevalentResourcesWithUserInteraction, median(sortedPrevalentResourcesWithoutUserInteraction, 0, 2, subframeUnderTopFrameOriginsGetter));
 }
     
-void WebResourceLoadStatisticsTelemetry::calculateAndSubmit(const ResourceLoadStatisticsStore& resourceLoadStatisticsStore)
+void WebResourceLoadStatisticsTelemetry::calculateAndSubmit(const WebResourceLoadStatisticsStore& resourceLoadStatisticsStore)
 {
     ASSERT(!RunLoop::isMain());
     

Modified: trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsTelemetry.h (219311 => 219312)


--- trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsTelemetry.h	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/UIProcess/WebResourceLoadStatisticsTelemetry.h	2017-07-10 23:00:42 UTC (rev 219312)
@@ -29,11 +29,11 @@
 
 namespace WebKit {
 
-class ResourceLoadStatisticsStore;
+class WebResourceLoadStatisticsStore;
 
 namespace WebResourceLoadStatisticsTelemetry {
     
-void calculateAndSubmit(const ResourceLoadStatisticsStore&);
+void calculateAndSubmit(const WebResourceLoadStatisticsStore&);
 void setNotifyPagesWhenTelemetryWasCaptured(bool);
     
 }

Modified: trunk/Source/WebKit2/UIProcess/WebsiteData/WebsiteDataStore.cpp (219311 => 219312)


--- trunk/Source/WebKit2/UIProcess/WebsiteData/WebsiteDataStore.cpp	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/UIProcess/WebsiteData/WebsiteDataStore.cpp	2017-07-10 23:00:42 UTC (rev 219312)
@@ -814,7 +814,7 @@
 #endif
 
     if (dataTypes.contains(WebsiteDataType::ResourceLoadStatistics) && m_resourceLoadStatistics)
-        m_resourceLoadStatistics->clearInMemoryAndPersistent(modifiedSince);
+        m_resourceLoadStatistics->scheduleClearInMemoryAndPersistent(modifiedSince);
 
     // There's a chance that we don't have any pending callbacks. If so, we want to dispatch the completion handler right away.
     callbackAggregator->callIfNeeded();
@@ -1083,7 +1083,7 @@
 #endif
 
     if (dataTypes.contains(WebsiteDataType::ResourceLoadStatistics) && m_resourceLoadStatistics)
-        m_resourceLoadStatistics->clearInMemoryAndPersistent();
+        m_resourceLoadStatistics->scheduleClearInMemoryAndPersistent();
 
     // There's a chance that we don't have any pending callbacks. If so, we want to dispatch the completion handler right away.
     callbackAggregator->callIfNeeded();

Modified: trunk/Source/WebKit2/WebKit2.xcodeproj/project.pbxproj (219311 => 219312)


--- trunk/Source/WebKit2/WebKit2.xcodeproj/project.pbxproj	2017-07-10 23:00:02 UTC (rev 219311)
+++ trunk/Source/WebKit2/WebKit2.xcodeproj/project.pbxproj	2017-07-10 23:00:42 UTC (rev 219312)
@@ -1355,7 +1355,6 @@
 		839902031BE9A02B000F3653 /* NetworkLoad.h in Headers */ = {isa = PBXBuildFile; fileRef = 839901FE1BE9A01B000F3653 /* NetworkLoad.h */; };
 		839A2F311E2067450039057E /* HighPerformanceGraphicsUsageSampler.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 839A2F2F1E2067390039057E /* HighPerformanceGraphicsUsageSampler.cpp */; };
 		839A2F321E2067450039057E /* HighPerformanceGraphicsUsageSampler.h in Headers */ = {isa = PBXBuildFile; fileRef = 839A2F301E2067390039057E /* HighPerformanceGraphicsUsageSampler.h */; };
-		83AFDC0A1F06CEEA00472815 /* ResourceLoadStatisticsStore.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 83AFDC081F06CEE000472815 /* ResourceLoadStatisticsStore.cpp */; };
 		83BDCCB91AC5FDB6003F6441 /* NetworkCacheStatistics.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 83BDCCB81AC5FDB6003F6441 /* NetworkCacheStatistics.cpp */; };
 		83BFAC421D96137C00433490 /* BlobDownloadClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 83BFAC401D96136000433490 /* BlobDownloadClient.h */; };
 		83BFAC431D96137C00433490 /* BlobDownloadClient.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 83BFAC411D96136000433490 /* BlobDownloadClient.cpp */; };
@@ -3652,8 +3651,6 @@
 		839901FF1BE9A01B000F3653 /* NetworkLoad.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = NetworkLoad.cpp; path = NetworkProcess/NetworkLoad.cpp; sourceTree = "<group>"; };
 		839A2F2F1E2067390039057E /* HighPerformanceGraphicsUsageSampler.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HighPerformanceGraphicsUsageSampler.cpp; sourceTree = "<group>"; };
 		839A2F301E2067390039057E /* HighPerformanceGraphicsUsageSampler.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HighPerformanceGraphicsUsageSampler.h; sourceTree = "<group>"; };
-		83AFDC081F06CEE000472815 /* ResourceLoadStatisticsStore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ResourceLoadStatisticsStore.cpp; sourceTree = "<group>"; };
-		83AFDC091F06CEE000472815 /* ResourceLoadStatisticsStore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ResourceLoadStatisticsStore.h; sourceTree = "<group>"; };
 		83BDCCB81AC5FDB6003F6441 /* NetworkCacheStatistics.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkCacheStatistics.cpp; sourceTree = "<group>"; };
 		83BFAC401D96136000433490 /* BlobDownloadClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = BlobDownloadClient.h; path = NetworkProcess/Downloads/BlobDownloadClient.h; sourceTree = "<group>"; };
 		83BFAC411D96136000433490 /* BlobDownloadClient.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = BlobDownloadClient.cpp; path = NetworkProcess/Downloads/BlobDownloadClient.cpp; sourceTree = "<group>"; };
@@ -4703,8 +4700,6 @@
 				1A1D8BA01731A36300141DA4 /* LocalStorageDatabase.h */,
 				1A8C728A1738477C000A6554 /* LocalStorageDatabaseTracker.cpp */,
 				1A8C728B1738477C000A6554 /* LocalStorageDatabaseTracker.h */,
-				83AFDC081F06CEE000472815 /* ResourceLoadStatisticsStore.cpp */,
-				83AFDC091F06CEE000472815 /* ResourceLoadStatisticsStore.h */,
 				1A44B95916B73F9F00B7BBD8 /* StorageManager.cpp */,
 				1A44B95A16B73F9F00B7BBD8 /* StorageManager.h */,
 				1AB31A9316BC65AB00F6DBC9 /* StorageManager.messages.in */,
@@ -9948,7 +9943,6 @@
 				7A821F4E1E2F67A800604577 /* LegacyCustomProtocolManagerClient.mm in Sources */,
 				5C14271D1C23F8CF00D41183 /* LegacyCustomProtocolManagerCocoa.mm in Sources */,
 				2984F588164BA095004BC0C6 /* LegacyCustomProtocolManagerMessageReceiver.cpp in Sources */,
-				83AFDC0A1F06CEEA00472815 /* ResourceLoadStatisticsStore.cpp in Sources */,
 				7A821F4A1E2F65E900604577 /* LegacyCustomProtocolManagerProxy.cpp in Sources */,
 				2984F57C164B915F004BC0C6 /* LegacyCustomProtocolManagerProxyMessageReceiver.cpp in Sources */,
 				1AFDE65D1954E8D500C48FFA /* LegacySessionStateCoding.cpp in Sources */,
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to