Title: [182935] trunk/Source/WebCore
- Revision
- 182935
- Author
- achristen...@apple.com
- Date
- 2015-04-16 18:12:51 -0700 (Thu, 16 Apr 2015)
Log Message
Use less memory when compiling content extensions
https://bugs.webkit.org/show_bug.cgi?id=143857
Reviewed by Benjamin Poulain.
When compiling a content extension, we convert the rule list into several intermediate forms:
1) A String.
2) A JSValue from JSONParse in loadEncodedRules.
3) A Vector of ContentExtensionRules.
4) A CombinedURLFilters object representing the pieces of the regular expressions from the triggers.
5) A Vector of NFAs.
6) A DFA for each NFA.
7) A Vector of DFABytecode.
Each one of these contains all the information contained in the content extension,
so we do not need to keep them all in memory at the same time like we are doing now.
When we are done with one, we can free that memory to greatly reduce the maximum memory usage while compiling.
The next step will be to reduce the copies of the original JSON String and to generate NFAs one at a time.
* contentextensions/CombinedURLFilters.cpp:
(WebCore::ContentExtensions::CombinedURLFilters::clear):
* contentextensions/CombinedURLFilters.h:
* contentextensions/ContentExtensionCompiler.cpp:
(WebCore::ContentExtensions::compileRuleList):
Clear structures when finished using them.
Modified Paths
Diff
Modified: trunk/Source/WebCore/ChangeLog (182934 => 182935)
--- trunk/Source/WebCore/ChangeLog 2015-04-17 01:03:55 UTC (rev 182934)
+++ trunk/Source/WebCore/ChangeLog 2015-04-17 01:12:51 UTC (rev 182935)
@@ -1,3 +1,32 @@
+2015-04-16 Alex Christensen <achristen...@webkit.org>
+
+ Use less memory when compiling content extensions
+ https://bugs.webkit.org/show_bug.cgi?id=143857
+
+ Reviewed by Benjamin Poulain.
+
+ When compiling a content extension, we convert the rule list into several intermediate forms:
+
+ 1) A String.
+ 2) A JSValue from JSONParse in loadEncodedRules.
+ 3) A Vector of ContentExtensionRules.
+ 4) A CombinedURLFilters object representing the pieces of the regular expressions from the triggers.
+ 5) A Vector of NFAs.
+ 6) A DFA for each NFA.
+ 7) A Vector of DFABytecode.
+
+ Each one of these contains all the information contained in the content extension,
+ so we do not need to keep them all in memory at the same time like we are doing now.
+ When we are done with one, we can free that memory to greatly reduce the maximum memory usage while compiling.
+ The next step will be to reduce the copies of the original JSON String and to generate NFAs one at a time.
+
+ * contentextensions/CombinedURLFilters.cpp:
+ (WebCore::ContentExtensions::CombinedURLFilters::clear):
+ * contentextensions/CombinedURLFilters.h:
+ * contentextensions/ContentExtensionCompiler.cpp:
+ (WebCore::ContentExtensions::compileRuleList):
+ Clear structures when finished using them.
+
2015-04-16 Brady Eidson <beid...@apple.com>
Compiling a content extension fails when user's home directory is on a different volume from /var/tmp.
Modified: trunk/Source/WebCore/contentextensions/CombinedURLFilters.cpp (182934 => 182935)
--- trunk/Source/WebCore/contentextensions/CombinedURLFilters.cpp 2015-04-17 01:03:55 UTC (rev 182934)
+++ trunk/Source/WebCore/contentextensions/CombinedURLFilters.cpp 2015-04-17 01:12:51 UTC (rev 182935)
@@ -52,6 +52,11 @@
{
}
+void CombinedURLFilters::clear()
+{
+ m_prefixTreeRoot = std::make_unique<PrefixTreeVertex>();
+}
+
void CombinedURLFilters::addPattern(uint64_t actionId, const Vector<Term>& pattern)
{
ASSERT_WITH_MESSAGE(!pattern.isEmpty(), "The parser should have excluded empty patterns before reaching CombinedURLFilters.");
Modified: trunk/Source/WebCore/contentextensions/CombinedURLFilters.h (182934 => 182935)
--- trunk/Source/WebCore/contentextensions/CombinedURLFilters.h 2015-04-17 01:03:55 UTC (rev 182934)
+++ trunk/Source/WebCore/contentextensions/CombinedURLFilters.h 2015-04-17 01:12:51 UTC (rev 182935)
@@ -45,6 +45,7 @@
void addPattern(uint64_t patternId, const Vector<Term>& pattern);
Vector<NFA> createNFAs() const;
+ void clear();
private:
std::unique_ptr<PrefixTreeVertex> m_prefixTreeRoot;
Modified: trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp (182934 => 182935)
--- trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp 2015-04-17 01:03:55 UTC (rev 182934)
+++ trunk/Source/WebCore/contentextensions/ContentExtensionCompiler.cpp 2015-04-17 01:12:51 UTC (rev 182935)
@@ -133,6 +133,9 @@
Vector<SerializedActionByte> actions;
Vector<unsigned> actionLocations = serializeActions(parsedRuleList, actions);
+ client.writeActions(WTF::move(actions));
+ actions.clear();
+
HashSet<uint64_t, DefaultHash<uint64_t>::Hash, WTF::UnsignedWithZeroKeyHashTraits<uint64_t>> universalActionLocations;
CombinedURLFilters combinedURLFilters;
@@ -161,6 +164,8 @@
if (contentExtensionRule.action().type() == ActionType::IgnorePreviousRules)
ignorePreviousRulesSeen = true;
}
+ parsedRuleList.clear();
+ actionLocations.clear();
#if CONTENT_EXTENSIONS_PERFORMANCE_REPORTING
double patternPartitioningEnd = monotonicallyIncreasingTime();
@@ -172,6 +177,7 @@
#endif
Vector<NFA> nfas = combinedURLFilters.createNFAs();
+ combinedURLFilters.clear();
if (!nfas.size() && universalActionLocations.size())
nfas.append(NFA());
@@ -226,6 +232,7 @@
DFABytecodeCompiler compiler(dfa, bytecode);
compiler.compile();
}
+ universalActionLocations.clear();
#if CONTENT_EXTENSIONS_PERFORMANCE_REPORTING
double totalNFAToByteCodeBuildTimeEnd = monotonicallyIncreasingTime();
@@ -233,9 +240,9 @@
dataLogF(" Bytecode size %zu\n", bytecode.size());
dataLogF(" DFA count %zu\n", nfas.size());
#endif
+ nfas.clear();
client.writeBytecode(WTF::move(bytecode));
- client.writeActions(WTF::move(actions));
return { };
}
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
