Diff
Modified: trunk/Source/_javascript_Core/CMakeLists.txt (184414 => 184415)
--- trunk/Source/_javascript_Core/CMakeLists.txt 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/CMakeLists.txt 2015-05-15 21:11:39 UTC (rev 184415)
@@ -227,7 +227,7 @@
dfg/DFGSpeculativeJIT64.cpp
dfg/DFGStackLayoutPhase.cpp
dfg/DFGStaticExecutionCountEstimationPhase.cpp
- dfg/DFGStoreBarrierElisionPhase.cpp
+ dfg/DFGStoreBarrierInsertionPhase.cpp
dfg/DFGStrengthReductionPhase.cpp
dfg/DFGStructureAbstractValue.cpp
dfg/DFGStructureRegistrationPhase.cpp
Modified: trunk/Source/_javascript_Core/ChangeLog (184414 => 184415)
--- trunk/Source/_javascript_Core/ChangeLog 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/ChangeLog 2015-05-15 21:11:39 UTC (rev 184415)
@@ -1,3 +1,61 @@
+2015-05-14 Filip Pizlo <[email protected]>
+
+ Insert store barriers late so that IR transformations don't have to worry about them
+ https://bugs.webkit.org/show_bug.cgi?id=145015
+
+ Reviewed by Geoffrey Garen.
+
+ We have had three kinds of bugs with store barriers. For the sake of discussion we say
+ that a store barrier is needed when we have something like:
+
+ base.field = value
+
+ - We sometimes fail to realize that we could remove a barrier when value is a non-cell.
+ This might happen if we prove value to be a non-cell even though in the FixupPhase it
+ wasn't predicted non-cell.
+
+ - We sometimes have a barrier in the wrong place after object allocation sinking. We
+ might sink an allocation to just above the store, but that puts it just after the
+ StoreBarrier that FixupPhase inserted.
+
+ - We don't remove redundant barriers across basic blocks.
+
+ This comprehensively fixes these issues by doing store barrier insertion late, and
+ removing the store barrier elision phase. Store barrier insertion uses an epoch-based
+ algorithm to determine when stores need barriers. Briefly, a barrier is not needed if
+ base is in the current GC epoch (i.e. was the last object that we allocated or had a
+ barrier since last GC) or if base has a newer GC epoch than value (i.e. value would have
+ always been allocated before base). We do conservative things when merging epoch state
+ between basic blocks, and we only do such inter-block removal in the FTL. FTL also
+ queries AI to determine what type we've proved about value, and avoids barriers when
+ value is not a cell. FixupPhase still inserts type checks on some stores, to maximize
+ the likelihood that this AI-based removal is effective.
+
+ * CMakeLists.txt:
+ * _javascript_Core.vcxproj/_javascript_Core.vcxproj:
+ * _javascript_Core.xcodeproj/project.pbxproj:
+ * dfg/DFGBlockMap.h:
+ (JSC::DFG::BlockMap::at):
+ * dfg/DFGConstantFoldingPhase.cpp:
+ (JSC::DFG::ConstantFoldingPhase::emitPutByOffset):
+ * dfg/DFGEpoch.h:
+ (JSC::DFG::Epoch::operator<):
+ (JSC::DFG::Epoch::operator>):
+ (JSC::DFG::Epoch::operator<=):
+ (JSC::DFG::Epoch::operator>=):
+ * dfg/DFGFixupPhase.cpp:
+ (JSC::DFG::FixupPhase::fixupNode):
+ (JSC::DFG::FixupPhase::speculateForBarrier):
+ (JSC::DFG::FixupPhase::insertStoreBarrier): Deleted.
+ * dfg/DFGPlan.cpp:
+ (JSC::DFG::Plan::compileInThreadImpl):
+ * dfg/DFGStoreBarrierElisionPhase.cpp: Removed.
+ * dfg/DFGStoreBarrierElisionPhase.h: Removed.
+ * dfg/DFGStoreBarrierInsertionPhase.cpp: Added.
+ (JSC::DFG::performFastStoreBarrierInsertion):
+ (JSC::DFG::performGlobalStoreBarrierInsertion):
+ * dfg/DFGStoreBarrierInsertionPhase.h: Added.
+
2015-05-15 Benjamin Poulain <[email protected]>
[ARM64] Do not fail branchConvertDoubleToInt32 when the result is zero and not negative zero
Modified: trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj (184414 => 184415)
--- trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj 2015-05-15 21:11:39 UTC (rev 184415)
@@ -467,7 +467,7 @@
<ClCompile Include="..\dfg\DFGSSALoweringPhase.cpp" />
<ClCompile Include="..\dfg\DFGStackLayoutPhase.cpp" />
<ClCompile Include="..\dfg\DFGStaticExecutionCountEstimationPhase.cpp" />
- <ClCompile Include="..\dfg\DFGStoreBarrierElisionPhase.cpp" />
+ <ClCompile Include="..\dfg\DFGStoreBarrierInsertionPhase.cpp" />
<ClCompile Include="..\dfg\DFGStrengthReductionPhase.cpp" />
<ClCompile Include="..\dfg\DFGStructureAbstractValue.cpp" />
<ClCompile Include="..\dfg\DFGStructureRegistrationPhase.cpp" />
@@ -1173,7 +1173,7 @@
<ClInclude Include="..\dfg\DFGSSALoweringPhase.h" />
<ClInclude Include="..\dfg\DFGStackLayoutPhase.h" />
<ClInclude Include="..\dfg\DFGStaticExecutionCountEstimationPhase.h" />
- <ClInclude Include="..\dfg\DFGStoreBarrierElisionPhase.h" />
+ <ClInclude Include="..\dfg\DFGStoreBarrierInsertionPhase.h" />
<ClInclude Include="..\dfg\DFGStrengthReductionPhase.h" />
<ClInclude Include="..\dfg\DFGStructureAbstractValue.h" />
<ClInclude Include="..\dfg\DFGStructureClobberState.h" />
Modified: trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj (184414 => 184415)
--- trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj 2015-05-15 21:11:39 UTC (rev 184415)
@@ -462,6 +462,8 @@
0F9D339B1803ADB70073C2BC /* FTLStackMaps.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9D33991803ADB70073C2BC /* FTLStackMaps.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9D36941AE9CC33000D4DFB /* DFGCleanUpPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9D36921AE9CC33000D4DFB /* DFGCleanUpPhase.cpp */; };
0F9D36951AE9CC33000D4DFB /* DFGCleanUpPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9D36931AE9CC33000D4DFB /* DFGCleanUpPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
+ 0F9E32631B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9E32611B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.cpp */; };
+ 0F9E32641B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9E32621B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9FB4F417FCB91700CB67F8 /* DFGStackLayoutPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */; };
0F9FB4F517FCB91700CB67F8 /* DFGStackLayoutPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F9FC8C314E1B5FE00D52AE0 /* PolymorphicPutByIdList.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F9FC8BF14E1B5FB00D52AE0 /* PolymorphicPutByIdList.cpp */; };
@@ -928,8 +930,6 @@
2AAD964A18569417001F93BE /* RecursiveAllocationScope.h in Headers */ = {isa = PBXBuildFile; fileRef = 2AAD964918569417001F93BE /* RecursiveAllocationScope.h */; };
2AC922BB18A16182003CE0FB /* FTLDWARFDebugLineInfo.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 2AC922B918A16182003CE0FB /* FTLDWARFDebugLineInfo.cpp */; };
2AC922BC18A16182003CE0FB /* FTLDWARFDebugLineInfo.h in Headers */ = {isa = PBXBuildFile; fileRef = 2AC922BA18A16182003CE0FB /* FTLDWARFDebugLineInfo.h */; };
- 2ACCF3DE185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 2ACCF3DC185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp */; };
- 2ACCF3DF185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h in Headers */ = {isa = PBXBuildFile; fileRef = 2ACCF3DD185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h */; };
2AD2EDFB19799E38004D6478 /* EnumerationMode.h in Headers */ = {isa = PBXBuildFile; fileRef = 2AD2EDFA19799E38004D6478 /* EnumerationMode.h */; settings = {ATTRIBUTES = (Private, ); }; };
2AD8932B17E3868F00668276 /* HeapIterationScope.h in Headers */ = {isa = PBXBuildFile; fileRef = 2AD8932917E3868F00668276 /* HeapIterationScope.h */; };
2ADFA26318EF3540004F9FCC /* GCLogging.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 2ADFA26218EF3540004F9FCC /* GCLogging.cpp */; };
@@ -2201,6 +2201,8 @@
0F9D33991803ADB70073C2BC /* FTLStackMaps.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLStackMaps.h; path = ftl/FTLStackMaps.h; sourceTree = "<group>"; };
0F9D36921AE9CC33000D4DFB /* DFGCleanUpPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGCleanUpPhase.cpp; path = dfg/DFGCleanUpPhase.cpp; sourceTree = "<group>"; };
0F9D36931AE9CC33000D4DFB /* DFGCleanUpPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGCleanUpPhase.h; path = dfg/DFGCleanUpPhase.h; sourceTree = "<group>"; };
+ 0F9E32611B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStoreBarrierInsertionPhase.cpp; path = dfg/DFGStoreBarrierInsertionPhase.cpp; sourceTree = "<group>"; };
+ 0F9E32621B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStoreBarrierInsertionPhase.h; path = dfg/DFGStoreBarrierInsertionPhase.h; sourceTree = "<group>"; };
0F9FB4F217FCB91700CB67F8 /* DFGStackLayoutPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStackLayoutPhase.cpp; path = dfg/DFGStackLayoutPhase.cpp; sourceTree = "<group>"; };
0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStackLayoutPhase.h; path = dfg/DFGStackLayoutPhase.h; sourceTree = "<group>"; };
0F9FC8BF14E1B5FB00D52AE0 /* PolymorphicPutByIdList.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PolymorphicPutByIdList.cpp; sourceTree = "<group>"; };
@@ -2637,8 +2639,6 @@
2AAD964918569417001F93BE /* RecursiveAllocationScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RecursiveAllocationScope.h; sourceTree = "<group>"; };
2AC922B918A16182003CE0FB /* FTLDWARFDebugLineInfo.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = FTLDWARFDebugLineInfo.cpp; path = ftl/FTLDWARFDebugLineInfo.cpp; sourceTree = "<group>"; };
2AC922BA18A16182003CE0FB /* FTLDWARFDebugLineInfo.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = FTLDWARFDebugLineInfo.h; path = ftl/FTLDWARFDebugLineInfo.h; sourceTree = "<group>"; };
- 2ACCF3DC185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGStoreBarrierElisionPhase.cpp; path = dfg/DFGStoreBarrierElisionPhase.cpp; sourceTree = "<group>"; };
- 2ACCF3DD185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGStoreBarrierElisionPhase.h; path = dfg/DFGStoreBarrierElisionPhase.h; sourceTree = "<group>"; };
2AD2EDFA19799E38004D6478 /* EnumerationMode.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = EnumerationMode.h; sourceTree = "<group>"; };
2AD8932917E3868F00668276 /* HeapIterationScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HeapIterationScope.h; sourceTree = "<group>"; };
2ADFA26218EF3540004F9FCC /* GCLogging.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = GCLogging.cpp; sourceTree = "<group>"; };
@@ -5089,8 +5089,8 @@
0F9FB4F317FCB91700CB67F8 /* DFGStackLayoutPhase.h */,
0F4F29DD18B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.cpp */,
0F4F29DE18B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.h */,
- 2ACCF3DC185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp */,
- 2ACCF3DD185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h */,
+ 0F9E32611B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.cpp */,
+ 0F9E32621B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.h */,
0FC20CB31852E2C600C9E954 /* DFGStrengthReductionPhase.cpp */,
0FC20CB41852E2C600C9E954 /* DFGStrengthReductionPhase.h */,
0F893BDA1936E23C001211F4 /* DFGStructureAbstractValue.cpp */,
@@ -5894,8 +5894,8 @@
A7D89D0017A0B8CC00773AD8 /* DFGSSAConversionPhase.h in Headers */,
0FC20CBA18556A3500C9E954 /* DFGSSALoweringPhase.h in Headers */,
0F9FB4F517FCB91700CB67F8 /* DFGStackLayoutPhase.h in Headers */,
+ 0F9E32641B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.h in Headers */,
0F4F29E018B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.h in Headers */,
- 2ACCF3DF185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.h in Headers */,
0FC20CB61852E2C600C9E954 /* DFGStrengthReductionPhase.h in Headers */,
0F63947815DCE34B006A597C /* DFGStructureAbstractValue.h in Headers */,
0F2FCCFF18A60070001A27F8 /* DFGThreadData.h in Headers */,
@@ -7181,7 +7181,6 @@
0F9FB4F417FCB91700CB67F8 /* DFGStackLayoutPhase.cpp in Sources */,
0F4F29DF18B6AD1C0057BC15 /* DFGStaticExecutionCountEstimationPhase.cpp in Sources */,
0FE7211D193B9C590031F6ED /* DFGTransition.cpp in Sources */,
- 2ACCF3DE185FE26B0083E2AD /* DFGStoreBarrierElisionPhase.cpp in Sources */,
0FC20CB51852E2C600C9E954 /* DFGStrengthReductionPhase.cpp in Sources */,
0F2FCCFE18A60070001A27F8 /* DFGThreadData.cpp in Sources */,
0FC097A1146B28CA00CF2442 /* DFGThunks.cpp in Sources */,
@@ -7202,6 +7201,7 @@
0F2BDC5115228FFD00CD8910 /* DFGVariableEvent.cpp in Sources */,
0F2BDC4A1522809A00CD8910 /* DFGVariableEventStream.cpp in Sources */,
0FFFC95F14EF90BB00C72532 /* DFGVirtualRegisterAllocationPhase.cpp in Sources */,
+ 0F9E32631B05AB0400801ED5 /* DFGStoreBarrierInsertionPhase.cpp in Sources */,
0FC97F4118202119002C9B26 /* DFGWatchpointCollectionPhase.cpp in Sources */,
0FDB2CE7174830A2007B3C1B /* DFGWorklist.cpp in Sources */,
0F9D3370165DBB90005AD387 /* Disassembler.cpp in Sources */,
Modified: trunk/Source/_javascript_Core/dfg/DFGBlockMap.h (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGBlockMap.h 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGBlockMap.h 2015-05-15 21:11:39 UTC (rev 184415)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2014 Apple Inc. All rights reserved.
+ * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -58,6 +58,26 @@
return m_vector[blockIndex];
}
+ T& at(BlockIndex blockIndex)
+ {
+ return m_vector[blockIndex];
+ }
+
+ const T& at(BlockIndex blockIndex) const
+ {
+ return m_vector[blockIndex];
+ }
+
+ T& at(BasicBlock* block)
+ {
+ return m_vector[block->index];
+ }
+
+ const T& at(BasicBlock* block) const
+ {
+ return m_vector[block->index];
+ }
+
T& operator[](BlockIndex blockIndex)
{
return m_vector[blockIndex];
Modified: trunk/Source/_javascript_Core/dfg/DFGConstantFoldingPhase.cpp (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGConstantFoldingPhase.cpp 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGConstantFoldingPhase.cpp 2015-05-15 21:11:39 UTC (rev 184415)
@@ -597,7 +597,6 @@
Node* allocatePropertyStorage = m_insertionSet.insertNode(
indexInBlock, SpecNone, AllocatePropertyStorage,
origin, OpInfo(transition), childEdge);
- m_insertionSet.insertNode(indexInBlock, SpecNone, StoreBarrier, origin, Edge(node->child1().node(), KnownCellUse));
propertyStorage = Edge(allocatePropertyStorage);
} else {
ASSERT(variant.oldStructureForTransition()->outOfLineCapacity());
@@ -609,7 +608,6 @@
OpInfo(transition), childEdge,
Edge(m_insertionSet.insertNode(
indexInBlock, SpecNone, GetButterfly, origin, childEdge)));
- m_insertionSet.insertNode(indexInBlock, SpecNone, StoreBarrier, origin, Edge(node->child1().node(), KnownCellUse));
propertyStorage = Edge(reallocatePropertyStorage);
}
Modified: trunk/Source/_javascript_Core/dfg/DFGEpoch.h (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGEpoch.h 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGEpoch.h 2015-05-15 21:11:39 UTC (rev 184415)
@@ -87,6 +87,26 @@
return !(*this == other);
}
+ bool operator<(const Epoch& other) const
+ {
+ return m_epoch < other.m_epoch;
+ }
+
+ bool operator>(const Epoch& other) const
+ {
+ return other < *this;
+ }
+
+ bool operator<=(const Epoch& other) const
+ {
+ return !(*this > other);
+ }
+
+ bool operator>=(const Epoch& other) const
+ {
+ return !(*this < other);
+ }
+
void dump(PrintStream&) const;
private:
Modified: trunk/Source/_javascript_Core/dfg/DFGFixupPhase.cpp (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGFixupPhase.cpp 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGFixupPhase.cpp 2015-05-15 21:11:39 UTC (rev 184415)
@@ -722,7 +722,7 @@
case Array::SlowPutArrayStorage:
fixEdge<KnownCellUse>(child1);
fixEdge<Int32Use>(child2);
- insertStoreBarrier(m_indexInBlock, child1, child3);
+ speculateForBarrier(child3);
break;
default:
fixEdge<KnownCellUse>(child1);
@@ -760,7 +760,7 @@
break;
case Array::Contiguous:
case Array::ArrayStorage:
- insertStoreBarrier(m_indexInBlock, node->child1(), node->child2());
+ speculateForBarrier(node->child2());
break;
default:
break;
@@ -922,7 +922,6 @@
case PutStructure: {
fixEdge<KnownCellUse>(node->child1());
- insertStoreBarrier(m_indexInBlock, node->child1());
break;
}
@@ -935,7 +934,7 @@
case PutClosureVar:
case PutToArguments: {
fixEdge<KnownCellUse>(node->child1());
- insertStoreBarrier(m_indexInBlock, node->child1(), node->child2());
+ speculateForBarrier(node->child2());
break;
}
@@ -950,7 +949,6 @@
case AllocatePropertyStorage:
case ReallocatePropertyStorage: {
fixEdge<KnownCellUse>(node->child1());
- insertStoreBarrier(m_indexInBlock + 1, node->child1());
break;
}
@@ -986,7 +984,7 @@
case PutByIdFlush:
case PutByIdDirect: {
fixEdge<CellUse>(node->child1());
- insertStoreBarrier(m_indexInBlock, node->child1(), node->child2());
+ speculateForBarrier(node->child2());
break;
}
@@ -1029,13 +1027,13 @@
if (!node->child1()->hasStorageResult())
fixEdge<KnownCellUse>(node->child1());
fixEdge<KnownCellUse>(node->child2());
- insertStoreBarrier(m_indexInBlock, node->child2(), node->child3());
+ speculateForBarrier(node->child3());
break;
}
case MultiPutByOffset: {
fixEdge<CellUse>(node->child1());
- insertStoreBarrier(m_indexInBlock, node->child1(), node->child2());
+ speculateForBarrier(node->child2());
break;
}
@@ -1115,16 +1113,16 @@
case PutStack:
case KillStack:
case GetStack:
+ case StoreBarrier:
// These are just nodes that we don't currently expect to see during fixup.
// If we ever wanted to insert them prior to fixup, then we just have to create
// fixup rules for them.
- RELEASE_ASSERT_NOT_REACHED();
+ DFG_CRASH(m_graph, node, "Unexpected node during fixup");
break;
case PutGlobalVar: {
fixEdge<CellUse>(node->child1());
- insertStoreBarrier(
- m_indexInBlock, node->child1(), node->child2());
+ speculateForBarrier(node->child2());
break;
}
@@ -1298,7 +1296,6 @@
case Unreachable:
case ExtractOSREntryLocal:
case LoopHint:
- case StoreBarrier:
case MovHint:
case ZombieHint:
case BottomValue:
@@ -1762,37 +1759,35 @@
edge.setUseKind(useKind);
}
- void insertStoreBarrier(unsigned indexInBlock, Edge base, Edge value = Edge())
+ void speculateForBarrier(Edge value)
{
- if (!!value) {
- if (value->shouldSpeculateInt32()) {
- insertCheck<Int32Use>(indexInBlock, value.node());
- return;
- }
+ if (!isFTL(m_graph.m_plan.mode))
+ return;
+
+ if (value->shouldSpeculateInt32()) {
+ insertCheck<Int32Use>(m_indexInBlock, value.node());
+ return;
+ }
- if (value->shouldSpeculateBoolean()) {
- insertCheck<BooleanUse>(indexInBlock, value.node());
- return;
- }
+ if (value->shouldSpeculateBoolean()) {
+ insertCheck<BooleanUse>(m_indexInBlock, value.node());
+ return;
+ }
- if (value->shouldSpeculateOther()) {
- insertCheck<OtherUse>(indexInBlock, value.node());
- return;
- }
+ if (value->shouldSpeculateOther()) {
+ insertCheck<OtherUse>(m_indexInBlock, value.node());
+ return;
+ }
- if (value->shouldSpeculateNumber()) {
- insertCheck<NumberUse>(indexInBlock, value.node());
- return;
- }
+ if (value->shouldSpeculateNumber()) {
+ insertCheck<NumberUse>(m_indexInBlock, value.node());
+ return;
+ }
- if (value->shouldSpeculateNotCell()) {
- insertCheck<NotCellUse>(indexInBlock, value.node());
- return;
- }
+ if (value->shouldSpeculateNotCell()) {
+ insertCheck<NotCellUse>(m_indexInBlock, value.node());
+ return;
}
-
- m_insertionSet.insertNode(
- indexInBlock, SpecNone, StoreBarrier, m_currentNode->origin, base);
}
template<UseKind useKind>
Modified: trunk/Source/_javascript_Core/dfg/DFGObjectAllocationSinkingPhase.cpp (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGObjectAllocationSinkingPhase.cpp 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGObjectAllocationSinkingPhase.cpp 2015-05-15 21:11:39 UTC (rev 184415)
@@ -613,11 +613,7 @@
}
case StoreBarrier: {
- Node* target = node->child1().node();
- if (m_sinkCandidates.contains(target)) {
- ASSERT(target->isPhantomAllocation());
- node->remove();
- }
+ DFG_CRASH(m_graph, node, "Unexpected store barrier during sinking.");
break;
}
@@ -854,7 +850,6 @@
case MovHint:
case PutHint:
- case StoreBarrier:
break;
case PutStructure:
Modified: trunk/Source/_javascript_Core/dfg/DFGPlan.cpp (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGPlan.cpp 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGPlan.cpp 2015-05-15 21:11:39 UTC (rev 184415)
@@ -60,7 +60,7 @@
#include "DFGSSALoweringPhase.h"
#include "DFGStackLayoutPhase.h"
#include "DFGStaticExecutionCountEstimationPhase.h"
-#include "DFGStoreBarrierElisionPhase.h"
+#include "DFGStoreBarrierInsertionPhase.h"
#include "DFGStrengthReductionPhase.h"
#include "DFGStructureRegistrationPhase.h"
#include "DFGTierUpCheckInjectionPhase.h"
@@ -315,7 +315,7 @@
performTierUpCheckInjection(dfg);
- performStoreBarrierElision(dfg);
+ performFastStoreBarrierInsertion(dfg);
performCleanUp(dfg);
performCPSRethreading(dfg);
performDCE(dfg);
@@ -387,9 +387,9 @@
// about code motion assumes that it's OK to insert GC points in random places.
dfg.m_fixpointState = FixpointConverged;
- performStoreBarrierElision(dfg);
performLivenessAnalysis(dfg);
performCFA(dfg);
+ performGlobalStoreBarrierInsertion(dfg);
if (Options::enableMovHintRemoval())
performMovHintRemoval(dfg);
performCleanUp(dfg);
Modified: trunk/Source/_javascript_Core/dfg/DFGPredictionPropagationPhase.cpp (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGPredictionPropagationPhase.cpp 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGPredictionPropagationPhase.cpp 2015-05-15 21:11:39 UTC (rev 184415)
@@ -562,10 +562,11 @@
case MaterializeCreateActivation:
case PutStack:
case KillStack:
+ case StoreBarrier:
case GetStack: {
// This node should never be visible at this stage of compilation. It is
// inserted by fixup(), which follows this phase.
- RELEASE_ASSERT_NOT_REACHED();
+ DFG_CRASH(m_graph, node, "Unexpected node during prediction propagation");
break;
}
@@ -617,7 +618,6 @@
#ifndef NDEBUG
// These get ignored because they don't return anything.
- case StoreBarrier:
case PutByValDirect:
case PutByVal:
case PutClosureVar:
Deleted: trunk/Source/_javascript_Core/dfg/DFGStoreBarrierElisionPhase.cpp (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGStoreBarrierElisionPhase.cpp 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGStoreBarrierElisionPhase.cpp 2015-05-15 21:11:39 UTC (rev 184415)
@@ -1,150 +0,0 @@
-/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
- * THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "config.h"
-#include "DFGStoreBarrierElisionPhase.h"
-
-#if ENABLE(DFG_JIT)
-
-#include "DFGBasicBlock.h"
-#include "DFGClobberize.h"
-#include "DFGDoesGC.h"
-#include "DFGGraph.h"
-#include "DFGPhase.h"
-#include "JSCInlines.h"
-#include <wtf/HashSet.h>
-
-namespace JSC { namespace DFG {
-
-class StoreBarrierElisionPhase : public Phase {
-public:
- StoreBarrierElisionPhase(Graph& graph)
- : Phase(graph, "store barrier elision")
- , m_currentBlock(0)
- , m_currentIndex(0)
- {
- }
-
- bool run()
- {
- for (BlockIndex blockIndex = 0; blockIndex < m_graph.numBlocks(); ++blockIndex) {
- m_currentBlock = m_graph.block(blockIndex);
- if (!m_currentBlock)
- continue;
- handleBlock(m_currentBlock);
- }
- return true;
- }
-
-private:
- bool allocatesFreshObject(Node* node)
- {
- switch (node->op()) {
- case NewObject:
- case NewArray:
- case NewArrayWithSize:
- case NewArrayBuffer:
- case NewTypedArray:
- case NewRegexp:
- return true;
- default:
- return false;
- }
- }
-
- void noticeFreshObject(HashSet<Node*>& dontNeedBarriers, Node* node)
- {
- ASSERT(allocatesFreshObject(node));
- dontNeedBarriers.add(node);
- }
-
- Node* getBaseOfStore(Node* barrierNode)
- {
- ASSERT(barrierNode->isStoreBarrier());
- return barrierNode->child1().node();
- }
-
- bool shouldBeElided(HashSet<Node*>& dontNeedBarriers, Node* node)
- {
- ASSERT(node->isStoreBarrier());
- return dontNeedBarriers.contains(node->child1().node());
- }
-
- void elideBarrier(Node* node)
- {
- ASSERT(node->isStoreBarrier());
- node->remove();
- }
-
- void handleNode(HashSet<Node*>& dontNeedBarriers, Node* node)
- {
- if (doesGC(m_graph, node))
- dontNeedBarriers.clear();
-
- if (allocatesFreshObject(node))
- noticeFreshObject(dontNeedBarriers, node);
-
- if (!node->isStoreBarrier())
- return;
-
- if (shouldBeElided(dontNeedBarriers, node)) {
- elideBarrier(node);
- return;
- }
-
- Node* base = getBaseOfStore(node);
- if (!base)
- return;
-
- if (dontNeedBarriers.contains(base))
- return;
- dontNeedBarriers.add(base);
- }
-
- bool handleBlock(BasicBlock* block)
- {
- HashSet<Node*> dontNeedBarriers;
- for (unsigned indexInBlock = 0; indexInBlock < block->size(); ++indexInBlock) {
- m_currentIndex = indexInBlock;
- Node* node = block->at(indexInBlock);
- handleNode(dontNeedBarriers, node);
- }
- return true;
- }
-
- BasicBlock* m_currentBlock;
- unsigned m_currentIndex;
-};
-
-bool performStoreBarrierElision(Graph& graph)
-{
- SamplingRegion samplingRegion("DFG Store Barrier Elision Phase");
- return runPhase<StoreBarrierElisionPhase>(graph);
-}
-
-
-} } // namespace JSC::DFG
-
-#endif // ENABLE(DFG_JIT)
Deleted: trunk/Source/_javascript_Core/dfg/DFGStoreBarrierElisionPhase.h (184414 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGStoreBarrierElisionPhase.h 2015-05-15 21:10:11 UTC (rev 184414)
+++ trunk/Source/_javascript_Core/dfg/DFGStoreBarrierElisionPhase.h 2015-05-15 21:11:39 UTC (rev 184415)
@@ -1,37 +0,0 @@
-/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
- * THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#ifndef DFGStoreBarrierElisionPhase_h
-#define DFGStoreBarrierElisionPhase_h
-
-namespace JSC { namespace DFG {
-
-class Graph;
-
-bool performStoreBarrierElision(Graph&);
-
-} } // namespace JSC::DFG
-
-#endif // DFGStoreBarrierElisionPhase_h
Added: trunk/Source/_javascript_Core/dfg/DFGStoreBarrierInsertionPhase.cpp (0 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGStoreBarrierInsertionPhase.cpp (rev 0)
+++ trunk/Source/_javascript_Core/dfg/DFGStoreBarrierInsertionPhase.cpp 2015-05-15 21:11:39 UTC (rev 184415)
@@ -0,0 +1,528 @@
+/*
+ * Copyright (C) 2015 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "DFGStoreBarrierInsertionPhase.h"
+
+#if ENABLE(DFG_JIT)
+
+#include "DFGAbstractInterpreterInlines.h"
+#include "DFGBlockMapInlines.h"
+#include "DFGDoesGC.h"
+#include "DFGGraph.h"
+#include "DFGInPlaceAbstractState.h"
+#include "DFGInsertionSet.h"
+#include "DFGPhase.h"
+#include "JSCInlines.h"
+#include <wtf/CommaPrinter.h>
+#include <wtf/HashSet.h>
+
+namespace JSC { namespace DFG {
+
+namespace {
+
+bool verbose = false;
+
+enum class PhaseMode {
+ // Does only a local analysis for store barrier insertion and assumes that pointers live
+ // from predecessor blocks may need barriers. Assumes CPS conventions. Does not use AI for
+ // eliminating store barriers, but does a best effort to eliminate barriers when you're
+ // storing a non-cell value by using Node::result() and by looking at constants. The local
+ // analysis is based on GC epochs, so it will eliminate a lot of locally redundant barriers.
+ Fast,
+
+ // Does a global analysis for store barrier insertion. Reuses the GC-epoch-based analysis
+ // used by Fast, but adds a conservative merge rule for propagating information from one
+ // block to the next. This will ensure for example that if a value V coming from multiple
+ // predecessors in B didn't need any more barriers at the end of each predecessor (either
+ // because it was the last allocated object in that predecessor or because it just had a
+ // barrier executed), then until we hit another GC point in B, we won't need another barrier
+ // on V. Uses AI for eliminating barriers when we know that the value being stored is not a
+ // cell. Assumes SSA conventions.
+ Global
+};
+
+template<PhaseMode mode>
+class StoreBarrierInsertionPhase : public Phase {
+public:
+ StoreBarrierInsertionPhase(Graph& graph)
+ : Phase(graph, mode == PhaseMode::Fast ? "fast store barrier insertion" : "global store barrier insertion")
+ , m_insertionSet(graph)
+ {
+ }
+
+ bool run()
+ {
+ if (verbose) {
+ dataLog("Starting store barrier insertion:\n");
+ m_graph.dump();
+ }
+
+ switch (mode) {
+ case PhaseMode::Fast: {
+ DFG_ASSERT(m_graph, nullptr, m_graph.m_form != SSA);
+
+ m_graph.clearEpochs();
+ for (BasicBlock* block : m_graph.blocksInNaturalOrder())
+ handleBlock(block);
+ return true;
+ }
+
+ case PhaseMode::Global: {
+ DFG_ASSERT(m_graph, nullptr, m_graph.m_form == SSA);
+
+ m_state = std::make_unique<InPlaceAbstractState>(m_graph);
+ m_interpreter = std::make_unique<AbstractInterpreter<InPlaceAbstractState>>(m_graph, *m_state);
+
+ m_isConverged = false;
+
+ // First run the analysis. Inside basic blocks we use an epoch-based analysis that
+ // is very precise. At block boundaries, we just propagate which nodes may need a
+ // barrier. This gives us a very nice bottom->top fixpoint: we start out assuming
+ // that no node needs any barriers at block boundaries, and then we converge
+ // towards believing that all nodes need barriers. "Needing a barrier" is like
+ // saying that the node is in a past epoch. "Not needing a barrier" is like saying
+ // that the node is in the current epoch.
+ m_stateAtHead = std::make_unique<BlockMap<HashSet<Node*>>>(m_graph);
+ m_stateAtTail = std::make_unique<BlockMap<HashSet<Node*>>>(m_graph);
+
+ BlockList postOrder = m_graph.blocksInPostOrder();
+
+ bool changed = true;
+ while (changed) {
+ changed = false;
+
+ // Intentional backwards loop because we are using RPO.
+ for (unsigned blockIndex = postOrder.size(); blockIndex--;) {
+ BasicBlock* block = postOrder[blockIndex];
+
+ if (!handleBlock(block)) {
+ // If the block didn't finish, then it cannot affect the fixpoint.
+ continue;
+ }
+
+ // Construct the state-at-tail based on the epochs of live nodes and the
+ // current epoch. We grow state-at-tail monotonically to ensure convergence.
+ bool thisBlockChanged = false;
+ for (Node* node : block->ssa->liveAtTail) {
+ if (node->epoch() != m_currentEpoch) {
+ // If the node is older than the current epoch, then we may need to
+ // run a barrier on it in the future. So, add it to the state.
+ thisBlockChanged |= m_stateAtTail->at(block).add(node).isNewEntry;
+ }
+ }
+
+ if (!thisBlockChanged) {
+ // This iteration didn't learn anything new about this block.
+ continue;
+ }
+
+ // Changed things. Make sure that we loop one more time.
+ changed = true;
+
+ for (BasicBlock* successor : block->successors()) {
+ for (Node* node : m_stateAtTail->at(block))
+ m_stateAtHead->at(successor).add(node);
+ }
+ }
+ }
+
+ // Tell handleBlock() that it's time to actually insert barriers for real.
+ m_isConverged = true;
+
+ for (BasicBlock* block : m_graph.blocksInNaturalOrder())
+ handleBlock(block);
+
+ return true;
+ } }
+
+ RELEASE_ASSERT_NOT_REACHED();
+ return false;
+ }
+
+private:
+ bool handleBlock(BasicBlock* block)
+ {
+ if (verbose) {
+ dataLog("Dealing with block ", pointerDump(block), "\n");
+ if (reallyInsertBarriers())
+ dataLog(" Really inserting barriers.\n");
+ }
+
+ m_currentEpoch = Epoch::first();
+
+ if (mode == PhaseMode::Global) {
+ if (!block->cfaHasVisited)
+ return false;
+ m_state->beginBasicBlock(block);
+
+ for (Node* node : block->ssa->liveAtHead) {
+ if (m_stateAtHead->at(block).contains(node)) {
+ // If previous blocks tell us that this node may need a barrier in the
+ // future, then put it in the ancient primordial epoch. This forces us to
+ // emit a barrier on any possibly-cell store, regardless of the epoch of the
+ // stored value.
+ node->setEpoch(Epoch());
+ } else {
+ // If previous blocks aren't requiring us to run a barrier on this node,
+ // then put it in the current epoch. This means that we will skip barriers
+ // on this node so long as we don't allocate. It also means that we won't
+ // run barriers on stores to on one such node into another such node. That's
+ // fine, because nodes would be excluded from the state set if at the tails
+ // of all predecessors they always had the current epoch.
+ node->setEpoch(m_currentEpoch);
+ }
+ }
+ }
+
+ bool result = true;
+
+ for (m_nodeIndex = 0; m_nodeIndex < block->size(); ++m_nodeIndex) {
+ m_node = block->at(m_nodeIndex);
+
+ if (verbose) {
+ dataLog(
+ " ", m_currentEpoch, ": Looking at node ", m_node, " with children: ");
+ CommaPrinter comma;
+ m_graph.doToChildren(
+ m_node,
+ [&] (Edge edge) {
+ dataLog(comma, edge, " (", edge->epoch(), ")");
+ });
+ dataLog("\n");
+ }
+
+ if (mode == PhaseMode::Global) {
+ // Execute edges separately because we don't want to insert barriers if the
+ // operation doing the store does a check that ensures that the child is not
+ // a cell.
+ m_interpreter->startExecuting();
+ m_interpreter->executeEdges(m_node);
+ }
+
+ switch (m_node->op()) {
+ case PutByValDirect:
+ case PutByVal:
+ case PutByValAlias: {
+ switch (m_node->arrayMode().modeForPut().type()) {
+ case Array::Contiguous:
+ case Array::ArrayStorage:
+ case Array::SlowPutArrayStorage: {
+ Edge child1 = m_graph.varArgChild(m_node, 0);
+ Edge child3 = m_graph.varArgChild(m_node, 2);
+ considerBarrier(child1, child3);
+ break;
+ }
+ default:
+ break;
+ }
+ break;
+ }
+
+ case ArrayPush: {
+ switch (m_node->arrayMode().type()) {
+ case Array::Contiguous:
+ case Array::ArrayStorage:
+ considerBarrier(m_node->child1(), m_node->child2());
+ break;
+ default:
+ break;
+ }
+ break;
+ }
+
+ case PutStructure: {
+ considerBarrier(m_node->child1());
+ break;
+ }
+
+ case PutClosureVar:
+ case PutToArguments:
+ case PutById:
+ case PutByIdFlush:
+ case PutByIdDirect:
+ case MultiPutByOffset: {
+ considerBarrier(m_node->child1(), m_node->child2());
+ break;
+ }
+
+ case PutByOffset: {
+ considerBarrier(m_node->child2(), m_node->child3());
+ break;
+ }
+
+ case PutGlobalVar: {
+ considerBarrier(m_node->child1(), m_node->child2());
+ break;
+ }
+
+ default:
+ break;
+ }
+
+ if (doesGC(m_graph, m_node))
+ m_currentEpoch.bump();
+
+ switch (m_node->op()) {
+ case NewObject:
+ case NewArray:
+ case NewArrayWithSize:
+ case NewArrayBuffer:
+ case NewTypedArray:
+ case NewRegexp:
+ case MaterializeNewObject:
+ case MaterializeCreateActivation:
+ case NewStringObject:
+ case MakeRope:
+ case CreateActivation:
+ case CreateDirectArguments:
+ case CreateScopedArguments:
+ case CreateClonedArguments:
+ case NewFunction:
+ // Nodes that allocate get to set their epoch because for those nodes we know
+ // that they will be the newest object in the heap.
+ m_node->setEpoch(m_currentEpoch);
+ break;
+
+ case AllocatePropertyStorage:
+ case ReallocatePropertyStorage:
+ // These allocate but then run their own barrier.
+ insertBarrier(m_nodeIndex + 1, m_node->child1().node());
+ m_node->setEpoch(Epoch());
+ break;
+
+ case Upsilon:
+ m_node->phi()->setEpoch(m_node->epoch());
+ m_node->setEpoch(Epoch());
+ break;
+
+ default:
+ // For nodes that aren't guaranteed to allocate, we say that their return value
+ // (if there is one) could be arbitrarily old.
+ m_node->setEpoch(Epoch());
+ break;
+ }
+
+ if (verbose) {
+ dataLog(
+ " ", m_currentEpoch, ": Done with node ", m_node, " (", m_node->epoch(),
+ ") with children: ");
+ CommaPrinter comma;
+ m_graph.doToChildren(
+ m_node,
+ [&] (Edge edge) {
+ dataLog(comma, edge, " (", edge->epoch(), ")");
+ });
+ dataLog("\n");
+ }
+
+ if (mode == PhaseMode::Global) {
+ if (!m_interpreter->executeEffects(m_nodeIndex, m_node)) {
+ result = false;
+ break;
+ }
+ }
+ }
+
+ if (mode == PhaseMode::Global)
+ m_state->reset();
+
+ if (reallyInsertBarriers())
+ m_insertionSet.execute(block);
+
+ return result;
+ }
+
+ void considerBarrier(Edge base, Edge child)
+ {
+ if (verbose)
+ dataLog(" Considering adding barrier ", base, " => ", child, "\n");
+
+ // We don't need a store barrier if the child is guaranteed to not be a cell.
+ switch (mode) {
+ case PhaseMode::Fast: {
+ // Don't try too hard because it's too expensive to run AI.
+ if (child->hasConstant()) {
+ if (!child->asJSValue().isCell()) {
+ if (verbose)
+ dataLog(" Rejecting because of constant type.\n");
+ return;
+ }
+ } else {
+ switch (child->result()) {
+ case NodeResultNumber:
+ case NodeResultDouble:
+ case NodeResultInt32:
+ case NodeResultInt52:
+ case NodeResultBoolean:
+ if (verbose)
+ dataLog(" Rejecting because of result type.\n");
+ return;
+ default:
+ break;
+ }
+ }
+ break;
+ }
+
+ case PhaseMode::Global: {
+ // Go into rage mode to eliminate any chance of a barrier with a non-cell child. We
+ // can afford to keep around AI in Global mode.
+ if (!m_interpreter->needsTypeCheck(child, ~SpecCell)) {
+ if (verbose)
+ dataLog(" Rejecting because of AI type.\n");
+ return;
+ }
+ break;
+ } }
+
+ // We don't need a store barrier if the base is at least as new as the child. For
+ // example this won't need a barrier:
+ //
+ // var o = {}
+ // var p = {}
+ // p.f = o
+ //
+ // This is stronger than the currentEpoch rule in considerBarrier(Edge), because it will
+ // also eliminate barriers in cases like this:
+ //
+ // var o = {} // o.epoch = 1, currentEpoch = 1
+ // var p = {} // o.epoch = 1, p.epoch = 2, currentEpoch = 2
+ // var q = {} // o.epoch = 1, p.epoch = 2, q.epoch = 3, currentEpoch = 3
+ // p.f = o // p.epoch >= o.epoch
+ //
+ // This relationship works because if it holds then we are in one of the following
+ // scenarios. Note that we don't know *which* of these scenarios we are in, but it's
+ // one of them (though without loss of generality, you can replace "a GC happened" with
+ // "many GCs happened").
+ //
+ // 1) There is no GC between the allocation/last-barrier of base, child and now. Then
+ // we definitely don't need a barrier.
+ //
+ // 2) There was a GC after child was allocated but before base was allocated. Then we
+ // don't need a barrier, because base is still a new object.
+ //
+ // 3) There was a GC after both child and base were allocated. Then they are both old.
+ // We don't need barriers on stores of old into old. Note that in this case it
+ // doesn't matter if there was also a GC between the allocation of child and base.
+ //
+ // Note that barriers will lift an object into the current epoch. This is sort of weird.
+ // It means that later if you store that object into some other object, and that other
+ // object was previously newer object, you'll think that you need a barrier. We could
+ // avoid this by tracking allocation epoch and barrier epoch separately. For now I think
+ // that this would be overkill. But this does mean that there are the following
+ // possibilities when this relationship holds:
+ //
+ // 4) Base is allocated first. A GC happens and base becomes old. Then we allocate
+ // child. (Note that alternatively the GC could happen during the allocation of
+ // child.) Then we run a barrier on base. Base will appear to be as new as child
+ // (same epoch). At this point, we don't need another barrier on base.
+ //
+ // 5) Base is allocated first. Then we allocate child. Then we run a GC. Then we run a
+ // barrier on base. Base will appear newer than child. We don't need a barrier
+ // because both objects are old.
+ //
+ // Something we watch out for here is that the null epoch is a catch-all for objects
+ // allocated before we did any epoch tracking. Two objects being in the null epoch
+ // means that we don't know their epoch relationship.
+ if (!!base->epoch() && base->epoch() >= child->epoch()) {
+ if (verbose)
+ dataLog(" Rejecting because of epoch ordering.\n");
+ return;
+ }
+
+ considerBarrier(base);
+ }
+
+ void considerBarrier(Edge base)
+ {
+ if (verbose)
+ dataLog(" Considering adding barrier on ", base, "\n");
+
+ // We don't need a store barrier if the epoch of the base is identical to the current
+ // epoch. That means that we either just allocated the object and so it's guaranteed to
+ // be in newgen, or we just ran a barrier on it so it's guaranteed to be remembered
+ // already.
+ if (base->epoch() == m_currentEpoch) {
+ if (verbose)
+ dataLog(" Rejecting because it's in the current epoch.\n");
+ return;
+ }
+
+ if (verbose)
+ dataLog(" Inserting barrier.\n");
+ insertBarrier(m_nodeIndex, base.node());
+ }
+
+ void insertBarrier(unsigned nodeIndex, Node* base)
+ {
+ // If we're in global mode, we should only insert the barriers once we have converged.
+ if (!reallyInsertBarriers())
+ return;
+
+ // FIXME: We could support StoreBarrier(UntypedUse:). That would be sort of cool.
+ // But right now we don't need it.
+ m_insertionSet.insertNode(
+ nodeIndex, SpecNone, StoreBarrier, m_node->origin, Edge(base, CellUse));
+
+ base->setEpoch(m_currentEpoch);
+ }
+
+ bool reallyInsertBarriers()
+ {
+ return mode == PhaseMode::Fast || m_isConverged;
+ }
+
+ InsertionSet m_insertionSet;
+ Epoch m_currentEpoch;
+ unsigned m_nodeIndex;
+ Node* m_node;
+
+ // Things we only use in Global mode.
+ std::unique_ptr<InPlaceAbstractState> m_state;
+ std::unique_ptr<AbstractInterpreter<InPlaceAbstractState>> m_interpreter;
+ std::unique_ptr<BlockMap<HashSet<Node*>>> m_stateAtHead;
+ std::unique_ptr<BlockMap<HashSet<Node*>>> m_stateAtTail;
+ bool m_isConverged;
+};
+
+} // anonymous namespace
+
+bool performFastStoreBarrierInsertion(Graph& graph)
+{
+ SamplingRegion samplingRegion("DFG Fast Store Barrier Insertion Phase");
+ return runPhase<StoreBarrierInsertionPhase<PhaseMode::Fast>>(graph);
+}
+
+bool performGlobalStoreBarrierInsertion(Graph& graph)
+{
+ SamplingRegion samplingRegion("DFG Global Store Barrier Insertion Phase");
+ return runPhase<StoreBarrierInsertionPhase<PhaseMode::Global>>(graph);
+}
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
Added: trunk/Source/_javascript_Core/dfg/DFGStoreBarrierInsertionPhase.h (0 => 184415)
--- trunk/Source/_javascript_Core/dfg/DFGStoreBarrierInsertionPhase.h (rev 0)
+++ trunk/Source/_javascript_Core/dfg/DFGStoreBarrierInsertionPhase.h 2015-05-15 21:11:39 UTC (rev 184415)
@@ -0,0 +1,51 @@
+/*
+ * Copyright (C) 2015 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef DFGStoreBarrierInsertionPhase_h
+#define DFGStoreBarrierInsertionPhase_h
+
+#if ENABLE(DFG_JIT)
+
+namespace JSC { namespace DFG {
+
+class Graph;
+
+// Inserts store barriers in a block-local manner without consulting the abstract interpreter.
+// Uses a simple epoch-based analysis to avoid inserting redundant barriers. This phase requires
+// that we are not in SSA.
+bool performFastStoreBarrierInsertion(Graph&);
+
+// Inserts store barriers using a global analysis and consults the abstract interpreter. Uses a
+// simple epoch-based analysis to avoid inserting redundant barriers, but only propagates "same
+// epoch as current" property from one block to the next. This phase requires SSA. This phase
+// also requires having valid AI and liveness.
+bool performGlobalStoreBarrierInsertion(Graph&);
+
+} } // namespace JSC::DFG
+
+#endif // ENABLE(DFG_JIT)
+
+#endif // DFGStoreBarrierInsertionPhase_h
+
