Re: r246497 - [modules] Rework serialized DeclContext lookup table management. Instead of

2015-09-01 Thread Richard Smith via cfe-commits
On Tue, Sep 1, 2015 at 6:26 AM, Aaron Ballman 
wrote:

> On Tue, Sep 1, 2015 at 4:43 AM, İsmail Dönmez
>  wrote:
> > Hi,
> >
> > On Tue, Sep 1, 2015 at 1:17 AM, Richard Smith via cfe-commits
> >  wrote:
> >> Author: rsmith
> >> Date: Mon Aug 31 17:17:11 2015
> >> New Revision: 246497
> >>
> >> URL: http://llvm.org/viewvc/llvm-project?rev=246497&view=rev
> >> Log:
> >> [modules] Rework serialized DeclContext lookup table management.
> Instead of
> >> walking the loaded ModuleFiles looking for lookup tables for the
> context, store
> >> them all in one place, and merge them together if we find we have too
> many
> >> (currently, more than 4). If we do merge, include the merged form in our
> >> serialized lookup table, so that downstream readers never need to look
> at our
> >> imports' tables.
> >>
> >> This gives a huge performance improvement to builds with very large
> numbers of
> >> modules (in some cases, more than a 2x speedup was observed).
> >>
> >> Added:
> >> cfe/trunk/lib/Serialization/MultiOnDiskHashTable.h
> >
> > This doesn't seem to compile with VS2015:
> >
> > FAILED: C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe   /nologo /TP
> > /DWIN32 /D_WINDOWS   -wd4146 -wd4180 -wd4244 -wd4258 -wd4267 -wd4291
> > -wd4345 -wd4351 -wd4355 -wd4456 -wd4457 -wd4458 -wd4459 -wd4503
> > -wd4624 -wd4722 -wd4800 -wd4100 -wd4127 -wd4512 -wd4505 -wd4610
> > -wd4510 -wd4702 -wd4245 -wd4706 -wd4310 -wd4701 -wd4703 -wd4389
> > -wd4611 -wd4805 -wd4204 -wd4577 -wd4091 -wd4324 -w14062 -we4238 /W4
> > /Zc:inline /Zc:sizedDealloc- /MT /O2 /Ob2
> > -Itools\clang\lib\Serialization -I..\tools\clang\lib\Serialization
> > -I..\tools\clang\include -Itools\clang\include -Iinclude -I..\include
> >   -UNDEBUG  /EHs-c- /GR- /showIncludes -DCLANG_ENABLE_ARCMT
> > -DCLANG_ENABLE_OBJC_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER
> > -DGTEST_HAS_RTTI=0 -D_CRT_NONSTDC_NO_DEPRECATE
> > -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE
> > -D_CRT_SECURE_NO_WARNINGS -D_GNU_SOURCE -D_HAS_EXCEPTIONS=0
> > -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS
> > -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
> >
> /Fotools\clang\lib\Serialization\CMakeFiles\clangSerialization.dir\ASTReader.cpp.obj
> > /Fdtools\clang\lib\Serialization\CMakeFiles\clangSerialization.dir\
> > /FS -c ..\tools\clang\lib\Serialization\ASTReader.cpp
> >
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
> > error C2065: 'Files': undeclared identifier
> >
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(111):
> > note: while compiling class template member function 'void
> >
> clang::serialization::MultiOnDiskHashTable::removeOverriddenTables(void)'
> >
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(243):
> > note: see reference to function template instantiation 'void
> >
> clang::serialization::MultiOnDiskHashTable::removeOverriddenTables(void)'
> > being compiled
> >
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\ASTReaderInternals.h(114):
> > note: see reference to class template instantiation
> >
> 'clang::serialization::MultiOnDiskHashTable'
> > being compiled
> >
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
> > error C2228: left of '.count' must have class/struct/union
> >
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
> > note: type is 'unknown-type'
> > ninja: build stopped: subcommand failed.
>
> I have reverted r246497 (which required also reverting r246524 and
> r246521 to avoid merge conflicts) to get back to green. Commit was
> r246546.
>
> I'm not certain why MSVC is falling over on this code, but I suspect
> compiler bug. If anyone has a reduced testcase (which I may spend some
> time on if I have a moment), I would be happy to report it to
> Microsoft.


It's two compiler bugs.

In MSVC 2013, providing a move constructor apparently doesn't suppress the
implicit generation of copy operations in some cases. In particular, given:

  struct X { X(); X(X&&); X &operator=(X&&); ~X(); };
  struct Y { X x; };

... Y is copyable (and in my case, copying it led to a use-after-free,
because X had the equivalent of a pointer member).

In MSVC 2015, it appears that implicit lambda capture doesn't work reliably
inside a member function of a class template. The testcase looks something
like:

template struct S {
  typedef T type;
  type f() {
type n;
[&] { ++n; }();
return n;
  }
};
int k = S().f();

... though I don't know if that's enough to reproduce the rejects-valid.

Anyway, these are hopefully both worked around in r246582.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


Re: r246497 - [modules] Rework serialized DeclContext lookup table management. Instead of

2015-09-01 Thread Aaron Ballman via cfe-commits
On Tue, Sep 1, 2015 at 4:43 AM, İsmail Dönmez
 wrote:
> Hi,
>
> On Tue, Sep 1, 2015 at 1:17 AM, Richard Smith via cfe-commits
>  wrote:
>> Author: rsmith
>> Date: Mon Aug 31 17:17:11 2015
>> New Revision: 246497
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=246497&view=rev
>> Log:
>> [modules] Rework serialized DeclContext lookup table management. Instead of
>> walking the loaded ModuleFiles looking for lookup tables for the context, 
>> store
>> them all in one place, and merge them together if we find we have too many
>> (currently, more than 4). If we do merge, include the merged form in our
>> serialized lookup table, so that downstream readers never need to look at our
>> imports' tables.
>>
>> This gives a huge performance improvement to builds with very large numbers 
>> of
>> modules (in some cases, more than a 2x speedup was observed).
>>
>> Added:
>> cfe/trunk/lib/Serialization/MultiOnDiskHashTable.h
>
> This doesn't seem to compile with VS2015:
>
> FAILED: C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe   /nologo /TP
> /DWIN32 /D_WINDOWS   -wd4146 -wd4180 -wd4244 -wd4258 -wd4267 -wd4291
> -wd4345 -wd4351 -wd4355 -wd4456 -wd4457 -wd4458 -wd4459 -wd4503
> -wd4624 -wd4722 -wd4800 -wd4100 -wd4127 -wd4512 -wd4505 -wd4610
> -wd4510 -wd4702 -wd4245 -wd4706 -wd4310 -wd4701 -wd4703 -wd4389
> -wd4611 -wd4805 -wd4204 -wd4577 -wd4091 -wd4324 -w14062 -we4238 /W4
> /Zc:inline /Zc:sizedDealloc- /MT /O2 /Ob2
> -Itools\clang\lib\Serialization -I..\tools\clang\lib\Serialization
> -I..\tools\clang\include -Itools\clang\include -Iinclude -I..\include
>   -UNDEBUG  /EHs-c- /GR- /showIncludes -DCLANG_ENABLE_ARCMT
> -DCLANG_ENABLE_OBJC_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER
> -DGTEST_HAS_RTTI=0 -D_CRT_NONSTDC_NO_DEPRECATE
> -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE
> -D_CRT_SECURE_NO_WARNINGS -D_GNU_SOURCE -D_HAS_EXCEPTIONS=0
> -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS
> -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
> /Fotools\clang\lib\Serialization\CMakeFiles\clangSerialization.dir\ASTReader.cpp.obj
> /Fdtools\clang\lib\Serialization\CMakeFiles\clangSerialization.dir\
> /FS -c ..\tools\clang\lib\Serialization\ASTReader.cpp
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
> error C2065: 'Files': undeclared identifier
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(111):
> note: while compiling class template member function 'void
> clang::serialization::MultiOnDiskHashTable::removeOverriddenTables(void)'
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(243):
> note: see reference to function template instantiation 'void
> clang::serialization::MultiOnDiskHashTable::removeOverriddenTables(void)'
> being compiled
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\ASTReaderInternals.h(114):
> note: see reference to class template instantiation
> 'clang::serialization::MultiOnDiskHashTable'
> being compiled
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
> error C2228: left of '.count' must have class/struct/union
> c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
> note: type is 'unknown-type'
> ninja: build stopped: subcommand failed.

I have reverted r246497 (which required also reverting r246524 and
r246521 to avoid merge conflicts) to get back to green. Commit was
r246546.

I'm not certain why MSVC is falling over on this code, but I suspect
compiler bug. If anyone has a reduced testcase (which I may spend some
time on if I have a moment), I would be happy to report it to
Microsoft.

~Aaron

> ___
> cfe-commits mailing list
> cfe-commits@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


Re: r246497 - [modules] Rework serialized DeclContext lookup table management. Instead of

2015-09-01 Thread Rafael Espíndola via cfe-commits
Any chance this is what caused the following leak?

http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/8340/steps/check-clang%20asan/logs/stdio

Cheers,
Rafael


On 31 August 2015 at 18:17, Richard Smith via cfe-commits
 wrote:
> Author: rsmith
> Date: Mon Aug 31 17:17:11 2015
> New Revision: 246497
>
> URL: http://llvm.org/viewvc/llvm-project?rev=246497&view=rev
> Log:
> [modules] Rework serialized DeclContext lookup table management. Instead of
> walking the loaded ModuleFiles looking for lookup tables for the context, 
> store
> them all in one place, and merge them together if we find we have too many
> (currently, more than 4). If we do merge, include the merged form in our
> serialized lookup table, so that downstream readers never need to look at our
> imports' tables.
>
> This gives a huge performance improvement to builds with very large numbers of
> modules (in some cases, more than a 2x speedup was observed).
>
> Added:
> cfe/trunk/lib/Serialization/MultiOnDiskHashTable.h
> Modified:
> cfe/trunk/include/clang/Serialization/ASTBitCodes.h
> cfe/trunk/include/clang/Serialization/ASTReader.h
> cfe/trunk/include/clang/Serialization/ASTWriter.h
> cfe/trunk/include/clang/Serialization/Module.h
> cfe/trunk/lib/Serialization/ASTReader.cpp
> cfe/trunk/lib/Serialization/ASTReaderDecl.cpp
> cfe/trunk/lib/Serialization/ASTReaderInternals.h
> cfe/trunk/lib/Serialization/ASTWriter.cpp
> cfe/trunk/lib/Serialization/ASTWriterDecl.cpp
> cfe/trunk/lib/Serialization/Module.cpp
> cfe/trunk/test/Modules/cxx-templates.cpp
> cfe/trunk/test/Modules/merge-using-decls.cpp
>
> Modified: cfe/trunk/include/clang/Serialization/ASTBitCodes.h
> URL: 
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTBitCodes.h?rev=246497&r1=246496&r2=246497&view=diff
> ==
> --- cfe/trunk/include/clang/Serialization/ASTBitCodes.h (original)
> +++ cfe/trunk/include/clang/Serialization/ASTBitCodes.h Mon Aug 31 17:17:11 
> 2015
> @@ -1530,4 +1530,23 @@ namespace clang {
>}
>  } // end namespace clang
>
> +namespace llvm {
> +  template <> struct DenseMapInfo {
> +static clang::serialization::DeclarationNameKey getEmptyKey() {
> +  return clang::serialization::DeclarationNameKey(-1, 1);
> +}
> +static clang::serialization::DeclarationNameKey getTombstoneKey() {
> +  return clang::serialization::DeclarationNameKey(-1, 2);
> +}
> +static unsigned
> +getHashValue(const clang::serialization::DeclarationNameKey &Key) {
> +  return Key.getHash();
> +}
> +static bool isEqual(const clang::serialization::DeclarationNameKey &L,
> +const clang::serialization::DeclarationNameKey &R) {
> +  return L == R;
> +}
> +  };
> +}
> +
>  #endif
>
> Modified: cfe/trunk/include/clang/Serialization/ASTReader.h
> URL: 
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTReader.h?rev=246497&r1=246496&r2=246497&view=diff
> ==
> --- cfe/trunk/include/clang/Serialization/ASTReader.h (original)
> +++ cfe/trunk/include/clang/Serialization/ASTReader.h Mon Aug 31 17:17:11 2015
> @@ -282,9 +282,8 @@ class ReadMethodPoolVisitor;
>
>  namespace reader {
>class ASTIdentifierLookupTrait;
> -  /// \brief The on-disk hash table used for the DeclContext's Name lookup 
> table.
> -  typedef llvm::OnDiskIterableChainedHashTable
> -ASTDeclContextNameLookupTable;
> +  /// \brief The on-disk hash table(s) used for DeclContext name lookup.
> +  struct DeclContextLookupTable;
>  }
>
>  } // end namespace serialization
> @@ -507,6 +506,10 @@ private:
>/// \brief Map from the TU to its lexical contents from each module file.
>std::vector> TULexicalDecls;
>
> +  /// \brief Map from a DeclContext to its lookup tables.
> +  llvm::DenseMap + serialization::reader::DeclContextLookupTable> Lookups;
> +
>// Updates for visible decls can occur for other contexts than just the
>// TU, and when we read those update records, the actual context may not
>// be available yet, so have this pending map using the ID as a key. It
> @@ -514,7 +517,6 @@ private:
>struct PendingVisibleUpdate {
>  ModuleFile *Mod;
>  const unsigned char *Data;
> -unsigned BucketOffset;
>};
>typedef SmallVector DeclContextVisibleUpdates;
>
> @@ -1089,6 +1091,10 @@ public:
>  Visit(GetExistingDecl(ID));
>}
>
> +  /// \brief Get the loaded lookup tables for \p Primary, if any.
> +  const serialization::reader::DeclContextLookupTable *
> +  getLoadedLookupTables(DeclContext *Primary) const;
> +
>  private:
>struct ImportedModule {
>  ModuleFile *Mod;
> @@ -1870,6 +1876,13 @@ public:
>/// Note: overrides method in ExternalASTSource
>Module *getModule(unsigned ID) override;
>
> +  /// \brief 

Re: r246497 - [modules] Rework serialized DeclContext lookup table management. Instead of

2015-09-01 Thread İsmail Dönmez via cfe-commits
Hi,

On Tue, Sep 1, 2015 at 1:17 AM, Richard Smith via cfe-commits
 wrote:
> Author: rsmith
> Date: Mon Aug 31 17:17:11 2015
> New Revision: 246497
>
> URL: http://llvm.org/viewvc/llvm-project?rev=246497&view=rev
> Log:
> [modules] Rework serialized DeclContext lookup table management. Instead of
> walking the loaded ModuleFiles looking for lookup tables for the context, 
> store
> them all in one place, and merge them together if we find we have too many
> (currently, more than 4). If we do merge, include the merged form in our
> serialized lookup table, so that downstream readers never need to look at our
> imports' tables.
>
> This gives a huge performance improvement to builds with very large numbers of
> modules (in some cases, more than a 2x speedup was observed).
>
> Added:
> cfe/trunk/lib/Serialization/MultiOnDiskHashTable.h

This doesn't seem to compile with VS2015:

FAILED: C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe   /nologo /TP
/DWIN32 /D_WINDOWS   -wd4146 -wd4180 -wd4244 -wd4258 -wd4267 -wd4291
-wd4345 -wd4351 -wd4355 -wd4456 -wd4457 -wd4458 -wd4459 -wd4503
-wd4624 -wd4722 -wd4800 -wd4100 -wd4127 -wd4512 -wd4505 -wd4610
-wd4510 -wd4702 -wd4245 -wd4706 -wd4310 -wd4701 -wd4703 -wd4389
-wd4611 -wd4805 -wd4204 -wd4577 -wd4091 -wd4324 -w14062 -we4238 /W4
/Zc:inline /Zc:sizedDealloc- /MT /O2 /Ob2
-Itools\clang\lib\Serialization -I..\tools\clang\lib\Serialization
-I..\tools\clang\include -Itools\clang\include -Iinclude -I..\include
  -UNDEBUG  /EHs-c- /GR- /showIncludes -DCLANG_ENABLE_ARCMT
-DCLANG_ENABLE_OBJC_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER
-DGTEST_HAS_RTTI=0 -D_CRT_NONSTDC_NO_DEPRECATE
-D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE
-D_CRT_SECURE_NO_WARNINGS -D_GNU_SOURCE -D_HAS_EXCEPTIONS=0
-D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS
-D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
/Fotools\clang\lib\Serialization\CMakeFiles\clangSerialization.dir\ASTReader.cpp.obj
/Fdtools\clang\lib\Serialization\CMakeFiles\clangSerialization.dir\
/FS -c ..\tools\clang\lib\Serialization\ASTReader.cpp
c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
error C2065: 'Files': undeclared identifier
c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(111):
note: while compiling class template member function 'void
clang::serialization::MultiOnDiskHashTable::removeOverriddenTables(void)'
c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(243):
note: see reference to function template instantiation 'void
clang::serialization::MultiOnDiskHashTable::removeOverriddenTables(void)'
being compiled
c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\ASTReaderInternals.h(114):
note: see reference to class template instantiation
'clang::serialization::MultiOnDiskHashTable'
being compiled
c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
error C2228: left of '.count' must have class/struct/union
c:\cygwin64\home\ismail\src\llvm\tools\clang\lib\serialization\MultiOnDiskHashTable.h(117):
note: type is 'unknown-type'
ninja: build stopped: subcommand failed.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


r246497 - [modules] Rework serialized DeclContext lookup table management. Instead of

2015-08-31 Thread Richard Smith via cfe-commits
Author: rsmith
Date: Mon Aug 31 17:17:11 2015
New Revision: 246497

URL: http://llvm.org/viewvc/llvm-project?rev=246497&view=rev
Log:
[modules] Rework serialized DeclContext lookup table management. Instead of
walking the loaded ModuleFiles looking for lookup tables for the context, store
them all in one place, and merge them together if we find we have too many
(currently, more than 4). If we do merge, include the merged form in our
serialized lookup table, so that downstream readers never need to look at our
imports' tables.

This gives a huge performance improvement to builds with very large numbers of
modules (in some cases, more than a 2x speedup was observed).

Added:
cfe/trunk/lib/Serialization/MultiOnDiskHashTable.h
Modified:
cfe/trunk/include/clang/Serialization/ASTBitCodes.h
cfe/trunk/include/clang/Serialization/ASTReader.h
cfe/trunk/include/clang/Serialization/ASTWriter.h
cfe/trunk/include/clang/Serialization/Module.h
cfe/trunk/lib/Serialization/ASTReader.cpp
cfe/trunk/lib/Serialization/ASTReaderDecl.cpp
cfe/trunk/lib/Serialization/ASTReaderInternals.h
cfe/trunk/lib/Serialization/ASTWriter.cpp
cfe/trunk/lib/Serialization/ASTWriterDecl.cpp
cfe/trunk/lib/Serialization/Module.cpp
cfe/trunk/test/Modules/cxx-templates.cpp
cfe/trunk/test/Modules/merge-using-decls.cpp

Modified: cfe/trunk/include/clang/Serialization/ASTBitCodes.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTBitCodes.h?rev=246497&r1=246496&r2=246497&view=diff
==
--- cfe/trunk/include/clang/Serialization/ASTBitCodes.h (original)
+++ cfe/trunk/include/clang/Serialization/ASTBitCodes.h Mon Aug 31 17:17:11 2015
@@ -1530,4 +1530,23 @@ namespace clang {
   }
 } // end namespace clang
 
+namespace llvm {
+  template <> struct DenseMapInfo {
+static clang::serialization::DeclarationNameKey getEmptyKey() {
+  return clang::serialization::DeclarationNameKey(-1, 1);
+}
+static clang::serialization::DeclarationNameKey getTombstoneKey() {
+  return clang::serialization::DeclarationNameKey(-1, 2);
+}
+static unsigned
+getHashValue(const clang::serialization::DeclarationNameKey &Key) {
+  return Key.getHash();
+}
+static bool isEqual(const clang::serialization::DeclarationNameKey &L,
+const clang::serialization::DeclarationNameKey &R) {
+  return L == R;
+}
+  };
+}
+
 #endif

Modified: cfe/trunk/include/clang/Serialization/ASTReader.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Serialization/ASTReader.h?rev=246497&r1=246496&r2=246497&view=diff
==
--- cfe/trunk/include/clang/Serialization/ASTReader.h (original)
+++ cfe/trunk/include/clang/Serialization/ASTReader.h Mon Aug 31 17:17:11 2015
@@ -282,9 +282,8 @@ class ReadMethodPoolVisitor;
 
 namespace reader {
   class ASTIdentifierLookupTrait;
-  /// \brief The on-disk hash table used for the DeclContext's Name lookup 
table.
-  typedef llvm::OnDiskIterableChainedHashTable
-ASTDeclContextNameLookupTable;
+  /// \brief The on-disk hash table(s) used for DeclContext name lookup.
+  struct DeclContextLookupTable;
 }
 
 } // end namespace serialization
@@ -507,6 +506,10 @@ private:
   /// \brief Map from the TU to its lexical contents from each module file.
   std::vector> TULexicalDecls;
 
+  /// \brief Map from a DeclContext to its lookup tables.
+  llvm::DenseMap Lookups;
+
   // Updates for visible decls can occur for other contexts than just the
   // TU, and when we read those update records, the actual context may not
   // be available yet, so have this pending map using the ID as a key. It
@@ -514,7 +517,6 @@ private:
   struct PendingVisibleUpdate {
 ModuleFile *Mod;
 const unsigned char *Data;
-unsigned BucketOffset;
   };
   typedef SmallVector DeclContextVisibleUpdates;
 
@@ -1089,6 +1091,10 @@ public:
 Visit(GetExistingDecl(ID));
   }
 
+  /// \brief Get the loaded lookup tables for \p Primary, if any.
+  const serialization::reader::DeclContextLookupTable *
+  getLoadedLookupTables(DeclContext *Primary) const;
+
 private:
   struct ImportedModule {
 ModuleFile *Mod;
@@ -1870,6 +1876,13 @@ public:
   /// Note: overrides method in ExternalASTSource
   Module *getModule(unsigned ID) override;
 
+  /// \brief Retrieve the module file with a given local ID within the 
specified
+  /// ModuleFile.
+  ModuleFile *getLocalModuleFile(ModuleFile &M, unsigned ID);
+
+  /// \brief Get an ID for the given module file.
+  unsigned getModuleFileID(ModuleFile *M);
+
   /// \brief Return a descriptor for the corresponding module.
   llvm::Optional getSourceDescriptor(unsigned ID) 
override;
   /// \brief Return a descriptor for the module.

Modified: cfe/trunk/include/clang/Serialization/ASTWriter.h
URL: 
http://llvm.org/viewvc/llvm-project/cfe/tr