Re: [clang-tools-extra] r303977 - [clangd] Allow to use vfs::FileSystem for file accesses.

2017-05-27 Thread NAKAMURA Takumi via cfe-commits
Hopefully fixed in r304067.

See also; https://reviews.llvm.org/D33416#766533

On Sat, May 27, 2017 at 6:12 AM Vedant Kumar via cfe-commits <
cfe-commits@lists.llvm.org> wrote:

> Hi Ilya,
>
> The unit test introduced by this commit hits an assertion failure on our
> bots. Could you please take a look at the issue?
>
> lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6733
>
> I see:
>
> [ RUN  ] ClangdVFSTest.Reparse
>
> Assertion failed: (Unit && "Unit wasn't created"), function ClangdUnit,
> file
> /Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/tools/clang/tools/extra/clangd/ClangdUnit.cpp,
> line 58.
>
> 0  ClangdTests  0x000102fec2f8
> llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
> 1  ClangdTests  0x000102fec9e6 SignalHandler(int) + 454
> 2  libsystem_platform.dylib 0x7fff97ed352a _sigtramp + 26
> 3  libsystem_platform.dylib 00 _sigtramp + 1746062064
> 4  libsystem_c.dylib0x7fff97d776df abort + 129
> 5  libsystem_c.dylib0x7fff97d3edd8 basename + 0
> 6  ClangdTests  0x0001030c95dd
> clang::clangd::ClangdUnit::ClangdUnit(llvm::StringRef, llvm::StringRef,
> std::__1::shared_ptr,
> std::__1::vector std::__1::allocator >,
> llvm::IntrusiveRefCntPtr) + 1837
> 7  ClangdTests  0x0001030c824f
> std::__1::__function::__func llvm::StringRef)::$_1,
> std::__1::allocator llvm::StringRef)::$_1>, void ()>::operator()() + 975
> 8  ClangdTests  0x0001030c6e03 void*
> std::__1::__thread_proxy
> >(void*) + 579
> 9  libsystem_pthread.dylib  0x7fff8e26099d _pthread_body + 131
> 10 libsystem_pthread.dylib  0x7fff8e26091a _pthread_body + 0
> 11 libsystem_pthread.dylib  0x7fff8e25e351 thread_start + 13
>
> 
> FAIL: Extra Tools Unit Tests ::
> clangd/ClangdTests/ClangdVFSTest.ReparseOnHeaderChange (13778 of 37955)
>  TEST 'Extra Tools Unit Tests ::
> clangd/ClangdTests/ClangdVFSTest.ReparseOnHeaderChange' FAILED
> 
> Note: Google Test filter = ClangdVFSTest.ReparseOnHeaderChange
> [==] Running 1 test from 1 test case.
> [--] Global test environment set-up.
> [--] 1 test from ClangdVFSTest
> [ RUN  ] ClangdVFSTest.ReparseOnHeaderChange
>
> Assertion failed: (Unit && "Unit wasn't created"), function ClangdUnit,
> file
> /Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/tools/clang/tools/extra/clangd/ClangdUnit.cpp,
> line 58.
>
> 0  ClangdTests  0x0001005b72f8
> llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
> 1  ClangdTests  0x0001005b79e6 SignalHandler(int) + 454
> 2  libsystem_platform.dylib 0x7fff97ed352a _sigtramp + 26
> 3  libsystem_platform.dylib 00 _sigtramp + 1746062064
> 4  libsystem_c.dylib0x7fff97d776df abort + 129
> 5  libsystem_c.dylib0x7fff97d3edd8 basename + 0
> 6  ClangdTests  0x0001006945dd
> clang::clangd::ClangdUnit::ClangdUnit(llvm::StringRef, llvm::StringRef,
> std::__1::shared_ptr,
> std::__1::vector std::__1::allocator >,
> llvm::IntrusiveRefCntPtr) + 1837
> 7  ClangdTests  0x00010069324f
> std::__1::__function::__func llvm::StringRef)::$_1,
> std::__1::allocator llvm::StringRef)::$_1>, void ()>::operator()() + 975
> 8  ClangdTests  0x000100691e03 void*
> std::__1::__thread_proxy
> >(void*) + 579
> 9  libsystem_pthread.dylib  0x7fff8e26099d _pthread_body + 131
> 10 libsystem_pthread.dylib  0x7fff8e26091a _pthread_body + 0
> 11 libsystem_pthread.dylib  0x7fff8e25e351 thread_start + 13
>
> thanks,
> vedant
>
> On May 26, 2017, at 5:26 AM, Ilya Biryukov via cfe-commits <
> cfe-commits@lists.llvm.org> wrote:
>
> Author: ibiryukov
> Date: Fri May 26 07:26:51 2017
> New Revision: 303977
>
> URL: http://llvm.org/viewvc/llvm-project?rev=303977=rev
> Log:
> [clangd] Allow to use vfs::FileSystem for file accesses.
>
> Summary:
> Custom vfs::FileSystem is currently used for unit tests.
> This revision depends on https://reviews.llvm.org/D33397.
>
> Reviewers: bkramer, krasimir
>
> Reviewed By: bkramer, krasimir
>
> Subscribers: klimek, cfe-commits, mgorny
>
> Differential Revision: https://reviews.llvm.org/D33416
>
> Added:
>clang-tools-extra/trunk/unittests/clangd/
>clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
>clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
> Modified:
>

Re: [clang-tools-extra] r303977 - [clangd] Allow to use vfs::FileSystem for file accesses.

2017-05-26 Thread Vedant Kumar via cfe-commits
Hi Ilya,

The unit test introduced by this commit hits an assertion failure on our bots. 
Could you please take a look at the issue?

lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6733 


I see:

[ RUN  ] ClangdVFSTest.Reparse

Assertion failed: (Unit && "Unit wasn't created"), function ClangdUnit, file 
/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/tools/clang/tools/extra/clangd/ClangdUnit.cpp,
 line 58.

0  ClangdTests  0x000102fec2f8 
llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  ClangdTests  0x000102fec9e6 SignalHandler(int) + 454
2  libsystem_platform.dylib 0x7fff97ed352a _sigtramp + 26
3  libsystem_platform.dylib 00 _sigtramp + 1746062064
4  libsystem_c.dylib0x7fff97d776df abort + 129
5  libsystem_c.dylib0x7fff97d3edd8 basename + 0
6  ClangdTests  0x0001030c95dd 
clang::clangd::ClangdUnit::ClangdUnit(llvm::StringRef, llvm::StringRef, 
std::__1::shared_ptr, 
std::__1::vector, 
llvm::IntrusiveRefCntPtr) + 1837
7  ClangdTests  0x0001030c824f 
std::__1::__function::__func, void ()>::operator()() + 975
8  ClangdTests  0x0001030c6e03 void* 
std::__1::__thread_proxy
 >(void*) + 579
9  libsystem_pthread.dylib  0x7fff8e26099d _pthread_body + 131
10 libsystem_pthread.dylib  0x7fff8e26091a _pthread_body + 0
11 libsystem_pthread.dylib  0x7fff8e25e351 thread_start + 13


FAIL: Extra Tools Unit Tests :: 
clangd/ClangdTests/ClangdVFSTest.ReparseOnHeaderChange (13778 of 37955)
 TEST 'Extra Tools Unit Tests :: 
clangd/ClangdTests/ClangdVFSTest.ReparseOnHeaderChange' FAILED 

Note: Google Test filter = ClangdVFSTest.ReparseOnHeaderChange
[==] Running 1 test from 1 test case.
[--] Global test environment set-up.
[--] 1 test from ClangdVFSTest
[ RUN  ] ClangdVFSTest.ReparseOnHeaderChange

Assertion failed: (Unit && "Unit wasn't created"), function ClangdUnit, file 
/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/tools/clang/tools/extra/clangd/ClangdUnit.cpp,
 line 58.

0  ClangdTests  0x0001005b72f8 
llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  ClangdTests  0x0001005b79e6 SignalHandler(int) + 454
2  libsystem_platform.dylib 0x7fff97ed352a _sigtramp + 26
3  libsystem_platform.dylib 00 _sigtramp + 1746062064
4  libsystem_c.dylib0x7fff97d776df abort + 129
5  libsystem_c.dylib0x7fff97d3edd8 basename + 0
6  ClangdTests  0x0001006945dd 
clang::clangd::ClangdUnit::ClangdUnit(llvm::StringRef, llvm::StringRef, 
std::__1::shared_ptr, 
std::__1::vector, 
llvm::IntrusiveRefCntPtr) + 1837
7  ClangdTests  0x00010069324f 
std::__1::__function::__func, void ()>::operator()() + 975
8  ClangdTests  0x000100691e03 void* 
std::__1::__thread_proxy
 >(void*) + 579
9  libsystem_pthread.dylib  0x7fff8e26099d _pthread_body + 131
10 libsystem_pthread.dylib  0x7fff8e26091a _pthread_body + 0
11 libsystem_pthread.dylib  0x7fff8e25e351 thread_start + 13

thanks,
vedant

> On May 26, 2017, at 5:26 AM, Ilya Biryukov via cfe-commits 
>  wrote:
> 
> Author: ibiryukov
> Date: Fri May 26 07:26:51 2017
> New Revision: 303977
> 
> URL: http://llvm.org/viewvc/llvm-project?rev=303977=rev
> Log:
> [clangd] Allow to use vfs::FileSystem for file accesses.
> 
> Summary:
> Custom vfs::FileSystem is currently used for unit tests.
> This revision depends on https://reviews.llvm.org/D33397.
> 
> Reviewers: bkramer, krasimir
> 
> Reviewed By: bkramer, krasimir
> 
> Subscribers: klimek, cfe-commits, mgorny
> 
> Differential Revision: https://reviews.llvm.org/D33416
> 
> Added:
>clang-tools-extra/trunk/unittests/clangd/
>clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
>clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
> Modified:
>clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
>clang-tools-extra/trunk/clangd/ClangdServer.cpp
>clang-tools-extra/trunk/clangd/ClangdServer.h
>clang-tools-extra/trunk/clangd/ClangdUnit.cpp
>

[clang-tools-extra] r303977 - [clangd] Allow to use vfs::FileSystem for file accesses.

2017-05-26 Thread Ilya Biryukov via cfe-commits
Author: ibiryukov
Date: Fri May 26 07:26:51 2017
New Revision: 303977

URL: http://llvm.org/viewvc/llvm-project?rev=303977=rev
Log:
[clangd] Allow to use vfs::FileSystem for file accesses.

Summary:
Custom vfs::FileSystem is currently used for unit tests.
This revision depends on https://reviews.llvm.org/D33397.

Reviewers: bkramer, krasimir

Reviewed By: bkramer, krasimir

Subscribers: klimek, cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D33416

Added:
clang-tools-extra/trunk/unittests/clangd/
clang-tools-extra/trunk/unittests/clangd/CMakeLists.txt
clang-tools-extra/trunk/unittests/clangd/ClangdTests.cpp
Modified:
clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.cpp
clang-tools-extra/trunk/clangd/ClangdServer.h
clang-tools-extra/trunk/clangd/ClangdUnit.cpp
clang-tools-extra/trunk/clangd/ClangdUnit.h
clang-tools-extra/trunk/clangd/ClangdUnitStore.h
clang-tools-extra/trunk/unittests/CMakeLists.txt

Modified: clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp?rev=303977=303976=303977=diff
==
--- clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp Fri May 26 07:26:51 2017
@@ -199,6 +199,7 @@ ClangdLSPServer::ClangdLSPServer(JSONOut
 : Out(Out),
   Server(llvm::make_unique(),
  llvm::make_unique(*this),
+ llvm::make_unique(),
  RunSynchronously) {}
 
 void ClangdLSPServer::run(std::istream ) {

Modified: clang-tools-extra/trunk/clangd/ClangdServer.cpp
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.cpp?rev=303977=303976=303977=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.cpp (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.cpp Fri May 26 07:26:51 2017
@@ -58,6 +58,10 @@ Position clangd::offsetToPosition(String
   return {Lines, Cols};
 }
 
+IntrusiveRefCntPtr RealFileSystemProvider::getFileSystem() {
+  return vfs::getRealFileSystem();
+}
+
 ClangdScheduler::ClangdScheduler(bool RunSynchronously)
 : RunSynchronously(RunSynchronously) {
   if (RunSynchronously) {
@@ -135,8 +139,10 @@ void ClangdScheduler::addToEnd(std::func
 
 ClangdServer::ClangdServer(std::unique_ptr CDB,
std::unique_ptr DiagConsumer,
+   std::unique_ptr FSProvider,
bool RunSynchronously)
 : CDB(std::move(CDB)), DiagConsumer(std::move(DiagConsumer)),
+  FSProvider(std::move(FSProvider)),
   PCHs(std::make_shared()),
   WorkScheduler(RunSynchronously) {}
 
@@ -150,10 +156,11 @@ void ClangdServer::addDocument(PathRef F
 
 assert(FileContents.Draft &&
"No contents inside a file that was scheduled for reparse");
-Units.runOnUnit(
-FileStr, *FileContents.Draft, *CDB, PCHs, [&](ClangdUnit const ) {
-  DiagConsumer->onDiagnosticsReady(FileStr, 
Unit.getLocalDiagnostics());
-});
+Units.runOnUnit(FileStr, *FileContents.Draft, *CDB, PCHs,
+FSProvider->getFileSystem(), [&](ClangdUnit const ) {
+  DiagConsumer->onDiagnosticsReady(
+  FileStr, Unit.getLocalDiagnostics());
+});
   });
 }
 
@@ -168,15 +175,22 @@ void ClangdServer::removeDocument(PathRe
   });
 }
 
+void ClangdServer::forceReparse(PathRef File) {
+  // The addDocument schedules the reparse even if the contents of the file
+  // never changed, so we just call it here.
+  addDocument(File, getDocument(File));
+}
+
 std::vector ClangdServer::codeComplete(PathRef File,
Position Pos) {
   auto FileContents = DraftMgr.getDraft(File);
   assert(FileContents.Draft && "codeComplete is called for non-added 
document");
 
   std::vector Result;
+  auto VFS = FSProvider->getFileSystem();
   Units.runOnUnitWithoutReparse(
-  File, *FileContents.Draft, *CDB, PCHs, [&](ClangdUnit ) {
-Result = Unit.codeComplete(*FileContents.Draft, Pos);
+  File, *FileContents.Draft, *CDB, PCHs, VFS, [&](ClangdUnit ) {
+Result = Unit.codeComplete(*FileContents.Draft, Pos, VFS);
   });
   return Result;
 }

Modified: clang-tools-extra/trunk/clangd/ClangdServer.h
URL: 
http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clangd/ClangdServer.h?rev=303977=303976=303977=diff
==
--- clang-tools-extra/trunk/clangd/ClangdServer.h (original)
+++ clang-tools-extra/trunk/clangd/ClangdServer.h Fri May 26 07:26:51 2017
@@ -50,6 +50,17 @@ public:
   std::vector Diagnostics) = 0;
 };
 
+class