Re: [PATCH] D47597: IRGen: Write .dwo files when -split-dwarf-file is used together with -fthinlto-index.

[Teresa Johnson via cfe-commits]

On Thu, May 31, 2018 at 12:00 PM Peter Collingbourne 
wrote:

>
>
> On Thu, May 31, 2018 at 11:54 AM, David Blaikie 
> wrote:
>
>>
>>
>> On Thu, May 31, 2018 at 11:20 AM Peter Collingbourne via Phabricator <
>> revi...@reviews.llvm.org> wrote:
>>
>>> pcc created this revision.
>>> pcc added reviewers: tejohnson, dblaikie.
>>> Herald added subscribers: JDevlieghere, hiraditya, eraman, inglorion,
>>> mehdi_amini.
>>>
>>> https://reviews.llvm.org/D47597
>>>
>>> Files:
>>>   clang/lib/CodeGen/BackendUtil.cpp
>>>   clang/test/CodeGen/thinlto-split-dwarf.c
>>>   llvm/include/llvm/LTO/Config.h
>>>   llvm/lib/LTO/LTOBackend.cpp
>>>
>>>
>>> Index: llvm/lib/LTO/LTOBackend.cpp
>>> ===
>>> --- llvm/lib/LTO/LTOBackend.cpp
>>> +++ llvm/lib/LTO/LTOBackend.cpp
>>> @@ -291,14 +291,19 @@
>>>  return;
>>>
>>>std::unique_ptr DwoOut;
>>> +  SmallString<1024> DwoFile(Conf.DwoPath);
>>>if (!Conf.DwoDir.empty()) {
>>>  std::error_code EC;
>>>  if (auto EC = llvm::sys::fs::create_directories(Conf.DwoDir))
>>>report_fatal_error("Failed to create directory " + Conf.DwoDir +
>>> ": " +
>>>   EC.message());
>>>
>>> -SmallString<1024> DwoFile(Conf.DwoDir);
>>> +DwoFile = Conf.DwoDir;
>>>  sys::path::append(DwoFile, std::to_string(Task) + ".dwo");
>>> +  }
>>> +
>>> +  if (!DwoFile.empty()) {
>>> +std::error_code EC;
>>>  TM->Options.MCOptions.SplitDwarfFile = DwoFile.str().str();
>>>  DwoOut = llvm::make_unique(DwoFile, EC,
>>> sys::fs::F_None);
>>>  if (EC)
>>> Index: llvm/include/llvm/LTO/Config.h
>>> ===
>>> --- llvm/include/llvm/LTO/Config.h
>>> +++ llvm/include/llvm/LTO/Config.h
>>> @@ -76,6 +76,11 @@
>>>/// The directory to store .dwo files.
>>>std::string DwoDir;
>>>
>>> +  /// The path to write a .dwo file to. This should generally only be
>>> used when
>>> +  /// running an individual backend directly via thinBackend(), as
>>> otherwise
>>> +  /// all .dwo files will be written to the same path.
>>> +  std::string DwoPath;
>>> +
>>>/// Optimization remarks file path.
>>>std::string RemarksFilename = "";
>>>
>>> Index: clang/test/CodeGen/thinlto-split-dwarf.c
>>> ===
>>> --- /dev/null
>>> +++ clang/test/CodeGen/thinlto-split-dwarf.c
>>> @@ -0,0 +1,21 @@
>>> +// REQUIRES: x86-registered-target
>>> +
>>> +// RUN: %clang_cc1 -debug-info-kind=limited -triple
>>> x86_64-unknown-linux-gnu \
>>> +// RUN:   -flto=thin -emit-llvm-bc \
>>> +// RUN:   -o %t.o %s
>>> +
>>> +// RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
>>> +// RUN:   -o %t2.index \
>>> +// RUN:   -r=%t.o,main,px
>>> +
>>> +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu \
>>> +// RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
>>> +// RUN:   -o %t.native.o -split-dwarf-file %t.native.dwo -x ir %t.o
>>>
>>
>> Can this be written in a single IR file yet (rather than frontend
>> compiling, indexing, then backend compiling), now that Teresa's implemented
>> some of the summary IR syntax?
>>
>
> I think the parsing part isn't implemented yet, so that wouldn't work.
>

Regarding this - the parsing support is coming along, slowly but surely.
I'd say I am about 75% done.
Teresa


> Besides, we would need two separate files anyway: the bitcode file (%t.o)
> and the combined summary (%t.o.thinlto.bc).
>
> Peter
>
>
>>
>>
>>> +
>>> +// RUN: llvm-readobj -sections %t.native.o | FileCheck --check-prefix=O
>>> %s
>>> +// RUN: llvm-readobj -sections %t.native.dwo | FileCheck
>>> --check-prefix=DWO %s
>>> +
>>> +// O-NOT: .dwo
>>> +// DWO: .dwo
>>> +
>>> +int main() {}
>>> Index: clang/lib/CodeGen/BackendUtil.cpp
>>> ===
>>> --- clang/lib/CodeGen/BackendUtil.cpp
>>> +++ clang/lib/CodeGen/BackendUtil.cpp
>>> @@ -1161,6 +1161,7 @@
>>>Conf.DebugPassManager = CGOpts.DebugPassManager;
>>>Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness;
>>>Conf.RemarksFilename = CGOpts.OptRecordFile;
>>> +  Conf.DwoPath = CGOpts.SplitDwarfFile;
>>>switch (Action) {
>>>case Backend_EmitNothing:
>>>  Conf.PreCodeGenModuleHook = [](size_t Task, const Module ) {
>>>
>>>
>>>
>
>
> --
> --
> Peter
>


-- 
Teresa Johnson |  Software Engineer |  tejohn...@google.com |  408-460-2413
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D47597: IRGen: Write .dwo files when -split-dwarf-file is used together with -fthinlto-index.

[Peter Collingbourne via cfe-commits]

On Thu, May 31, 2018 at 11:54 AM, David Blaikie  wrote:

>
>
> On Thu, May 31, 2018 at 11:20 AM Peter Collingbourne via Phabricator <
> revi...@reviews.llvm.org> wrote:
>
>> pcc created this revision.
>> pcc added reviewers: tejohnson, dblaikie.
>> Herald added subscribers: JDevlieghere, hiraditya, eraman, inglorion,
>> mehdi_amini.
>>
>> https://reviews.llvm.org/D47597
>>
>> Files:
>>   clang/lib/CodeGen/BackendUtil.cpp
>>   clang/test/CodeGen/thinlto-split-dwarf.c
>>   llvm/include/llvm/LTO/Config.h
>>   llvm/lib/LTO/LTOBackend.cpp
>>
>>
>> Index: llvm/lib/LTO/LTOBackend.cpp
>> ===
>> --- llvm/lib/LTO/LTOBackend.cpp
>> +++ llvm/lib/LTO/LTOBackend.cpp
>> @@ -291,14 +291,19 @@
>>  return;
>>
>>std::unique_ptr DwoOut;
>> +  SmallString<1024> DwoFile(Conf.DwoPath);
>>if (!Conf.DwoDir.empty()) {
>>  std::error_code EC;
>>  if (auto EC = llvm::sys::fs::create_directories(Conf.DwoDir))
>>report_fatal_error("Failed to create directory " + Conf.DwoDir +
>> ": " +
>>   EC.message());
>>
>> -SmallString<1024> DwoFile(Conf.DwoDir);
>> +DwoFile = Conf.DwoDir;
>>  sys::path::append(DwoFile, std::to_string(Task) + ".dwo");
>> +  }
>> +
>> +  if (!DwoFile.empty()) {
>> +std::error_code EC;
>>  TM->Options.MCOptions.SplitDwarfFile = DwoFile.str().str();
>>  DwoOut = llvm::make_unique(DwoFile, EC,
>> sys::fs::F_None);
>>  if (EC)
>> Index: llvm/include/llvm/LTO/Config.h
>> ===
>> --- llvm/include/llvm/LTO/Config.h
>> +++ llvm/include/llvm/LTO/Config.h
>> @@ -76,6 +76,11 @@
>>/// The directory to store .dwo files.
>>std::string DwoDir;
>>
>> +  /// The path to write a .dwo file to. This should generally only be
>> used when
>> +  /// running an individual backend directly via thinBackend(), as
>> otherwise
>> +  /// all .dwo files will be written to the same path.
>> +  std::string DwoPath;
>> +
>>/// Optimization remarks file path.
>>std::string RemarksFilename = "";
>>
>> Index: clang/test/CodeGen/thinlto-split-dwarf.c
>> ===
>> --- /dev/null
>> +++ clang/test/CodeGen/thinlto-split-dwarf.c
>> @@ -0,0 +1,21 @@
>> +// REQUIRES: x86-registered-target
>> +
>> +// RUN: %clang_cc1 -debug-info-kind=limited -triple
>> x86_64-unknown-linux-gnu \
>> +// RUN:   -flto=thin -emit-llvm-bc \
>> +// RUN:   -o %t.o %s
>> +
>> +// RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
>> +// RUN:   -o %t2.index \
>> +// RUN:   -r=%t.o,main,px
>> +
>> +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu \
>> +// RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
>> +// RUN:   -o %t.native.o -split-dwarf-file %t.native.dwo -x ir %t.o
>>
>
> Can this be written in a single IR file yet (rather than frontend
> compiling, indexing, then backend compiling), now that Teresa's implemented
> some of the summary IR syntax?
>

I think the parsing part isn't implemented yet, so that wouldn't work.
Besides, we would need two separate files anyway: the bitcode file (%t.o)
and the combined summary (%t.o.thinlto.bc).

Peter


>
>
>> +
>> +// RUN: llvm-readobj -sections %t.native.o | FileCheck --check-prefix=O
>> %s
>> +// RUN: llvm-readobj -sections %t.native.dwo | FileCheck
>> --check-prefix=DWO %s
>> +
>> +// O-NOT: .dwo
>> +// DWO: .dwo
>> +
>> +int main() {}
>> Index: clang/lib/CodeGen/BackendUtil.cpp
>> ===
>> --- clang/lib/CodeGen/BackendUtil.cpp
>> +++ clang/lib/CodeGen/BackendUtil.cpp
>> @@ -1161,6 +1161,7 @@
>>Conf.DebugPassManager = CGOpts.DebugPassManager;
>>Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness;
>>Conf.RemarksFilename = CGOpts.OptRecordFile;
>> +  Conf.DwoPath = CGOpts.SplitDwarfFile;
>>switch (Action) {
>>case Backend_EmitNothing:
>>  Conf.PreCodeGenModuleHook = [](size_t Task, const Module ) {
>>
>>
>>


-- 
-- 
Peter
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D47597: IRGen: Write .dwo files when -split-dwarf-file is used together with -fthinlto-index.

[David Blaikie via cfe-commits]

On Thu, May 31, 2018 at 11:20 AM Peter Collingbourne via Phabricator <
revi...@reviews.llvm.org> wrote:

> pcc created this revision.
> pcc added reviewers: tejohnson, dblaikie.
> Herald added subscribers: JDevlieghere, hiraditya, eraman, inglorion,
> mehdi_amini.
>
> https://reviews.llvm.org/D47597
>
> Files:
>   clang/lib/CodeGen/BackendUtil.cpp
>   clang/test/CodeGen/thinlto-split-dwarf.c
>   llvm/include/llvm/LTO/Config.h
>   llvm/lib/LTO/LTOBackend.cpp
>
>
> Index: llvm/lib/LTO/LTOBackend.cpp
> ===
> --- llvm/lib/LTO/LTOBackend.cpp
> +++ llvm/lib/LTO/LTOBackend.cpp
> @@ -291,14 +291,19 @@
>  return;
>
>std::unique_ptr DwoOut;
> +  SmallString<1024> DwoFile(Conf.DwoPath);
>if (!Conf.DwoDir.empty()) {
>  std::error_code EC;
>  if (auto EC = llvm::sys::fs::create_directories(Conf.DwoDir))
>report_fatal_error("Failed to create directory " + Conf.DwoDir + ":
> " +
>   EC.message());
>
> -SmallString<1024> DwoFile(Conf.DwoDir);
> +DwoFile = Conf.DwoDir;
>  sys::path::append(DwoFile, std::to_string(Task) + ".dwo");
> +  }
> +
> +  if (!DwoFile.empty()) {
> +std::error_code EC;
>  TM->Options.MCOptions.SplitDwarfFile = DwoFile.str().str();
>  DwoOut = llvm::make_unique(DwoFile, EC,
> sys::fs::F_None);
>  if (EC)
> Index: llvm/include/llvm/LTO/Config.h
> ===
> --- llvm/include/llvm/LTO/Config.h
> +++ llvm/include/llvm/LTO/Config.h
> @@ -76,6 +76,11 @@
>/// The directory to store .dwo files.
>std::string DwoDir;
>
> +  /// The path to write a .dwo file to. This should generally only be
> used when
> +  /// running an individual backend directly via thinBackend(), as
> otherwise
> +  /// all .dwo files will be written to the same path.
> +  std::string DwoPath;
> +
>/// Optimization remarks file path.
>std::string RemarksFilename = "";
>
> Index: clang/test/CodeGen/thinlto-split-dwarf.c
> ===
> --- /dev/null
> +++ clang/test/CodeGen/thinlto-split-dwarf.c
> @@ -0,0 +1,21 @@
> +// REQUIRES: x86-registered-target
> +
> +// RUN: %clang_cc1 -debug-info-kind=limited -triple
> x86_64-unknown-linux-gnu \
> +// RUN:   -flto=thin -emit-llvm-bc \
> +// RUN:   -o %t.o %s
> +
> +// RUN: llvm-lto2 run -thinlto-distributed-indexes %t.o \
> +// RUN:   -o %t2.index \
> +// RUN:   -r=%t.o,main,px
> +
> +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu \
> +// RUN:   -emit-obj -fthinlto-index=%t.o.thinlto.bc \
> +// RUN:   -o %t.native.o -split-dwarf-file %t.native.dwo -x ir %t.o
>

Can this be written in a single IR file yet (rather than frontend
compiling, indexing, then backend compiling), now that Teresa's implemented
some of the summary IR syntax?


> +
> +// RUN: llvm-readobj -sections %t.native.o | FileCheck --check-prefix=O %s
> +// RUN: llvm-readobj -sections %t.native.dwo | FileCheck
> --check-prefix=DWO %s
> +
> +// O-NOT: .dwo
> +// DWO: .dwo
> +
> +int main() {}
> Index: clang/lib/CodeGen/BackendUtil.cpp
> ===
> --- clang/lib/CodeGen/BackendUtil.cpp
> +++ clang/lib/CodeGen/BackendUtil.cpp
> @@ -1161,6 +1161,7 @@
>Conf.DebugPassManager = CGOpts.DebugPassManager;
>Conf.RemarksWithHotness = CGOpts.DiagnosticsWithHotness;
>Conf.RemarksFilename = CGOpts.OptRecordFile;
> +  Conf.DwoPath = CGOpts.SplitDwarfFile;
>switch (Action) {
>case Backend_EmitNothing:
>  Conf.PreCodeGenModuleHook = [](size_t Task, const Module ) {
>
>
>
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits