hoy added inline comments.
================
Comment at: clang/test/CodeGen/unique-internal-linkage-names.cpp:48-49
+// LPIPELINE: Unique Internal Linkage Names
+// NPIPELINE: Running pass: UniqueInternalLinkageNamesPass
// PLAIN: @_ZL4glob = internal global
----------------
dblaikie wrote:
> aeubanks wrote:
> > dblaikie wrote:
> > > hoy wrote:
> > > > aeubanks wrote:
> > > > > hoy wrote:
> > > > > > aeubanks wrote:
> > > > > > > dblaikie wrote:
> > > > > > > > hoy wrote:
> > > > > > > > > dblaikie wrote:
> > > > > > > > > > Does this test validate the new behavior? (ie: does this
> > > > > > > > > > test fail without the LLVM changes and pass with it) Not
> > > > > > > > > > that it necessarily has to - since Clang isn't here to test
> > > > > > > > > > the LLVM behavior - perhaps this test is sufficient in
> > > > > > > > > > Clang to test that the code in BackendUtil works to enable
> > > > > > > > > > this pass.
> > > > > > > > > >
> > > > > > > > > > This could possibly be staged as independent commits -
> > > > > > > > > > adding the LLVM functionality in one commit, which would be
> > > > > > > > > > a no-op for Clang because it wouldn't be setting
> > > > > > > > > > PTO.UniqueLinkageNames - then committing the Clang change
> > > > > > > > > > that would remove the custom pass addition and set
> > > > > > > > > > PTO.UniqueLinkageNames - and then it'd probably be
> > > > > > > > > > reasonable to have this test be made a bit more explicit
> > > > > > > > > > (testing the pass manager structure/order) to show that
> > > > > > > > > > that Clang change had an effect: Moving the pass to the
> > > > > > > > > > desired location in the pass pipeline.
> > > > > > > > > This is a good question. No, this test does not validate the
> > > > > > > > > pipeline change on the LLVM side, since Clang shouldn't have
> > > > > > > > > knowledge about how the pipelines are arranged in LLVM. As
> > > > > > > > > you pointed out, the test here is to test if the specific
> > > > > > > > > pass is run and gives expected results.
> > > > > > > > >
> > > > > > > > > Thanks for the suggestion to break the Clang changes and LLVM
> > > > > > > > > changes apart which would make the testing more specific. The
> > > > > > > > > pipeline ordering could be tested with a LLVM test but that
> > > > > > > > > would require a LLVM switch setup for UniqueLinkageNames and
> > > > > > > > > I'm not sure there's a need for that switch except for
> > > > > > > > > testing.
> > > > > > > > > No, this test does not validate the pipeline change on the
> > > > > > > > > LLVM side, since Clang shouldn't have knowledge about how the
> > > > > > > > > pipelines are arranged in LLVM.
> > > > > > > >
> > > > > > > > "ish" - but Clang should have tests for changes to Clang,
> > > > > > > > ideally. Usually they can simply be testing LLVM's IR output
> > > > > > > > before it goes to LLVM for optimization/codegen - but for
> > > > > > > > features that don't have this serialization boundary that makes
> > > > > > > > testing and isolation clear/simple, it becomes a bit fuzzier.
> > > > > > > >
> > > > > > > > In this case, there is a clang change - from adding the pass
> > > > > > > > explicitly in Clang, to setting a parameter about how LLVM will
> > > > > > > > add the pass, and it has an observable effect. One way to test
> > > > > > > > this change while isolating the Clang test from further changes
> > > > > > > > to the pipeline in LLVM, would be to test that the pass ends up
> > > > > > > > somewhere in the LLVM-created part of the pass pipeline - the
> > > > > > > > parts that you can't get to from the way the original pass
> > > > > > > > addition was written in Clang. At least I assume that's the
> > > > > > > > case/what motivated the change from adding it in Clang to
> > > > > > > > adding it in LLVM?
> > > > > > > >
> > > > > > > > eg: if LLVM always forms passes {x, y, z} and Clang is able to
> > > > > > > > add passes before/after, say it always adds 'a' before and 'b'
> > > > > > > > after, to make {a, x, y, z, b} - and this new pass u was
> > > > > > > > previously added at the start to make {u, a, x, y, z, b} but
> > > > > > > > now needs to go in {a, x, y, u, z, b} you could test that 'u'
> > > > > > > > is after 'a' and before 'b', or between 'x' and 'z', etc. If
> > > > > > > > there's some other more clear/simple/reliable marker of where
> > > > > > > > the LLVM-created passes start/end in the structured dump,
> > > > > > > > that'd be good to use as a landmark to make such a test more
> > > > > > > > robust. If there's some meaningful pass that this pass always
> > > > > > > > needs to go after - testing that might be OK, even if it's
> > > > > > > > somewhat an implementation detail of LLVM - whatever's likely
> > > > > > > > to make the test more legible and more reliable/resilient to
> > > > > > > > unrelated changes would be good.
> > > > > > > >
> > > > > > > > > As you pointed out, the test here is to test if the specific
> > > > > > > > > pass is run and gives expected results.
> > > > > > > >
> > > > > > > > If that's the case, this test could be committed standalone,
> > > > > > > > before any of these other changes?
> > > > > > > >
> > > > > > > > > The pipeline ordering could be tested with a LLVM test but
> > > > > > > > > that would require a LLVM switch setup for UniqueLinkageNames
> > > > > > > > > and I'm not sure there's a need for that switch except for
> > > > > > > > > testing.
> > > > > > > >
> > > > > > > > That's OK, the entire 'opt' tool and all its switches only
> > > > > > > > exist for testing. eg:
> > > > > > > > https://github.com/llvm/llvm-project/blob/master/llvm/tools/opt/NewPMDriver.cpp#L284
> > > > > > > The point of this change is that UniqueInternalLinkageNamesPass
> > > > > > > should run before SampleProfileProbePass. That must make a
> > > > > > > difference in the output of something like `clang -emit-llvm
> > > > > > > -O1`, right? Maybe we can add a new clang test that checks for
> > > > > > > that new change in IR, no need to check -fdebug-pass-manager.
> > > > > > > (I'm not familiar with the passes, correct me if I'm wrong)
> > > > > > Maybe we can just keep the Clang test unchanged? What do you think?
> > > > > > Since it's basically testing the command line switch
> > > > > > `-funique-internal-linkage-names` works as expected, i.e, giving
> > > > > > unique linkage names, it probably shouldn't care where the renaming
> > > > > > happens exactly. Checking the pass order sounds a job to LLVM. I'll
> > > > > > make the LLVM test do that.
> > > > > >
> > > > > >
> > > > > > > The point of this change is that UniqueInternalLinkageNamesPass
> > > > > > > should run before SampleProfileProbePass.
> > > > > >
> > > > > > Yeah, that's the point. We should probably make an LLVM test for it
> > > > > > instead of a Clang test.
> > > > > An LLVM test sounds good, though you'll need a new cl::opt that the
> > > > > new option in PipelineTuningOptions defaults to (like other options
> > > > > in PipelineTuningOptions).
> > > > Yeah, I was thinking about that too. I will also need a switch to
> > > > trigger SampleProfileProbePass, like the exiting
> > > > `-new-pm-debug-info-for-profiling`.
> > > I'm a bit confused by this thread of discussion.
> > >
> > > Some fairly fundamental test architecture issues in the LLVM project:
> > > Code changes within a project should be tested within that project.
> > > Ideally they should test so as narrowly as possible so as not to produce
> > > failures due to unrelated changes.
> > >
> > > This is usually fairly easy with anything in IR (test that Clang produces
> > > certain IR, test that optimization passes optimize that IR in certain
> > > ways, test that certain IR produces certain machine code, etc) - but
> > > harder with things that are represented only in API surface area (ie:
> > > there's no serialization of PipelineTuningOptions between Clang and LLVM
> > > - if there was, we could test that given a clang command line argument,
> > > the PTO has a certain property - then separately in LLVM we'd test that,
> > > given that PTO, a certain pass pipeline is constructed with the relevant
> > > features). In the absence of a serialization layer, we make a best effort
> > > in some way or another.
> > >
> > > I think the best effort for a clang test for this clang change would be
> > > to dump the pass pipeline and ensure it has the property that's important
> > > - whatever property wasn't true before this change and is being made true
> > > by this change. Such as, as @aeubanks said, testing that
> > > UniqueInternalLinkageNamesPass comes before SampleProfileProbePass.
> > >
> > > I think it is important that this is tested in Clang and separately that
> > > the functionality is tested in LLVM (by exposing the PTO parameter
> > > through opt, like other PTO parameters), probably in a similar manner
> > > (testing that given this PTO parameter, the pass pipeline has a certain
> > > shape). All of that separate from testing the pass itself does certain
> > > things when it is run (& that testing would be done in isolation - just
> > > running the specified pass).
> > I'm not a fan of clang tests checking the output of -debug-pass-manager,
> > it's checking implementation details that clang doesn't control. I'd prefer
> > clang to just check that the pass ran somehow by examining the output IR
> > given the clang cc1 flag. For example, maybe some function has `__uniq` in
> > the name (maybe this test already checks something along these lines).
> > Checking the exact PTO doesn't seem important. And for this change IMO
> > clang doesn't need to test that some passes ran in some specific order,
> > that's now an LLVM implementation detail.
> >
> > The specifics of running UniqueInternalLinkageNamesPass before
> > SampleProfileProbePass is now an LLVM thing, so an LLVM test should test
> > that, whether it's checking -debug-pass-manager, or even better, checking
> > the IR for certain properties.
> There's certainly no great answers here, imho. It's going to be tradeoffs for
> sure.
>
> Clang tests executing the whole LLVM pipeline and checking the right answer
> out the otehr end means a lot more code under test - a lot more places that
> can have bugs that cause this test to fail that aren't just the one line in
> Clang the test is intended to test (it's not meant to test the LLVM
> functionality, that is tested in LLVM).
>
> Clang does control some aspects of the pass pipeline - in this case moving
> the pass being added by clang explicitly, to asking LLVM to do it.
> Admittedly, yeah, no there's other aspects of implementation detail - Clang
> doesn't need to have any knowledge of specific pass names, or that this
> functionality is implemented by a pass.
>
> All that said, as much as I don't find it great (tradeoffs for all answers
> here), yeah, I'm not going to veto an end-to-end test. I've certainly written
> them in the past when there really wasn't any other option
> (-fdebug-types-section, if I recall - MC flag with no observable effect until
> assembly is generated... no pass pipeline differences, etc (actually, maybe I
> just didn't test that at all, I forget which way I went - not ideal either
> way, to be sure)).
Thanks for all the discussion and suggestions here. I think we all agree on
making a LLVM test that checks the pipeline order as well as the output of that
particular pass. Regarding the Clang test, since there's no for-sure answer,
I'm inclined to leave it as is, i.e, without checking the exact PTO. This
sounds a bit more robust to me since we'd like to isolate LLVM changes from
Clang testing failures.
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D93656/new/
https://reviews.llvm.org/D93656
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
