[lldb-dev] [Bug 45642] New: Step over misbehaves with multiple threads on Linux

[via lldb-dev]

https://bugs.llvm.org/show_bug.cgi?id=45642

Bug ID: 45642
   Summary: Step over misbehaves with multiple threads on Linux
   Product: lldb
   Version: 10.0
  Hardware: PC
OS: Linux
Status: NEW
  Severity: normal
  Priority: P
 Component: All Bugs
  Assignee: lldb-dev@lists.llvm.org
  Reporter: ja...@google.com
CC: jdevliegh...@apple.com, llvm-b...@lists.llvm.org

Stepping over a line can report a spurious SIGTRAP when the thread hits a real
(public) breakpoint while another thread hits the step-over plan's private
breakpoint.

A repro is shown below (it is not 100% deterministic, but seems to repro in
more than 50% of cases). We have two threads; with one thread, we step over a
function call and hits a public breakpoint there, the other thread hits a
private breakpoint that was placed by the first thread's step-over plan (more
precisely, by its step-out child plan). When the debugger stops, it selects
that thread that hit the private breakpoint and confusingly reports SIGTRAP. 

The repro console session:

$ cat u.cc
#include 

void g() {
  printf(".");
}

void f() {
  while (true) {
g();  // Break here, continue, then step over twice.
  }
}

int main() {
  std::thread t1(f);
  std::thread t2(f);

  t1.join();
  t2.join();
  return 0;
}

$ clang++ -O0 -g -pthread u.cc
$ lldb --version
lldb version 11.0.0
  clang revision cfd235e6fc7a2306644ee63bfea310d79084ef66
  llvm revision cfd235e6fc7a2306644ee63bfea310d79084ef66
$ lldb a.out
(lldb) target create "a.out"
Current executable set to 'a.out' (x86_64).
(lldb) b u.cc:9
Breakpoint 1: where = a.out`f() + 9 at u.cc:9:5, address = 0x004011f9
(lldb) r
Process 229470 launched: 'a.out' (x86_64)
Process 229470 stopped
* thread #2, name = 'a.out', stop reason = breakpoint 1.1
frame #0: 0x004011f9 a.out`f() at u.cc:9:5
   6
   7void f() {
   8  while (true) {
-> 9g();  // Break here, then step over.
   10 }
   11   }
   12   
(lldb) c
Process 229470 resuming
Process 229470 stopped
* thread #3, name = 'a.out', stop reason = breakpoint 1.1
frame #0: 0x004011f9 a.out`f() at u.cc:9:5
   6
   7void f() {
   8  while (true) {
-> 9g();  // Break here, then step over.
   10 }
   11   }
   12   
(lldb) n
Process 229470 stopped
* thread #3, name = 'a.out', stop reason = step over
frame #0: 0x004011fe a.out`f() at u.cc:8:3
   5}
   6
   7void f() {
-> 8  while (true) {
   9g();  // Break here, then step over.
   10 }
   11   }
(lldb) n
Process 229470 stopped
* thread #2, name = 'a.out', stop reason = signal SIGTRAP
frame #0: 0x004011fe a.out`f() at u.cc:8:3
   5}
   6
   7void f() {
-> 8  while (true) {
   9g();  // Break here, then step over.
   10 }
   11   }
  thread #3, name = 'a.out', stop reason = breakpoint 1.1
frame #0: 0x004011f9 a.out`f() at u.cc:9:5
   6
   7void f() {
   8  while (true) {
-> 9g();  // Break here, then step over.
   10 }
   11   }
   12


There is also a related problem where step-over seemingly steps into a
function. This happens when another thread hits a public breakpoint at the same
time as the step over plan single-steps.

$ lldb a.out
(lldb) target create "a.out"
Current executable set to 'a.out' (x86_64).
(lldb) b t.cc:9
Breakpoint 1: where = a.out`f() + 9 at t.cc:9:5, address = 0x00401209
(lldb) r
Process 193890 launched: 'jarin/a.out' (x86_64)
Process 193890 stopped
* thread #2, name = 'a.out', stop reason = breakpoint 1.1
frame #0: 0x00401209 a.out`f() at t.cc:9:5
   6
   7void f() {
   8  while (true) {
-> 9g();
   10 }
   11   }
   12   
(lldb) n
Process 193890 stopped
* thread #2, name = 'a.out', stop reason = step over
frame #0: 0x0040120e a.out`f() at t.cc:8:3
   5}
   6
   7void f() {
-> 8  while (true) {
   9g();
   10 }
   11   }
  thread #3, name = 'a.out', stop reason = breakpoint 1.1
frame #0: 0x00401209 a.out`f() at t.cc:9:5
   6
   7void f() {
   8  while (true) {
-> 9g();
   10 }
   11   }
   12   
(lldb) n
Process 193890 stopped
* thread #2, name = 'a.out', stop reason = breakpoint 1.1
frame #0: 0x00401209 a.out`f() at t.cc:9:5
   6
   7void f() {
   8  while (true) {
-> 9g();
   10 }
   11   }
   12   
  thread #3, name = 'a.out', stop reason = breakpoint 1.1
frame #0: 0x00401209 a.out`f() at t.cc:9:5
   6
   7void f() {
   8  while (true) {
-> 9g();
   10 }
   11   }
   12   
(lldb) 
Process 193890 stopped
* thread #2, name = 'a.out', stop reason = step over
frame #0: 0x0040120e a.out`f() at t.cc:8:3
   5}
   6
   7void f() {
-> 8  while (true) {
   9g();
   10 }
   11   }

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Philip Reames via lldb-dev]

On 4/22/20 2:35 PM, Richard Smith wrote:
On Wed, 22 Apr 2020 at 09:45, Philip Reames via cfe-dev 
mailto:cfe-...@lists.llvm.org>> wrote:


On 4/21/20 6:50 PM, Richard Smith wrote:


On Tue, 21 Apr 2020 at 17:00, Tom Stellard via llvm-dev
mailto:llvm-...@lists.llvm.org>> wrote:

On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev
mailto:cfe-...@lists.llvm.org>
>> wrote:
>
>     +1 to James's take
>
>     I'd prefer simplicity of implementation over perfection
here.
>
> If we end up with two different bug numbering systems,
that's a problem that we will be paying for for many years.
It's worth some investment now to avoid that problem. And it
doesn't seem like it really requires much investment.
>
> Here's another path we could take:
>
> 1) Fork the llvm repository to a private "bugs" repository.
Mirror the bugzilla issues there. Iterate until we're happy,
as per James's proposal.
> 2) Sync the forked repository to the llvm repository,
delete the llvm repository, rename "bugs" to "llvm", and make
it public.
>
> Then we'll have the first N bugs in llvm-project/llvm being
*exactly* the bugzilla bugs, and we'll have excised the
existing github issues that we want to pretend never existed
anyway.
>
>
> I think we've missed an important step in the planning
here: we've not agreed on a set of goals for the transition.
Here are mine:
>
>  * We end up with one single issue tracking system
containing all issues, both old and new, both open and closed.
>  * All links and references to existing bugs still work.
>  * We have a single bug numbering system covering all bugs,
and old bugs retain their numbers.

Why are the bug numbers important?


These numbers appear all over our codebase. PR[0-9] appears 3592
times in Clang testcases, plus 45 times in Clang source code and
119 times more as the file names of Clang testcases. If we add
inconvenience to looking up all of those, that makes maintenance
harder each time someone wants to look one of those up. (That's
probably a ~weekly occurrence for me.)


For this use case, a simple script and bulk change to update
references in source repo means the numbering can change
arbitrarily.  ~4k small mechanical changes is just not that much
to review for a one time update assuming you trust the number
remapping script and are just looking for overly aggressive regex
matches.

It's not quite as straightforward as you're suggesting: such a simple 
script would break a bunch of our CodeGen tests that match mangled 
names, if the length of any bug identifier changes. A grep for 
'_Z.*PR[0-9]' indicates that there are at least 254 of those that 
might need manual updating if we took this path.
We have an auto-updater for most llc scripts, but point taken.  My main 
point was this was one time, not that the one time was trivial.


(I don't have any quick fixes for your other mentioned cases.)

Another case I didn't think of before, but that seems very important: 
bug numbers appear in commit messages, and are primary context in 
understanding what the commit is doing and why. [We *could* go on a 
bulk history editing spree to fix those commit messages up (git 
filter-branch actually makes this fairly easy) -- but that too would 
create a little churn as everyone would needs to rebase all their work 
in progress on the rewritten master, and honestly, that sounds a lot 
scarier than any of the other things we've considered in this thread :)]
Agreed, history rewrite as a solution here should be rejected out of 
hand.  :)



Also, bug numbers appear in other bugs. I would assume we're not
going to be able to reliably figure out which numbers appearing
in a bug are bug numbers during the import process, so those
numbers will persist into the github issues world.

(In addition, I'm sure multiple groups have their own tracking
systems, web pages, documentation, etc. that contain references
to LLVM PR numbers. But maybe we shouldn't worry too much about
that.)

Could you help give some example use cases that require having
a non-intersecting set of bug numbers for bugzilla bugs and
github issues?


It makes conversing about bug numbers more difficult if you need
to clarify which system you're talking about. As a minor example,
we'd have to avoid saying "PR" for the new system in order to
avoid confusion, and get used to some new terminology, and
probably not use "bug 1234" to describe either system, because
  

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Richard Smith via lldb-dev]

On Wed, 22 Apr 2020 at 09:45, Philip Reames via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> On 4/21/20 6:50 PM, Richard Smith wrote:
>
> On Tue, 21 Apr 2020 at 17:00, Tom Stellard via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
>
>> On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>> > On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
>> cfe-...@lists.llvm.org > wrote:
>> >
>> > +1 to James's take
>> >
>> > I'd prefer simplicity of implementation over perfection here.
>> >
>> > If we end up with two different bug numbering systems, that's a problem
>> that we will be paying for for many years. It's worth some investment now
>> to avoid that problem. And it doesn't seem like it really requires much
>> investment.
>> >
>> > Here's another path we could take:
>> >
>> > 1) Fork the llvm repository to a private "bugs" repository. Mirror the
>> bugzilla issues there. Iterate until we're happy, as per James's proposal.
>> > 2) Sync the forked repository to the llvm repository, delete the llvm
>> repository, rename "bugs" to "llvm", and make it public.
>> >
>> > Then we'll have the first N bugs in llvm-project/llvm being *exactly*
>> the bugzilla bugs, and we'll have excised the existing github issues that
>> we want to pretend never existed anyway.
>> >
>> >
>> > I think we've missed an important step in the planning here: we've not
>> agreed on a set of goals for the transition. Here are mine:
>> >
>> >  * We end up with one single issue tracking system containing all
>> issues, both old and new, both open and closed.
>> >  * All links and references to existing bugs still work.
>> >  * We have a single bug numbering system covering all bugs, and old
>> bugs retain their numbers.
>>
>> Why are the bug numbers important?
>
>
> These numbers appear all over our codebase. PR[0-9] appears 3592 times in
> Clang testcases, plus 45 times in Clang source code and 119 times more as
> the file names of Clang testcases. If we add inconvenience to looking up
> all of those, that makes maintenance harder each time someone wants to look
> one of those up. (That's probably a ~weekly occurrence for me.)
>
> For this use case, a simple script and bulk change to update references in
> source repo means the numbering can change arbitrarily.  ~4k small
> mechanical changes is just not that much to review for a one time update
> assuming you trust the number remapping script and are just looking for
> overly aggressive regex matches.
>
It's not quite as straightforward as you're suggesting: such a simple
script would break a bunch of our CodeGen tests that match mangled names,
if the length of any bug identifier changes. A grep for '_Z.*PR[0-9]'
indicates that there are at least 254 of those that might need manual
updating if we took this path.

(I don't have any quick fixes for your other mentioned cases.)
>
Another case I didn't think of before, but that seems very important: bug
numbers appear in commit messages, and are primary context in understanding
what the commit is doing and why. [We *could* go on a bulk history editing
spree to fix those commit messages up (git filter-branch actually makes
this fairly easy) -- but that too would create a little churn as everyone
would needs to rebase all their work in progress on the rewritten master,
and honestly, that sounds a lot scarier than any of the other things we've
considered in this thread :)]

Also, bug numbers appear in other bugs. I would assume we're not going to
> be able to reliably figure out which numbers appearing in a bug are bug
> numbers during the import process, so those numbers will persist into the
> github issues world.
>
> (In addition, I'm sure multiple groups have their own tracking systems,
> web pages, documentation, etc. that contain references to LLVM PR numbers.
> But maybe we shouldn't worry too much about that.)
>
> Could you help give some example use cases that require having
>> a non-intersecting set of bug numbers for bugzilla bugs and github issues?
>>
>
> It makes conversing about bug numbers more difficult if you need to
> clarify which system you're talking about. As a minor example, we'd have to
> avoid saying "PR" for the new system in order to avoid confusion, and get
> used to some new terminology, and probably not use "bug 1234" to describe
> either system, because that would be ambiguous. None of these individual
> factors seems like a huge disruption, but they all seem like inconvenience
> we should prefer to avoid if possible.
>
> -Tom
>>
>>
>> >
>> > It sounds like we don't all agree that the last point is important, but
>> if we can achieve it without any significant additional cost, why not do so?
>> >
>> > Philip
>> >
>> > On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>> >> In a previous discussion, one other suggestion had been to migrate
>> all the bugzilla bugs to a separate initially-private "bug archive"
>> repository in github. This has a few 

Re: [lldb-dev] [llvm-dev] [cfe-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Tom Stellard via lldb-dev]

On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev 
> mailto:cfe-...@lists.llvm.org>> wrote:
> 
> +1 to James's take
> 
> I'd prefer simplicity of implementation over perfection here.
> 
> If we end up with two different bug numbering systems, that's a problem that 
> we will be paying for for many years. It's worth some investment now to avoid 
> that problem. And it doesn't seem like it really requires much investment.
> 
> Here's another path we could take:
> 
> 1) Fork the llvm repository to a private "bugs" repository. Mirror the 
> bugzilla issues there. Iterate until we're happy, as per James's proposal.
> 2) Sync the forked repository to the llvm repository, delete the llvm 
> repository, rename "bugs" to "llvm", and make it public.
> 

I would be nervous about deleting the current repo for any reason.
It may work, but the implications of doing this are unclear.  I think
an alternative would be to copy the existing GitHub issues from the llvm repo
into the private "bugs" repository after the bugzilla import is done.

Then we can just copy and overwrite any existing GitHub issues in the
llvm repository with the bugzilla data for the corresponding ID.  e.g. For
GitHub issue #1, delete all the comments and replace them with the
comments (and other data) from bugzilla issue #1.

-Tom

> Then we'll have the first N bugs in llvm-project/llvm being *exactly* the 
> bugzilla bugs, and we'll have excised the existing github issues that we want 
> to pretend never existed anyway.
> 
> 
> I think we've missed an important step in the planning here: we've not agreed 
> on a set of goals for the transition. Here are mine:
> 
>  * We end up with one single issue tracking system containing all issues, 
> both old and new, both open and closed.
>  * All links and references to existing bugs still work.
>  * We have a single bug numbering system covering all bugs, and old bugs 
> retain their numbers.
> 
> It sounds like we don't all agree that the last point is important, but if we 
> can achieve it without any significant additional cost, why not do so?
> 
> Philip
> 
> On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>> In a previous discussion, one other suggestion had been to migrate all 
>> the bugzilla bugs to a separate initially-private "bug archive" repository 
>> in github. This has a few benefits:
>> 1. If the migration is messed up, the repo can be deleted, and the 
>> process run again, until we get a result we like.
>> 2. The numbering can be fully-controlled.
>> Once the bugs are migrated to /some/ github repository, individual 
>> issues can then be "moved" between repositories, and github will redirect 
>> from the movefrom-repository's bug to the target repository's bug.
>>
>> We could also just have llvm.org/PR###  be 
>> the url only for legacy bugzilla issue numbers -- and have it use a file 
>> listing the mappings of bugzilla id -> github id to generate the redirects. 
>> (GCC just did this recently for svn revision number redirections, 
>> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
>>
>> Then we could introduce a new naming scheme for github issue shortlinks.
>>
>> On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev 
>> mailto:llvm-...@lists.llvm.org>> wrote:
>>
>> On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev 
>> mailto:llvm-...@lists.llvm.org>> wrote:
>>
>> Hi,
>>
>> I wanted to continue discussing the plan to migrate from 
>> Bugzilla to Github.
>> It was suggested that I start a new thread and give a summary of 
>> the proposal
>> and what has changed since it was originally proposed in October.
>>
>> == Here is the original proposal:
>>
>> http://lists.llvm.org/pipermail/llvm-dev/2019-October/136162.html
>>
>> == What has changed:
>>
>> * You will be able to subscribe to notifications for a specific 
>> issue
>>   labels.  We have a proof of concept notification system using 
>> github actions
>>   that will be used for this.
>>
>> * Emails will be sent to llvm-bugs when issues are opened or 
>> closed.
>>
>> * We have the initial list of labels: 
>> https://github.com/llvm/llvm-project/labels
>>
>> == Remaining issue:
>>
>> * There is one remaining issue that I don't feel we have 
>> consensus on,
>> and that is what to do with bugs in the existing bugzilla.  Here 
>> are some options
>> that we have discussed:
>>
>> 1. Switch to GitHub issues for new bugs only.  Bugs filed in 
>> bugzilla that are
>> still active will be updated there until they are closed.  This 
>> means that over
>> time the number of active bugs in bugzilla will slowly decrease 
>> as bugs are closed
>>   

Re: [lldb-dev] [llvm-dev] [cfe-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Tom Stellard via lldb-dev]

On 04/21/2020 06:50 PM, Richard Smith wrote:
> On Tue, 21 Apr 2020 at 17:00, Tom Stellard via llvm-dev 
> mailto:llvm-...@lists.llvm.org>> wrote:
> 
> On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> > On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev 
> mailto:cfe-...@lists.llvm.org> 
> >> wrote:
> >
> > +1 to James's take
> >
> > I'd prefer simplicity of implementation over perfection here.
> >
> > If we end up with two different bug numbering systems, that's a problem 
> that we will be paying for for many years. It's worth some investment now to 
> avoid that problem. And it doesn't seem like it really requires much 
> investment.
> >
> > Here's another path we could take:
> >
> > 1) Fork the llvm repository to a private "bugs" repository. Mirror the 
> bugzilla issues there. Iterate until we're happy, as per James's proposal.
> > 2) Sync the forked repository to the llvm repository, delete the llvm 
> repository, rename "bugs" to "llvm", and make it public.
> >
> > Then we'll have the first N bugs in llvm-project/llvm being *exactly* 
> the bugzilla bugs, and we'll have excised the existing github issues that we 
> want to pretend never existed anyway.
> >
> >
> > I think we've missed an important step in the planning here: we've not 
> agreed on a set of goals for the transition. Here are mine:
> >
> >  * We end up with one single issue tracking system containing all 
> issues, both old and new, both open and closed.
> >  * All links and references to existing bugs still work.
> >  * We have a single bug numbering system covering all bugs, and old 
> bugs retain their numbers.
> 
> Why are the bug numbers important?
> 
> 
> These numbers appear all over our codebase. PR[0-9] appears 3592 times in 
> Clang testcases, plus 45 times in Clang source code and 119 times more as the 
> file names of Clang testcases. If we add inconvenience to looking up all of 
> those, that makes maintenance harder each time someone wants to look one of 
> those up. (That's probably a ~weekly occurrence for me.)
> 

Having a new number scheme does not mean we have to break all the
llvm.org/PR links.  These could still be used to look up old
bugs even if we change bug notation to GH-.

> Also, bug numbers appear in other bugs. I would assume we're not going to be 
> able to reliably figure out which numbers appearing in a bug are bug numbers 
> during the import process, so those numbers will persist into the github 
> issues world.
> 

This is a good point and not something that I had thought of.
I think maintaining the bug links in the bugs is something that could
be done during the import.   e.g replace references to bug# x with
links to llvm.org/PR.  We will have to investigate this more.

Thanks,
Tom

> (In addition, I'm sure multiple groups have their own tracking systems, web 
> pages, documentation, etc. that contain references to LLVM PR numbers. But 
> maybe we shouldn't worry too much about that.)
> 
> Could you help give some example use cases that require having
> a non-intersecting set of bug numbers for bugzilla bugs and github issues?
> 
> 
> It makes conversing about bug numbers more difficult if you need to clarify 
> which system you're talking about. As a minor example, we'd have to avoid 
> saying "PR" for the new system in order to avoid confusion, and get used to 
> some new terminology, and probably not use "bug 1234" to describe either 
> system, because that would be ambiguous. None of these individual factors 
> seems like a huge disruption, but they all seem like inconvenience we should 
> prefer to avoid if possible.
> 
> -Tom
> 
> 
> >
> > It sounds like we don't all agree that the last point is important, but 
> if we can achieve it without any significant additional cost, why not do so?
> >
> > Philip
> >
> > On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
> >> In a previous discussion, one other suggestion had been to migrate 
> all the bugzilla bugs to a separate initially-private "bug archive" 
> repository in github. This has a few benefits:
> >> 1. If the migration is messed up, the repo can be deleted, and the 
> process run again, until we get a result we like.
> >> 2. The numbering can be fully-controlled.
> >> Once the bugs are migrated to /some/ github repository, individual 
> issues can then be "moved" between repositories, and github will redirect 
> from the movefrom-repository's bug to the target repository's bug.
> >>
> >> We could also just have llvm.org/PR 
>   be the url only for 
> legacy bugzilla issue numbers -- and have it use a file listing the mappings 
> of bugzilla id -> github id to generate the redirects. (GCC just did 

Re: [lldb-dev] [llvm-dev] [cfe-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Philip Reames via lldb-dev]

On 4/21/20 6:50 PM, Richard Smith wrote:
On Tue, 21 Apr 2020 at 17:00, Tom Stellard via llvm-dev 
mailto:llvm-...@lists.llvm.org>> wrote:


On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev
mailto:cfe-...@lists.llvm.org>
>>
wrote:
>
>     +1 to James's take
>
>     I'd prefer simplicity of implementation over perfection here.
>
> If we end up with two different bug numbering systems, that's a
problem that we will be paying for for many years. It's worth some
investment now to avoid that problem. And it doesn't seem like it
really requires much investment.
>
> Here's another path we could take:
>
> 1) Fork the llvm repository to a private "bugs" repository.
Mirror the bugzilla issues there. Iterate until we're happy, as
per James's proposal.
> 2) Sync the forked repository to the llvm repository, delete the
llvm repository, rename "bugs" to "llvm", and make it public.
>
> Then we'll have the first N bugs in llvm-project/llvm being
*exactly* the bugzilla bugs, and we'll have excised the existing
github issues that we want to pretend never existed anyway.
>
>
> I think we've missed an important step in the planning here:
we've not agreed on a set of goals for the transition. Here are mine:
>
>  * We end up with one single issue tracking system containing
all issues, both old and new, both open and closed.
>  * All links and references to existing bugs still work.
>  * We have a single bug numbering system covering all bugs, and
old bugs retain their numbers.

Why are the bug numbers important?


These numbers appear all over our codebase. PR[0-9] appears 3592 times 
in Clang testcases, plus 45 times in Clang source code and 119 times 
more as the file names of Clang testcases. If we add inconvenience to 
looking up all of those, that makes maintenance harder each time 
someone wants to look one of those up. (That's probably a ~weekly 
occurrence for me.)


For this use case, a simple script and bulk change to update references 
in source repo means the numbering can change arbitrarily.  ~4k small 
mechanical changes is just not that much to review for a one time update 
assuming you trust the number remapping script and are just looking for 
overly aggressive regex matches.


(I don't have any quick fixes for your other mentioned cases.)



Also, bug numbers appear in other bugs. I would assume we're not going 
to be able to reliably figure out which numbers appearing in a bug are 
bug numbers during the import process, so those numbers will persist 
into the github issues world.


(In addition, I'm sure multiple groups have their own tracking 
systems, web pages, documentation, etc. that contain references to 
LLVM PR numbers. But maybe we shouldn't worry too much about that.)


Could you help give some example use cases that require having
a non-intersecting set of bug numbers for bugzilla bugs and github
issues?


It makes conversing about bug numbers more difficult if you need to 
clarify which system you're talking about. As a minor example, we'd 
have to avoid saying "PR" for the new system in order to avoid 
confusion, and get used to some new terminology, and probably not use 
"bug 1234" to describe either system, because that would be ambiguous. 
None of these individual factors seems like a huge disruption, but 
they all seem like inconvenience we should prefer to avoid if possible.


-Tom


>
> It sounds like we don't all agree that the last point is
important, but if we can achieve it without any significant
additional cost, why not do so?
>
>     Philip
>
>     On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>>     In a previous discussion, one other suggestion had been to
migrate all the bugzilla bugs to a separate initially-private "bug
archive" repository in github. This has a few benefits:
>>     1. If the migration is messed up, the repo can be deleted,
and the process run again, until we get a result we like.
>>     2. The numbering can be fully-controlled.
>>     Once the bugs are migrated to /some/ github repository,
individual issues can then be "moved" between repositories, and
github will redirect from the movefrom-repository's bug to the
target repository's bug.
>>
>>     We could also just have llvm.org/PR###
  be the url
only for legacy bugzilla issue numbers -- and have it use a file
listing the mappings of bugzilla id -> github id to generate the
redirects. (GCC just did this recently for svn revision number
redirections,
https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
>>
>>     Then we could introduce a new 

Re: [lldb-dev] [llvm-dev] [cfe-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[James Y Knight via lldb-dev]

Custom prefixes are intended for autolinks to external systems -- I suspect
it would not work properly (but have not tested) if you used it to refer
back to github itself. E.g. putting reverse links in issues you refer to,
or closing an issue when writing "closes CUSTOM-123" in a commit message.

On Wed, Apr 22, 2020 at 8:32 AM Anton Korobeynikov 
wrote:

> GitHub also supports custom prefixes for the issues. However, here is
> another limitation: the prefix must be at least 3 letters, so we
> cannot, for example, autolink PR1234 issues. Already asked whether
> this restriction could be lifted.
>
> On Wed, Apr 22, 2020 at 3:15 PM James Y Knight via llvm-dev
>  wrote:
> >
> > GitHub canonically uses "#NNN" to refer to its bugs or pull requests,
> and also supports "GH-NNN". We'll want to switch to one of those schemes,
> so that automatic linking works properly. So, in that case, PR1234 ==
> legacy issue, #1234 or GH-1234 == new issue.
> >
> > (See
> https://help.github.com/en/github/writing-on-github/autolinked-references-and-urls
> )
> >
> > On Tue, Apr 21, 2020, 10:43 PM Johannes Doerfert via cfe-dev <
> cfe-...@lists.llvm.org> wrote:
> >>
> >>
> >> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
> >> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> >> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
> cfe-...@lists.llvm.org > wrote:
> >> >>
> >> >>  +1 to James's take
> >> >>
> >> >>  I'd prefer simplicity of implementation over perfection here.
> >> >>
> >> >> If we end up with two different bug numbering systems, that's a
> problem that we will be paying for for many years. It's worth some
> investment now to avoid that problem. And it doesn't seem like it really
> requires much investment.
> >> >>
> >> >> Here's another path we could take:
> >> >>
> >> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror
> the bugzilla issues there. Iterate until we're happy, as per James's
> proposal.
> >> >> 2) Sync the forked repository to the llvm repository, delete the
> llvm repository, rename "bugs" to "llvm", and make it public.
> >> >>
> >> >> Then we'll have the first N bugs in llvm-project/llvm being
> *exactly* the bugzilla bugs, and we'll have excised the existing github
> issues that we want to pretend never existed anyway.
> >> >>
> >> >>
> >> >> I think we've missed an important step in the planning here: we've
> not agreed on a set of goals for the transition. Here are mine:
> >> >>
> >> >>   * We end up with one single issue tracking system containing all
> issues, both old and new, both open and closed.
> >> >>   * All links and references to existing bugs still work.
> >> >>   * We have a single bug numbering system covering all bugs, and old
> bugs retain their numbers.
> >> > Why are the bug numbers important?  Could you help give some example
> use cases that require having
> >> > a non-intersecting set of bug numbers for bugzilla bugs and github
> issues?
> >>
> >>
> >> While I have no experience in bugzilla or github tooling, the two step
> >> process described by Richard doesn't seem to be very complicated.
> >>
> >>
> >> As mentioned by others, we have commits and tests (and sometimes source
> >> files) that explicitly mention bug numbers. I do regularly look up bugs
> >> from a decade ago to determine if a test or some code still has
> >> relevance or not. If PR3214 can be one of two bugs, it does not only
> >> increase lookup time but also add confusion to everyone involved.
> >>
> >>
> >> Cheers,
> >>
> >>Johannes
> >>
> >>
> >>
> >> > -Tom
> >> >
> >> >
> >> >> It sounds like we don't all agree that the last point is important,
> but if we can achieve it without any significant additional cost, why not
> do so?
> >> >>
> >> >>  Philip
> >> >>
> >> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
> >> >>>  In a previous discussion, one other suggestion had been to
> migrate all the bugzilla bugs to a separate initially-private "bug archive"
> repository in github. This has a few benefits:
> >> >>>  1. If the migration is messed up, the repo can be deleted, and
> the process run again, until we get a result we like.
> >> >>>  2. The numbering can be fully-controlled.
> >> >>>  Once the bugs are migrated to /some/ github repository,
> individual issues can then be "moved" between repositories, and github will
> redirect from the movefrom-repository's bug to the target repository's bug.
> >> >>>
> >> >>>  We could also just have llvm.org/PR###
>   be the url only
> for legacy bugzilla issue numbers -- and have it use a file listing the
> mappings of bugzilla id -> github id to generate the redirects. (GCC just
> did this recently for svn revision number redirections,
> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
> >> >>>
> >> >>>  Then we could introduce a new naming scheme for github issue
> 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[James Henderson via lldb-dev]

Github may do things in a canonical way, but I think you'll find that lots
of people will refer to them by other means in review comments, email
threads, etc. Let's avoid any risk of ambiguity...

Also, there's no guarantee in the future that Github won't decide to start
auto-linking PR1234 as well, which would just get even more confusing.

On Wed, 22 Apr 2020 at 13:14, James Y Knight via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> GitHub canonically uses "#NNN" to refer to its bugs or pull requests, and
> also supports "GH-NNN". We'll want to switch to one of those schemes, so
> that automatic linking works properly. So, in that case, PR1234 == legacy
> issue, #1234 or GH-1234 == new issue.
>
> (See
> https://help.github.com/en/github/writing-on-github/autolinked-references-and-urls
> )
>
> On Tue, Apr 21, 2020, 10:43 PM Johannes Doerfert via cfe-dev <
> cfe-...@lists.llvm.org> wrote:
>
>>
>> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
>> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
>> cfe-...@lists.llvm.org > wrote:
>> >>
>> >>  +1 to James's take
>> >>
>> >>  I'd prefer simplicity of implementation over perfection here.
>> >>
>> >> If we end up with two different bug numbering systems, that's a
>> problem that we will be paying for for many years. It's worth some
>> investment now to avoid that problem. And it doesn't seem like it really
>> requires much investment.
>> >>
>> >> Here's another path we could take:
>> >>
>> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the
>> bugzilla issues there. Iterate until we're happy, as per James's proposal.
>> >> 2) Sync the forked repository to the llvm repository, delete the llvm
>> repository, rename "bugs" to "llvm", and make it public.
>> >>
>> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly*
>> the bugzilla bugs, and we'll have excised the existing github issues that
>> we want to pretend never existed anyway.
>> >>
>> >>
>> >> I think we've missed an important step in the planning here: we've not
>> agreed on a set of goals for the transition. Here are mine:
>> >>
>> >>   * We end up with one single issue tracking system containing all
>> issues, both old and new, both open and closed.
>> >>   * All links and references to existing bugs still work.
>> >>   * We have a single bug numbering system covering all bugs, and old
>> bugs retain their numbers.
>> > Why are the bug numbers important?  Could you help give some example
>> use cases that require having
>> > a non-intersecting set of bug numbers for bugzilla bugs and github
>> issues?
>>
>>
>> While I have no experience in bugzilla or github tooling, the two step
>> process described by Richard doesn't seem to be very complicated.
>>
>>
>> As mentioned by others, we have commits and tests (and sometimes source
>> files) that explicitly mention bug numbers. I do regularly look up bugs
>> from a decade ago to determine if a test or some code still has
>> relevance or not. If PR3214 can be one of two bugs, it does not only
>> increase lookup time but also add confusion to everyone involved.
>>
>>
>> Cheers,
>>
>>Johannes
>>
>>
>>
>> > -Tom
>> >
>> >
>> >> It sounds like we don't all agree that the last point is important,
>> but if we can achieve it without any significant additional cost, why not
>> do so?
>> >>
>> >>  Philip
>> >>
>> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>> >>>  In a previous discussion, one other suggestion had been to
>> migrate all the bugzilla bugs to a separate initially-private "bug archive"
>> repository in github. This has a few benefits:
>> >>>  1. If the migration is messed up, the repo can be deleted, and
>> the process run again, until we get a result we like.
>> >>>  2. The numbering can be fully-controlled.
>> >>>  Once the bugs are migrated to /some/ github repository,
>> individual issues can then be "moved" between repositories, and github will
>> redirect from the movefrom-repository's bug to the target repository's bug.
>> >>>
>> >>>  We could also just have llvm.org/PR###
>>   be the url only
>> for legacy bugzilla issue numbers -- and have it use a file listing the
>> mappings of bugzilla id -> github id to generate the redirects. (GCC just
>> did this recently for svn revision number redirections,
>> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
>> >>>
>> >>>  Then we could introduce a new naming scheme for github issue
>> shortlinks.
>> >>>
>> >>>  On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev <
>> llvm-...@lists.llvm.org > wrote:
>> >>>
>> >>>  On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev <
>> llvm-...@lists.llvm.org > wrote:
>> >>>
>> >>>  Hi,
>> >>>
>> >>> 

Re: [lldb-dev] [llvm-dev] [cfe-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Anton Korobeynikov via lldb-dev]

GitHub also supports custom prefixes for the issues. However, here is
another limitation: the prefix must be at least 3 letters, so we
cannot, for example, autolink PR1234 issues. Already asked whether
this restriction could be lifted.

On Wed, Apr 22, 2020 at 3:15 PM James Y Knight via llvm-dev
 wrote:
>
> GitHub canonically uses "#NNN" to refer to its bugs or pull requests, and 
> also supports "GH-NNN". We'll want to switch to one of those schemes, so that 
> automatic linking works properly. So, in that case, PR1234 == legacy issue, 
> #1234 or GH-1234 == new issue.
>
> (See 
> https://help.github.com/en/github/writing-on-github/autolinked-references-and-urls)
>
> On Tue, Apr 21, 2020, 10:43 PM Johannes Doerfert via cfe-dev 
>  wrote:
>>
>>
>> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
>> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev 
>> >> mailto:cfe-...@lists.llvm.org>> wrote:
>> >>
>> >>  +1 to James's take
>> >>
>> >>  I'd prefer simplicity of implementation over perfection here.
>> >>
>> >> If we end up with two different bug numbering systems, that's a problem 
>> >> that we will be paying for for many years. It's worth some investment now 
>> >> to avoid that problem. And it doesn't seem like it really requires much 
>> >> investment.
>> >>
>> >> Here's another path we could take:
>> >>
>> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the 
>> >> bugzilla issues there. Iterate until we're happy, as per James's proposal.
>> >> 2) Sync the forked repository to the llvm repository, delete the llvm 
>> >> repository, rename "bugs" to "llvm", and make it public.
>> >>
>> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly* the 
>> >> bugzilla bugs, and we'll have excised the existing github issues that we 
>> >> want to pretend never existed anyway.
>> >>
>> >>
>> >> I think we've missed an important step in the planning here: we've not 
>> >> agreed on a set of goals for the transition. Here are mine:
>> >>
>> >>   * We end up with one single issue tracking system containing all 
>> >> issues, both old and new, both open and closed.
>> >>   * All links and references to existing bugs still work.
>> >>   * We have a single bug numbering system covering all bugs, and old bugs 
>> >> retain their numbers.
>> > Why are the bug numbers important?  Could you help give some example use 
>> > cases that require having
>> > a non-intersecting set of bug numbers for bugzilla bugs and github issues?
>>
>>
>> While I have no experience in bugzilla or github tooling, the two step
>> process described by Richard doesn't seem to be very complicated.
>>
>>
>> As mentioned by others, we have commits and tests (and sometimes source
>> files) that explicitly mention bug numbers. I do regularly look up bugs
>> from a decade ago to determine if a test or some code still has
>> relevance or not. If PR3214 can be one of two bugs, it does not only
>> increase lookup time but also add confusion to everyone involved.
>>
>>
>> Cheers,
>>
>>Johannes
>>
>>
>>
>> > -Tom
>> >
>> >
>> >> It sounds like we don't all agree that the last point is important, but 
>> >> if we can achieve it without any significant additional cost, why not do 
>> >> so?
>> >>
>> >>  Philip
>> >>
>> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>> >>>  In a previous discussion, one other suggestion had been to migrate 
>> >>> all the bugzilla bugs to a separate initially-private "bug archive" 
>> >>> repository in github. This has a few benefits:
>> >>>  1. If the migration is messed up, the repo can be deleted, and the 
>> >>> process run again, until we get a result we like.
>> >>>  2. The numbering can be fully-controlled.
>> >>>  Once the bugs are migrated to /some/ github repository, individual 
>> >>> issues can then be "moved" between repositories, and github will 
>> >>> redirect from the movefrom-repository's bug to the target repository's 
>> >>> bug.
>> >>>
>> >>>  We could also just have llvm.org/PR###  
>> >>> be the url only for legacy bugzilla issue numbers -- and have it use a 
>> >>> file listing the mappings of bugzilla id -> github id to generate the 
>> >>> redirects. (GCC just did this recently for svn revision number 
>> >>> redirections, https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
>> >>>
>> >>>  Then we could introduce a new naming scheme for github issue 
>> >>> shortlinks.
>> >>>
>> >>>  On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev 
>> >>> mailto:llvm-...@lists.llvm.org>> wrote:
>> >>>
>> >>>  On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev 
>> >>> mailto:llvm-...@lists.llvm.org>> wrote:
>> >>>
>> >>>  Hi,
>> >>>
>> >>>  I wanted to continue discussing the plan to migrate from 
>> >>> Bugzilla to Github.
>> >>>  It was suggested that I 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[James Y Knight via lldb-dev]

GitHub canonically uses "#NNN" to refer to its bugs or pull requests, and
also supports "GH-NNN". We'll want to switch to one of those schemes, so
that automatic linking works properly. So, in that case, PR1234 == legacy
issue, #1234 or GH-1234 == new issue.

(See
https://help.github.com/en/github/writing-on-github/autolinked-references-and-urls
)

On Tue, Apr 21, 2020, 10:43 PM Johannes Doerfert via cfe-dev <
cfe-...@lists.llvm.org> wrote:

>
> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
> cfe-...@lists.llvm.org > wrote:
> >>
> >>  +1 to James's take
> >>
> >>  I'd prefer simplicity of implementation over perfection here.
> >>
> >> If we end up with two different bug numbering systems, that's a problem
> that we will be paying for for many years. It's worth some investment now
> to avoid that problem. And it doesn't seem like it really requires much
> investment.
> >>
> >> Here's another path we could take:
> >>
> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the
> bugzilla issues there. Iterate until we're happy, as per James's proposal.
> >> 2) Sync the forked repository to the llvm repository, delete the llvm
> repository, rename "bugs" to "llvm", and make it public.
> >>
> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly*
> the bugzilla bugs, and we'll have excised the existing github issues that
> we want to pretend never existed anyway.
> >>
> >>
> >> I think we've missed an important step in the planning here: we've not
> agreed on a set of goals for the transition. Here are mine:
> >>
> >>   * We end up with one single issue tracking system containing all
> issues, both old and new, both open and closed.
> >>   * All links and references to existing bugs still work.
> >>   * We have a single bug numbering system covering all bugs, and old
> bugs retain their numbers.
> > Why are the bug numbers important?  Could you help give some example use
> cases that require having
> > a non-intersecting set of bug numbers for bugzilla bugs and github
> issues?
>
>
> While I have no experience in bugzilla or github tooling, the two step
> process described by Richard doesn't seem to be very complicated.
>
>
> As mentioned by others, we have commits and tests (and sometimes source
> files) that explicitly mention bug numbers. I do regularly look up bugs
> from a decade ago to determine if a test or some code still has
> relevance or not. If PR3214 can be one of two bugs, it does not only
> increase lookup time but also add confusion to everyone involved.
>
>
> Cheers,
>
>Johannes
>
>
>
> > -Tom
> >
> >
> >> It sounds like we don't all agree that the last point is important, but
> if we can achieve it without any significant additional cost, why not do so?
> >>
> >>  Philip
> >>
> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
> >>>  In a previous discussion, one other suggestion had been to
> migrate all the bugzilla bugs to a separate initially-private "bug archive"
> repository in github. This has a few benefits:
> >>>  1. If the migration is messed up, the repo can be deleted, and
> the process run again, until we get a result we like.
> >>>  2. The numbering can be fully-controlled.
> >>>  Once the bugs are migrated to /some/ github repository,
> individual issues can then be "moved" between repositories, and github will
> redirect from the movefrom-repository's bug to the target repository's bug.
> >>>
> >>>  We could also just have llvm.org/PR###
>   be the url only
> for legacy bugzilla issue numbers -- and have it use a file listing the
> mappings of bugzilla id -> github id to generate the redirects. (GCC just
> did this recently for svn revision number redirections,
> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
> >>>
> >>>  Then we could introduce a new naming scheme for github issue
> shortlinks.
> >>>
> >>>  On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev <
> llvm-...@lists.llvm.org > wrote:
> >>>
> >>>  On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev <
> llvm-...@lists.llvm.org > wrote:
> >>>
> >>>  Hi,
> >>>
> >>>  I wanted to continue discussing the plan to migrate from
> Bugzilla to Github.
> >>>  It was suggested that I start a new thread and give a
> summary of the proposal
> >>>  and what has changed since it was originally proposed in
> October.
> >>>
> >>>  == Here is the original proposal:
> >>>
> >>>
> http://lists.llvm.org/pipermail/llvm-dev/2019-October/136162.html
> >>>
> >>>  == What has changed:
> >>>
> >>>  * You will be able to subscribe to notifications for a
> specific 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Anton Korobeynikov via lldb-dev]

Hi Konrad,

Thanks for the scripts – look useful! For the record, here is the
result of previous experiments
https://github.com/asl/llvm-bugzilla/issues

On Wed, Apr 22, 2020 at 2:21 PM Konrad Kleine via cfe-dev
 wrote:
>
> I wanted to try importing llvm bugs into a fresh github repo and here's my 
> result so far (import is still running): 
> https://github.com/kwk/test-llvm-bz-import-4 . I've written the scripts 
> (https://github.com/kwk/bz2gh) myself because I wanted to remain in control 
> and don't make my life more complicated than it needs to be. The README.md 
> describes in great length what's imported and how. For now I import the 
> bugzillas just as placeholder issues. Then I lock those issues in github to 
> avoid messing with them. Before I created labels based on 
> /. Those are assigned to each issue. It should give a 
> good start to at least reserve all github #IDs so they map 1:1 to LLVM BZs.
>
> On Wed, 22 Apr 2020 at 09:23, Dimitry Andric via cfe-dev 
>  wrote:
>>
>> Since Bugzilla numbers are all under 50,000 (at least for now:), can't we 
>> simply bump the GitHub issue/pull request numbers to 50,000, and start from 
>> there?
>>
>> Then it would be easy to identify: < 5 means Bugzilla, >= 5 means 
>> GitHub.
>>
>> Now somebody's only gotta find a way to file 5-200 bogus GitHub tickets. 
>> :)  (Or ask GitHub support to bump the number synthetically.)
>>
>> -Dimitry
>>
>> On 22 Apr 2020, at 09:10, James Henderson via cfe-dev 
>>  wrote:
>>
>> Similar to other people's experiences, I've worked on a common code base 
>> that supported three different platforms, and each platform used a different 
>> bugzilla with it's own numbering scheme. I regularly came across references 
>> to "BZ123456" with no indication as to which of the three systems that 
>> referred to. This would often mean having to go to each in turn and seeing 
>> if the corresponding bug looked like it had anything to do with the related 
>> topic. Fortunately, given that there were many other things using the same 
>> bugzilla instances, this was usually pretty clear, but not always. Typos in 
>> bug numbers sometimes made things even harder, since you had to spend three 
>> times as long trying to guess.
>>
>> In other words +1 to using unique numbers, however we do it.
>>
>> On Wed, 22 Apr 2020 at 03:44, Johannes Doerfert via cfe-dev 
>>  wrote:
>>>
>>>
>>> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
>>> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>>> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev 
>>> >> mailto:cfe-...@lists.llvm.org>> wrote:
>>> >>
>>> >>  +1 to James's take
>>> >>
>>> >>  I'd prefer simplicity of implementation over perfection here.
>>> >>
>>> >> If we end up with two different bug numbering systems, that's a problem 
>>> >> that we will be paying for for many years. It's worth some investment 
>>> >> now to avoid that problem. And it doesn't seem like it really requires 
>>> >> much investment.
>>> >>
>>> >> Here's another path we could take:
>>> >>
>>> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the 
>>> >> bugzilla issues there. Iterate until we're happy, as per James's 
>>> >> proposal.
>>> >> 2) Sync the forked repository to the llvm repository, delete the llvm 
>>> >> repository, rename "bugs" to "llvm", and make it public.
>>> >>
>>> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly* 
>>> >> the bugzilla bugs, and we'll have excised the existing github issues 
>>> >> that we want to pretend never existed anyway.
>>> >>
>>> >>
>>> >> I think we've missed an important step in the planning here: we've not 
>>> >> agreed on a set of goals for the transition. Here are mine:
>>> >>
>>> >>   * We end up with one single issue tracking system containing all 
>>> >> issues, both old and new, both open and closed.
>>> >>   * All links and references to existing bugs still work.
>>> >>   * We have a single bug numbering system covering all bugs, and old 
>>> >> bugs retain their numbers.
>>> > Why are the bug numbers important?  Could you help give some example use 
>>> > cases that require having
>>> > a non-intersecting set of bug numbers for bugzilla bugs and github issues?
>>>
>>>
>>> While I have no experience in bugzilla or github tooling, the two step
>>> process described by Richard doesn't seem to be very complicated.
>>>
>>>
>>> As mentioned by others, we have commits and tests (and sometimes source
>>> files) that explicitly mention bug numbers. I do regularly look up bugs
>>> from a decade ago to determine if a test or some code still has
>>> relevance or not. If PR3214 can be one of two bugs, it does not only
>>> increase lookup time but also add confusion to everyone involved.
>>>
>>>
>>> Cheers,
>>>
>>>Johannes
>>>
>>>
>>>
>>> > -Tom
>>> >
>>> >
>>> >> It sounds like we don't all agree that the last point is important, but 
>>> >> if we can achieve it without any significant 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Konrad Kleine via lldb-dev]

I wanted to try importing llvm bugs into a fresh github repo and here's my
result so far (import is still running):
https://github.com/kwk/test-llvm-bz-import-4 . I've written the scripts (
https://github.com/kwk/bz2gh) myself because I wanted to remain in control
and don't make my life more complicated than it needs to be. The README.md
describes in great length what's imported and how. For now I import the
bugzillas just as placeholder issues. Then I lock those issues in github to
avoid messing with them. Before I created labels based on
/. Those are assigned to each issue. It should give
a good start to at least reserve all github #IDs so they map 1:1 to LLVM
BZs.

On Wed, 22 Apr 2020 at 09:23, Dimitry Andric via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> Since Bugzilla numbers are all under 50,000 (at least for now:), can't we
> simply bump the GitHub issue/pull request numbers to 50,000, and start from
> there?
>
> Then it would be easy to identify: < 5 means Bugzilla, >= 5 means
> GitHub.
>
> Now somebody's only gotta find a way to file 5-200 bogus GitHub
> tickets. :)  (Or ask GitHub support to bump the number synthetically.)
>
> -Dimitry
>
> On 22 Apr 2020, at 09:10, James Henderson via cfe-dev <
> cfe-...@lists.llvm.org> wrote:
>
> Similar to other people's experiences, I've worked on a common code base
> that supported three different platforms, and each platform used a
> different bugzilla with it's own numbering scheme. I regularly came across
> references to "BZ123456" with no indication as to which of the three
> systems that referred to. This would often mean having to go to each in
> turn and seeing if the corresponding bug looked like it had anything to do
> with the related topic. Fortunately, given that there were many other
> things using the same bugzilla instances, this was usually pretty clear,
> but not always. Typos in bug numbers sometimes made things even harder,
> since you had to spend three times as long trying to guess.
>
> In other words +1 to using unique numbers, however we do it.
>
> On Wed, 22 Apr 2020 at 03:44, Johannes Doerfert via cfe-dev <
> cfe-...@lists.llvm.org> wrote:
>
>>
>> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
>> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
>> cfe-...@lists.llvm.org > wrote:
>> >>
>> >>  +1 to James's take
>> >>
>> >>  I'd prefer simplicity of implementation over perfection here.
>> >>
>> >> If we end up with two different bug numbering systems, that's a
>> problem that we will be paying for for many years. It's worth some
>> investment now to avoid that problem. And it doesn't seem like it really
>> requires much investment.
>> >>
>> >> Here's another path we could take:
>> >>
>> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the
>> bugzilla issues there. Iterate until we're happy, as per James's proposal.
>> >> 2) Sync the forked repository to the llvm repository, delete the llvm
>> repository, rename "bugs" to "llvm", and make it public.
>> >>
>> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly*
>> the bugzilla bugs, and we'll have excised the existing github issues that
>> we want to pretend never existed anyway.
>> >>
>> >>
>> >> I think we've missed an important step in the planning here: we've not
>> agreed on a set of goals for the transition. Here are mine:
>> >>
>> >>   * We end up with one single issue tracking system containing all
>> issues, both old and new, both open and closed.
>> >>   * All links and references to existing bugs still work.
>> >>   * We have a single bug numbering system covering all bugs, and old
>> bugs retain their numbers.
>> > Why are the bug numbers important?  Could you help give some example
>> use cases that require having
>> > a non-intersecting set of bug numbers for bugzilla bugs and github
>> issues?
>>
>>
>> While I have no experience in bugzilla or github tooling, the two step
>> process described by Richard doesn't seem to be very complicated.
>>
>>
>> As mentioned by others, we have commits and tests (and sometimes source
>> files) that explicitly mention bug numbers. I do regularly look up bugs
>> from a decade ago to determine if a test or some code still has
>> relevance or not. If PR3214 can be one of two bugs, it does not only
>> increase lookup time but also add confusion to everyone involved.
>>
>>
>> Cheers,
>>
>>Johannes
>>
>>
>>
>> > -Tom
>> >
>> >
>> >> It sounds like we don't all agree that the last point is important,
>> but if we can achieve it without any significant additional cost, why not
>> do so?
>> >>
>> >>  Philip
>> >>
>> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>> >>>  In a previous discussion, one other suggestion had been to
>> migrate all the bugzilla bugs to a separate initially-private "bug archive"
>> repository in github. This has a few 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Dimitry Andric via lldb-dev]

Since Bugzilla numbers are all under 50,000 (at least for now:), can't we 
simply bump the GitHub issue/pull request numbers to 50,000, and start from 
there?

Then it would be easy to identify: < 5 means Bugzilla, >= 5 means 
GitHub.

Now somebody's only gotta find a way to file 5-200 bogus GitHub tickets. :) 
 (Or ask GitHub support to bump the number synthetically.)

-Dimitry

> On 22 Apr 2020, at 09:10, James Henderson via cfe-dev 
>  wrote:
> 
> Similar to other people's experiences, I've worked on a common code base that 
> supported three different platforms, and each platform used a different 
> bugzilla with it's own numbering scheme. I regularly came across references 
> to "BZ123456" with no indication as to which of the three systems that 
> referred to. This would often mean having to go to each in turn and seeing if 
> the corresponding bug looked like it had anything to do with the related 
> topic. Fortunately, given that there were many other things using the same 
> bugzilla instances, this was usually pretty clear, but not always. Typos in 
> bug numbers sometimes made things even harder, since you had to spend three 
> times as long trying to guess.
> 
> In other words +1 to using unique numbers, however we do it.
> 
> On Wed, 22 Apr 2020 at 03:44, Johannes Doerfert via cfe-dev 
> mailto:cfe-...@lists.llvm.org>> wrote:
> 
> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev 
> >> mailto:cfe-...@lists.llvm.org> 
> >> >> wrote:
> >>
> >>  +1 to James's take
> >>
> >>  I'd prefer simplicity of implementation over perfection here.
> >>
> >> If we end up with two different bug numbering systems, that's a problem 
> >> that we will be paying for for many years. It's worth some investment now 
> >> to avoid that problem. And it doesn't seem like it really requires much 
> >> investment.
> >>
> >> Here's another path we could take:
> >>
> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the 
> >> bugzilla issues there. Iterate until we're happy, as per James's proposal.
> >> 2) Sync the forked repository to the llvm repository, delete the llvm 
> >> repository, rename "bugs" to "llvm", and make it public.
> >>
> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly* the 
> >> bugzilla bugs, and we'll have excised the existing github issues that we 
> >> want to pretend never existed anyway.
> >>
> >>
> >> I think we've missed an important step in the planning here: we've not 
> >> agreed on a set of goals for the transition. Here are mine:
> >>
> >>   * We end up with one single issue tracking system containing all issues, 
> >> both old and new, both open and closed.
> >>   * All links and references to existing bugs still work.
> >>   * We have a single bug numbering system covering all bugs, and old bugs 
> >> retain their numbers.
> > Why are the bug numbers important?  Could you help give some example use 
> > cases that require having
> > a non-intersecting set of bug numbers for bugzilla bugs and github issues?
> 
> 
> While I have no experience in bugzilla or github tooling, the two step
> process described by Richard doesn't seem to be very complicated.
> 
> 
> As mentioned by others, we have commits and tests (and sometimes source
> files) that explicitly mention bug numbers. I do regularly look up bugs
> from a decade ago to determine if a test or some code still has
> relevance or not. If PR3214 can be one of two bugs, it does not only
> increase lookup time but also add confusion to everyone involved.
> 
> 
> Cheers,
> 
>Johannes
> 
> 
> 
> > -Tom
> >
> >
> >> It sounds like we don't all agree that the last point is important, but if 
> >> we can achieve it without any significant additional cost, why not do so?
> >>
> >>  Philip
> >>
> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
> >>>  In a previous discussion, one other suggestion had been to migrate 
> >>> all the bugzilla bugs to a separate initially-private "bug archive" 
> >>> repository in github. This has a few benefits:
> >>>  1. If the migration is messed up, the repo can be deleted, and the 
> >>> process run again, until we get a result we like.
> >>>  2. The numbering can be fully-controlled.
> >>>  Once the bugs are migrated to /some/ github repository, individual 
> >>> issues can then be "moved" between repositories, and github will redirect 
> >>> from the movefrom-repository's bug to the target repository's bug.
> >>>
> >>>  We could also just have llvm.org/PR###  
> >>> > be the url only 
> >>> for legacy bugzilla issue numbers -- and have it use a file listing the 
> >>> mappings of bugzilla id -> github id to generate the redirects. (GCC just 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[James Henderson via lldb-dev]

Similar to other people's experiences, I've worked on a common code base
that supported three different platforms, and each platform used a
different bugzilla with it's own numbering scheme. I regularly came across
references to "BZ123456" with no indication as to which of the three
systems that referred to. This would often mean having to go to each in
turn and seeing if the corresponding bug looked like it had anything to do
with the related topic. Fortunately, given that there were many other
things using the same bugzilla instances, this was usually pretty clear,
but not always. Typos in bug numbers sometimes made things even harder,
since you had to spend three times as long trying to guess.

In other words +1 to using unique numbers, however we do it.

On Wed, 22 Apr 2020 at 03:44, Johannes Doerfert via cfe-dev <
cfe-...@lists.llvm.org> wrote:

>
> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
> cfe-...@lists.llvm.org > wrote:
> >>
> >>  +1 to James's take
> >>
> >>  I'd prefer simplicity of implementation over perfection here.
> >>
> >> If we end up with two different bug numbering systems, that's a problem
> that we will be paying for for many years. It's worth some investment now
> to avoid that problem. And it doesn't seem like it really requires much
> investment.
> >>
> >> Here's another path we could take:
> >>
> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the
> bugzilla issues there. Iterate until we're happy, as per James's proposal.
> >> 2) Sync the forked repository to the llvm repository, delete the llvm
> repository, rename "bugs" to "llvm", and make it public.
> >>
> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly*
> the bugzilla bugs, and we'll have excised the existing github issues that
> we want to pretend never existed anyway.
> >>
> >>
> >> I think we've missed an important step in the planning here: we've not
> agreed on a set of goals for the transition. Here are mine:
> >>
> >>   * We end up with one single issue tracking system containing all
> issues, both old and new, both open and closed.
> >>   * All links and references to existing bugs still work.
> >>   * We have a single bug numbering system covering all bugs, and old
> bugs retain their numbers.
> > Why are the bug numbers important?  Could you help give some example use
> cases that require having
> > a non-intersecting set of bug numbers for bugzilla bugs and github
> issues?
>
>
> While I have no experience in bugzilla or github tooling, the two step
> process described by Richard doesn't seem to be very complicated.
>
>
> As mentioned by others, we have commits and tests (and sometimes source
> files) that explicitly mention bug numbers. I do regularly look up bugs
> from a decade ago to determine if a test or some code still has
> relevance or not. If PR3214 can be one of two bugs, it does not only
> increase lookup time but also add confusion to everyone involved.
>
>
> Cheers,
>
>Johannes
>
>
>
> > -Tom
> >
> >
> >> It sounds like we don't all agree that the last point is important, but
> if we can achieve it without any significant additional cost, why not do so?
> >>
> >>  Philip
> >>
> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
> >>>  In a previous discussion, one other suggestion had been to
> migrate all the bugzilla bugs to a separate initially-private "bug archive"
> repository in github. This has a few benefits:
> >>>  1. If the migration is messed up, the repo can be deleted, and
> the process run again, until we get a result we like.
> >>>  2. The numbering can be fully-controlled.
> >>>  Once the bugs are migrated to /some/ github repository,
> individual issues can then be "moved" between repositories, and github will
> redirect from the movefrom-repository's bug to the target repository's bug.
> >>>
> >>>  We could also just have llvm.org/PR###
>   be the url only
> for legacy bugzilla issue numbers -- and have it use a file listing the
> mappings of bugzilla id -> github id to generate the redirects. (GCC just
> did this recently for svn revision number redirections,
> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
> >>>
> >>>  Then we could introduce a new naming scheme for github issue
> shortlinks.
> >>>
> >>>  On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev <
> llvm-...@lists.llvm.org > wrote:
> >>>
> >>>  On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev <
> llvm-...@lists.llvm.org > wrote:
> >>>
> >>>  Hi,
> >>>
> >>>  I wanted to continue discussing the plan to migrate from
> Bugzilla to Github.
> >>>  It was suggested that I 

[lldb-dev] Fwd: [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Konrad Kleine via lldb-dev]

-- Forwarded message -
From: Konrad Kleine 
Date: Tue, 21 Apr 2020 at 09:39
Subject: Re: [llvm-dev] RFC: Switching from Bugzilla to Github Issues
[UPDATED]
To: Tom Stellard 


Hi Tom.

I haven't read all the replies before mine. Sorry if my idea overlaps with
someone else's.

I have a proposal for the issue migration from bugzilla to github. You
mentioned in the ML thread that proposal 1 and 2 were favored because they
can be done sooner than proposal 3.

In order to begin with proposal 1 and 2, what if you created dummy issues
in github for the number of bug-ranges 1-X, where X is the highest
number in bugzilla? When this is done by a script you could add a special
label to those issues that mark them as bugs that are still managed by
bugzilla. For the description you could put the link to the bugzilla
instance and a text that "this bug is still managed by bugzilla".

If you immediately close those bugs, then they don't show up. BUT IIRC
github has this odd way of numbering bugs and issues. Issues and Pull
Requests share the same pool of numbers. I'm sure you know that. But as
long as you disallow creation of NEW bugs in bugzilla, have created the
dummy issues in github, and don't open any PR in github this should be good.

Then at some point you can move over all the bugs in bugzilla to their
dummy issues in github. The latter just serve as placeholders until then.

I hope this makes sense.

Regards
Konrad

On Mon, 20 Apr 2020 at 21:30, Tom Stellard via llvm-dev <
llvm-...@lists.llvm.org> wrote:

> Hi,
>
> I wanted to continue discussing the plan to migrate from Bugzilla to
> Github.
> It was suggested that I start a new thread and give a summary of the
> proposal
> and what has changed since it was originally proposed in October.
>
> == Here is the original proposal:
>
> http://lists.llvm.org/pipermail/llvm-dev/2019-October/136162.html
>
> == What has changed:
>
> * You will be able to subscribe to notifications for a specific issue
>   labels.  We have a proof of concept notification system using github
> actions
>   that will be used for this.
>
> * Emails will be sent to llvm-bugs when issues are opened or closed.
>
> * We have the initial list of labels:
> https://github.com/llvm/llvm-project/labels
>
> == Remaining issue:
>
> * There is one remaining issue that I don't feel we have consensus on,
> and that is what to do with bugs in the existing bugzilla.  Here are some
> options
> that we have discussed:
>
> 1. Switch to GitHub issues for new bugs only.  Bugs filed in bugzilla that
> are
> still active will be updated there until they are closed.  This means that
> over
> time the number of active bugs in bugzilla will slowly decrease as bugs
> are closed
> out.  Then at some point in the future, all of the bugs from bugzilla will
> be archived
> into their own GitHub repository that is separate from the llvm-project
> repo.
>
> 2. Same as 1, but also create a migration script that would allow anyone to
> manually migrate an active bug from bugzilla to a GitHub issue in the
> llvm-project
> repo.  The intention with this script is that it would be used to migrate
> high-traffic
> or important bugs from bugzilla to GitHub to help increase the visibility
> of the bug.
> This would not be used for mass migration of all the bugs.
>
> 3. Do a mass bug migration from bugzilla to GitHub and enable GitHub
> issues at the same time.
> Closed or inactive bugs would be archived into their own GitHub
> repository, and active bugs
> would be migrated to the llvm-project repo.
>
>
> The key difference between proposal 1,2 and 3, is when bugs will be
> archived from bugzilla
> to GitHub.  Delaying the archiving of bugs (proposals 1 and 2) means that
> we can migrate
> to GitHub issues sooner (within 1-2 weeks), whereas trying to archive bugs
> during the
> transition (proposal 3) will delay the transition for a while (likely
> several months)
> while we evaluate the various solutions for moving bugs from bugzilla to
> GitHub.
>
>
> The original proposal was to do 1 or 2, however there were some concerns
> raised on the list
> that having 2 different places to search for bugs for some period of time
> would
> be very inconvenient.  So, I would like to restart this discussion and
> hopefully we can
> come to some kind of conclusion about the best way forward.
>
> Thanks,
> Tom
>
> ___
> LLVM Developers mailing list
> llvm-...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev