Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-16 Thread Antoine Musso

On 03/04/15 17:52, Monty Taylor wrote:

Could do better: ACLs for Vulnerability management
--

snip


I'd love to learn how wikimedia is working with this.

http://www.mediawiki.org/wiki/Phabricator/Security

tl;dr They have added a security dropdown to task filing that triggers
some policy actions. I betcha we could copy theirs.


Hello,

At Wikimedia we definitely had the requirement of having internal bugs, 
we have two kinds of them:


* security vulnerability that will eventually be disclosed / made public
* private / sensitive information we want to keep in (contracts, 
personal informations etc)


We ended up writing our own extension which is in our Gerrit as 
phabricator/extensions/security.git the README:


http://git.wikimedia.org/blob/phabricator%2Fextensions%2Fsecurity.git/master/README

The wiki page you found is appropriate.

The main author is Mukunda Modell or twentyafterfour on IRC.  I am not 
sure how much available spare time he has though.



--
Antoine hashar Musso

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-16 Thread Antoine Musso

On 03/04/15 18:06, Jeremy Stanley wrote:

On 2015-04-03 11:54:00 -0400 (-0400), Sean Dague wrote:
[...]

2) is there an event stream of changes (either real time or rss) that
can be consumed by said tools? Having the change stream would be really
helpful.


Which relates to a feature request we hear all the time: is there a
way to have bug events spammed to our IRC channel? If LP had a
proper event stream, we'd probably already be doing that.


Hello,

The fab python module wraps around the Phabricator Conduit system:
 https://pypi.python.org/pypi/fab

A few volunteers from the Wikimedia community created a python bot that 
consumes event, store them in a redis queue, format and route messages 
to IRC channels.



The repo is hosted on our Gerrit:
  https://gerrit.wikimedia.org/r/p/labs/tools/wikibugs2.git

Web view:
  http://git.wikimedia.org/summary/?r=labs/tools/wikibugs2.git

Our routing definition:

http://git.wikimedia.org/blob/labs%2Ftools%2Fwikibugs2.git/master/channels.yaml

There is some sparse documentation at:
  https://www.mediawiki.org/wiki/wikibugs


You can find the three authors in #wikimedia-labs, their IRC nicks, 
location and potential timezones are:


legoktm SF, PST
valhallasw  Europe, CET
YuviPanda   SF, PST

I am sure they are willing to take patches and give you more informations.


--
Antoine hashar Musso

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-16 Thread Antoine Musso

On 03/04/15 17:57, Monty Taylor wrote:

On 04/03/2015 11:44 AM, Michael Krotscheck wrote:

This proposal is all well and good, however (no offense intended) Monty's
got a history of putting out neat proposals and leaving someone else to
support it. Without identifying a dedicated person/resource that will
maintain phabricator, any discussion is moot.

This is actually an excellent point - and one of the reasons that
running our own Launchpad is not being suggested. It has been
communicated that the operational burden of running a launchpad is
extensive and WOULD require dedicated resources.

I propose that we not have any developers dedicated to developing
phabricator.

I propose that the infra-team as a whole would support and maintain it -
similar to how we support and maintain gerrit, jenkins, etherpad, ELK,
graphite, cgit, mailman, and soon zanata - which are all substantial
pieces of software that we did not write ourselves. Since phabricator is
actually designed and intended to be able to be run CD from master, the
overall operational cost should not be particularly harder than any of
the rest of the software we run.


Hello,

One of the reason Wikimedia migrated from Bugzilla to Phabricator is 
that it is a PHP application and we have a ton of PHP developers.  With 
OpenStack being all python, maintaining a PHP application might add to 
your operations burden.


As you said in your initial mail, building a bug tracker is not in 
OpenStack infra team core duties.  May I add that hosting such app is 
probably not either?


Phabricator was build for Facebook internal usage but has spawn in its 
own little company known as Phacility.  They seem to provide hosting 
service which would save you from having to maintain it.  Might end up 
being cheaper than adding workload to the current team / hire one more.


Hosting infos:  http://www.phabricator.org/hosting/
Staff page: http://phacility.com/about/

We have invited Evan Priestley at the Wikimedia office to present us 
Phabricator.  Might be worth getting in touch with them and organize a 
quick tour and find out what they can offer.


--
Antoine hashar Musso


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


[OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-03 Thread Monty Taylor
As a follow up to my previous email about thinking about ceasing to use
storyboard, I have done some work towards investigating using phabricator.

phabricator came out of Facebook, but has been spun out completely and
is now managed as an Open Source project. There is a company that has
formed around it doing support and hosted installs ... so it has a
vibrant and active developer community working on it.

In our broader infra ecosystem, we do lots of things with the Wikimedia
Foundation. They are close collaborators on Jenkins Job Builder, and are
also zuul users. They have recently migrated their bug tracking to
phabricator ... so one could imagine that our collaboration could
continue nicely.

There are several phabricator features we are not interested in - such
as code review. Luckily it is easy to turn them off.

The phabricator model is not exactly what we want, but there are some
nice things about it. It doesn't fully encompass things our
project-group/project/branch hierarchy as it relates to things. However,
it does allow for an issue to have multiple other issues it's blocked
on, as well as multiple issues it blocks. And an issue can be associated
with more than one project

A project is actually more like an arbitrary tag. That's bad in that
it's not a structured concept. It's good in that it's flexible. A
project also carries with it an kanban-like workboard, that allows for
different priority setting on a per-project basis.

In any case - it's hard to think about those things without something to
look at, so I set up a phabricator instance and did a data
transformation of the current (ish) storyboard database:

http://15.126.194.141/

If you have ever logged in to storyboard, you have a login here with a
password of password. As you might imagine, I do not consider the data
in this instance precious - I may or may not blow it away and reload it
from newer database dumps or from improved data migration scripts at any
time.

You may want to note the workboard concept:

The issue:

http://15.126.194.141/T2298

Is in both the openstack-infra/system-config and the openstack-ci
projects. Each of those have a workboard:

http://15.126.194.141/tag/openstack-infra_system-config/
http://15.126.194.141/tag/openstack-ci/

T2298 is in the backlog for openstack-infra/system-config but listed in
priority efforts for openstack-ci.

Also, you can see that T2298 has a pholio mock -
http://15.126.194.141/M1 - which is an image with design conversation
associated with it.

The code to deploy this as well as do the data transformation can be
found here:

https://github.com/emonty/puppet-phabricator

It's not perfect - but I figured that a conversation can't really be had
without something to point at.

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-03 Thread Thierry Carrez
Monty Taylor wrote:
 As a follow up to my previous email about thinking about ceasing to use
 storyboard, I have done some work towards investigating using phabricator.
 [...]

For the record, I'll repost some of the analysis I did for mordred on this.


Main blocker: Ability to track tasks across projects/branches
-

One of the main features we need from a task tracker in OpenStack is the
ability to track the completion of tasks across projects and branches.
That affects vulnerability management, bug backports, and generally
cross-project work. The fact that Launchpad has a limitation there (on
the number of tasks it supports before timeouting) is issue #2 with LP.
The problem is stated once, then a number of tasks affecting different
groups are created, and the issue is closed when the last task is completed.

In Maniphest as it stands, that would translate into one of those:

1- Create a single task that affects multiple projects
2- Create a main task and multiple subtasks, one for each project

The main issue with (1) is that you can't track partial completion (i.e.
which groups have completed their task and which haven't), which is the
main goal of using a common task tracker.

The main issue with (2) is that it's difficult to see what groups are
affected from the main task (since it only lists task names), and the
main task is not autoclosed when the last subtask is closed (requiring
manual intervention).

In both cases, the UI allows multiple projects to apply to a given
task, which is great for tagging, but not so great when you want to
track tasks per affected repository and branches. Finally, there is no
concept of branches -- we would have to create one project for every
repo/branch combinations.

Finally, projects have no relationship between them. So even if you
associated a task to project openstack/nova--master you would have to
also manually associate it to project nova (as in openstack/nova +
openstack/python-novaclient) so that Nova team members could also find
it there. There is basically no projectgroup concept, which is issue #3
we have with Launchpad.

In summary, I think Phabricator's projects concept is a (really) neat
*tagging* system, with associated dashboard and all. It's just not meant
to cover multiple projects in the OpenStack sense. Our projects have a
few properties which make them difficult to emulate with Phabricator's
projects: there should be only one per task, they have an affected
branch (master, juno...), and they are grouped into higher-level
structures that can be searched and browsed as well.

Phabricator seems to be sized for single separate projects, and it's
project concept is just a way to organize work within that single
project. It is pretty obvious when you see what happens if you click one
of their projects: you end up in a dashboard with one item per bug.
Imagine clicking openstack/nova and landing on a dashboard with
thousands of bugs: it's just not meant to be used for that. It is
excellent for tracking a subset of tasks and organizing them, but not as
a higher-level structure.

So in order to make it work, we'd need a whole new concept added, let's
call it repository. Each repository may have multiple branches
associated with it. When you create a task, you select a repository and
a branch (and only one). That field is clearly shown in the task name
(no need to drill down to the subtask to find which repo/branch is
actually affected by the task). Clicking on that will bring you to a
list of bugs affecting that repository. Repositories can be grouped
arbitrarily into repogroups which let you search and navigate issues in
a set of repositories, rather than force everyone to rebuild queries to
find, say, all infra issues.

Then we could encode a rule so that when the last subtask of a repo-less
task is closed, the parent task is auto-closed. Then that /could/ all
work for task coordination for OpenStack, although it would be slightly
less convenient than Launchpad or StoryBoard (which do not require you
to drill down to update subtasks, and keep the discussion in one single
place).


Could do a lot better: CLI/API
--

One of the main pain points with Launchpad is its partial API.
Phabricator comes with the arc tool, but it is designed to drive the
DVCS git-like featureset. There is no CLI for Maniphest, only a shortcut
to make raw Conduit (API) calls. The Conduit API itself looks
mostly undocumented and (2) making Conduit more robust isn't currently
a major project priority because there isn't much demand for it outside
of internal scripts.

We rely on a lot of scripting to interface with Launchpad, which is why
a complete and clean API was such a significant part in StoryBoard
design. It looks like the Maniphest story on APIs is actually worse than
Launchpad.


Could do better: Tracking feature development
-

A feature is a 

Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-03 Thread Monty Taylor
On 04/03/2015 11:54 AM, Sean Dague wrote:
 On 04/03/2015 11:00 AM, Monty Taylor wrote:
 As a follow up to my previous email about thinking about ceasing to use
 storyboard, I have done some work towards investigating using phabricator.

 phabricator came out of Facebook, but has been spun out completely and
 is now managed as an Open Source project. There is a company that has
 formed around it doing support and hosted installs ... so it has a
 vibrant and active developer community working on it.

 In our broader infra ecosystem, we do lots of things with the Wikimedia
 Foundation. They are close collaborators on Jenkins Job Builder, and are
 also zuul users. They have recently migrated their bug tracking to
 phabricator ... so one could imagine that our collaboration could
 continue nicely.

 There are several phabricator features we are not interested in - such
 as code review. Luckily it is easy to turn them off.

 The phabricator model is not exactly what we want, but there are some
 nice things about it. It doesn't fully encompass things our
 project-group/project/branch hierarchy as it relates to things. However,
 it does allow for an issue to have multiple other issues it's blocked
 on, as well as multiple issues it blocks. And an issue can be associated
 with more than one project

 A project is actually more like an arbitrary tag. That's bad in that
 it's not a structured concept. It's good in that it's flexible. A
 project also carries with it an kanban-like workboard, that allows for
 different priority setting on a per-project basis.

 In any case - it's hard to think about those things without something to
 look at, so I set up a phabricator instance and did a data
 transformation of the current (ish) storyboard database:

 http://15.126.194.141/

 If you have ever logged in to storyboard, you have a login here with a
 password of password. As you might imagine, I do not consider the data
 in this instance precious - I may or may not blow it away and reload it
 from newer database dumps or from improved data migration scripts at any
 time.

 You may want to note the workboard concept:

 The issue:

 http://15.126.194.141/T2298

 Is in both the openstack-infra/system-config and the openstack-ci
 projects. Each of those have a workboard:

 http://15.126.194.141/tag/openstack-infra_system-config/
 http://15.126.194.141/tag/openstack-ci/

 T2298 is in the backlog for openstack-infra/system-config but listed in
 priority efforts for openstack-ci.

 Also, you can see that T2298 has a pholio mock -
 http://15.126.194.141/M1 - which is an image with design conversation
 associated with it.

 The code to deploy this as well as do the data transformation can be
 found here:

 https://github.com/emonty/puppet-phabricator

 It's not perfect - but I figured that a conversation can't really be had
 without something to point at.
 
 2 specific phabricator questions (which I'm running into in dealing with
 the pile of Nova bugs).
 
 1) what's the REST API support look like for building tools outside of
 tree?

As ttx said, there is a thing called conduit which is the thing that
does this. Link:

https://secure.phabricator.com/book/phabdev/article/conduit/

The full quote that's relevant:

Hopefully, this should improve over time, but making Conduit more
robust isn't currently a major project priority because there isn't much
demand for it outside of internal scripts. If you want to use Conduit to
build things on top of Phabricator, let us know so we can adjust
priorities.

I would imagine we'd let them know that it's important to us so that
they can adjust their priorities.

You can see the available API calls here;

http://15.126.194.141/conduit/

 2) is there an event stream of changes (either real time or rss) that
 can be consumed by said tools? Having the change stream would be really
 helpful.

There is a feed:

http://15.126.194.141/feed/

But there is no RSS support for it:

https://secure.phabricator.com/T4825

I have not looked into real-time web hook like things. However, I do
know that everything that happens produces an event internally, and
those events are then picked up by queue processing daemons and
dispatched. This is how email support is implemented. So, worst case
scenario developing another event stream consumer that would publish a
real-time feed wouldn't be terrible. There are docs here:

https://secure.phabricator.com/book/phabricator/article/events/



___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-03 Thread Monty Taylor
On 04/03/2015 12:06 PM, Jeremy Stanley wrote:
 On 2015-04-03 11:54:00 -0400 (-0400), Sean Dague wrote:
 [...]
 2) is there an event stream of changes (either real time or rss) that
 can be consumed by said tools? Having the change stream would be really
 helpful.
 
 Which relates to a feature request we hear all the time: is there a
 way to have bug events spammed to our IRC channel? If LP had a
 proper event stream, we'd probably already be doing that.
 

We don't even necessarily need an external thing that consumes an event
stream - we can deploy an internal agent:

https://secure.phabricator.com/book/phabdev/class/PhabricatorBot/

That could spam IRC. OR - that could inject events into a gearman queue
that one of our existing IRC bots could then spam into channel. Totally
agree ... would be a nice improvement over the current world.

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-03 Thread Sean Dague
On 04/03/2015 12:12 PM, Monty Taylor wrote:
 On 04/03/2015 12:06 PM, Jeremy Stanley wrote:
 On 2015-04-03 11:54:00 -0400 (-0400), Sean Dague wrote:
 [...]
 2) is there an event stream of changes (either real time or rss) that
 can be consumed by said tools? Having the change stream would be really
 helpful.

 Which relates to a feature request we hear all the time: is there a
 way to have bug events spammed to our IRC channel? If LP had a
 proper event stream, we'd probably already be doing that.

 
 We don't even necessarily need an external thing that consumes an event
 stream - we can deploy an internal agent:
 
 https://secure.phabricator.com/book/phabdev/class/PhabricatorBot/
 
 That could spam IRC. OR - that could inject events into a gearman queue
 that one of our existing IRC bots could then spam into channel. Totally
 agree ... would be a nice improvement over the current world.

Yeh, looking at the feed.http-hooks -
https://secure.phabricator.com/T5462 this seems actually reasonably done.

Poking around a bit it also looks like Phabricator is more designed
around the idea you'd write a plugin within it for some of this
functionality. I think living with launchpad as a blackbox-as-a-service
for so long it will take a little getting used to the fact that we can
actually do this kind of thing in service.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-03 Thread Michael Krotscheck
This proposal is all well and good, however (no offense intended) Monty's
got a history of putting out neat proposals and leaving someone else to
support it. Without identifying a dedicated person/resource that will
maintain phabricator, any discussion is moot.

Michael

On Fri, Apr 3, 2015 at 8:16 AM Thierry Carrez thie...@openstack.org wrote:

 Monty Taylor wrote:
  As a follow up to my previous email about thinking about ceasing to use
  storyboard, I have done some work towards investigating using
 phabricator.
  [...]

 For the record, I'll repost some of the analysis I did for mordred on this.


 Main blocker: Ability to track tasks across projects/branches
 -

 One of the main features we need from a task tracker in OpenStack is the
 ability to track the completion of tasks across projects and branches.
 That affects vulnerability management, bug backports, and generally
 cross-project work. The fact that Launchpad has a limitation there (on
 the number of tasks it supports before timeouting) is issue #2 with LP.
 The problem is stated once, then a number of tasks affecting different
 groups are created, and the issue is closed when the last task is
 completed.

 In Maniphest as it stands, that would translate into one of those:

 1- Create a single task that affects multiple projects
 2- Create a main task and multiple subtasks, one for each project

 The main issue with (1) is that you can't track partial completion (i.e.
 which groups have completed their task and which haven't), which is the
 main goal of using a common task tracker.

 The main issue with (2) is that it's difficult to see what groups are
 affected from the main task (since it only lists task names), and the
 main task is not autoclosed when the last subtask is closed (requiring
 manual intervention).

 In both cases, the UI allows multiple projects to apply to a given
 task, which is great for tagging, but not so great when you want to
 track tasks per affected repository and branches. Finally, there is no
 concept of branches -- we would have to create one project for every
 repo/branch combinations.

 Finally, projects have no relationship between them. So even if you
 associated a task to project openstack/nova--master you would have to
 also manually associate it to project nova (as in openstack/nova +
 openstack/python-novaclient) so that Nova team members could also find
 it there. There is basically no projectgroup concept, which is issue #3
 we have with Launchpad.

 In summary, I think Phabricator's projects concept is a (really) neat
 *tagging* system, with associated dashboard and all. It's just not meant
 to cover multiple projects in the OpenStack sense. Our projects have a
 few properties which make them difficult to emulate with Phabricator's
 projects: there should be only one per task, they have an affected
 branch (master, juno...), and they are grouped into higher-level
 structures that can be searched and browsed as well.

 Phabricator seems to be sized for single separate projects, and it's
 project concept is just a way to organize work within that single
 project. It is pretty obvious when you see what happens if you click one
 of their projects: you end up in a dashboard with one item per bug.
 Imagine clicking openstack/nova and landing on a dashboard with
 thousands of bugs: it's just not meant to be used for that. It is
 excellent for tracking a subset of tasks and organizing them, but not as
 a higher-level structure.

 So in order to make it work, we'd need a whole new concept added, let's
 call it repository. Each repository may have multiple branches
 associated with it. When you create a task, you select a repository and
 a branch (and only one). That field is clearly shown in the task name
 (no need to drill down to the subtask to find which repo/branch is
 actually affected by the task). Clicking on that will bring you to a
 list of bugs affecting that repository. Repositories can be grouped
 arbitrarily into repogroups which let you search and navigate issues in
 a set of repositories, rather than force everyone to rebuild queries to
 find, say, all infra issues.

 Then we could encode a rule so that when the last subtask of a repo-less
 task is closed, the parent task is auto-closed. Then that /could/ all
 work for task coordination for OpenStack, although it would be slightly
 less convenient than Launchpad or StoryBoard (which do not require you
 to drill down to update subtasks, and keep the discussion in one single
 place).


 Could do a lot better: CLI/API
 --

 One of the main pain points with Launchpad is its partial API.
 Phabricator comes with the arc tool, but it is designed to drive the
 DVCS git-like featureset. There is no CLI for Maniphest, only a shortcut
 to make raw Conduit (API) calls. The Conduit API itself looks
 mostly undocumented and (2) making Conduit more robust isn't currently
 a 

Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-03 Thread Monty Taylor
On 04/03/2015 11:16 AM, Thierry Carrez wrote:
 Monty Taylor wrote:
 As a follow up to my previous email about thinking about ceasing to use
 storyboard, I have done some work towards investigating using phabricator.
 [...]
 
 For the record, I'll repost some of the analysis I did for mordred on this.
 
 
 Main blocker: Ability to track tasks across projects/branches
 -
 
 One of the main features we need from a task tracker in OpenStack is the
 ability to track the completion of tasks across projects and branches.
 That affects vulnerability management, bug backports, and generally
 cross-project work. The fact that Launchpad has a limitation there (on
 the number of tasks it supports before timeouting) is issue #2 with LP.
 The problem is stated once, then a number of tasks affecting different
 groups are created, and the issue is closed when the last task is completed.
 
 In Maniphest as it stands, that would translate into one of those:
 
 1- Create a single task that affects multiple projects
 2- Create a main task and multiple subtasks, one for each project
 
 The main issue with (1) is that you can't track partial completion (i.e.
 which groups have completed their task and which haven't), which is the
 main goal of using a common task tracker.
 
 The main issue with (2) is that it's difficult to see what groups are
 affected from the main task (since it only lists task names), and the
 main task is not autoclosed when the last subtask is closed (requiring
 manual intervention).

I think 2 is the way phabricator expects this to be modeled. It's
certainly different, but it also gives us a view of a thing we've NEVER
had that I honestly think might be more useful to more people. If you
look at:

https://secure.phabricator.com/T1536

You can see quite quickly which tasks are done and which are still
blocking this main task from being done. That those aren't broken out by
project there is actually a little nice to me - we are One Project
after all.

Looking at how wikipedia are using the feature as well may be
enlightening. I believe they similarly have multiple projects as well as
multiple vertical efforts.

In fact, one of the things I particularly like is that it's NOT tied to
repos or branches ... because things like product management have
interests that are particularly clunky when we're tied to a thinking
everything is related to a git repository... as is thinking that having
multiple git repositories means multiple bug projects. The mapping is
actually potentially different given the project.

 In both cases, the UI allows multiple projects to apply to a given
 task, which is great for tagging, but not so great when you want to
 track tasks per affected repository and branches. Finally, there is no
 concept of branches -- we would have to create one project for every
 repo/branch combinations.

I'm not so sure about this. I actually think we could create a nova
project and a stable/kilo project. Or we may need a nova-kilo - I
don't know - but I think there are some iterations we should investigate.

 Finally, projects have no relationship between them. So even if you
 associated a task to project openstack/nova--master you would have to
 also manually associate it to project nova (as in openstack/nova +
 openstack/python-novaclient) so that Nova team members could also find
 it there. There is basically no projectgroup concept, which is issue #3
 we have with Launchpad.
 
 In summary, I think Phabricator's projects concept is a (really) neat
 *tagging* system, with associated dashboard and all. 

Yes. It is, in fact, explicitly a tagging system.

 It's just not meant
 to cover multiple projects in the OpenStack sense. Our projects have a
 few properties which make them difficult to emulate with Phabricator's
 projects: there should be only one per task, they have an affected
 branch (master, juno...), and they are grouped into higher-level
 structures that can be searched and browsed as well.
 
 Phabricator seems to be sized for single separate projects, and it's
 project concept is just a way to organize work within that single
 project. It is pretty obvious when you see what happens if you click one
 of their projects: you end up in a dashboard with one item per bug.
 Imagine clicking openstack/nova and landing on a dashboard with
 thousands of bugs: it's just not meant to be used for that. It is
 excellent for tracking a subset of tasks and organizing them, but not as
 a higher-level structure.
 
 So in order to make it work, we'd need a whole new concept added, let's
 call it repository. Each repository may have multiple branches
 associated with it. When you create a task, you select a repository and
 a branch (and only one). That field is clearly shown in the task name
 (no need to drill down to the subtask to find which repo/branch is
 actually affected by the task). Clicking on that will bring you to a
 list of bugs affecting 

Re: [OpenStack-Infra] A proposal to use phabricator for issue tracking

2015-04-03 Thread Sean Dague
On 04/03/2015 11:00 AM, Monty Taylor wrote:
 As a follow up to my previous email about thinking about ceasing to use
 storyboard, I have done some work towards investigating using phabricator.
 
 phabricator came out of Facebook, but has been spun out completely and
 is now managed as an Open Source project. There is a company that has
 formed around it doing support and hosted installs ... so it has a
 vibrant and active developer community working on it.
 
 In our broader infra ecosystem, we do lots of things with the Wikimedia
 Foundation. They are close collaborators on Jenkins Job Builder, and are
 also zuul users. They have recently migrated their bug tracking to
 phabricator ... so one could imagine that our collaboration could
 continue nicely.
 
 There are several phabricator features we are not interested in - such
 as code review. Luckily it is easy to turn them off.
 
 The phabricator model is not exactly what we want, but there are some
 nice things about it. It doesn't fully encompass things our
 project-group/project/branch hierarchy as it relates to things. However,
 it does allow for an issue to have multiple other issues it's blocked
 on, as well as multiple issues it blocks. And an issue can be associated
 with more than one project
 
 A project is actually more like an arbitrary tag. That's bad in that
 it's not a structured concept. It's good in that it's flexible. A
 project also carries with it an kanban-like workboard, that allows for
 different priority setting on a per-project basis.
 
 In any case - it's hard to think about those things without something to
 look at, so I set up a phabricator instance and did a data
 transformation of the current (ish) storyboard database:
 
 http://15.126.194.141/
 
 If you have ever logged in to storyboard, you have a login here with a
 password of password. As you might imagine, I do not consider the data
 in this instance precious - I may or may not blow it away and reload it
 from newer database dumps or from improved data migration scripts at any
 time.
 
 You may want to note the workboard concept:
 
 The issue:
 
 http://15.126.194.141/T2298
 
 Is in both the openstack-infra/system-config and the openstack-ci
 projects. Each of those have a workboard:
 
 http://15.126.194.141/tag/openstack-infra_system-config/
 http://15.126.194.141/tag/openstack-ci/
 
 T2298 is in the backlog for openstack-infra/system-config but listed in
 priority efforts for openstack-ci.
 
 Also, you can see that T2298 has a pholio mock -
 http://15.126.194.141/M1 - which is an image with design conversation
 associated with it.
 
 The code to deploy this as well as do the data transformation can be
 found here:
 
 https://github.com/emonty/puppet-phabricator
 
 It's not perfect - but I figured that a conversation can't really be had
 without something to point at.

2 specific phabricator questions (which I'm running into in dealing with
the pile of Nova bugs).

1) what's the REST API support look like for building tools outside of
tree?

2) is there an event stream of changes (either real time or rss) that
can be consumed by said tools? Having the change stream would be really
helpful.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra