Thanks for your roundup here (very useful). You are making it clear that this
is something you might want to spend time on - I'll try and answer your final
question ("is there already any argument to say that inevitably the answer of
the proof of concept will be negative?"). The short answer is yes and no. Below
I spend most of the time explaining the "no", if you want the short version
skip to the "yes" in the last para.
Speaking entirely personally, I have always argued against using numbers to
judge the health of a project (which is the only natural outcome from
collecting such numbers). For an ASF project it's not absolute activity that is
important. It's the strength and behavior of the community and its governance
that is important. Metrics do not provide this information, and indeed can
detract from the community health issues.
For example, many years ago we had a podling that didn't graduate for a long
time because we had evolved into having a hard metric on what "diversity"
meant. It did graduate in the end and today is a thriving and productive TLP.
The significant expansion of the community didn't happen until after
graduation, a time when newcomers feel it is safe to invest in the project.
We've since scrapped the diversity metric and reverted to the original intent
of requiring a project to behave in a way that is welcoming to diverse
interests (and thus capable of satisfying a diversity metric given time). My
point is that while some metrics can provide indicators of something that needs
looking into we need to ensure that the metrics do not become more important
than community health.
Personally, I do use metrics when evaluating a project, but I use ones that are
readily available already through a number of other services. These are not
"official" or "sanctioned" and therefore say nothing, from an ASF perspective,
about the health of a project. The danger I see is that providing official
metrics a) provides a level of "authority" to the metrics which most newcomers
are ill-equipped to evaluate and b) could lead to shortcut rules like "the
metrics must show there is X level of diversity/activity/volume/foobar"
replacing proper evaluation of the project and its community.
In summary, I'm not against metrics per se, I'm cautious about them becoming
more important than they should be. I can imagine the tools being somewhat
useful *internally* where we can ensure that expectations are properly managed.
I am very cautious about using such metrics externally where they can be quoted
out of context and/or misrepresented.
All that being said, sponsors are increasingly asking us for metrics. For this
reason I'm vary interested in cross-foundational statistics rather than
statistics about specific projects. That is if you were to roll up the data
from across the projects into valuable data about the foundation as a whole I
can see real value with minimal risk. Sally, over on press@ is currently
looking at the kind of data that would be useful to report.
Ross
-----Original Message-----
From: Sergio Fernández [mailto:sergio.fernan...@salzburgresearch.at]
Sent: Sunday, November 23, 2014 11:37 AM
To: general@incubator.apache.org
Subject: Re: dashboarding incubator
After awaiting for feedback about my proposal, I understand there are three
different aspects that should be discussed:
* Cost: As Ross pointed, the potential prize is important to evaluate a
solution. Although I'd love to use the professional services of the company,
the toolkit is open/free software and be freely used, which moves more
attention to the next point.
* Infrastructure requirements: Specially in the case we decide to provide all
by ourselves, such service would have some infrastructure requirements that
need to be studied, as David correctly pointed.
* Technical proposition: In the end the first two aspect should not be critical
if the proposition brings some value, to the project-level, Incubator or ASF.
I really see strong arguments against the proposal regarding the first two
aspects. The third is not that easy, since I do not see how such metrics should
be used for evaluating projects, rather than just bringing some indicators.
Before taking the discussion to the next level, where costs and resources need
to be evaluated, I opened this discussion proposing my time and personal
resources to provide a simple proof of concept. Then we should have more
arguments (how much resources are actually required, how useful are the
indicators the dashboard provides, etc...) to move the discussion to the next
level.
But of course I'd like to have the good pleasure before investing time.
So I'd like to ask the following question: is there already any argument to say
that inevitably the answer of the proof of concept will be negative?
Cheers,
On 21/11/14 21:27, Ross Gardler (MS OPEN TECH) wrote:
> We already evaluated the Bitergia offering - it is expensive and does
> not provide sufficient benefit for the money (don't get me started on
> how metrics are not a good evaluator of open source code...)
>
> I fully agree with the comments below.
>
> Ross
>
> -----Original Message-----
> From: jan i [mailto:j...@apache.org]
> Sent: Friday, November 21, 2014 12:24 PM
> To: general@incubator.apache.org
> Subject: Re: dashboarding incubator
>
> On 21 November 2014 20:44, Bertrand Delacretaz
> <bdelacre...@apache.org>
> wrote:
>
>> Hi,
>>
>> On Fri, Nov 21, 2014 at 3:35 PM, David Nalley <da...@gnsa.us> wrote:
>>> ...I am generally against us standing up our own service that does this.
>>> We've had a couple of these systems over the years. (pulse.a,o for
>>> instance). It takes a non-trivial amount of work to setup and
>>> maintain such a system, and invariably it falls apart....
>>
>> I agree, OTOH if someone wants to help third parties get the data
>> that they need to implement such services externally that might be fine.
>>
> Having our own service will only marginally provide us with something better,
> and will cost (in endeffect) contractor resources, so I agree with david.
>
> rgds
> jan i.
>
>
>>
>> -Bertrand
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>> For additional commands, e-mail: general-h...@incubator.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
--
Sergio Fernández
Senior Researcher
Knowledge and Media Technologies
Salzburg Research Forschungsgesellschaft mbH Jakob-Haringer-Straße 5/3 | 5020
Salzburg, Austria
T: +43 662 2288 318 | M: +43 660 2747 925 sergio.fernan...@salzburgresearch.at
http://www.salzburgresearch.at
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
