[Wikitech-l] Re: [Wikimedia-l] Re: Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

2022-01-12 Thread Strainu 
În mar., 11 ian. 2022 la 08:01, Kunal Mehta  a scris:
>
> So I think the status quo can be changed by just about anyone who is
> motivated to do so, not by trying to convince the WMF to change its
> prioritization, but just by doing the work. We should be empowering
> those people rather than continuing to further entrench a WMF technical
> monopoly.
>

Counterexample:
https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/G2QTRJFAUKLE45SFTFUHOOTOBR6G3DP3/
(this was the situation that I quoted in my first email on this thread
as the WMF refusing to even do reviews).

Maybe it's just the multimedia part that it's in this desperate
situation, but I can totally see volunteer developers getting
discouraged quickly if their patches are outright ignored.

Strainu
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Wikimedia-l] Re: Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

2022-01-11 Thread Amir Sarabadani 
(Speaking in my volunteer capacity)
I doubt there is any malicious intent by WMF. I personally think the
underlying problem is time. Let me explain.

Fixing a big issue in software takes time (I wrote a long essay about it in
this thread) so it makes sense WMF annual planning to focus on issues
before they get to a level that hinders community's work. The problem is
that an issue doesn't get enough attention if it's not severe enough to
affect users so the cycle of frustration continues. For example, I sent an
email in February 2021, at the start of annual planning, to one of the
directors at product outlining all of the issues of multimedia stack.
Because at that point, it wasn't this bad, it didn't make it to FY21-22
plans. Now I feel like a cassandra. We have similar issues in lots of other
places that will lead to frustration. Load balancers (pybal), dumps, beta
cluster, flagged revs, patrolling tools, etc. etc.

On Tue, Jan 11, 2022 at 8:21 AM bawolff  wrote:

> Honestly, I find the "not in the annual plan" thing more damning than the
> actual issue at hand.
>
> The core competency of WMF is supposed to be keeping the site running. WMF
> does a lot of things, some of them very useful, others less so, but at its
> core its mission is to keep the site going. Everything else should be
> secondary to that.
>
> It should be obvious that running a 300 TB+ media store servicing 70
> billion requests a month requires occasional investment and maintenance
>
> And yet, this was not only not in this year's annual plan, it has been
> ignored in the annual plan for many many years. We didn't get to this state
> by just 1 year of neglect.
>
> Which raises the question - If wmf is not in the business of keeping the
> Wikimedia sites going, what is it in the business of?
>
> On Tue, Jan 11, 2022 at 6:01 AM Kunal Mehta  wrote:
>
>> Hi,
>>
>> On 1/1/22 12:10, Asaf Bartov wrote:
>> > It seems to me there are *very few* people who could change status quo,
>> > not much more than a handful: the Foundation's executive leadership (in
>> > its annual planning work, coming up this first quarter of 2022), and
>> the
>> > Board of Trustees.
>>
>> If the goal is to get paid WMF staff to fix the issues, then you're
>> correct. However, I do not believe that as a solution is healthy
>> long-term. The WMF isn't perfect and I don't think it's desirable to
>> have a huge WMF that tries to do everything and has a monopoly on
>> technical prioritization.
>>
>> The technical stack must be co-owned by volunteers and paid staff from
>> different orgs at all levels. It's significantly more straightforward
>> now for trusted volunteers to get NDA/deployment access than it used to
>> be, there are dedicated training sessions, etc.
>>
>> Given that the multimedia stack is neglected and the WMF has given no
>> indication it intends to work on/fix the problem, we should be
>> recruiting people outside the WMF's paid staff who are interested in
>> working on this and give them the necessary access/mentorship to get it
>> done. Given the amount of work on e.g. T40010[1] to develop an
>> alternative SVG renderer, I'm sure those people exist.
>>
>> Take moving Thumbor to Buster[2] for example. That requires
>> forward-porting some Debian packages written Python, and then testing in
>> WMCS that there's no horrible regressions in newer imagemagick, librsvg,
>> etc. I'm always happy to mentor people w/r to Debian packaging (and have
>> done so in the past), and there are a decent amount of people in our
>> community who know Python, and likely others from the Commons community
>> who would be willing to help with testing and dealing with whatever
>> fallout.
>>
>> So I think the status quo can be changed by just about anyone who is
>> motivated to do so, not by trying to convince the WMF to change its
>> prioritization, but just by doing the work. We should be empowering
>> those people rather than continuing to further entrench a WMF technical
>> monopoly.
>>
>> [1] https://phabricator.wikimedia.org/T40010
>> [2] https://phabricator.wikimedia.org/T216815
>>
>> -- Legoktm
>> ___
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/



-- 
Amir (he/him)
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Wikimedia-l] Re: Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

2022-01-10 Thread bawolff 
Honestly, I find the "not in the annual plan" thing more damning than the
actual issue at hand.

The core competency of WMF is supposed to be keeping the site running. WMF
does a lot of things, some of them very useful, others less so, but at its
core its mission is to keep the site going. Everything else should be
secondary to that.

It should be obvious that running a 300 TB+ media store servicing 70
billion requests a month requires occasional investment and maintenance

And yet, this was not only not in this year's annual plan, it has been
ignored in the annual plan for many many years. We didn't get to this state
by just 1 year of neglect.

Which raises the question - If wmf is not in the business of keeping the
Wikimedia sites going, what is it in the business of?

On Tue, Jan 11, 2022 at 6:01 AM Kunal Mehta  wrote:

> Hi,
>
> On 1/1/22 12:10, Asaf Bartov wrote:
> > It seems to me there are *very few* people who could change status quo,
> > not much more than a handful: the Foundation's executive leadership (in
> > its annual planning work, coming up this first quarter of 2022), and the
> > Board of Trustees.
>
> If the goal is to get paid WMF staff to fix the issues, then you're
> correct. However, I do not believe that as a solution is healthy
> long-term. The WMF isn't perfect and I don't think it's desirable to
> have a huge WMF that tries to do everything and has a monopoly on
> technical prioritization.
>
> The technical stack must be co-owned by volunteers and paid staff from
> different orgs at all levels. It's significantly more straightforward
> now for trusted volunteers to get NDA/deployment access than it used to
> be, there are dedicated training sessions, etc.
>
> Given that the multimedia stack is neglected and the WMF has given no
> indication it intends to work on/fix the problem, we should be
> recruiting people outside the WMF's paid staff who are interested in
> working on this and give them the necessary access/mentorship to get it
> done. Given the amount of work on e.g. T40010[1] to develop an
> alternative SVG renderer, I'm sure those people exist.
>
> Take moving Thumbor to Buster[2] for example. That requires
> forward-porting some Debian packages written Python, and then testing in
> WMCS that there's no horrible regressions in newer imagemagick, librsvg,
> etc. I'm always happy to mentor people w/r to Debian packaging (and have
> done so in the past), and there are a decent amount of people in our
> community who know Python, and likely others from the Commons community
> who would be willing to help with testing and dealing with whatever
> fallout.
>
> So I think the status quo can be changed by just about anyone who is
> motivated to do so, not by trying to convince the WMF to change its
> prioritization, but just by doing the work. We should be empowering
> those people rather than continuing to further entrench a WMF technical
> monopoly.
>
> [1] https://phabricator.wikimedia.org/T40010
> [2] https://phabricator.wikimedia.org/T216815
>
> -- Legoktm
> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Wikimedia-l] Re: Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

2022-01-10 Thread Kunal Mehta 

Hi,

On 1/1/22 12:10, Asaf Bartov wrote:
It seems to me there are *very few* people who could change status quo, 
not much more than a handful: the Foundation's executive leadership (in 
its annual planning work, coming up this first quarter of 2022), and the 
Board of Trustees.


If the goal is to get paid WMF staff to fix the issues, then you're 
correct. However, I do not believe that as a solution is healthy 
long-term. The WMF isn't perfect and I don't think it's desirable to 
have a huge WMF that tries to do everything and has a monopoly on 
technical prioritization.


The technical stack must be co-owned by volunteers and paid staff from 
different orgs at all levels. It's significantly more straightforward 
now for trusted volunteers to get NDA/deployment access than it used to 
be, there are dedicated training sessions, etc.


Given that the multimedia stack is neglected and the WMF has given no 
indication it intends to work on/fix the problem, we should be 
recruiting people outside the WMF's paid staff who are interested in 
working on this and give them the necessary access/mentorship to get it 
done. Given the amount of work on e.g. T40010[1] to develop an 
alternative SVG renderer, I'm sure those people exist.


Take moving Thumbor to Buster[2] for example. That requires 
forward-porting some Debian packages written Python, and then testing in 
WMCS that there's no horrible regressions in newer imagemagick, librsvg, 
etc. I'm always happy to mentor people w/r to Debian packaging (and have 
done so in the past), and there are a decent amount of people in our 
community who know Python, and likely others from the Commons community 
who would be willing to help with testing and dealing with whatever fallout.


So I think the status quo can be changed by just about anyone who is 
motivated to do so, not by trying to convince the WMF to change its 
prioritization, but just by doing the work. We should be empowering 
those people rather than continuing to further entrench a WMF technical 
monopoly.


[1] https://phabricator.wikimedia.org/T40010
[2] https://phabricator.wikimedia.org/T216815

-- Legoktm
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: [Wikimedia-l] Re: Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

2022-01-01 Thread Mike Peel 

Hi Asaf,

That's a good response, but I'm not sure it provides a practical way 
forward. How can volunteers bring this issue to the attention of the WMF 
leadership to get the allocation of the time of Wikimedia staff who can 
take ownership implement changes here?


Presumably emails on these lists have relatively little impact at the 
most senior levels, so they aren't a good way forward - and similarly on 
Phabricator.


The Wishlist provides a way of showcasing issues and a relatively clear 
way forward to get them implemented, but with really limited capacity.


How would a case for technical support be made apart from that? It's not 
clear if a simple survey would be sufficient. Would an RfC and 
discussion on meta help? Does it need the media to be involved to make 
it a public crisis? Or should it be proposed as a grant request, perhaps 
for a Wikimedia affiliate to implement? Or is there another avenue that 
could be persued? Bearing in mind that there's no practical way for 
community members to propose changes to the WMF annual plan for multiple 
years now.


Sorry to defocus things and express more frustration, but I think there 
should be a clear way forward with this type of issue, which isn't 
obvious right now. Personally, my hopes are on the Wishlist, although 
I'll be reposting a 14-year-old issue there for the fifth time when that 
process opens on the 10th January...


Thanks,
Mike

On 1/1/22 20:10:43, Asaf Bartov wrote:

Writing in my volunteer capacity:

On Sat, 1 Jan 2022, 08:43 Amir Sarabadani > wrote:



Honestly, the situation is more dire than you think. For example,
until a couple months ago, we didn't have backups for the media
files. There was a live copy in the secondary datacenter but for
example if due to a software issue, we lost some files, they were
gone. I would like to thank Jaime Crespo for pushing for it and
implementing the backups.

But I beat my drum again, it's not something you can fix overnight.
I'm sure people are monitoring this mailing list and are aware of
the problem.


[My goal in this post is to ficus effort and reduce frustration.]

Yes, people reading here are aware, and absolutely none of them expects 
this (i.e. multimedia technical debt and missing features) to be fixed 
overnight.


What's lacking, as you pointed out, is ownership of the problem.  To own 
the problem, one must have *both* technical understanding of the issues 
*and* a mandate to devote resources to addressing them.


It is this *combination* that we don't have at the moment. Lots of 
technical people are aware, and some of them quite willing to work 
toward addressing the issues, but they are not empowered to set 
priorities and commit resources for an effort of that scale, and the 
problems, for the most part, don't easily lend themselves to volunteer 
development.


It seems to me there are *very few* people who could change status quo, 
not much more than a handful: the Foundation's executive leadership (in 
its annual planning work, coming up this first quarter of 2022), and the 
Board of Trustees.


Therefore, the greatest contribution the rest of us could make toward 
seeing this work get resourced is to help make the case to the 
executives (including the new CEO, just entering the role) with clear 
and compelling illustrations of the *mission impact* of such investment. 
In parallel, interested engineers and middle managers could help by 
offering rough effort estimates for some components, a roadmap, an 
overview of alternatives considered and a rationale for a recommended 
approach, etc.


But this would all be preparatory and supporting work toward *a 
resourcing decision* being made. So long as such a decision isn't made, 
no significant work on this can happen.


Finally, while it is easy to agree that *this* is necessary and useful 
on its own, to actual resource it in the coming annual plan it would be 
necessary to argue that it is *more* useful and necessary than some 
*other* work, itself also necessary and useful.


Another thing that may help is being explicit about just how important 
this is, even literally saying things like "this would have far more 
impact on our X goal than initiative A, B, or C", naming actual 
resourced or potentially resourced things. It is sometimes difficult for 
managers who aren't practicing Wikimedia volunteers to assess just how 
necessary different necessary things are, from different community 
perspectives.


And of course, one such opinion, or a handful, would not be a solid base 
for resourcing decisions, so perhaps a large-scale ranking survey of 
some sort would be helpful, as SJ implicitly suggested in the original post.


Cheers,

     A.
     (In my volunteer capacity)

___
Wikimedia-l mailing list -- wikimedi...@lists.wikimedia.org, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and