[Wikimedia-l] Re: Community Wishlist Survey 2022 is coming. Help us and prepare

[Inductiveload]

On Thu, 30 Dec 2021 at 11:52,  wrote:
> This year, we (that is, the WMF using movement funds) spent a huge amount of 
> money ($4.5 million) just directly donating to external knowledge equity 
> funds.

>From https://meta.wikimedia.org/wiki/Knowledge_Equity_Fund:

> Many of the barriers that prevent people from accessing and contributing to 
> free knowledge are rooted in systems of racial oppression.

What about many of the barriers to contributing thrown up by broken or
defective systems? This disproportionately affects people without the
technical skills to circumvent the issues and/or implement fixes
and/or gain political capital to agitate for a fix from the "system".
Considering the well-documented inequalities both on racial and gender
grounds in STEM subjects and computing in particular, these technical
defects strike harder at the already-disadvantaged groups.

Combine this with the adversarial system of actually getting things
done by having to fight, grab and compete for the scant resources
thrown down, combined with the need to advocate for your issue just-so
to get what you want (which again disadvantages "system-outsiders")
and again this is disadvantageous to the already-disadvantaged.

Even if we just ignore race or gender, also consider that deficient
technical platforms are also a brutal filter against _anyone_ without
a STEM background: also people the knowledge equity stuff is supposed
to help. The WMF has contrived a system that almost seems perfectly
designed to filter out many of the disadvantaged or vulnerable groups
they profess to want to help by failing to actually provide support
for them to make their contributions in the first place. A few bright
pictures and fluffy press releases and slinging money around on
wishy-washy initiatives surely keeps everyone looking busy, fills some
nice FTE positions and gets Twitter cachet. However, a bug's a bug and
a missing feature is a missing feature.

A simple example: there's not even a "blessed" way to ask for
technical help: you get ignored at COM:VPT, ignored at Mediawiki.org's
dead manual pages, find out that the Discord isn't even official,
bounce around several random IRC channels (once you figure out IRC in
the first place, let's not pretend most people in the world have even
heard of it) looking for one that can help you, eventually off to
Phabricator, ignored again if your issue isn't written up '"just so"
to align with the priorities of some "team" that isn't actually
documented anywhere and then...you give up.

Another example closer to my home wiki: the smaller Wikisources are
very disadvantaged by technical limitations which the big Wikisources
work around on an ad-hoc basis using local users' skills. Many
features that the big ones have would be _more_ useful for the smaller
ones. A good example here is the OCR, which for years worked only at a
handful of big Wikisources until CommTech integrated the OCR tool this
year into the Wikisource extension. But the big wikis didn't really
need the OCR as much (though they still needed it), because upstream
OCR is usually better in English or French, than, say, Marathi. I had
a wish to "upstream" more local features to the core extensions so all
can benefit. But it's taken so long to get anything done, that I am
out of time and have to go to a day-job soon, so it's not going to
happen. Who has the luxury of tens or hundreds of hours spare to learn
PHP from scratch to fix an issue, submit, resubmit, rebase and massage
patches? Guess what: not the knowledge-equity-disadvantaged.

Perhaps instead of a firehose of money at fighting the "far enemy" of
racial/social/etc. injustice, a more directed attitude of fixing the
platforms and making it attractive and/or even possible for these
groups to contribute in the first place is in order. Some of it's not
even hard, certainly not $4,500,000-hard. It doesn't matter, in wiki
terms (and those are terms that donors believe their money is to be
used) how well you train a group of Jordanian journalists if when they
come to upload their video, the upload fails, does it? Or PattyPan, or
whatever tool has been created to address the lack of "native" tools
is broken _again_.

Even if the "magic money tree" doesn't extend to more than one
engineering or technical staff member, just a single FTE rotating a
day per week though the projects, dedicated to helping others
contribute, linking up issues with teams and driving things forward
when they stall out (rather than just waiting for the would-be
contributor to give up and abandon it all) would be a huge force
multiplier. Allegedly, there are _already_ Developer Advocates, but
whatever it is they do does not seem to actually involve advocating
for the ability of would-be contributors to actually contribute.

The most frustrating thing is that this could all work well: Mediawiki
is a solid piece of engineering, running on a solid operational
foundation. There is CI, code review, relatively clean code, a

[Wikimedia-l] Re: [Commons-l] Re: Re: [Wikitech-l] Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

[Gnangarra]

I see part of the problem is that the contributors experiencing the biggest
impact arent the same contributors that have the technical skill sets to
appropriately explain and understand the issues. Adding the the need to be
able to make comparisons between other areas of need just makes its even
more difficult.  None of us want to be putting forward arguments that say
that the WMF should neglect supporting Wikidata functions so that repairs
can be made to Commons functions, this is a loss for everyone.

I know that the experience of the volunteers who spent 3 months in limbo
trying to get the 2021 Wikimania videos converted and uploaded  will feed
back through to WMF hierarchy highlighting, but whether that taken has a
priority needing to be fixed or bug to swatted is unknown.  The underlying
issue isnt so much that we need to fix software(though we do) as it is that
we have structural problems in the way the WMF technical team interacts
with each project. With that its ability to keep with the growth and
maintenance necessary to function effectively.

The point I raised is that like many other aspects the software and
technical support along with its communication channels havent effectively
kept up with the needs of the community, not even the wishlist itself can
keep up with it.  This is why I said we need to pause and rethink the whole
process, focus on clearing whats on the phabricator while we do so.

The frustration comes from being able to upload a video to the likes of
youtube or vimeo in about 15-20 minutes, where as its takes 30 hours to
convert to webm on proprietary software which I have to pay for and then
10-12 attempts over the space of a week or two to upload the video to
commons.  The available tools like Videoconvertor, and Video2Commons are so
unstable that they dont survive the 30 hour conversion process.


On Sun, 2 Jan 2022 at 04:38, Mike Peel  wrote:

> Hi Asaf,
>
> That's a good response, but I'm not sure it provides a practical way
> forward. How can volunteers bring this issue to the attention of the WMF
> leadership to get the allocation of the time of Wikimedia staff who can
> take ownership implement changes here?
>
> Presumably emails on these lists have relatively little impact at the
> most senior levels, so they aren't a good way forward - and similarly on
> Phabricator.
>
> The Wishlist provides a way of showcasing issues and a relatively clear
> way forward to get them implemented, but with really limited capacity.
>
> How would a case for technical support be made apart from that? It's not
> clear if a simple survey would be sufficient. Would an RfC and
> discussion on meta help? Does it need the media to be involved to make
> it a public crisis? Or should it be proposed as a grant request, perhaps
> for a Wikimedia affiliate to implement? Or is there another avenue that
> could be persued? Bearing in mind that there's no practical way for
> community members to propose changes to the WMF annual plan for multiple
> years now.
>
> Sorry to defocus things and express more frustration, but I think there
> should be a clear way forward with this type of issue, which isn't
> obvious right now. Personally, my hopes are on the Wishlist, although
> I'll be reposting a 14-year-old issue there for the fifth time when that
> process opens on the 10th January...
>
> Thanks,
> Mike
>
> On 1/1/22 20:10:43, Asaf Bartov wrote:
> > Writing in my volunteer capacity:
> >
> > On Sat, 1 Jan 2022, 08:43 Amir Sarabadani  > > wrote:
> >
> >
> > Honestly, the situation is more dire than you think. For example,
> > until a couple months ago, we didn't have backups for the media
> > files. There was a live copy in the secondary datacenter but for
> > example if due to a software issue, we lost some files, they were
> > gone. I would like to thank Jaime Crespo for pushing for it and
> > implementing the backups.
> >
> > But I beat my drum again, it's not something you can fix overnight.
> > I'm sure people are monitoring this mailing list and are aware of
> > the problem.
> >
> >
> > [My goal in this post is to ficus effort and reduce frustration.]
> >
> > Yes, people reading here are aware, and absolutely none of them expects
> > this (i.e. multimedia technical debt and missing features) to be fixed
> > overnight.
> >
> > What's lacking, as you pointed out, is ownership of the problem.  To own
> > the problem, one must have *both* technical understanding of the issues
> > *and* a mandate to devote resources to addressing them.
> >
> > It is this *combination* that we don't have at the moment. Lots of
> > technical people are aware, and some of them quite willing to work
> > toward addressing the issues, but they are not empowered to set
> > priorities and commit resources for an effort of that scale, and the
> > problems, for the most part, don't easily lend themselves to volunteer
> > development.
> >
> > It seems to me there are 

[Wikimedia-l] Re: [Wikitech-l] Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

[Mike Peel]

Hi Asaf,

That's a good response, but I'm not sure it provides a practical way 
forward. How can volunteers bring this issue to the attention of the WMF 
leadership to get the allocation of the time of Wikimedia staff who can 
take ownership implement changes here?


Presumably emails on these lists have relatively little impact at the 
most senior levels, so they aren't a good way forward - and similarly on 
Phabricator.


The Wishlist provides a way of showcasing issues and a relatively clear 
way forward to get them implemented, but with really limited capacity.


How would a case for technical support be made apart from that? It's not 
clear if a simple survey would be sufficient. Would an RfC and 
discussion on meta help? Does it need the media to be involved to make 
it a public crisis? Or should it be proposed as a grant request, perhaps 
for a Wikimedia affiliate to implement? Or is there another avenue that 
could be persued? Bearing in mind that there's no practical way for 
community members to propose changes to the WMF annual plan for multiple 
years now.


Sorry to defocus things and express more frustration, but I think there 
should be a clear way forward with this type of issue, which isn't 
obvious right now. Personally, my hopes are on the Wishlist, although 
I'll be reposting a 14-year-old issue there for the fifth time when that 
process opens on the 10th January...


Thanks,
Mike

On 1/1/22 20:10:43, Asaf Bartov wrote:

Writing in my volunteer capacity:

On Sat, 1 Jan 2022, 08:43 Amir Sarabadani > wrote:



Honestly, the situation is more dire than you think. For example,
until a couple months ago, we didn't have backups for the media
files. There was a live copy in the secondary datacenter but for
example if due to a software issue, we lost some files, they were
gone. I would like to thank Jaime Crespo for pushing for it and
implementing the backups.

But I beat my drum again, it's not something you can fix overnight.
I'm sure people are monitoring this mailing list and are aware of
the problem.


[My goal in this post is to ficus effort and reduce frustration.]

Yes, people reading here are aware, and absolutely none of them expects 
this (i.e. multimedia technical debt and missing features) to be fixed 
overnight.


What's lacking, as you pointed out, is ownership of the problem.  To own 
the problem, one must have *both* technical understanding of the issues 
*and* a mandate to devote resources to addressing them.


It is this *combination* that we don't have at the moment. Lots of 
technical people are aware, and some of them quite willing to work 
toward addressing the issues, but they are not empowered to set 
priorities and commit resources for an effort of that scale, and the 
problems, for the most part, don't easily lend themselves to volunteer 
development.


It seems to me there are *very few* people who could change status quo, 
not much more than a handful: the Foundation's executive leadership (in 
its annual planning work, coming up this first quarter of 2022), and the 
Board of Trustees.


Therefore, the greatest contribution the rest of us could make toward 
seeing this work get resourced is to help make the case to the 
executives (including the new CEO, just entering the role) with clear 
and compelling illustrations of the *mission impact* of such investment. 
In parallel, interested engineers and middle managers could help by 
offering rough effort estimates for some components, a roadmap, an 
overview of alternatives considered and a rationale for a recommended 
approach, etc.


But this would all be preparatory and supporting work toward *a 
resourcing decision* being made. So long as such a decision isn't made, 
no significant work on this can happen.


Finally, while it is easy to agree that *this* is necessary and useful 
on its own, to actual resource it in the coming annual plan it would be 
necessary to argue that it is *more* useful and necessary than some 
*other* work, itself also necessary and useful.


Another thing that may help is being explicit about just how important 
this is, even literally saying things like "this would have far more 
impact on our X goal than initiative A, B, or C", naming actual 
resourced or potentially resourced things. It is sometimes difficult for 
managers who aren't practicing Wikimedia volunteers to assess just how 
necessary different necessary things are, from different community 
perspectives.


And of course, one such opinion, or a handful, would not be a solid base 
for resourcing decisions, so perhaps a large-scale ranking survey of 
some sort would be helpful, as SJ implicitly suggested in the original post.


Cheers,

     A.
     (In my volunteer capacity)

___
Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and 

[Wikimedia-l] Re: [Wikitech-l] Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

[Asaf Bartov]

Writing in my volunteer capacity:

On Sat, 1 Jan 2022, 08:43 Amir Sarabadani  wrote:


Honestly, the situation is more dire than you think. For example, until a
couple months ago, we didn't have backups for the media files. There was a
live copy in the secondary datacenter but for example if due to a software
issue, we lost some files, they were gone. I would like to thank Jaime
Crespo for pushing for it and implementing the backups.

But I beat my drum again, it's not something you can fix overnight. I'm
sure people are monitoring this mailing list and are aware of the problem.


[My goal in this post is to ficus effort and reduce frustration.]

Yes, people reading here are aware, and absolutely none of them expects
this (i.e. multimedia technical debt and missing features) to be fixed
overnight.

What's lacking, as you pointed out, is ownership of the problem.  To own
the problem, one must have *both* technical understanding of the issues
*and* a mandate to devote resources to addressing them.

It is this *combination* that we don't have at the moment. Lots of
technical people are aware, and some of them quite willing to work toward
addressing the issues, but they are not empowered to set priorities and
commit resources for an effort of that scale, and the problems, for the
most part, don't easily lend themselves to volunteer development.

It seems to me there are *very few* people who could change status quo, not
much more than a handful: the Foundation's executive leadership (in its
annual planning work, coming up this first quarter of 2022), and the Board
of Trustees.

Therefore, the greatest contribution the rest of us could make toward
seeing this work get resourced is to help make the case to the executives
(including the new CEO, just entering the role) with clear and compelling
illustrations of the *mission impact* of such investment. In parallel,
interested engineers and middle managers could help by offering rough
effort estimates for some components, a roadmap, an overview of
alternatives considered and a rationale for a recommended approach, etc.

But this would all be preparatory and supporting work toward *a resourcing
decision* being made. So long as such a decision isn't made, no significant
work on this can happen.

Finally, while it is easy to agree that *this* is necessary and useful on
its own, to actual resource it in the coming annual plan it would be
necessary to argue that it is *more* useful and necessary than some *other*
work, itself also necessary and useful.

Another thing that may help is being explicit about just how important this
is, even literally saying things like "this would have far more impact on
our X goal than initiative A, B, or C", naming actual resourced or
potentially resourced things. It is sometimes difficult for managers who
aren't practicing Wikimedia volunteers to assess just how necessary
different necessary things are, from different community perspectives.

And of course, one such opinion, or a handful, would not be a solid base
for resourcing decisions, so perhaps a large-scale ranking survey of some
sort would be helpful, as SJ implicitly suggested in the original post.

Cheers,

A.
(In my volunteer capacity)
___
Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: 
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and 
https://meta.wikimedia.org/wiki/Wikimedia-l
Public archives at 
https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/XQ2INJOXSLW76CQ7UXN5ZMIADUZM7HWI/
To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org

[Wikimedia-l] Re: [Wikitech-l] Re: Uplifting the multimedia stack (was: Community Wishlist Survery)

[Peter Southwood]

“until a couple months ago, we didn't have backups for the media files”

How is that even possible?

Cheers, Peter

 

From: Amir Sarabadani [mailto:ladsgr...@gmail.com] 
Sent: 01 January 2022 08:42
To: Wikitech-l
Cc: Wikimedia Mailing List; Wikimedia Commons Discussion List
Subject: [Wikimedia-l] Re: [Wikitech-l] Re: Uplifting the multimedia stack 
(was: Community Wishlist Survery)

 

 

 

On Fri, Dec 31, 2021 at 12:42 AM Strainu  wrote:

> So where is the best current place to discuss scaling Commons, and all that 
> entails? 

My impression is that we don't have one. All we hear is "it needs to be 
planned", but there is no transparency on what that planning involves or when 
it actually happens.

> I'd be surprised if the bottleneck were people or budget 

The main problem I see is that we end up in this kind of situation. Scaling and 
bug fixing critical features should be part of the annual budget. Each line of 
code deployed to production wikis should have an owner and associated 
maintenance budget each year. Without this, the team will not even commit 
reviews - see the thread on wikitech a few months back where a volunteer 
programmer willing to work on Upload Wizard was basically told "We will not 
review your code. Go fork." 

 

There is "code stewardship program" and its goal is to find owners for 
components that don't have an owner (or undeploy them). Sometimes it's 
successful, sometimes it's not. I have been asking for a maintainer for 
FlaggedRevs for four years now, beta cluster is suffering from a similar 
situation, etc. etc. It takes time to find an owner for everything, to fill the 
gaps in places we don't have a team to handle those (e.g. Multimedia, you can't 
just hand over that to team responsible for security for example). More info at 
https://www.mediawiki.org/wiki/Code_stewardship_reviews


> Some examples from recent discussions  

Also improvements to the Upload Wizard. There are quite a few open items in 
Phab on this. 

+1 


I really hope you will have better luck than others with bringing this issue up 
in the priority list for next year - multimedia support is growing more 
outdated by the minute. 

Honestly, the situation is more dire than you think. For example, until a 
couple months ago, we didn't have backups for the media files. There was a live 
copy in the secondary datacenter but for example if due to a software issue, we 
lost some files, they were gone. I would like to thank Jaime Crespo for pushing 
for it and implementing the backups.

 

But I beat my drum again, it's not something you can fix overnight. I'm sure 
people are monitoring this mailing list and are aware of the problem.

 

Best


Strainu 

Pe joi, 30 decembrie 2021, Samuel Klein  a scris:
> Separate thread.  I'm not sure which list is appropriate. 
> ... but not all the way to sentience.
>
> The annual community wishlist survey (implemented by a small team, possibly 
> in isolation?) may not be the mechanism for prioritizing large changes, but 
> the latter also deserves a community-curated priority queue.  To complement 
> the staff-maintained priorities in phab ~
> For core challenges (like Commons stability and capacity), I'd be surprised 
> if the bottleneck were people or budget.  We do need a shared understanding 
> of what issues are most important and most urgent, and how to solve them. For 
> instance, a way to turn Amir's recent email about the problem (and related 
> phab tickets) into a family of persistent, implementable specs and proposals 
> and their articulated obstacles.
> An issue tracker like phab is good for tracking the progress and dependencies 
> of agreed-upon tasks, but weak for discussing what is important, what we know 
> about it, how to address it. And weak for discussing ecosystem-design issues 
> that are important and need persistent updating but don't have a simple 
> checklist of steps.
> So where is the best current place to discuss scaling Commons, and all that 
> entails?  Some examples from recent discussions (most from the wm-l thread 
> below):
> - Uploads: Support for large file uploads / Keeping bulk upload tools online
> - Video: Debugging + rolling out the videojs player
> - Formats: Adding support for CML and dozens of other common high-demand file 
> formats
> - Thumbs: Updating thumbor and librsvg
> - Search: WCQS still down, noauth option wanted for tools
> - General: Finish implementing redesign of the image table
>
> SJ
> On Wed, Dec 29, 2021 at 6:26 AM Amir Sarabadani  wrote:
>>
>> I'm not debating your note. It is very valid that we lack proper support for 
>> multimedia stack. I myself wrote a detailed rant on how broken it is [1] but 
>> three notes:
>>  - Fixing something like this takes time, you need to assign the budget for 
>> it (which means it has to be done during the annual planning) and if gets 
>> approved, you need to start it with the fiscal year (meaning July 2022) and 
>> then hire (meaning, write JD, do recruitment,