[Wikitech-l] Re: Wiki content and other dumps new ownership, feedback requested on new version!

[Dennis During]

Thanks for setting an example for this kind of work should be done.

On Fri, Sep 29, 2023, 7:43 PM Steven Walling 
wrote:

> +1 to Dan. Ariel thank you for so many years of doing this essential work.
>
> On Thu, Sep 28, 2023 at 4:14 AM Dan Andreescu 
> wrote:
>
>> Ariel, your dedication to dumps has always been an inspiration to me,
>> both in how you handled the technology and how you fused that with your
>> care for the ideas behind it.  Thank you, and hope we can do it justice
>> with the next phase of Dumps.
>>
>> On Thu, Sep 28, 2023 at 12:39 AM Ariel Glenn WMF 
>> wrote:
>>
>>> Hello folks!
>>>
>>> For some years now, I've been the main or only point of contact for the
>>> Wiki project sql/xml dumps semimonthly, as well as for a number of
>>> miscellaneous weekly datasets.
>>>
>>> This work is now passing to Data Platform Engineering (DPE), and your
>>> new points of contact, starting right away, will be Will Doran
>>> (email:wdoran) and Virginia Poundstone (email:vpoundstone). I'll still be
>>> lending a hand in the background for a little while but by the end of the
>>> month I'll have transitioned into a new role at the Wikimedia Foundation,
>>> working more directly on MediaWiki itself.
>>>
>>> The Data Products team, a subteam of DPE, will be managing the current
>>> dumps day-to-day, as well as working on a new dumps system intended to
>>> replace and greatly improve the current one. What formats will it produce,
>>> and what content, and in what bundles?  These are all great questions, and
>>> you have a chance to help decide on the answers. The team is gathering
>>> feedback right now; follow this link [
>>> https://docs.google.com/forms/d/e/1FAIpQLScp2KzkcTF7kE8gilCeSogzpeoVN-8yp_SY6Q47eEbuYfXzsw/viewform?usp=sf_link]
>>> to give your input!
>>>
>>> If you want to follow along on work being done on the new dumps system,
>>> you can check the phabricator workboard at
>>> https://phabricator.wikimedia.org/project/board/6630/ and look for
>>> items with the "Dumps 2.0" tag.
>>>
>>> Members of the Data Products team are already stepping up to manage the
>>> xmldatadumps-l mailing list, so you should not notice any changes as far as
>>> that goes.
>>>
>>> And as always, for dumps-related questions people on this list cannot
>>> answer, and which are not covered in the docs at
>>> https://meta.wikimedia.org/wiki/Data_dumps or
>>> https://wikitech.wikimedia.org/wiki/Dumps you can always email
>>> ops-dumps (at) wikimedia.org.
>>>
>>> See you on the wikis!
>>>
>>> Ariel Glenn
>>> ar...@wikimedia.org
>>> ___
>>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>>
>>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>>
>> ___
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: Wiki content and other dumps new ownership, feedback requested on new version!

[Steven Walling]

+1 to Dan. Ariel thank you for so many years of doing this essential work.

On Thu, Sep 28, 2023 at 4:14 AM Dan Andreescu 
wrote:

> Ariel, your dedication to dumps has always been an inspiration to me, both
> in how you handled the technology and how you fused that with your care for
> the ideas behind it.  Thank you, and hope we can do it justice with the
> next phase of Dumps.
>
> On Thu, Sep 28, 2023 at 12:39 AM Ariel Glenn WMF 
> wrote:
>
>> Hello folks!
>>
>> For some years now, I've been the main or only point of contact for the
>> Wiki project sql/xml dumps semimonthly, as well as for a number of
>> miscellaneous weekly datasets.
>>
>> This work is now passing to Data Platform Engineering (DPE), and your new
>> points of contact, starting right away, will be Will Doran (email:wdoran)
>> and Virginia Poundstone (email:vpoundstone). I'll still be lending a hand
>> in the background for a little while but by the end of the month I'll have
>> transitioned into a new role at the Wikimedia Foundation, working more
>> directly on MediaWiki itself.
>>
>> The Data Products team, a subteam of DPE, will be managing the current
>> dumps day-to-day, as well as working on a new dumps system intended to
>> replace and greatly improve the current one. What formats will it produce,
>> and what content, and in what bundles?  These are all great questions, and
>> you have a chance to help decide on the answers. The team is gathering
>> feedback right now; follow this link [
>> https://docs.google.com/forms/d/e/1FAIpQLScp2KzkcTF7kE8gilCeSogzpeoVN-8yp_SY6Q47eEbuYfXzsw/viewform?usp=sf_link]
>> to give your input!
>>
>> If you want to follow along on work being done on the new dumps system,
>> you can check the phabricator workboard at
>> https://phabricator.wikimedia.org/project/board/6630/ and look for items
>> with the "Dumps 2.0" tag.
>>
>> Members of the Data Products team are already stepping up to manage the
>> xmldatadumps-l mailing list, so you should not notice any changes as far as
>> that goes.
>>
>> And as always, for dumps-related questions people on this list cannot
>> answer, and which are not covered in the docs at
>> https://meta.wikimedia.org/wiki/Data_dumps or
>> https://wikitech.wikimedia.org/wiki/Dumps you can always email ops-dumps
>> (at) wikimedia.org.
>>
>> See you on the wikis!
>>
>> Ariel Glenn
>> ar...@wikimedia.org
>> ___
>> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
>> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
>>
>> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
>
> ___
> Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
> To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
> https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] MediaWiki Insights - second monthly email

[Birgit Müller]

Hello All,

Welcome to the second edition of the monthly MediaWiki Insights
 email!


Since the beginning of July, the Foundation has dedicated MediaWiki product
leadership and a new MediaWiki engineering group
.

In August
,
we’ve shared a broad overview on the focus for the next few quarters:


   1. Building up the new MW Engineering group and MW Product function
   2. Developing a strategy for MediaWiki - by June 30th, 2024 [WMF Annual
   Plan, WE3
   

   ]
   3. Reaching a 20% increase of authors to selected MediaWiki repositories
   deployed in Wikimedia production - by June 30th, 2024 [WE3.2]
   4. Investing in developer experiences and reduce fragmentation of
   developer workflows [WE 3.1] - continuous work with specific deliverables
   in 2023/24
   5. Exploring and resolving a set of questions around stewardship and
   Open Source strategy (goes beyond MediaWiki) [WE3.3]

We’re still in the process of “settling in” (1.), but also made progress on
a few things that we wanted to start tackling early:

Stewardship

We’ve had conversations within the MediaWiki Engineering group
 on which
components we should prioritize initially/own directly and what the things
are that we’d primarily provide guidance on. While this is work in progress
and also touches on bigger questions, one notable decision is that MW
Engineering takes on stewardship for the authentication-related components
(in MW core and extensions), with support from the Security team. We’ve
resolved outstanding code stewardship requests for the CentralAuth
 and Oauth
 extensions as a consequence of
this decision. These changes and other updates are reflected on the developers
and maintainers page 
on MW.org.

MediaWiki within Wikimedia’s ecosystem: Update on interviews

So far we’ve interviewed about 40 people on their experiences with
MediaWiki within Wikimedia’s ecosystem and plan a few more interviews. We
expect to be able to wrap up this first round of research in October and to
share the outcome and conclusions in November. These conversations have
been incredibly helpful - many thanks to all the people who took the time
to share their thoughts or still will do so <3


You can find a tentative timeline and overview on research planned
throughout the next few quarters on this page

.


Project snapshot: Source Maps and top-level autologin

Over the past few weeks, the MediaWiki Engineering group has been working
on a mix of onboarding tasks (i.e. ResourceLoader, ActionAPI, CentralAuth),
production errors, long term initiatives (Parsoid Read Views, RESTBase
deprecation), consultancy, code review for staff and volunteers’ patches,
and completed projects that had been in the making for a while. A few
snapshots:


Source Maps  aim to make debugging in web
development easier. It’s a technique for mapping combined and minified
JavaScript back to the original files. Support for source maps is now
implemented in ResourceLoader
, to aid with debugging
ResourceLoader in production. You can learn more about this work in this
ticket.  <3 to Tim, Timo and
others for their work on this!


Browsers increasingly roll out anti-tracking measures and limitations on
third-party cookie use. An unfortunate side effect of this is that it also
impacts CentralAuth autologin. One way to mitigate the effects and to allow
auto-login when the browser blocks third-party cookies is to attempt
central auto-login via top-level navigation. This has been enabled in
September. You can learn more about this work in this ticket
. <3 to Gergö and others for the
work on this!


Onboarding, among other means, has continued via the weekly Code Mob
sessions: Check out the recordings on this page
 if you
want to follow along.


Next: Enable more people to know MediaWiki and contribute effectively

A key question this year is how we can grow the number of people willing
and able to contribute to MediaWiki. So far we’ve explored approaches and
focus areas, turned some aspects of this already into active practice
through code review and consultancy for teams whose projects touch
MediaWiki core; and came up with first ideas that may help new Me

[Wikitech-l] Re: New developer feature: $wgUseXssLanguage / x-xss language code

[Bartosz Dziewoński]

On 2023-09-29 19:55, bawolff wrote:

This is clearly yielding some interesting results.

One of the patterns i've noticed is that several of the examples seem to 
involve mustache templates. I think there are two reasons for this:


* mustache templates cannot currently be checked by phan-taint-check
* Because they are a separate file, the escaping is now fairly far away 
from the context where the variable is used. Its easy to lose track of 
if a specific variable is supposed to be escaped between the template 
file and the call into TemplateProcessor.


Let's not go too easy on Mustache, there are several more reasons why 
these templates are full of security gaps:


* Escaping or failing to escape HTML is the difference between {{ }} and 
{{{ }}}, and unless you spent your whole life writing Mustache 
templates, you won't remember which is which.


* Mustache has no concept of HTML structure, or any structure, or 
variable types; it just concatenates strings, so it's difficult to 
automatically detect any problems.



Anyways, i'd like to propose a naming convention. Any mustache variable 
that is used as raw html should have some sort of easily identifiable 
prefix so it is easy to keep track of which parameters are escaped and 
which are not. e.g. instead of naming the parameter foo, it would be 
named something like HTMLFoo.


We already do this, at least! Most Mustache variables used as raw HTML 
are prefixed with 'html-'. Vector is pretty consistent about this [1], 
but even it has some exceptions. Other code is not all so good.


[1] 
https://codesearch.wmcloud.org/search/?q={{{&files=\.mustache%24&excludeFiles=&repos=Skin%3AVector



--
Bartosz Dziewoński
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: New developer feature: $wgUseXssLanguage / x-xss language code

[bawolff]

This is clearly yielding some interesting results.

One of the patterns i've noticed is that several of the examples seem to
involve mustache templates. I think there are two reasons for this:

* mustache templates cannot currently be checked by phan-taint-check
* Because they are a separate file, the escaping is now fairly far away
from the context where the variable is used. Its easy to lose track of if a
specific variable is supposed to be escaped between the template file and
the call into TemplateProcessor.

Anyways, i'd like to propose a naming convention. Any mustache variable
that is used as raw html should have some sort of easily identifiable
prefix so it is easy to keep track of which parameters are escaped and
which are not. e.g. instead of naming the parameter foo, it would be named
something like HTMLFoo.

Thoughts?
--
Brian


On Thu, Sep 28, 2023 at 9:01 AM Lucas Werkmeister <
lucas.werkmeis...@wikimedia.de> wrote:

> Hi all! This is an announcement for a new developer feature in MediaWiki.
> If you don’t develop MediaWiki core, extensions or skins, you can stop
> reading :)
>
> MediaWiki interface messages are generally “safe” to edit: when they
> contain markup, it is either parsed (as wikitext), sanitized, or fully
> HTML-escaped. For this reason, administrators are allowed to edit normal
> messages on-wiki in the MediaWiki: namespace, while editing JS code (which
> is more dangerous) is restricted to interface administrators. (A few
> exceptions, messages that are not escaped and which can only be edited by
> interface administrators, are configured in $wgRawHtmlMessages
> .)
> Occasionally, a bug in the software means that a message isn’t properly
> escaped, which can in theory be abused by administrators to effectively
> gain interface administrator powers (by editing a MediaWiki: page for a
> message to contain 

[Wikitech-l] Re: SemanticCite

[tdvit]

https://github.com/Knowledge-Wiki/SemanticCite/tree/master

 
 

Sent: Friday, September 29, 2023 at 7:01 PM
From: "Sorin Gheorghiu" 
To: wikitech-l@lists.wikimedia.org
Subject: [Wikitech-l] SemanticCite

Hello folks!

does anybody know if this extension is still maintained? The latest commit is from May 16, 2020.

There are older tickets about SCI being not compatible with SWM 4.0.0 (and therefore not compatible with MW1.39 which will become EOL end of 2023).

Sorin




___ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/




 

 ___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: SemanticCite

[novemlinguae]

Convenience link: https://www.mediawiki.org/wiki/Extension:Semantic_Cite

 

It does look pretty abandoned. The extension’s listed author MWJames last 
edited in 2015  . Extension is 
not deployed to Wikimedia production and is not listed at 
https://www.mediawiki.org/wiki/Developers/Maintainers. Last human commit is Nov 
2022: https://github.com/SemanticMediaWiki/SemanticCite/commits/master.

 

The listed maintainer is “SMW Project”, with a link to 
https://www.semantic-mediawiki.org/wiki/Help:SMW_Project. Asking the SMW 
Project folks might be worth a try. Could also try emailing or posting on the 
user talk of MWJames.

 

Sincerely,

Novem Linguae

 

From: Sorin Gheorghiu  
Sent: Friday, September 29, 2023 8:02 AM
To: wikitech-l@lists.wikimedia.org
Subject: [Wikitech-l] SemanticCite

 

Hello folks!

does anybody know if this extension is still maintained? The latest commit is 
from May 16, 2020.

There are older tickets about SCI being not compatible with SWM 4.0.0 (and 
therefore not compatible with MW1.39 which will become EOL end of 2023).

Sorin





___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] SemanticCite

[Sorin Gheorghiu]

Hello folks!

does anybody know if this extension is still maintained? The latest 
commit is from May 16, 2020.


There are older tickets about SCI being not compatible with SWM 4.0.0 
(and therefore not compatible with MW1.39 which will become EOL end of 
2023).


Sorin



___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Re: New developer feature: $wgUseXssLanguage / x-xss language code

[Yaron Koren]

>
> If the developer setting $wgUseXssLanguage is set to true, then an “x-xss”
> language code becomes available and can be selected with *?uselang=x-xss*
> in the URL. When using this language code, all messages become “malicious”:
> every message is replaced by a snippet of HTML that tries to run alert('
> *message-key*').


Clever feature - this will be great for testing. Thank you!

-Yaron

-- 
WikiWorks · MediaWiki Consulting · http://wikiworks.com
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Talk to the Search Platform / Query Service Team—October 4, 2023

[Guillaume Lederrey]

Hello all!

The Search Platform Team usually holds an open meeting on the first
Wednesday of each month. Come talk to us about anything related to
Wikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons Query
Service (WCQS), etc.!

Feel free to add your items to the Etherpad Agenda for the next meeting.

Details for our next meeting:
Date: Wednesday, October 4, 2023
Time: 15:00-16:00 UTC / 08:00 PT / 11:00 EDT / 17:00 CET
Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours
Google Meet link: https://meet.google.com/vgj-bbeb-uyi
Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927

Have fun and see you soon!

   Guillaume

-- 
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation 
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/