[Wikitech-l] Production Excellence #30: March 2021

[Krinkle]

How’d we do in our strive for operational excellence last month? Read on to
find out!

Read on Phabricator at
https://phabricator.wikimedia.org/phame/post/view/229/
Incidents

2 documented incidents. That's average for this time of year, when we usually
had  1-4 incidents.

Learn about recent incidents at Incident status
 on Wikitech, or
Preventive
measures  in
Phabricator.

---

Trends

In March we made significant progress on the outstanding errors of previous
months. Several of the 2020 months are finally starting to empty out. But
with over 30 new tasks from March itself remaining, we did not break even,
and ended up slightly higher than last month. This could be reversing two
positive trends, but I hope not.

Firstly, there was a steep increase in the number of new production errors
that were not resolved within the same month. This is counter the positive
trend we started in November. The past four months typically saw 10-20
errors outlive their month of discovery, and this past month saw 34 of its
48 new errors remain unresolved.

Secondly, we saw the overall number of unresolved errors increase again. This
January

began a downward trend for the first time in thirteen months, which
continued nicely through February. But, this past month we broke even and
even pushed upward by one task. I hope this is just a breather and we can
continue our way downward.

Figure 1, Figure 2: Unresolved error reports stacked by month.


---

Outstanding errors

Take a look at the workboard
 and
look for tasks that could use your help.

Summary over recent months, per spreadsheet

:
Jul 2019 (0 of 18 left) ✅ Last two tasks resolved! -2
Aug 2019 (1 of 14 left) ⚠️ *Unchanged (and over one year old).*
Oct 2019 (3 of 12 left) ⬇️ One task resolved. -1
Nov 2019 (0 of 5 left) ✅ Last task resolved! -1
Dec 2019 (0 of 9 left) ✅ Last task resolved! -1
Jan 2020 (2 of 7 left) ⬇️ One task resolved. -1
Feb 2020 (0 of 7 left) ✅ Last task resolved! -1
Mar 2020 (2 of 2 left) ⚠️ *Unchanged (and over one year old).*
Apr 2020 (5 of 14 left) ⬇️ Four tasks resolved. -4
May 2020 (5 of 14 left) ⬇️ One task resolved. -1
Jun 2020 (6 of 14 left) ⬇️ One task resolved. -1
Jul 2020 (5 of 24 issues
) ⬇️
Four tasks resolved. -4
Aug 2020 (15 of 53 issues
) ⬇️
Five tasks resolved. -5
Sep 2020 (7 of 33 issues
) ⬇️ One
task resolved. -1
Oct 2020 (22 of 69 issues
) ⬇️
Four tasks resolved. -4
Nov 2020 (9 of 38 issues
) ⬇️ Two
tasks resolved. -2
Dec 2020 (11 of 33 issues
) ⬇️ One
task resolved. -1
Jan 2021 (4 of 50 issues
) ⬇️ One
task resolved. -1
Feb 2021 (9 of 20 issues
) ⬇️ Two
tasks resolved. -2
*Mar 2021* (34 of 48 issues
) 34 new
tasks survived and remain unresolved. +48; -14

---
Tally
138 issues open, as of Excellence #29

(6 Mar 2021).
-33 issues closed, of the previous 138 open issues.
+34 new issues that survived March 2021.
139 issues open, as of today (2 Apr 2021).
---Thanks!

Thank you to everyone who helped by reporting, investigating, or resolving
problems in Wikimedia production. Thanks!

Until next time,

– Timo Tijhof


Footnotes:

Incident status, Wikitech
.
Wikimedia incident stats by Krinkle, CodePen
.
Production Excellence: Month-over-month spreadsheet and plot

.
Report charts for Wikimedia-production-error project, Phabricator
.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Does anyone know about bulk loads on Wikibase/Wikidata?

[Yuri Astrakhan]

I have been doing it from open street map data using a python script.

At first it generates about 100gb of zipped ttl files, imports them with a
specialized blazegraph tool and afterwards uses direct sparql statements to
do batch updates when new data becomes available.

https://github.com/Sophox/sophox/tree/main/osm2rdf


On Wed, Apr 14, 2021, 5:07 PM Henry Rosales  wrote:

> Hi all!
>
> I hope you are doing very well! I’m working on constructing an RDF
> dataset, which I’m ingesting to Wikibase and leverage its ecosystem:
> visualization, SPARQL endpoint, a script for dump creation, etc. For
> ingestion, I’m using the API (explained in
> https://www.wikidata.org/w/api.php), and it is going well, except for the
> time performance.
>
> I was exploring the Quick Statements module of Wikibase, which allows the
> ingestion through the UI. However, I need a way to ingest data
> automatically. I wonder if Wikibase/Wikidata has any script for bulk loads.
>
> Any comment or suggestion will be welcome.
>
> Best Regards,
> Henry Rosales
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Does anyone know about bulk loads on Wikibase/Wikidata?

[Henry Rosales]

Hi all!

I hope you are doing very well! I’m working on constructing an RDF dataset,
which I’m ingesting to Wikibase and leverage its ecosystem: visualization,
SPARQL endpoint, a script for dump creation, etc. For ingestion, I’m using
the API (explained in  https://www.wikidata.org/w/api.php), and it is going
well, except for the time performance.

I was exploring the Quick Statements module of Wikibase, which allows the
ingestion through the UI. However, I need a way to ingest data
automatically. I wonder if Wikibase/Wikidata has any script for bulk loads.

Any comment or suggestion will be welcome.

Best Regards,
Henry Rosales
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] April 2021 Technical Community Newsletter

[Sarah R]

Hi Everyone,

We’re happy to announce the  April 2021 edition of the Technical Community
Newsletter

is now available. The newsletter is compiled by the Wikimedia Developer
Advocacy Team. It aims to share highlights, news, and information of
interest from and about the Wikimedia technical community.

Check it out, and learn about what technical contributors have been up to
this past quarter, upcoming conferences & calls for papers, and how to get
involved.

The Wikimedia Technical Community is large and diverse, and we know we
can't capture everything perfectly. We welcome your ideas for future
newsletters. Let us know what you would like to see or highlights you would
like us to include.

Subscribe to the Technical Community Newsletter
,
if you'd like to keep up with essential updates and information


Kindly,

Sarah R. Rodlund
Senior Technical Writer, Developer Advocacy

srodl...@wikimedia.org
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] see stacktrace from functions within mediawiki

[Sandeep Gupta]

Hello,

I have recently started looking into mediawiki source code and trying to 
understand various codeflow. I am new to php as well.

I want to see the stacktrace from database functions. 
I have put following debug line in the function :

ob_start();
debug_print_backtrace();
$data = ob_get_clean();
fwrite(STDERR, $data  . PHP_EOL);

But I don't see any output in /var/log/nginx/mediawiki.error ( in current setup 
mediawiki is served by nginx).

Any help would be great. Or any other approach to access the stacktrace would 
also work.

Thanks
Sandeep
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] 2021-04-14 Scrum of Scrums meeting notes

[James Forrester]

https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-04-14

= 2021-04-14 =

Grace/Deb have conflicts today. Please self-organize :)

== Callouts ==

* MediaWiki 1.36-beta has been branched:
https://lists.wikimedia.org/pipermail/wikitech-l/2021-April/094424.html

== Gerrit patches or GitHub Pull Requests for reviews or feedback ==

* None.

=== No updates ===
Community Tech, Anti-Harassment Tools, Library, Site Reliability Engineering

=== '''No notes provided''' ===
Editing, Growth, iOS native app, Android native app, Web, Product
Infrastructure, Parsing, Language, Inuka, Analytics, Cloud Services,
 Platform, Quality and Test Engineering, Search Platform, Security

== SoS Meeting Bookkeeping ==

* Updates:

== Product ==

=== Structured Data ===

* Thank yous:
** Jon Robson for getting Vue errors into logstash and helping us identify
some issues in our code
* Updates:
** Working to make the new MediaSearch extension (not yet deployed on
Commons) more usable on other wikis (
https://www.mediawiki.org/wiki/Extension:MediaSearch)
** Image recommendations design work

=== Abstract Wikipedia ===

* Thank yous:
** Thank you to Brennen and Jeena in RelEng for discussing options for
testing MW<->service integrations in CI.
* Updates:
** Working on Phase δ (delta):
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases
** Working on our TDMP pitch for asynchronous parser fragments.

=== Vue.js ===

* Updates:
** Working on components design inventory:
https://phabricator.wikimedia.org/T277047
** Starting to work on unifying various component implementations so we can
add them to the experimental WMF component library (WVUI)
*** Button: https://phabricator.wikimedia.org/T278509
*** Binary inputs: https://phabricator.wikimedia.org/T279714

== Technology ==

=== Fundraising Tech ===

* Updates:
** More work on email prefs page https://phabricator.wikimedia.org/T268510
** Planning for integration with new API of backup card processor
** Work to migrate custom CRM code off drupal 7
** CiviCRM contact deduplication enhancements
** Audit / reconciliation file processing improvements:
https://phabricator.wikimedia.org/T277244,
https://phabricator.wikimedia.org/T265545

=== Engineering Productivity ===

 Performance 

* Blocking:
** WMSE with performance review of WikiSpeech (in progress, almost finished)
* Thank yous:
** Geno from Abstract Wikipedia for how engaged she is with the matrixing
on our team
* Updates:
** Cancelling our Kobiton subscription, we will build our own device lab
hosted at the office instead

 Release Engineering 

* Updates:
** [All] Deployments/Covid-19
https://wikitech.wikimedia.org/wiki/Deployments/Covid-19
** MediaWiki 1.36-beta has been branched:
https://lists.wikimedia.org/pipermail/wikitech-l/2021-April/094424.html
** Train Health
*** Last week: 1.36.0-wmf.38 [[phab:T278343]]
*** This week: 1.37.0-wmf.1 [[phab:T278344]]
*** This week: 1.37.0-wmf.2 [[phab:T278345]]

=== WMDE Technical Wishes ===

* Updates:
** Line numbering was deployed. Currently only on template namespace, due
to feedback by the editing team. Discussion about other namespaces is still
open.

== Cross-cutting ==

* Blocked by:
** [long term] Search Platform: PHP 8.0 work is long-term blocked on the
migration to ElasticSearch 7.0 https://phabricator.wikimedia.org/T263142
(or at least 6.7).
* Thank yous:
* Updates:
** Nothing major.
** CI tools
*** Next release of mediawiki-codesniffer likely soon.
*** CI tools' upgrade status:
https://libraryupgrader2.wmcloud.org/status?branch=master
** PHP 8.0:
*** Our target is REL1_35 (and thus also REL1_36) as well as master.
*** Upstream libraries: Elastica-related PHP code is theoretically the last
one.
*** Core: Some unit and integration tests still fail; thank you to everyone
working on fixing them.
[End]

Yours,
-- 
*James D. Forrester* (he/him  or they/themself
)
Wikimedia Foundation 
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] No train the week of 2021-04-19

[Tyler Cipriani]

Hi All

There will be no train next week (the week of 2021-04-19) due to a wmf
holiday on 2021-04-22.

There is a long-term calendar of upcoming deployment disruptions available
on Wikitech: https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar

Thanks!
– Tyler
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Translation request

[Timur Vorkul]

Hello, dear translators!

Your help with a short outreach message is needed. It is the announcement
of the new suggested values feature to be deployed on all wikis. It will
make filling in templates which expect specific inputs much easier to do in
the VisualEditor.


https://meta.wikimedia.org/wiki/User:Timur_Vorkul_(WMDE)/Suggested_values_announcement

I plan to send the message to all wikis on April 27, so having many
translations by April 26 would be awesome!

Thanks a lot!

Best,

Timur


-- 
Timur Vorkul
Technische Wünsche

Wikimedia Deutschland e. V. | Tempelhofer Ufer 23–24 | 10963 Berlin
Tel. (030) 219 158 26-0
https://wikimedia.de

Bleiben Sie auf dem neuesten Stand! Aktuelle Nachrichten und spannende
Geschichten rund um Wikimedia, Wikipedia und Freies Wissen im Newsletter: Zur
Anmeldung .

Unsere Vision ist eine Welt, in der alle Menschen am Wissen der Menschheit
teilhaben, es nutzen und mehren können. Helfen Sie uns dabei!
https://spenden.wikimedia.de

Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Problems with the CI

[Zoran Dori]

Hello,
thank you very much for providing informations, I appreciate it and I hope
that the problems will be resolved soon. :)

Best regards,
Zoran
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Problems with the CI

[Daimona]

Indeed. I should also note that the issue reported at
https://phabricator.wikimedia.org/T280064 is about xdebug not being found.
I don't have an explanation for this; perhaps something was missed while
switching the job to PHP 7.3. OTOH, my proposed solution will make it
possible to use pcov instead of xdebug, thus basically ignoring the current
issue with xdebug. The rationale being, I don't think it's necessary to
unbreak xdebug support if we're migrating away from it anyway.

Il giorno mer 14 apr 2021 alle ore 07:05 Kunal Mehta 
ha scritto:

> Hi,
>
> On 4/13/21 7:47 PM, Zoran Dori wrote:
>  > 
> > I would like to know why is this happening.
>
> See https://phabricator.wikimedia.org/T280064 and
> https://phabricator.wikimedia.org/T234020#6989290. Daimona has proposed
> https://gerrit.wikimedia.org/r/c/integration/config/+/678383/ to
> hopefully fix it.
>
> -- Legoktm
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>


-- 
https://meta.wikimedia.org/wiki/User:Daimona_Eaytoy
"Daimona" is not my real name -- he/him
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l