from:"Timo Tijhof"

[Wikitech-l] New flamegraphs selector (now with two year range!)

2023-04-24 Thread Timo Tijhof

== *What* ==

Flamegraphs provide insight to where and how your MediaWiki component or 
extension spends its time. We instrument real web requests on the MediaWiki app 
servers at WMF (using php-excimer, a low-overhead sampling profiler) and 
continually publish interactive SVG visuals by entrypoint (e.g. index.php, 
JobQueue, ResourceLoader load.php, or rest.php API). [1] [2]

Read more about the Arc Lamp pipeline, introduced by Ori during the 2014 HHVM 
migration [3], through our blog post:
https://techblog.wikimedia.org/2021/03/03/profiling-php-in-production-at-scale/

== *What's New* ==

The user interface at https://performance.wikimedia.org/php-profiling/ has 
undergone major changes.

 • Date: You can now choose any date in the last *2+ years*. Previously, this 
was limited to 2-3 weeks. In the dropdown menu, choose "Other", and select any 
date using the native datepicker. Existing URLs remain compatible.
 • Source: Real-time performance data from the *MediaWiki-on-Kubernetes* 
deployment is now available through here (excimer-k8s and excimer-k8s-wall). 
While web traffic remains largely on bare metal (except for test2wiki), 
internal MediaWiki API traffic is already partly on Kubernetes. Refer to 
T333120 <https://phabricator.wikimedia.org/T333120> for work by SRE ServiceOps 
(e.g. CXServer uses MediaWiki-on-Kubernetes).
 • Entrypoint: *JobQueue* (RunSingleJob) is now included in the selector, plus 
the new detailed breakdowns <https://phabricator.wikimedia.org/T253679> for 
*EditAction*, *PreSend*, and *PostSend* are now promoted here. These were 
previously available via the file browser only.

== *How* ==

The new interface is an HTML form with vanilla JavaScript to navigate you to 
the right SVG.
Details at https://gerrit.wikimedia.org/r/c/performance/docroot/+/908374.

As before, we promote "good" defaults. You don't have to make any choices to 
start exploring the data. Press the big blue button to instantly view the most 
recent index.php flamegraph with the default settings.

--
Timo Tijhof,
Performance Team,
Wikimedia Foundation.

[1] https://wikitech.wikimedia.org/wiki/Performance/Runbook/Arc_Lamp_service
[2] https://www.mediawiki.org/wiki/Excimer
[3] 
https://techblog.wikimedia.org/2014/12/29/how-we-made-editing-wikipedia-twice-as-fast/___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Performance Guides 2023

2023-03-16 Thread Timo Tijhof

The mission of Wikimedia Performance is for our sites to transcend
socioeconomic barriers around reliable and fast access to find and contribute
knowledge. We provide tools and expertise to empower developers, and directly
inform or undertake high-yield engineering projects. [1][2]

The below is a periodic introduction and summary of recent changes to our
guides. If you haven't read these before, or if it's been more than six months,
I recommend taking a fresh look. Especially if you work on frontend or backend
components in a MediaWiki extension or MediaWiki core.

== *Current best practices* ==

The practices guides help set direction. Use them to guide new developments, or
to identify areas for improvement in current code. If you're short on time,
focus on the "Getting started" section atop each guide.

*Frontend*:
https://wikitech.wikimedia.org/wiki/Performance/Guides/Frontend_performance_practices

* The introduction details the principles that drive our platform's
architecture, and how to get the most out of it.

* Changed: The CSS "@embed" optimisation is now only recommended for very small
icons (up to 0.3KB). The guide explains why and how.

*Backend*:
https://wikitech.wikimedia.org/wiki/Performance/Guides/Backend_performance_practices

* New "Getting started" section, with pointers to specific chapters for
detailed guidance.
* Rewritten "Shared resources" chapter, now with a more accessible explanation
of MySQL deadlocks and how to avoid them.
* Update "Multi-datacenter deployment" guidance. (No changes are needed to
existing code.) We first adopted Multi-DC practices in 2015. WANObjectCache and
JobQueue interfaces have gotten simpler since. Cross-DC purges and job queuing
"just work", with no awareness or responsibility on calling code. MediaWiki now
automatically pins a browser to the primary DC for a few seconds after
publishing an edit. This allowed us to remove cross-DC complexity around the
ChronologyProtector
<https://doc.wikimedia.org/mediawiki-core/master/php/classWikimedia_1_1Rdbms_1_1ChronologyProtector.html#details>.

== *Measuring* ==

The new "measure" guides help assess performance of existing code, and can help
iterate development of proposed changes.

Frontend includes browser dev tools and continuous monitoring through dedicated
perf testing infrastructure:
https://wikitech.wikimedia.org/wiki/Performance/Guides/Measure_frontend_performance

Backend includes flame graphs, benchmarking, and automatic Grafana stats if you
adopt WANObjectCache:
https://wikitech.wikimedia.org/wiki/Performance/Guides/Measure_backend_performance

== *More* ==

The above guides and an overview of datasets, tools, recommended Grafana
dashboards, and infrastructure diagrams are available at:

https://wikitech.wikimedia.org/wiki/Performance

On behalf of the Performance Team,
-- Timo Tijhof.

[1] https://techblog.wikimedia.org/2018/12/12/why-performance-matters/
[2] https://www.mediawiki.org/wiki/Wikimedia_Performance_Team#Mission
___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] Change to PHPUnit's cache config for MediaWiki

2023-03-09 Thread Timo Tijhof

TLDR: Previously, PHPUnit enabled all caches except WANObjectCache. This is now
enabled as well. If you encounter a new test failure this week on an extension
repo that is not covered by the automated "wmf CI gate", then that may be the
reason. This could point to a genuine bug in your code. However, you can choose
to turn of all caching layers in the relevant test cases by calling
$this->setMainCache( CACHE_NONE ).

Hi,

MediaWiki provides broudly four layers of object caching
<https://www.mediawiki.org/wiki/Object_cache>, that are individually
configurable (to optimise large wiki farms like Wikipedia). Each of these
auto-configures themselves or falls back to $wgMainCacheType
<https://doc.wikimedia.org/mediawiki-core/master/php/classMediaWiki_1_1MainConfigSchema.html#a0b3d0535aa2fc03327944280797b1c54>,
thus in practice you don't need to be aware of their differing backends. They
have a simple unified interface: BagOStuff.

In our PHPUnit context, we set each layer to 'hash' (for HashBagOStuff) which
represents an in-memory cache that we automatically reset between each test and
each data set. In addition to the four levels of caching, this therefore also
ensures realistic caching behaviour during your tests for services such as
ParserCache, MessageCache, SessionStore, MainStash, etc.

For example, your extension might have a test that asserts "no database
queries" made during a cache hit, or it might assert that if an edit is made
the same value is returned as before, and then you call your code's purge
method, and assert the updated value.

By default and during local development, (other) CI jobs, beta cluster, and
production - WANObjectCache is explicitly a wrapper for the main cache (e.g.
Memcached at WMF). It is never configured separately from the main cache, nor
is there any known use case (among both third parties, WMF, and local
development) for doing so.

As of this week, configuring WANObjectCache separately from the "main" cache is
no longer supported. This improves defaults for sysadmins and reduces the
number things one has to get right for optimal performance. It also removes an
apparent concept from the code going forward to ease debugging by not having a
level of indirection that doesn't translate to a specific narrative or purpose.

In dong so, we found one place that does configure the WANCache backend, namely
our PHPUnit config. While expected to override it, it overrode it to wrap the
no-op "none" EmptyBagOStuff instead instead of the in-memory "hash"
HashBagOStuff. This quirky setup was surprising a times to developers and
required numerous tests to copy boilerplate around in order to opt-in to to the
"normal" cache behaviour that we expect and follow in other cache levels
already. The boiler plate in question was also atypical in that it wasn't
covered by our standard ServiceWiring and its automatic resets (the main cache
is now recognised an injectable internal service).

Our shared "wmf CI gate" job identified two extensions where this was relied
upon, and I've updated those before making the change
<https://gerrit.wikimedia.org/r/c/mediawiki/core/+/889244> (example 1
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GlobalPreferences/+/894115>,
and example 2
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/894228>).

If you encounter a new test failure this week on an extension repo that is not
covered by the CI gate, then this may be the reason. Tests that fail with
caching enabled, may expose a bug in the underlying code (e.g. after a cache it
should probably not use the cache for subsequent reads of the same data, unless
it is expected that it will be stale), or may reflect that the code (or the
test) is missing a call to your "purge" method, or that a service reset is
needed if the test is meant to reflect what a subsequent web request would see,
etc. You can (temporarily or not) choose to turn of all caching layers in the
relevant test cases by calling $this->setMainCache( CACHE_NONE ). Example
1-line patch
<https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GlobalPreferences/+/894115>.

--
Timo Tijhof
Performance Team,
Wikimedia Foundation.___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l]  Fresh 22.11.2 released!

2023-03-02 Thread Timo Tijhof

The fresh-node16 command receives a minor Node.js update, and major upgrades to 
both Firefox and Chromium.

Get started by updating, installing, or learning more, at:
https://gerrit.wikimedia.org/g/fresh

Changelog:
https://gerrit.wikimedia.org/g/fresh/+/22.11.2/CHANGELOG.md

The same updates were applied to WMF CI last week. Thanks to Kosta Harlan for 
making this release!

Fresh is a fast way to create isolated environments from your terminal. These 
can be used to work more responsibly with 'npm' developer tools such as ESLint, 
QUnit, Grunt, Selenium, and more. Example guide: 
https://www.mediawiki.org/wiki/Manual:JavaScript_unit_testing. To report issues 
or browse past and current tasks, check Phabricator at 
https://phabricator.wikimedia.org/tag/fresh/.

--
Timo Tijhof,
Performance Team,
Wikimedia Foundation.___
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-le...@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/

[Wikitech-l] [BREAKING CHANGE] Require MySQL 5.6 for pt-heartbeat lag detection in MW 1.36

2021-03-12 Thread Timo Tijhof

TLDR:
* We'd like to drop MySQL 5.5 support for the
"lagDetectionMethod=pt-heartbeart"
option in $wgLBFactoryConf.
* If you have not configured the lagDetectionMethod option, or use MySQL
5.6+, then this does not affect you.

Hi,

If you host MediaWiki using MySQL (or MariaDB) and have a cluster of two or
more database hosts (using replication), then MediaWiki uses lag detection,
via the wikimedia/rdbms library. For example, if lag is too high, MediaWiki
tries to connect to a different replica database, or it may automatically
set the wiki in read-only mode for a few seconds until replication catches
up.

By default, MediaWiki uses the MySQL built-in "Seconds_Behind_Master"
feature to power this lag detection.

We also support use of Percona's pt-heartbeat service, which can be more
accurate, [1] and is what WMF currently uses. However, we are currently
unable to use its accuracy very well, because TIMESTAMPDIFF is not
supported in MySQL 5.5 (it was introduced in MySQL 5.6.4). [2]

In order to make pt-heartbeat more useful as a lag detection method, whilst
avoiding expensive roundtrips or runtime detection (which are not
acceptable in this hot code path), we'd either have to support three lag
detection methods, or to change the current one to require MySQL 5.6.

MySQL 5.5 reached end-of-life in Dec 2018, and there is an open proposal at
T273375 to drop support for it completely in a future version of MediaWiki.
[3] However, I'd like to propose we drop support for it now with the
pt-heartbeat feature specifically, as being opt-in and rarely used, so as
to not require additional complexity and maintenance for us in the form a
third lag detection method. Especially, as it might not be used by anyone.

If you're affected by this, then the upcoming MediaWiki 1.36 release would
require that you either set MW to use the default lag detection instead, or
upgrade to MySQL 5.6+.

Task: https://phabricator.wikimedia.org/T248481
Patch: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/657471

--
Timo Tijhof
Performance Team
Wikimedia Foundation

[1]
https://www.percona.com/blog/2014/05/02/how-to-identify-and-cure-mysql-replication-slave-lag/
[2] https://dev.mysql.com/doc/relnotes/mysql/5.6/en/news-5-6-4.html
[3] https://phabricator.wikimedia.org/T273375
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] TechCom meeting 2020-12-16

2020-12-16 Thread Timo Tijhof

The minutes from TechCom's triage meeting on 16 December 2020.

Present:  Daniel K., Tim S., Timo T.
Create WikiTeq group on Gerrit

   - https://phabricator.wikimedia.org/T267213
   - Approved.
   - TechCom to amend the policy to add the exemption.
   - Gerrit admins to create the group and explain that they can decide who
   to be added, but that Phab tickets will be required still.

RFC: Expand API title generator to support other generated data

   - https://phabricator.wikimedia.org/T263841
   - Matthias has proposed wording for the API policy.
   - Put on Last Call until Jan 6.

RFC: Introduce PageIdentity to be used instead of WikiPage.

   - https://phabricator.wikimedia.org/T259771
   - Daniel to document concern about adopting false advertisement.
   - Daniel to add some kind of intermediary Local-only interface to adopt
   first, then on Last Call until Jan 6.

RFC: Drop support for older database upgrades

   - https://phabricator.wikimedia.org/T259771
   - Looks good to go, but we already have two RFCs on Last Call.
   - Expected to go on Last Call in January.

Upcoming holidaysThere will also be no TechCom triage meeting on 23
December and 30 December due to holidays. The next triage and review will
be in the week of 6 January 2021. Next week IRC office hours

No IRC discussion scheduled for next week.

You can also find our meeting minutes at
https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes

-- Timo

On Tue, Dec 15, 2020 at 12:49 PM Niklas Laxström 
wrote:

> This is the weekly TechCom board review in preparation of our meeting on
> Wednesday. If there are additional topics for TechCom to review, please let
> us know by replying to this email. However, please keep discussion about
> individual RFCs to the Phabricator tickets.
>
> Activity since Tuesday 2020-12-08 on the following boards:
>
> https://phabricator.wikimedia.org/tag/techcom/
> https://phabricator.wikimedia.org/tag/techcom-rfc/
>
> Committee inbox: (none)
>
> Committee board activity:
>
>- T42787  Remove legacy ajax
>interface
>   - I moved from Inbox to Watching
>- T267213  Create WikiTeq
>group on Gerrit
>- Kizule is asking for an update.
>   - It seems there are two things to do: create the group and update
>   the policy.
>
> New RFCs: (none)
>
> Phase progression: (none)
>
> IRC meeting request: (none)
>
> Other RFC activity:
>
>- T263841  RFC: Expand API
>title generator to support other generated data
>- There is a new proposal text. Please comment.
>- T259771 : RFC: Drop
>support for database upgrade older than two LTS releases.
>- Discussion on the task. To me it looks mostly in favor or commenting
>   that some of the problems are not fully solved by this proposal.
>- T268326  RFC: Amendment
>to the Stable interface policy (November 2020)
>- Feedback given about the proposed 3 month minimum period
>- T119173 : RFC: Discourage
>use of MySQL's ENUM type.
>- Amir showed how to automatically generate on-wiki documentation for
>   tables.
>- T133452 : RFC: Create
>temporary accounts for anonymous editors.
>- Many comments after Tim proposed to use cloaks (see his comment in
>   the task for details)
>- T214362 : RFC: Store
>WikibaseQualityConstraint check data in persistent storage.
>- Krinkle asked for clarifications. Lucas responded with a comment and
>   Lydia offered to have it explained in a call.
>- T208776 : RFC: Introduce
>PageIdentity to be used instead of WikiPage.
>- Krinkle and Daniel discuss implementation details in context of
>   what's best for type safety, clarity and migration.
>
>
>   -Niklas
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] TechCom meeting 2020-12-09

2020-12-16 Thread Timo Tijhof

The minutes from TechCom's triage meeting on 2020-12-09.

Present:  Daniel K, Tim S, Dan A, Timo T.
RFC: PHP microservice for containerized shell execution

   - https://phabricator.wikimedia.org/T260330
   - Approved

Create WikiTeq group on Gerrit

   - https://phabricator.wikimedia.org/T267213
   - Daniel to update the page
   - Tim to create the Gerrit group

RFC: Amendment to the Stable interface policy (Nov 2020)

   - https://phabricator.wikimedia.org/T268326
   - Ready for Last Call?
   - Discussed deprecation logistics.  Currently it’s possible to have only
   one day between soft and hard deprecation.  Talking about increasing it to
   at least 3 months or one full release cycle (from start to finish, aka at
   least 6 months).

RFC: Introduce PageIdentity to be used instead of WikiPage

   - https://phabricator.wikimedia.org/T208776
   - Ready for Last Call?
   - Some discussion about PageIdentity being global (with a Wiki ID) and
   not being a perfect drop-in replacement for Title because of that, because
   it suggests classes using PageIdentity support non-local pages.
   - Not ready for last call :)  Will walk through some concrete examples
   for the transition from Title to PageIdentity to unblock.  A suggested
   alternative: transition to LocalPageIdentity instead, and later to
   PageIdentity for users of Title that begin to support non-local pages.

RFC: Expand API title generator to support other generated data

   - https://phabricator.wikimedia.org/T263841
   - Tim will comment and ready for last call

You can also find our meeting minutes at
https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes

-- Timo

On Tue, Dec 8, 2020 at 2:52 PM Daniel Kinzler 
wrote:

> This is the weekly TechCom board review in preparation of our meeting on
> Wednesday. If there are additional topics for TechCom to review, please let
> us know by replying to this email. However, please keep discussion about
> individual RFCs to the Phabricator tickets.
>
> Activity since Monday 2020-11-30 on the following boards:
>
> https://phabricator.wikimedia.org/tag/techcom/
> https://phabricator.wikimedia.org/tag/techcom-rfc/
>
> Committee inbox:
>
>- Remove legacy ajax interface T42787
>
>   - Very old task. We should just do it.
>
> Committee board activity:
>
>- Create WikiTeq group on Gerrit T267213
>
>   - We should just add WikiTeq to the policy page
>
> New RFCs: none
>
> Phase progression:
>
>- PHP microservice for containerized shell execution (aka ShellBox)
>T260330 
>- Last call should have ended last week. Some discussion during last
>   call period. New comment asking why this should be written in PHP.
>   - Code is already merged, though not used yet.
>   - Can probably be approved.
>   - Amendment to the Stable interface policy T268326
>
>   - Question about new requirement to wait at least thee month
>   between hard deprecation and removal.
>   - Moved to phase 4
>- PageIdentity T208776 
>- Thiemo (WMDE) asking about some details
>   - Daniel would like this to go on last call.
>
> Other RFC activity:
>
>- Expand API title generator to support other generated data T263841
>
>   - Tim says this is fine and should go ahead
>   - Do we need a last call?
>- Provide mechanism for defining and utilizing configuration sets for
>local development and browser / API-testing tests T267928
>
>   - Continued discussion after IRC meeting
>   - Adam Wight emphasizes the need to have config controlled
>   per-request, rather than changing config on disk.
>- Store WikibaseQualityConstraint check data in persistent storage
>T214362 
>   - WMDE would like this to move forward.  Resourcing is not really
>   clear.
>
> --
> Daniel Kinzler
> Principal Software Engineer, Core Platform
> Wikimedia Foundation
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] TechCom meeting 2020-10-06

2020-10-07 Thread Timo Tijhof

The minutes from TechCom's triage meeting on 7 October 2020.

Present: Daniel K, Giuseppe L, Dan A, Timo T.
RFC T260714: Parsoid Extension API

   -

   https://phabricator.wikimedia.org/T260714
   -

   Last Call ended. No objections on ticket nor otherwise brought to us.
   -

   Approved.

RFC T487: Associated namespaces

   -

   https://phabricator.wikimedia.org/T487
   -

   Last Call ended. Discussion about the process. Task to remain open, but
   decline as proposed.

FYI: Heavy Wing - Machine Learning at WMF

   -


   
https://docs.google.com/document/d/1U8PKfotPUFmEXH1EXrQwqlccFE8i7smlQtxMrTAM6GE/edit#heading=h.5ha1j1agkk55

Controlling HTTP cache around cookies and privileged content is hard

   -

   https://phabricator.wikimedia.org/T264631
   -

   Yes, it’s hard.
   -

   TT: We have three or four layers in MW to defend against this going
   wrong. I’ll comment about past work around cookie blocks and error pages
   which relates to this.

Next week IRC office hours

No IRC discussion scheduled for next week.

On Tue, Oct 6, 2020 at 11:19 AM Giuseppe Lavagetto 
wrote:

> This is the weekly TechCom board review in preparation of our meeting on
> Wednesday. If there are additional topics for TechCom to review, please let
> us know by replying to this email. However, please keep discussion about
> individual RFCs to the Phabricator tickets.
>
> Activity since Monday 2020-09-28 on the following boards:
>
> https://phabricator.wikimedia.org/tag/techcom/
>
> https://phabricator.wikimedia.org/tag/techcom-rfc/
>
> Committee inbox:
>
>-
>
>T264334 : Could the
>registered module manifest be removed from the client?
>-
>
>   New task about the possibility of removing the huge module registry
>   from the js sent to the client. The idea is being discussed.
>
> Committee board activity: Nothing to report, besides inbox
>
> New RFCs: none.
>
> Phase progression:
>
>-
>
>T262946 : Bump Firefox
>version in basic support to 3.6 or newer
>-
>
>   Moves to P3 (explore)
>   -
>
>   It is pointed out that we’ve dropped support in production for TLS
>   1.0/1.1 in january, so de facto only Firefox 27+ is able to connect to 
> the
>   wikimedia sites
>   -
>
>   In light of that, it’s suggested that we might bump the minimum
>   supported versions of browsers further.
>
> IRC meeting request: none
>
> Other RFC activity:
>
>-
>
>T260714 : Parsoid Extension
>API.
>-
>
>   Last call to be approved, that will end on October 7 (tomorrow)
>   -
>
>T487 : RfC: Associated
>namespaces.
>-
>
>   On last call to be declined, there is some opposition to the
>   opportunity of marking it as declined on phabricator. Last call should 
> end
>   on October 7 (tomorrow)
>   -
>
>T263841 : RFC: Expand API
>title generator to support other generated data.
>-
>
>   Erik asks if this is going to be generally applied to all
>   generators or not.
>
> Cheers,
> Giuseppe
> --
> Giuseppe Lavagetto
> Principal Site Reliability Engineer, Wikimedia Foundation
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] TechCom meeting 2020-09-30

2020-09-30 Thread Timo Tijhof

This is the weekly TechCom board review in preparation of our meeting. If
there are additional topics for TechCom to review, please let us know by
replying to this email. However, please keep discussion about individual
RFCs to the Phabricator tickets.

Activity since Monday 2020-09-21 on the following boards:

https://phabricator.wikimedia.org/tag/techcom/
https://phabricator.wikimedia.org/tag/techcom-rfc/

Committee inbox:

   - T263904 : Are traits part
   of the stable interface?
  - New question by @DannyS712.

Committee board activity:

   - T227776 : General
   ParserCache service class for large "current" page-derived data
  - @daniel's team will soon start work on this project.
   - T244058 : Strategy for
   storing old revisions in ParserCache.
  - @daniel analyzed various code paths relating to this.

New RFCs:

   - T263841 : RFC: Expand API
   title generator to support other generated data
  - @matthiasmullie started an RFC with interest from Structured Data
  and Search Platform teams.

Phase progression:

   - T240775 : RFC: Support PHP
   7.4 preload
  - Moves to P3.
  - Interest from the Performance Team.
   - T260714:  RFC: Parsoid
   Extension API
  - Moves to Last Call to be *Approved* on 7 Oct.
  - T487 : RFC: Associated
   namespaces
  - Moves to Last Call to be *Declined* on 7 Oct.
  - @tgr mentions that Bask Wikipedia (eu.wikipedia.org) has a "
  Txikipedia "
  namespace that could benefit from this.

IRC meeting request: (none)

Other RFC activity:

   - T260330:  RFC: PHP
   microservice for containerized shell execution
  - @tstarling wrote an example to demonstrate how the Score extension
  could work.
   - T250406 : RFC: Hybrid
   extension management
  - @RHeigl asks about next steps.
   - T120085 : RFC: Serve Main
   Page of Wikimedia wikis from a consistent URL
  - @bblack indicates their team considers this an Epic Idea™. Thank
  you Brandon.


-- Timo
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Allow HTML email

2020-09-22 Thread Timo Tijhof

+1 for both of these. This would save me some time in the drafting of
Wikitech-l emails.

You will of course still reserve the
right to insert line breaks at your
own discretion to achieve a
particular
visual
fit.-- Timo





On Wed, Sep 23, 2020 at 4:03 AM Tim Starling 
wrote:

> * Should Mailman collapse multipart/alternative to its first part content?
> * Should Mailman convert text/html parts to plain text? This
> conversion happens after MIME attachments have been stripped.
>
> These mailman options are both enabled for this list. I would argue
> that they should both be disabled. It is nice to write emails with
> cutting-edge features like client-side word wrapping and links.
>
> Are there any objections to this change?
>
> -- Tim Starling
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Welcome to Wikimedia development! (Summer of 2020)

2020-07-21 Thread Timo Tijhof

Hello fellow contributors, developers, and colleagues,

This is a semiannual refresher on Wikimedia development policies. Whether
you joined last week, or have been here for a decade, I hope this helps you
navigate our current practices. (See changelog below.) Feel free to use
this thread to ask or answer questions about any of these!

TL;DR:
* The Code of Conduct applies to how all of us behave.
https://mediawiki.org/wiki/Code_of_Conduct
* If you can merge code changes, please familiarize yourself with the
Privilege policy. https://mediawiki.org/wiki/Gerrit/Privilege_policy
* The Support policy for PHP describes what we support and for how long. It
allows site operators and developers to plan ahead (incl WMF).
https://mediawiki.org/wiki/Support_policy_for_PHP
* The Stable interface is about code that should stay compatible, and is
safe to use in an extension.
https://mediawiki.org/wiki/Stable_interface_policy
* Ask for help! Either here onlist, or on IRC chat.
https://mediawiki.org/wiki/MediaWiki_on_IRC

### Gerrit privilege policy

As maintainer you take final responsibility over code before it ships to
the servers and the personal devices of a billion people. No pressure! :-)
https://mediawiki.org/wiki/Gerrit/Privilege_policy

### MediaWiki stability and PHP support

Before you use internal code or extend core in an undocumented way, it’s
best to start a conversation on Phabricator first. Explain your use case
and decide together how to proceed. Also involve the maintainers or code
stewards of the component in question.
https://mediawiki.org/wiki/Developers/Maintainers

### Data access and privacy

Wikimedia takes pride in its commitment to protecting user privacy. If you
build feature instrumentation or store personal data, think about how the
data will be removed or aggregated.
https://meta.wikimedia.org/wiki/Data_retention_guidelines

If you have shell access, see also the Server Access and Data Access
responsibilities.
https://phabricator.wikimedia.org/L3
https://wikitech.wikimedia.org/wiki/Data_access#Responsibilities

### Best practices and ask for help

You can find pages about coding style, security, performance, localisation,
and more; on https://mediawiki.org/wiki/Development_guidelines

Use these pages to your advantage, and feel free to ask questions here on
Wikitech-l, or chat in #mediawiki on Freenode IRC.
https://mediawiki.org/wiki/MediaWiki_on_IRC

### Changes since 2019

For those of you who know most of this already, here’s what’s changed:

* February 2019: The "+2 policy" was updated and is now the Gerrit
privilege policy. – https://mediawiki.org/wiki/Gerrit/Privilege_policy,
https://phabricator.wikimedia.org/T216295

* March 2019: Established principles around the creation of new services.
The idea for this started at Wikimedia Technical Conference in Portland
(2018).  – https://mediawiki.org/wiki/Wikimedia_services_guideline,
https://phabricator.wikimedia.org/T208524

* September 2019: There is a new general policy for PHP support changes. We
no longer need dedicated RFCs, such when MediaWiki required PHP 5.5+ in
2016, or PHP 7 in 2017. The policy was first used to require PHP 7.2+ for
MediaWiki 1.34. – https://mediawiki.org/wiki/Support_policy_for_PHP,
https://phabricator.wikimedia.org/T228342

* March 2020: The “Deprecation policy” was updated and is now the Stable
interface policy. – https://mediawiki.org/wiki/Stable_interface_policy,
https://phabricator.wikimedia.org/T193613

* April 2020: We added some structure, boilerplate, and documented common
stakeholder questions. These changes enable RFC authors to go through the
process quicker, and to steer more of it on their own. –
https://mediawiki.org/wiki/Requests_for_comment,
https://phabricator.wikimedia.org/T216308

--
Timo Tijhof
Principal Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l