Re: [Wikitech-l] CodeReview to be removed from mediawiki.org, dumps available

[Zoran Dori]

Hello,
> We're planning to review the CodeReview extension, which was used
> during the SVN era, from mediawiki.org soon.

You mean to "remove extension" from mediawiki.org, right? :)

Best regards,
Zoran Dori
volunteer, Wikimedia Serbia
s: zoranzoki21.github.io e: zorandori4...@gmail.com
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] CodeReview to be removed from mediawiki.org, dumps available

[Kunal Mehta]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hi everyone,

We're planning to review the CodeReview extension, which was used
during the SVN era, from mediawiki.org soon.

A static HTML dump of the code review comments is available at
 and SQL dumps are available
at .

The next steps will be to set up redirects from
Special:CodeReview/[...] to the new static dump website and remove the
CodeReview extension.

Please let us know if you see anything missing from those sets of
dumps so we can figure out an archival plan before removal.

The main Phabricator task for tracking this is
.

Thanks,
- -- Kunal / Legoktm
-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEE+h6fmkHn9DUCyl1jUvyOe+23/KIFAl7DAXAACgkQUvyOe+23
/KIybw//c82E3PkEFjTCXM8Ti0nNMRJeaNXr1+YHIoMzPmYsv4MYaSHgJ6YGADTO
3DK4L6YO6PUTkxnnLsELd8cQgAKQzPNeSSjvhOb0zUw38ohKPKo/5ns2+jmTMIwB
BIUfL8/06S8SZ6izADsZ7G5Z6BttRNSsgvRy8hkxTwwfHy7V87ySeThRxLhSjFpr
cKKExxACC9icdy/dpY7HhONADsj7EAc1+YYpcDT9U7bucw2l8E1rXoP90tBEIYKF
IVgnrnGm7C2sUewcK1gJHYkJhqeIykxsnD5skGhOd0x3cAPtPTlFeAcEhwYXJQTU
nI613jNkOR+FYRVkguzYYMr2mRmOyX0NdW8638gVTtVWhUW99DuyyNKGLfS71csY
bv9ovYs0wC44WUwS6iFuF5I0FLjqUAeCA9XIlfVZivPSUgsv0TVGCCoxQeByb+cS
VBe1PY5RGz5lOwEGY8RDmbNCTLGh6S5lswk94GYtb+MvCu5GYQh+Byandtt25KvW
7xdyhfXG7KqaLiF8kVFwarBj3Y/LcERLJeo278EqgtX+t1Tv0/ZRtccSdXvmOhDw
o5LdFDtkQxDyKyZAeIY8o1Y2r/1iESA8Ai/TrnC8UTi5Jqs2sxPyuD8yo55/KRz2
VYmJpOxgIhh5nGtjpmTk/eWYBai7C+gs21z/ELZytLmFjg3owYs=
=8Uj5
-END PGP SIGNATURE-

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Namespace Localisations & Updates

[RhinosF1 -]

I would suggest a special window.

I normally do the late window so I can see if Catrope can come an hour
early for one and we do it then.

Samuel

On Mon, 18 May 2020 at 20:48, Tyler Cipriani 
wrote:

> On 20-05-18 16:32:59, RhinosF1 - wrote:
> >My only concern with that is that between the 2 tasks it would be 5/6
> >patches in the SWAT window.
>
> Yes, this work looks like it will consume a whole window easily :(
>
> >It would also be my first mediawiki core + extensions SWAT so are the
> >patches safe to +2 during / just before SWAT or Should I get that done
> >before?
>
> On process I think might work:
>
> * Merge core changes to master early in the week you plan to SWAT (but
> after branch cut for the week)
> * Prepare cherry-picks to backport to current stable + branch to go out
> that week
> * Add cherry-picks to deploy window
>
> It's likely this is more than 6 patches :\
>
> Syncing with a SWATter prior to SWAT and ensuring that you pick a window
> with some time after it (should you need more time to deploy) OR making
> a special window on the deployment calendar[0] for this set of changes
> would be best.
>
> -- Tyler
>
> [0]: 
>
> >On Mon, 18 May 2020 at 16:25, Tyler Cipriani 
> >wrote:
> >
> >> Hi Samuel
> >>
> >> On 20-05-18 09:57:54, RhinosF1 - wrote:
> >> >On https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/, it was
> >> >raised correctly that namespaceDupes.php would need to be ran.
> >> >
> >> >https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/ and
> >> >https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 can mostly run with
> the
> >> >train (expect the mediawiki config patch) but all require
> namespaceDupes
> >> to
> >> >be ran and on https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 could
> do
> >> >with being deployed as close together as possible to avoid
> >> >inconsistently translated namespaces.
> >> >
> >> >Would as mentioned SWAT be better for all 5 patches or should we let
> what
> >> >can ride with the train and deploy the one config patch shortly after
> and
> >> >run for both wikis in that window? or could we ask the train runners
> to do
> >> >that?
> >>
> >> What you're describing sounds like it would be a good candidate for SWAT
> >> deployment. My reasoning is that (1) it is atypical to run maintenance
> >> scripts as part of the train and (2) there are no guarantees that a
> >> train won't rollback.
> >>
> >> That is, backporting to a version that is stable ensures that we don't
> >> end up having rolled forward to all wikis, run the maintenance script,
> >> and then having to rollback due to an unrelated problem. Additionally,
> >> the log triage that follows a train window may mean that we can't
> >> guarantee a timely deploy of the configuration change following train.
> >>
> >> To me, this feels safer/faster/easier as a SWAT deployment; even though
> >> this might make for a particularly long SWAT window.
> >>
> >> Thanks!
> >> -- Tyler
> >> ___
> >> Wikitech-l mailing list
> >> Wikitech-l@lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> >--
> >Thanks,
> >Samuel
> >___
> >Wikitech-l mailing list
> >Wikitech-l@lists.wikimedia.org
> >https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

-- 
Thanks,
Samuel
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Namespace Localisations & Updates

[Tyler Cipriani]

On 20-05-18 16:32:59, RhinosF1 - wrote:

My only concern with that is that between the 2 tasks it would be 5/6
patches in the SWAT window.


Yes, this work looks like it will consume a whole window easily :(


It would also be my first mediawiki core + extensions SWAT so are the
patches safe to +2 during / just before SWAT or Should I get that done
before?


On process I think might work:

* Merge core changes to master early in the week you plan to SWAT (but 
after branch cut for the week)
* Prepare cherry-picks to backport to current stable + branch to go out 
that week

* Add cherry-picks to deploy window

It's likely this is more than 6 patches :\

Syncing with a SWATter prior to SWAT and ensuring that you pick a window 
with some time after it (should you need more time to deploy) OR making 
a special window on the deployment calendar[0] for this set of changes 
would be best.


-- Tyler

[0]: 


On Mon, 18 May 2020 at 16:25, Tyler Cipriani 
wrote:


Hi Samuel

On 20-05-18 09:57:54, RhinosF1 - wrote:
>On https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/, it was
>raised correctly that namespaceDupes.php would need to be ran.
>
>https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/ and
>https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 can mostly run with the
>train (expect the mediawiki config patch) but all require namespaceDupes
to
>be ran and on https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 could do
>with being deployed as close together as possible to avoid
>inconsistently translated namespaces.
>
>Would as mentioned SWAT be better for all 5 patches or should we let what
>can ride with the train and deploy the one config patch shortly after and
>run for both wikis in that window? or could we ask the train runners to do
>that?

What you're describing sounds like it would be a good candidate for SWAT
deployment. My reasoning is that (1) it is atypical to run maintenance
scripts as part of the train and (2) there are no guarantees that a
train won't rollback.

That is, backporting to a version that is stable ensures that we don't
end up having rolled forward to all wikis, run the maintenance script,
and then having to rollback due to an unrelated problem. Additionally,
the log triage that follows a train window may mean that we can't
guarantee a timely deploy of the configuration change following train.

To me, this feels safer/faster/easier as a SWAT deployment; even though
this might make for a particularly long SWAT window.

Thanks!
-- Tyler
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


--
Thanks,
Samuel
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


signature.asc
Description: PGP signature
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Namespace Localisations & Updates

[RhinosF1 -]

I’m quite happy go with a SWAT deployment and I agree that it’s probably a
lot safer.

My only concern with that is that between the 2 tasks it would be 5/6
patches in the SWAT window.

It would also be my first mediawiki core + extensions SWAT so are the
patches safe to +2 during / just before SWAT or Should I get that done
before?

Thanks,
Samuel

On Mon, 18 May 2020 at 16:25, Tyler Cipriani 
wrote:

> Hi Samuel
>
> On 20-05-18 09:57:54, RhinosF1 - wrote:
> >On https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/, it was
> >raised correctly that namespaceDupes.php would need to be ran.
> >
> >https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/ and
> >https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 can mostly run with the
> >train (expect the mediawiki config patch) but all require namespaceDupes
> to
> >be ran and on https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 could do
> >with being deployed as close together as possible to avoid
> >inconsistently translated namespaces.
> >
> >Would as mentioned SWAT be better for all 5 patches or should we let what
> >can ride with the train and deploy the one config patch shortly after and
> >run for both wikis in that window? or could we ask the train runners to do
> >that?
>
> What you're describing sounds like it would be a good candidate for SWAT
> deployment. My reasoning is that (1) it is atypical to run maintenance
> scripts as part of the train and (2) there are no guarantees that a
> train won't rollback.
>
> That is, backporting to a version that is stable ensures that we don't
> end up having rolled forward to all wikis, run the maintenance script,
> and then having to rollback due to an unrelated problem. Additionally,
> the log triage that follows a train window may mean that we can't
> guarantee a timely deploy of the configuration change following train.
>
> To me, this feels safer/faster/easier as a SWAT deployment; even though
> this might make for a particularly long SWAT window.
>
> Thanks!
> -- Tyler
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

-- 
Thanks,
Samuel
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Namespace Localisations & Updates

[Tyler Cipriani]

Hi Samuel

On 20-05-18 09:57:54, RhinosF1 - wrote:

On https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/, it was
raised correctly that namespaceDupes.php would need to be ran.

https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/ and
https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 can mostly run with the
train (expect the mediawiki config patch) but all require namespaceDupes to
be ran and on https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 could do
with being deployed as close together as possible to avoid
inconsistently translated namespaces.

Would as mentioned SWAT be better for all 5 patches or should we let what
can ride with the train and deploy the one config patch shortly after and
run for both wikis in that window? or could we ask the train runners to do
that?


What you're describing sounds like it would be a good candidate for SWAT 
deployment. My reasoning is that (1) it is atypical to run maintenance 
scripts as part of the train and (2) there are no guarantees that a 
train won't rollback.


That is, backporting to a version that is stable ensures that we don't 
end up having rolled forward to all wikis, run the maintenance script, 
and then having to rollback due to an unrelated problem. Additionally, 
the log triage that follows a train window may mean that we can't 
guarantee a timely deploy of the configuration change following train.


To me, this feels safer/faster/easier as a SWAT deployment; even though 
this might make for a particularly long SWAT window.


Thanks!
-- Tyler


signature.asc
Description: PGP signature
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Abstract Schema and Schema changes, request for help

[Gergo Tisza]

Thank you so much for working on this, it was one of the most painful
aspects of core development!

It might be worth using a consistent gerrit topic or hashtag to make
finding the relevant patches easy.

On Sat, May 9, 2020 at 3:21 AM Amir Sarabadani  wrote:

> Hello,
> In case you haven't done any changes on database schema of mediawiki core,
> let me explain the process to you (if you know this, feel free to skip this
> paragraph):
> * Mediawiki core supports three types of RDBMS: MySQL, Sqlite, Postgres. It
> used to be five (plus Oracle and MSSQL)
> * For each one of these types, you need to do three parts: 1- Change the
> tables.sql file so new installations get the new schema 2- Make .sql schema
> change file, like an "ALTER TABLE" for current installations so they can
> upgrade. 3- Wire that schema change file into *Updater.php file.
> * For example, this is a patch to drop a column:
> https://gerrit.wikimedia.org/r/c/mediawiki/core/+/473601 This file touches
> 14 different files, adds 94 lines and removes 30.
>
> This is bad for several reasons:
> * It is extremely complicated to do a even a simple schema change. Usually
> something as simple as adding an column takes a whole day for me. There are
> lots of complicating factors, like Sqlite doesn't have ALTER TABLE, so when
> you want to make a patch for adding a column, you need to make a temporary
> table with the new column, copy the old table data to it, drop the old
> table and then rename the old table.
> ** Imagine the pain and sorrow when you want to normalize a table meaning
> you need to do several schema changes: 1- Add a table, 2- Add a column on
> the old table, 3- make the column not-nullable when it's filled and make
> the old column nullable instead 4- drop the old column.
> * It's almost impossible to test all DBMS types, I don't have MSSQL or
> Oracle installed and I don't even know their differences with MySQL. I
> assume most other developers are good in one type, not all.
> * Writing raw sqls, specially duplicated ones, and doubly specially when we
> don't have CI to test (because we won't install propriety software in our
> infra) is pretty much prone to error. My favourite one was that a new
> column on a table was actually added to the wrong table in MSSQL and it
> went unnoticed for two years (four releases, including one LTS).
> * It's impossible to support more DBMS types through extensions or other
> third party systems. Because the maintainer needs to keep up with all
> patches we add to core and write their equivalents.
> * For lots of reasons, these schemas are diverging, there have been several
> work to just reduce this to a minimum.
>
> There was a RFC to introduce abstract schema and schema changes and it got
> accepted and I have been working to implement this:
> https://phabricator.wikimedia.org/T191231
>
> This is not a small task, and like any big work, it's important to cut it
> to small pieces and gradually improve things. So my plan is first, I
> abstract the schema (tables.sql files), then slowly I abstract schema
> changes. For now, the plan is to make these .sql files automatically
> generated through maintenance scripts. So we will have a file called
> tables.json and when running something like:
> php maintenance/generateSchemaSql.php --json maintenance/tables.json --sql
> maintenance/tables-generated.sql --type=mysql
> It would produce tables-generated.sql file. The code that produces it is
> Doctrine DBAL and this is already installed as a dev dependency of core
> because you would need Doctrine if you want to make a schema change, if you
> maintain an instance, you should not need anything. Most of the work for
> automatically generating schema is already merged and the last part that
> wires it (and migrates two tables) is up for review:
> https://gerrit.wikimedia.org/r/c/mediawiki/core/+/595240
>
> My request is that I need to make lots of patches and since I'm doing this
> in my volunteer capacity, I need developers to review (and potentially help
> with the work if you're excited about this like me). Let me know if you're
> willing to be added in future patches and the current patch also welcomes
> any feedback: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/595240
>
> I have added the documentation in
> https://www.mediawiki.org/wiki/Manual:Schema_changes for the plan and
> future changes. The ideal goal is that when you want to do a schema change,
> you just change tables.json and create a json file that is snapshot of
> before and after table (remember, sqlite doesn't have alter table, meaning
> it has to know the whole table). Also, once we are in a good shape in
> migrating mediawiki core, we can start cleaning up extensions.
>
> Any feedback is also welcome.
>
> Best
> --
> Amir (he/him)
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Namespace Localisations & Updates

[RhinosF1 -]

Hello all,

I hope everyone is doing well.

On https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/, it was
raised correctly that namespaceDupes.php would need to be ran.

https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/596424/ and
https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 can mostly run with the
train (expect the mediawiki config patch) but all require namespaceDupes to
be ran and on https://gerrit.wikimedia.org/r/#/q/bug:%20T251287 could do
with being deployed as close together as possible to avoid
inconsistently translated namespaces.

Would as mentioned SWAT be better for all 5 patches or should we let what
can ride with the train and deploy the one config patch shortly after and
run for both wikis in that window? or could we ask the train runners to do
that?

Thanks,
Samuel
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l