[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 519211 abandoned by Alaa Sarhan: Switch Property Terms migration to WRITE_NEW on test wikidata https://gerrit.wikimedia.org/r/519211 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. We are now reading property terms from new store. We probably should see some drop in fetching terms from old store, since property terms are read way more often than item terms. Will be checking it in few days

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-08-26T11:10:54Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:527087|Switch property terms migration to WRITE_NEW on client wikis (T225053 )]] (duration:

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-08-20T12:05:55Z] Synchronized php-1.34.0-wmf.17/extensions/Wikibase: SWAT: [[gerrit:530845|Initialize DatabaseTermIdsResolver and DatabaseTypeIdsStore with repo database name in client. (T230119

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 530845 **merged** by jenkins-bot: [mediawiki/extensions/Wikibase@wmf/1.34.0-wmf.17] Initialize DatabaseTermIdsResolver and DatabaseTypeIdsStore with repo database name in client. https://gerrit.wikimedia.org/r/530845 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 530845 had a related patch set uploaded (by Alaa Sarhan; owner: Alaa Sarhan): [mediawiki/extensions/Wikibase@wmf/1.34.0-wmf.17] Initialize DatabaseTermIdsResolver and DatabaseTypeIdsStore with repo database name in client.

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. Rescheduled for deployment 20th of August, with a priory backport https://wikitech.wikimedia.org/wiki/Deployments#deploycal-item-20190820T1100 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Urbanecm added a comment. Reverted. TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: alaa_wmde, Urbanecm Cc: Urbanecm, Addshore, Reedy, Marostegui, hoo, Lucas_Werkmeister_WMDE, Ladsgroup,

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-08-19T11:46:58Z] Synchronized wmf-config/InitialiseSettings.php: Revert 483691c (T225053 )

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. In T225053#5421218 , @Stashbot wrote: > Mentioned in SAL (#wikimedia-operations) [2019-08-19T11:39:06Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: 483691c

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-08-07T19:16:03Z] Synchronized wmf-config/InitialiseSettings.php: Revert Switch property terms migration to WRITE_NEW on client wikis T225053 (duration: 00m 58s) TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 528909 **merged** by jenkins-bot: [operations/mediawiki-config@master] Revert "Switch property terms migration to WRITE_NEW on client wikis" https://gerrit.wikimedia.org/r/528909 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 528909 had a related patch set uploaded (by Reedy; owner: Reedy): [operations/mediawiki-config@master] Revert "Switch property terms migration to WRITE_NEW on client wikis" https://gerrit.wikimedia.org/r/528909 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-08-07T11:33:40Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:527087|Switch property terms migration to WRITE_NEW on client wikis (T225053 )]] (duration:

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 527087 **merged** by jenkins-bot: [operations/mediawiki-config@master] Switch property terms migration to WRITE_NEW on client wikis https://gerrit.wikimedia.org/r/527087 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-08-05T12:13:24Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:526657|Switch property terms migration to WRITE_NEW on production wikidata (T225053 )]]

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-08-02T09:17:05Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:526657|Revert: Switch property terms migration to WRITE_NEW on production wikidata (T225053

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Marostegui added a comment. F29937909: mysql.png While having the `read_key` handler being used isn't a bad thing, I think there is a pattern there that needs some investigation as the spikes are huge and very consistent - there's something

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 527087 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [operations/mediawiki-config@master] Switch property terms migration to WRITE_NEW on client wikis https://gerrit.wikimedia.org/r/527087 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-08-01T11:23:49Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: c164132: Revert "Revert "Switch property terms migration to WRITE_NEW on production wikidata"" (T225053

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. This seems good to me: wikiadmin@10.64.0.96(wikidatawiki)> EXPLAIN SELECT wbxl_language as term_language, wby_name as term_type, wbx_text as term_text FROM wbt_property_terms INNER JOIN wbt_term_in_lang ON wbpt_term_in_lang_id =

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-07-31T12:19:36Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:526657|Revert: Switch property terms migration to WRITE_NEW on production wikidata (T225053

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-07-31T12:06:11Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:519212|Switch property terms migration to WRITE_NEW on production wikidata (T225053 )]]

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 519212 **merged** by jenkins-bot: [operations/mediawiki-config@master] Switch property terms migration to WRITE_NEW on production wikidata https://gerrit.wikimedia.org/r/519212 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-07-31T12:05:02Z] sync-file aborted: SWAT: [[gerrit:519212|Switch property terms migration to WRITE_NEW on production wikidata (T225053 )]] (duration: 00m 03s) TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 526653 **merged** by jenkins-bot: [operations/mediawiki-config@master] labs: Set tmpPropertyTermsMigrationStage to MIGRATION_WRITE_NEW in wikidata https://gerrit.wikimedia.org/r/526653 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 526653 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [operations/mediawiki-config@master] labs: Set tmpPropertyTermsMigrationStage to MIGRATION_WRITE_NEW in wikidata https://gerrit.wikimedia.org/r/526653 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. After coordinating with our DBA, this will go live in Wednesday. TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc: hoo, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. > Are you connecting to the right database? we are getting the db and passing it the same way it was done for old TermIndex implementation .. not sure where we are failing to achieve the same effect. The only difference that I spotted from just reading

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. It's not federation, it's also clients as well. For example try to invoke to get label of a property in English Wikipedia in beta cluster (Add `{{#invoke:Wikidata test|hello}}` in any page and try to save it). It fatals with this: A database query error has

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 520292 **merged** by jenkins-bot: [operations/mediawiki-config@master] labs: Temporary disable read new for wikibase term store of commons https://gerrit.wikimedia.org/r/520292 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 520292 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [operations/mediawiki-config@master] labs: Temporary disable read new for wikibase term store of commons https://gerrit.wikimedia.org/r/520292 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. It should not query term store at all. It's a bug needs fixing, I can disable it in commons for now. TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc:

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

hoo added a comment. This broke beta commons: https://commons.wikimedia.beta.wmflabs.org/wiki/File:CLD_test.webm (`Error: 1146 Table 'commonswiki.wbt_property_terms' doesn't exist (172.16.4.147:3306)`) TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. > 2. Clear all cached terms of that property They have a pretty short TTL, I don't think that's needd > 3. Hide all terms of that property from wb_terms table (you can use `sql wikidatawiki` or `mwscript sql.php wikidatawiki` to execute sql): You

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. Tested in https://wikidata.beta.wmflabs.org/wiki/Property:P174944 and it seems it's okay https://wikidata.beta.wmflabs.org/wiki/Property:P175360 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 520220 **merged** by jenkins-bot: [operations/mediawiki-config@master] labs: Set wmgWikibaseTmpPropertyTermsMigrationStage to MIGRATION_WRITE_NEW https://gerrit.wikimedia.org/r/520220 TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 520220 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup): [operations/mediawiki-config@master] labs: Set wmgWikibaseTmpPropertyTermsMigrationStage to MIGRATION_WRITE_NEW https://gerrit.wikimedia.org/r/520220 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. In T225053#5299613 , @alaa_wmde wrote: > > We are changing query plans of around 10k queries per second, in case anything is wrong with the read, it explodes logstash, elastic, database and or Wikipedia and

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. > We are changing query plans of around 10k queries per second, in case anything is wrong with the read, it explodes logstash, elastic, database and or Wikipedia and all of its projects. I don't say we should wait and look at walls, we should set the read true in

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. In T225053#5299336 , @alaa_wmde wrote: > Why should we wait on read? I'd prefer to see it failing, if it will, asap actually. Waiting will just delay delivery here, or? We are changing query plans of

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. In T225053#5287520 , @Ladsgroup wrote: > Sure, so it means WRITE_NEW is actually write_both + read_new? if that's case and given that huge amount of reads on wb_terms table is actually for properties, we

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. Sure, so it means WRITE_NEW is actually write_both + read_new? if that's case and given that huge amount of reads on wb_terms table is actually for properties, we should still wait a little. TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. The entry point to where you can look at the implementation of all that is probably this https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Wikibase/+/refs/heads/master/repo/includes/WikibaseRepo.php#1805 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. Reading the documentation of the constants in core Defines.php they don't match the previously explained process to me. /**@{ * Schema change migration flags. * * Used as values of a feature flag for an orderly transition from an old *

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. no I actually let the confusing terms confuse me again. but yea WRITE_NEW should write to both by definition as far as I understand, shouldn't it? TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. I'm confused now, Setting it to WRITE_NEW makes it to write to both? Can I see the codepath? TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup Cc:

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. Which of course makes it redundant.. it is here just a transient state to keep going in the same process of migration as usual TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

alaa_wmde added a comment. effectively this is doing the same thing as WRITE_BOTH.. it will keep writing to both stores. so it wouldn't make a difference really. TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

Ladsgroup added a comment. Can this wait for a couple of days until we make sure there's no bug with WRITE_BOTH mode, we just turned it on today. TASK DETAIL https://phabricator.wikimedia.org/T225053 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 519212 had a related patch set uploaded (by Alaa Sarhan; owner: Alaa Sarhan): [operations/mediawiki-config@master] Switch property terms migration to WRITE_NEW on production wikidata https://gerrit.wikimedia.org/r/519212 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Commented On] T225053: Switch `tmpPropertyTermsMigrationStage` to MIGRATION_WRITE_NEW

gerritbot added a comment. Change 519211 had a related patch set uploaded (by Alaa Sarhan; owner: Alaa Sarhan): [operations/mediawiki-config@master] Switch Property Terms migration to WRITE_NEW on test wikidata https://gerrit.wikimedia.org/r/519211 TASK DETAIL