[Wikidata-bugs] [Maniphest] [Updated] T198350: Rising lock wait timeout SQL errors upon 1.32.0-wmf.10 group1 deployment

[Tgr]

Tgr added a comment.

Here's a stack trace for a Commons error, which is more typical (probably. We really need a frequent stack traces view):

#0 /srv/mediawiki/php-1.32.0-wmf.10/includes/libs/rdbms/database/Database.php(1413): Wikimedia\Rdbms\Database->makeQueryException(string, integer, string, string)
#1 /srv/mediawiki/php-1.32.0-wmf.10/includes/libs/rdbms/database/Database.php(1186): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#2 /srv/mediawiki/php-1.32.0-wmf.10/includes/libs/rdbms/database/Database.php(2010): Wikimedia\Rdbms\Database->query(string, string)
#3 /srv/mediawiki/php-1.32.0-wmf.10/includes/CommentStore.php(549): Wikimedia\Rdbms\Database->insert(string, array, string)
#4 /srv/mediawiki/php-1.32.0-wmf.10/includes/Storage/RevisionStore.php(668): Closure$CommentStore::insertInternal(integer)
#5 /srv/mediawiki/php-1.32.0-wmf.10/includes/Storage/RevisionStore.php(476): MediaWiki\Storage\RevisionStore->insertRevisionRowOn(Wikimedia\Rdbms\DatabaseMysqli, MediaWiki\Storage\MutableRevisionRecord, Title, integer)
#6 /srv/mediawiki/php-1.32.0-wmf.10/includes/Storage/RevisionStore.php(417): MediaWiki\Storage\RevisionStore->insertRevisionInternal(MediaWiki\Storage\MutableRevisionRecord, Wikimedia\Rdbms\DatabaseMysqli, User, CommentStoreComment, Title, integer, integer)
#7 /srv/mediawiki/php-1.32.0-wmf.10/includes/libs/rdbms/database/Database.php(3726): Closure$MediaWiki\Storage\RevisionStore::insertRevisionOn(Wikimedia\Rdbms\DatabaseMysqli, string)
#8 /srv/mediawiki/php-1.32.0-wmf.10/includes/libs/rdbms/database/DBConnRef.php(49): Wikimedia\Rdbms\Database->doAtomicSection(string, Closure$MediaWiki\Storage\RevisionStore::insertRevisionOn;3861)
#9 /srv/mediawiki/php-1.32.0-wmf.10/includes/libs/rdbms/database/DBConnRef.php(523): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#10 /srv/mediawiki/php-1.32.0-wmf.10/includes/Storage/RevisionStore.php(419): Wikimedia\Rdbms\DBConnRef->doAtomicSection(string, Closure$MediaWiki\Storage\RevisionStore::insertRevisionOn;3861)
#11 /srv/mediawiki/php-1.32.0-wmf.10/includes/Storage/PageUpdater.php(953): MediaWiki\Storage\RevisionStore->insertRevisionOn(MediaWiki\Storage\MutableRevisionRecord, Wikimedia\Rdbms\DBConnRef)
#12 /srv/mediawiki/php-1.32.0-wmf.10/includes/Storage/PageUpdater.php(724): MediaWiki\Storage\PageUpdater->doModify(CommentStoreComment, User, integer)
#13 /srv/mediawiki/php-1.32.0-wmf.10/includes/page/WikiPage.php(1845): MediaWiki\Storage\PageUpdater->saveRevision(CommentStoreComment, integer)
#14 /srv/mediawiki/php-1.32.0-wmf.10/includes/EditPage.php(2241): WikiPage->doEditContent(WikitextContent, CommentStoreComment, integer, boolean, User, string, array, integer)
#15 /srv/mediawiki/php-1.32.0-wmf.10/includes/EditPage.php(1529): EditPage->internalAttemptSave(array, boolean)
#16 /srv/mediawiki/php-1.32.0-wmf.10/includes/api/ApiEditPage.php(392): EditPage->attemptSave(array)
#17 /srv/mediawiki/php-1.32.0-wmf.10/includes/api/ApiMain.php(1584): ApiEditPage->execute()
#18 /srv/mediawiki/php-1.32.0-wmf.10/includes/api/ApiMain.php(535): ApiMain->executeAction()
#19 /srv/mediawiki/php-1.32.0-wmf.10/includes/api/ApiMain.php(506): ApiMain->executeActionWithErrorHandling()
#20 /srv/mediawiki/php-1.32.0-wmf.10/api.php(83): ApiMain->execute()
#21 /srv/mediawiki/w/api.php(3): include(string)
#22 {main}

(logstash)

Per the investigation @jcrespo did in P7314, this seems like a conflict between INSERT INTO revision_comment_temp (revcomment_rev, revcomment_comment_id) VALUES (<revision id>, <comment id>) (in CommentStore::insertInternal, on page edit or upload) and SELECT 1 FROM revision, revision_comment_temp WHERE rev_page = <page id> FOR UPDATE (in WikiPage::doDeleteArticleReal, in preparation for page deletion).

The edit process looks like this:


lock (FOR UPDATE) the page row
insert a comment row
insert a revision row
insert a revision_comment_temp row with the IDs from the two previous rows
...some more stuff normally, but this is where the exception above was thrown


The delete process looks like this:


lock (FOR UPDATE) the page row
lock all revision, revision_comment_temp, revision_actor_temp rows belonging to the page
copy all those revision rows to archive. (It seems the comments are inserted again in the process. That could be improved but is irrelevant to this task.)
delete the page row
delete all the revision rows
delete all the revision_comment_temp rows
delete all the revision_actor_temp rows
delete related ip_changes rows
insert logging row


In theory this seems safe: both actions lock page (and nothing else, initially) which acts as a semaphore.

TASK DETAIL

https://phabricator.wikimedia.org/T198350

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dduvall, Tgr
Cc: Danmichaelo, jcrespo, Ankry, Nikerabbit, Marostegui, Anomie, cscott, daniel, Tgr, Addshore, Herzi.Pinki, DC, Yann, Raymond, DerHexer, matmarex, AlexisJazz, Aklapper, greg, Jdforrester-WMF, Stashbot, gerritbot, MarcoAurelio, revi, dduvall, AndyTan, Rayssa-, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair

_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs