[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Only rerender summary after mobile-sections were rerendered.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/406058 ) Change subject: Only rerender summary after mobile-sections were rerendered. .. Only rerender summary after mobile-sections were rerendered. Bug: T184753 Change-Id: I610611f2a64997e72d9049b8c10da44c19ade2ce --- M scap/templates/config.yaml.j2 1 file changed, 21 insertions(+), 54 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/deploy refs/changes/58/406058/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index db503d3..96ba6e4 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -72,9 +72,10 @@ cases: - match: meta: -uri: '/^https?:\/\/[^\/]+\/api\/rest_v1\/page\/html\/([^\/]+)$/' +uri: '/^https?:\/\/[^\/]+\/api\/rest_v1\/page\/mobile-sections\/([^\/]+)$/' tags: - restbase +- mobile-sections match_not: - meta: domain: '/wiktionary\.org$/' @@ -92,10 +93,11 @@ meta: # These URIs are coming from RESTBase, so we know that article titles will be normalized # and main namespace articles will not have : (uri-encoded, so %3a or %3A) -uri: '/^https?:\/\/[^\/]+\/api\/rest_v1\/page\/html\/((?:(?!%3a|%3A|\/).)+)$/' +uri: '/^https?:\/\/[^\/]+\/api\/rest_v1\/page\/mobile-sections\/((?:(?!%3a|%3A|\/).)+)$/' domain: '/^en\.wiktionary\.org$/' tags: - restbase +- mobile-sections exec: method: get # Don't encode title since it should be already encoded @@ -112,7 +114,6 @@ match: meta: uri: '/^https?:\/\/[^\/]+\/api\/rest_v1\/page\/html\/([^/]+)$/' -domain: '/\.wikipedia\.org$/' tags: - restbase exec: @@ -164,14 +165,8 @@ if-unmodified-since: '{{date(message.meta.dt)}}' query: redirect: false -# The HTML might not change but sometimes editors use a purge to drop incorrectly rendered summary/MCS +# The HTML might not change but sometimes editors use a purge to drop incorrectly rendered MCS # content, so let's purge them as well just in case. The rate is low. - - method: get -uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/summary/{decode(match.meta.uri.title)}' -headers: - cache-control: no-cache -query: - redirect: false - method: get uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/mobile-sections/{decode(match.meta.uri.title)}' headers: @@ -205,14 +200,8 @@ if-unmodified-since: '{{date(message.meta.dt)}}' query: redirect: false -# The HTML might not change but sometimes editors use a purge to drop incorrectly rendered summary/MCS +# The HTML might not change but sometimes editors use a purge to drop incorrectly rendered MCS # content, so let's purge them as well just in case. The rate is low. - - method: get -uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/summary/{decode(match.meta.uri.title)}' -headers: - cache-control: no-cache -query: - redirect: false - method: get uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/mobile-sections/{decode(match.meta.uri.title)}' headers: @@ -661,38 +650,20 @@ on_wikidata_description_change: topic: change-prop.wikidata.resource-change cases: - - match: - meta: -uri: '/https?:\/\/[^\/]+\/wiki\/(?.+)/' -domain: '/\.wikipedia\.org$/' - tags: [ 'wikidata' ] -exec: - - method: get -uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/summary/{{match.meta.uri.title}}' -headers: - cache-control: no-cache -query: -
[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: RESTBase: Update the config
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/404956 ) Change subject: RESTBase: Update the config .. RESTBase: Update the config Change-Id: I03fb0f855181340f31fc2f99051513427fa999af --- M puppet/modules/restbase/templates/config.yaml.erb 1 file changed, 10 insertions(+), 3 deletions(-) Approvals: Ppchelko: Looks good to me, approved jenkins-bot: Verified diff --git a/puppet/modules/restbase/templates/config.yaml.erb b/puppet/modules/restbase/templates/config.yaml.erb index 41eba30..a95f3dd 100644 --- a/puppet/modules/restbase/templates/config.yaml.erb +++ b/puppet/modules/restbase/templates/config.yaml.erb @@ -8,12 +8,10 @@ options: &default_options table: backend: sqlite - dbname: <%= @dbdir %>/mw-vagrant.sqlite3 -table_ng: - backend: sqlite dbname: <%= @dbdir %>/mw-vagrant-ng.sqlite3 parsoid: host: http://localhost:<%= scope['::parsoid::port'] %> + grace_ttl: 3600 action: apiUriTemplate: "{{'http://localhost<%= scope['::port_fragment'] %>/w/api.php'}}" baseUriTemplate: "{{'http://{domain}:7231/{domain}/v1'}}" @@ -25,6 +23,10 @@ cache-control: s-maxage=864000, max-age=86400 mobileapps: host: http://localhost:<%= @mobileapps_port %> +summary: + protocol: http + implementation: mw_api # TODO: change to mcs soon + host: http://localhost:<%= @mobileapps_port %> events: eventlogging_service: uri: http://localhost:<%= @eventlogging_service_port %>/v1/events @@ -35,10 +37,15 @@ host: http://localhost:<%= @citoid_port %> transform: cx_host: http://localhost:<%= @cxserver_port %> +# 10 days Varnish caching, one day client-side +purged_cache_control: s-maxage=864000, max-age=86400 +# Cache control for purged endpoints allowing short-term client caching +purged_cache_control_client_cache: s-maxage=1209600, max-age=300 pdf: cache_control: s-maxage=600, max-age=600 uri: http://localhost:<%= @pdf_service_port %>/ secret: <%= @pdf_service_secret %> + scheme: http wikimedia.org: &wikimedia x-modules: -- To view, visit https://gerrit.wikimedia.org/r/404956 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I03fb0f855181340f31fc2f99051513427fa999af Gerrit-PatchSet: 2 Gerrit-Project: mediawiki/vagrant Gerrit-Branch: master Gerrit-Owner: Mobrovac Gerrit-Reviewer: BryanDavis Gerrit-Reviewer: Dduvall Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Only set the comment field if the comment is non-null.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/404897 ) Change subject: Only set the comment field if the comment is non-null. .. Only set the comment field if the comment is non-null. For null comments we shouldn't set the field cause it's not required, but the event validation fails if we set it to null. Change-Id: I97a2b87f268e40beca5cee48c8d824c6f9aa75a2 --- M EventBus.hooks.php 1 file changed, 11 insertions(+), 9 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/97/404897/1 diff --git a/EventBus.hooks.php b/EventBus.hooks.php index 4b11438..ef25a51 100644 --- a/EventBus.hooks.php +++ b/EventBus.hooks.php @@ -155,7 +155,6 @@ // Common Mediawiki entity fields 'database' => $wgDBname, 'performer' => EventBus::createPerformerAttrs( $user ), - 'comment'=> $reason, // page entity fields 'page_id'=> $id, @@ -174,6 +173,7 @@ } if ( !is_null( $reason ) ) { + $attrs['comment'] = $reason; $attrs['parsedcomment'] = Linker::formatComment( $reason, $wikiPage->getTitle() ); } @@ -211,7 +211,6 @@ // Common Mediawiki entity fields 'database' => $wgDBname, 'performer' => EventBus::createPerformerAttrs( $performer ), - 'comment'=> $comment, // page entity fields 'page_id'=> $title->getArticleID(), @@ -236,6 +235,7 @@ } if ( !is_null( $comment ) ) { + $attrs['comment'] = $comment; $attrs['parsedcomment'] = Linker::formatComment( $comment, $title ); } @@ -280,7 +280,6 @@ // Common Mediawiki entity fields 'database' => $wgDBname, 'performer' => EventBus::createPerformerAttrs( $user ), - 'comment'=> $reason, // page entity fields 'page_id'=> $pageid, @@ -312,6 +311,7 @@ } if ( !is_null( $reason ) ) { + $attrs['comment'] = $reason; $attrs['parsedcomment'] = Linker::formatComment( $reason, $newTitle ); } @@ -376,7 +376,6 @@ // Common Mediawiki entity fields: 'database' => $wgDBname, 'performer' => EventBus::createPerformerAttrs( $performer ), - 'comment'=> $revision->getComment(), // revision entity fields: 'page_id'=> $revision->getPage(), @@ -407,10 +406,10 @@ $attrs['page_is_redirect'] = false; } - if ( !is_null( $revision->getComment() ) ) { - $attrs['parsedcomment'] = Linker::formatComment( - $revision->getComment(), - $revision->getTitle() ); + $comment = $revision->getComment(); + if ( !is_null( $comment ) ) { + $attrs['comment'] = $comment; + $attrs['parsedcomment'] = Linker::formatComment( $comment, $revision->getTitle() ); } $events[] = EventBus::createEvent( @@ -544,9 +543,12 @@ // Common Mediawiki entity fields: 'database' => $wgDBname, 'performer' => EventBus::createPerformerAttrs( $user ), - 'comment'=> $block->mReason, ]; + if ( !is_null( $block->mReason ) ) { + $attrs['comment'] = $block->mReason; + } + // user entity fields: // Note that, except for null, it is always safe to treat the target -- To view, visit https://gerrit.wikimedia.org/r/404897 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I97a2b87f268e40beca5cee48c8d824c6f9aa75a2 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Bra
[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Remove wmgDebugJobQueueEventBus config parameter.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/404888 ) Change subject: Remove wmgDebugJobQueueEventBus config parameter. .. Remove wmgDebugJobQueueEventBus config parameter. The parameter was introduced temporary to be able to gradually roll our the debug queue feature for JobQueue Kafka backend project. Since the debug queue is rolled out on all wikis that have EventBus, additional parameter is not needed any more. Change-Id: Ia90fc83576e62b39ffac7a1c4895e4378daac6ab --- M wmf-config/InitialiseSettings.php M wmf-config/jobqueue.php 2 files changed, 1 insertion(+), 9 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config refs/changes/88/404888/1 diff --git a/wmf-config/InitialiseSettings.php b/wmf-config/InitialiseSettings.php index a13a765..06dff2f 100644 --- a/wmf-config/InitialiseSettings.php +++ b/wmf-config/InitialiseSettings.php @@ -19634,14 +19634,6 @@ 'votewiki' => false, ], -'wmgDebugJobQueueEventBus' => [ - 'default' => true, - 'wikitech' => false, - 'private' => false, - 'loginwiki' => false, - 'votewiki' => false, -], - // Temporary code for gradually integrating htmlCacheUpdate // jobs into the EventBus JobQueue - T182023 // Step 2: enable everywhere but wikidata, enwiki and commons diff --git a/wmf-config/jobqueue.php b/wmf-config/jobqueue.php index a685ac9..a4c1fd6 100644 --- a/wmf-config/jobqueue.php +++ b/wmf-config/jobqueue.php @@ -41,7 +41,7 @@ 'maxPartitionsTry' => 5 // always covers 2+ servers ]; -if ( $wmgUseEventBus && $wmgDebugJobQueueEventBus ) { +if ( $wmgUseEventBus ) { $wgJobTypeConf['MessageIndexRebuildJob'] = $wgJobTypeConf['flaggedrevs_CacheUpdate'] = $wgJobTypeConf['deleteLinks'] = -- To view, visit https://gerrit.wikimedia.org/r/404888 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia90fc83576e62b39ffac7a1c4895e4378daac6ab Gerrit-PatchSet: 1 Gerrit-Project: operations/mediawiki-config Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update change-propagation to 214d48e
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/404881 ) Change subject: Update change-propagation to 214d48e .. Update change-propagation to 214d48e List of changes: 341eb53 Fixed sha1-based deduplication and added dedupe tests cdc9d97 Added jobqueue configuration 09d0338 Copy the correct config files into the container 283d689 Rerender mobile-sectionss and summary on MW purge and null edit. f82d46b Update node-rdkafka driver to version 2 xxx Update node module dependencies Change-Id: I81b62d1dc912c8ddad505b80d2920f2aeef2b36f --- M node_modules/.bin/rc M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js M node_modules/ajv/lib/ajv.d.ts M node_modules/ajv/package.json M node_modules/core-js/bower.json M node_modules/core-js/client/core.js M node_modules/core-js/client/core.min.js M node_modules/core-js/client/core.min.js.map M node_modules/core-js/client/library.js M node_modules/core-js/client/library.min.js M node_modules/core-js/client/library.min.js.map M node_modules/core-js/client/shim.js M node_modules/core-js/client/shim.min.js M node_modules/core-js/client/shim.min.js.map M node_modules/core-js/library/modules/_core.js M node_modules/core-js/library/modules/_iter-define.js M node_modules/core-js/library/modules/_microtask.js M node_modules/core-js/library/modules/_partial.js M node_modules/core-js/library/modules/_set-collection-of.js M node_modules/core-js/library/modules/_typed-buffer.js A node_modules/core-js/library/modules/_user-agent.js M node_modules/core-js/library/modules/es6.array.slice.js M node_modules/core-js/library/modules/es6.promise.js M node_modules/core-js/library/modules/es6.symbol.js M node_modules/core-js/library/modules/es7.observable.js M node_modules/core-js/library/modules/es7.string.pad-end.js M node_modules/core-js/library/modules/es7.string.pad-start.js M node_modules/core-js/library/modules/web.timers.js M node_modules/core-js/library/stage/2.js M node_modules/core-js/modules/_core.js M node_modules/core-js/modules/_iter-define.js M node_modules/core-js/modules/_microtask.js M node_modules/core-js/modules/_partial.js M node_modules/core-js/modules/_set-collection-of.js M node_modules/core-js/modules/_typed-buffer.js A node_modules/core-js/modules/_user-agent.js M node_modules/core-js/modules/es6.array.slice.js M node_modules/core-js/modules/es6.promise.js M node_modules/core-js/modules/es6.symbol.js M node_modules/core-js/modules/es7.observable.js M node_modules/core-js/modules/es7.string.pad-end.js M node_modules/core-js/modules/es7.string.pad-start.js M node_modules/core-js/modules/web.timers.js M node_modules/core-js/package.json M node_modules/core-js/stage/2.js M node_modules/dtrace-provider/.npmignore A node_modules/dtrace-provider/binding.gyp A node_modules/dtrace-provider/build.sh A node_modules/dtrace-provider/build/DTraceProviderStub.target.mk A node_modules/dtrace-provider/build/Makefile A node_modules/dtrace-provider/build/Release/.deps/Release/obj.target/DTraceProviderStub.stamp.d A node_modules/dtrace-provider/build/Release/obj.target/DTraceProviderStub.stamp A node_modules/dtrace-provider/build/binding.Makefile A node_modules/dtrace-provider/build/config.gypi D node_modules/dtrace-provider/compile.py M node_modules/dtrace-provider/dtrace-provider.js D node_modules/dtrace-provider/libusdt-arch.js D node_modules/dtrace-provider/libusdt-build.sh M node_modules/dtrace-provider/package.json D node_modules/dtrace-provider/scripts/install.js A node_modules/dtrace-provider/src/binding.gyp R node_modules/dtrace-provider/src/dtrace_argument.cc R node_modules/dtrace-provider/src/dtrace_probe.cc R node_modules/dtrace-provider/src/dtrace_provider.cc R node_modules/dtrace-provider/src/dtrace_provider.h A node_modules/dtrace-provider/suppress-error.js D node_modules/dtrace-provider/test/32probe-char.test.js D node_modules/dtrace-provider/test/32probe-char_fire.js D node_modules/dtrace-provider/test/32probe.test.js D node_modules/dtrace-provider/test/32probe_fire.js D node_modules/dtrace-provider/test/add-probes.test.js D node_modules/dtrace-provider/test/add-probes_fire.js D node_modules/dtrace-provider/test/args-after-cb-1.test.js D node_modules/dtrace-provider/test/args-after-cb-1_fire.js D node_modules/dtrace-provider/test/args-after-cb-2.test.js D node_modules/dtrace-provider/test/args-after-cb-2_fire.js D node_modules/dtrace-provider/test/basic.test.js D node_modules/dtrace-provider/test/basic_fire.js D node_modules/dtrace-provider/test/create-destroy.test.js D node_modules/dtrace-provider/test/create-destroy_fire.js D node_modules/dtrace-provider/test/disambiguation.test.js D node_modules/dtrace-provider/test/disambiguation_fire.js D node_modules/dtrace-provider/test/dtrace-test.js D node_modules/dtrace-provider/test/enabled-disabled.test.js D no
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update change-propagation to 214d48e
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/404881 ) Change subject: Update change-propagation to 214d48e .. Update change-propagation to 214d48e List of changes: 341eb53 Fixed sha1-based deduplication and added dedupe tests cdc9d97 Added jobqueue configuration 09d0338 Copy the correct config files into the container 283d689 Rerender mobile-sectionss and summary on MW purge and null edit. f82d46b Update node-rdkafka driver to version 2 xxx Update node module dependencies Change-Id: I81b62d1dc912c8ddad505b80d2920f2aeef2b36f --- M node_modules/.bin/rc M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js M node_modules/ajv/lib/ajv.d.ts M node_modules/ajv/package.json M node_modules/core-js/bower.json M node_modules/core-js/client/core.js M node_modules/core-js/client/core.min.js M node_modules/core-js/client/core.min.js.map M node_modules/core-js/client/library.js M node_modules/core-js/client/library.min.js M node_modules/core-js/client/library.min.js.map M node_modules/core-js/client/shim.js M node_modules/core-js/client/shim.min.js M node_modules/core-js/client/shim.min.js.map M node_modules/core-js/library/modules/_core.js M node_modules/core-js/library/modules/_iter-define.js M node_modules/core-js/library/modules/_microtask.js M node_modules/core-js/library/modules/_partial.js M node_modules/core-js/library/modules/_set-collection-of.js M node_modules/core-js/library/modules/_typed-buffer.js A node_modules/core-js/library/modules/_user-agent.js M node_modules/core-js/library/modules/es6.array.slice.js M node_modules/core-js/library/modules/es6.promise.js M node_modules/core-js/library/modules/es6.symbol.js M node_modules/core-js/library/modules/es7.observable.js M node_modules/core-js/library/modules/es7.string.pad-end.js M node_modules/core-js/library/modules/es7.string.pad-start.js M node_modules/core-js/library/modules/web.timers.js M node_modules/core-js/library/stage/2.js M node_modules/core-js/modules/_core.js M node_modules/core-js/modules/_iter-define.js M node_modules/core-js/modules/_microtask.js M node_modules/core-js/modules/_partial.js M node_modules/core-js/modules/_set-collection-of.js M node_modules/core-js/modules/_typed-buffer.js A node_modules/core-js/modules/_user-agent.js M node_modules/core-js/modules/es6.array.slice.js M node_modules/core-js/modules/es6.promise.js M node_modules/core-js/modules/es6.symbol.js M node_modules/core-js/modules/es7.observable.js M node_modules/core-js/modules/es7.string.pad-end.js M node_modules/core-js/modules/es7.string.pad-start.js M node_modules/core-js/modules/web.timers.js M node_modules/core-js/package.json M node_modules/core-js/stage/2.js M node_modules/dtrace-provider/.npmignore A node_modules/dtrace-provider/binding.gyp A node_modules/dtrace-provider/build.sh A node_modules/dtrace-provider/build/DTraceProviderStub.target.mk A node_modules/dtrace-provider/build/Makefile A node_modules/dtrace-provider/build/Release/.deps/Release/obj.target/DTraceProviderStub.stamp.d A node_modules/dtrace-provider/build/Release/obj.target/DTraceProviderStub.stamp A node_modules/dtrace-provider/build/binding.Makefile A node_modules/dtrace-provider/build/config.gypi D node_modules/dtrace-provider/compile.py M node_modules/dtrace-provider/dtrace-provider.js D node_modules/dtrace-provider/libusdt-arch.js D node_modules/dtrace-provider/libusdt-build.sh M node_modules/dtrace-provider/package.json D node_modules/dtrace-provider/scripts/install.js A node_modules/dtrace-provider/src/binding.gyp R node_modules/dtrace-provider/src/dtrace_argument.cc R node_modules/dtrace-provider/src/dtrace_probe.cc R node_modules/dtrace-provider/src/dtrace_provider.cc R node_modules/dtrace-provider/src/dtrace_provider.h A node_modules/dtrace-provider/suppress-error.js D node_modules/dtrace-provider/test/32probe-char.test.js D node_modules/dtrace-provider/test/32probe-char_fire.js D node_modules/dtrace-provider/test/32probe.test.js D node_modules/dtrace-provider/test/32probe_fire.js D node_modules/dtrace-provider/test/add-probes.test.js D node_modules/dtrace-provider/test/add-probes_fire.js D node_modules/dtrace-provider/test/args-after-cb-1.test.js D node_modules/dtrace-provider/test/args-after-cb-1_fire.js D node_modules/dtrace-provider/test/args-after-cb-2.test.js D node_modules/dtrace-provider/test/args-after-cb-2_fire.js D node_modules/dtrace-provider/test/basic.test.js D node_modules/dtrace-provider/test/basic_fire.js D node_modules/dtrace-provider/test/create-destroy.test.js D node_modules/dtrace-provider/test/create-destroy_fire.js D node_modules/dtrace-provider/test/disambiguation.test.js D node_modules/dtrace-provider/test/disambiguation_fire.js D node_modules/dtrace-provider/test/dtrace-test.js D node_modules/dtrace-provider/test/enabled-disabled.test.js D node_modul
[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: [JobQueue] Enable htmlCacheUpdate on new infrastructure for ...
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/404598 ) Change subject: [JobQueue] Enable htmlCacheUpdate on new infrastructure for all projects. .. [JobQueue] Enable htmlCacheUpdate on new infrastructure for all projects. Previously htmlCacheUpdate job was running on the new infrastructure for all projects except enwiki, commons and wikidats. This change switches processing of all the jobs to the new infrastructure. Bug: T182023 Depends-On: Ibbe8c11b4177f9671c6cd4e0681e9851c8b999e5 Change-Id: I8072c59453c69e959d1b07a1809d72b853012985 --- M wmf-config/InitialiseSettings.php M wmf-config/jobqueue.php 2 files changed, 5 insertions(+), 21 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config refs/changes/98/404598/1 diff --git a/wmf-config/InitialiseSettings.php b/wmf-config/InitialiseSettings.php index 9767eb5..649fa0e 100644 --- a/wmf-config/InitialiseSettings.php +++ b/wmf-config/InitialiseSettings.php @@ -19655,20 +19655,6 @@ 'votewiki' => false, ], -// Temporary code for gradually integrating htmlCacheUpdate -// jobs into the EventBus JobQueue - T182023 -// Step 2: enable everywhere but wikidata, enwiki and commons -'wmgDisableHTMLCacheUpdateInRedis' => [ - 'default' => true, - 'wikitech' => false, - 'private' => false, - 'loginwiki' => false, - 'votewiki' => false, - 'commonswiki' => false, - 'enwiki' => false, - 'wikidatawiki' => false, -], - 'wmgUseCapiunto' => [ 'default' => false, 'testwiki' => true, diff --git a/wmf-config/jobqueue.php b/wmf-config/jobqueue.php index a685ac9..29d6683 100644 --- a/wmf-config/jobqueue.php +++ b/wmf-config/jobqueue.php @@ -42,16 +42,14 @@ ]; if ( $wmgUseEventBus && $wmgDebugJobQueueEventBus ) { - $wgJobTypeConf['MessageIndexRebuildJob'] = + $wgJobTypeConf['deleteLinks'] = $wgJobTypeConf['flaggedrevs_CacheUpdate'] = - $wgJobTypeConf['deleteLinks'] = - $wgJobTypeConf['updateBetaFeaturesUserCounts'] = + $wgJobTypeConf['htmlCacheUpdate'] = + $wgJobTypeConf['MessageIndexRebuildJob'] = $wgJobTypeConf['RecordLintJob'] = - $wgJobTypeConf['wikibase-addUsagesForPage'] = + $wgJobTypeConf['updateBetaFeaturesUserCounts'] = + $wgJobTypeConf['wikibase-addUsagesForPage'] = [ 'class' => 'JobQueueEventBus' ]; - if ( $wmgDisableHTMLCacheUpdateInRedis ) { - $wgJobTypeConf['htmlCacheUpdate'] = [ 'class' => 'JobQueueEventBus' ]; - } $wgJobTypeConf['default'] = [ 'class' => 'JobQueueSecondTestQueue', 'mainqueue' => $jobQueueFederatedConfig, -- To view, visit https://gerrit.wikimedia.org/r/404598 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8072c59453c69e959d1b07a1809d72b853012985 Gerrit-PatchSet: 1 Gerrit-Project: operations/mediawiki-config Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Enable htmlCacheUpdate job processing for all wikis
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/404596 ) Change subject: [Config] Enable htmlCacheUpdate job processing for all wikis .. [Config] Enable htmlCacheUpdate job processing for all wikis Previously htmlCacheUpdate job was running on the new infrastructure for all projects except enwiki, commons and wikidats. This change switches processing of all the jobs to the new infrastructure. Bug: T182023 Change-Id: Ibbe8c11b4177f9671c6cd4e0681e9851c8b999e5 --- M scap/templates/config.yaml.j2 M scap/vars.yaml 2 files changed, 6 insertions(+), 18 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/96/404596/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index cf20ab3..5bd73bc 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -49,22 +49,6 @@ agentOptions: keepAlive: true <%- endfor %> -# Temporary section - we are only enabling the htmlCacheUpdate for all but enwiki, commons and wikidata -# Later this will be moved to the vars.yaml as all the other jobs. - htmlCacheUpdate: -topic: 'mediawiki.job.htmlCacheUpdate' -match_not: - meta: -domain: '/(?:^en\.wikipedia\.org$)|(?:^commons\.wikimedia\.org$)|(?:.*wikidata\.org$)/' -exec: - method: post - uri: '<%= jobrunner_uri %>' - headers: -content-type: 'application/json' - body: '{{globals.message}}' - agentOptions: -keepAlive: true -# End of temporary section <%- else %> job: topic: '/^mediawiki\.job\..*/' diff --git a/scap/vars.yaml b/scap/vars.yaml index bb782da..fefc489 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -21,11 +21,15 @@ kafka_max_bytes: 4194304 jobs_enabled: deleteLinks: {} - MessageIndexRebuildJob: {} flaggedrevs_CacheUpdate: {} - updateBetaFeaturesUserCounts: {} + htmlCacheUpdate: +# The avg exec time of the job is 60ms +# so this gives us 150 jobs/s - 3 times the avg rate +concurrency: 10 + MessageIndexRebuildJob: {} RecordLintJob: concurrency: 100 consumer_batch_size: 10 + updateBetaFeaturesUserCounts: {} wikibase-addUsagesForPage: {} -- To view, visit https://gerrit.wikimedia.org/r/404596 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibbe8c11b4177f9671c6cd4e0681e9851c8b999e5 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: [Config] Rerender mobile-sectionss and summary on MW purge a...
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/399692 ) Change subject: [Config] Rerender mobile-sectionss and summary on MW purge and null edit. .. [Config] Rerender mobile-sectionss and summary on MW purge and null edit. Sometimes due to software bugs MCS output could be wrong. If it's cached, the editors will naturally try to purge it with MW purge or a null edit. But since HTML might not have changed, the optimization in RESTBase that doesn't issue purges if HTML's unchanged will not be issued. This config change fixes the situation. Bug: T182953 Change-Id: I4580bfab99f4df6c3f27bb12e6383718579bf4b7 --- M scap/templates/config.yaml.j2 1 file changed, 46 insertions(+), 19 deletions(-) Approvals: Mobrovac: Looks good to me, but someone else must approve Ppchelko: Verified; Looks good to me, approved diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index bca8e38..db503d3 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -155,15 +155,29 @@ meta: domain: /\.wikidata\.org$/ exec: - method: get - # This even comes directly from MediaWiki, so title is encoded in MW-specific way. - # Re-encode the title in standard `encodeURIComponent` encoding. - uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{decode(match.meta.uri.title)}' - headers: -cache-control: no-cache -if-unmodified-since: '{{date(message.meta.dt)}}' - query: -redirect: false + - method: get +# This even comes directly from MediaWiki, so title is encoded in MW-specific way. +# Re-encode the title in standard `encodeURIComponent` encoding. +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache + if-unmodified-since: '{{date(message.meta.dt)}}' +query: + redirect: false +# The HTML might not change but sometimes editors use a purge to drop incorrectly rendered summary/MCS +# content, so let's purge them as well just in case. The rate is low. + - method: get +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/summary/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache +query: + redirect: false + - method: get +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/mobile-sections/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache +query: + redirect: false null_edit: topic: resource_change @@ -182,16 +196,29 @@ meta: domain: /\.wikidata\.org$/ exec: - method: get - # This even comes directly from MediaWiki, so title is encoded in MW-specific way. - # Re-encode the title in standard `encodeURIComponent` encoding. - uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{decode(match.meta.uri.title)}' - headers: -cache-control: no-cache -if-unmodified-since: '{{date(message.meta.dt)}}' - query: -redirect: false - + - method: get +# This even comes directly from MediaWiki, so title is encoded in MW-specific way. +# Re-encode the title in standard `encodeURIComponent` encoding. +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache + if-unmodified-since: '{{date(message.meta.dt)}}' +query: + redirect: false +# The HTML might not change but sometimes editors use a purge to drop incorrectly rendered summary/MCS +# content, so let's purge them as well just in case. The rate is low. + - method: get +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/summary/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache +query: + redirect: false + - method: get +
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: [Config] Rerender mobile-sectionss and summary on MW purge a...
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/399692 ) Change subject: [Config] Rerender mobile-sectionss and summary on MW purge and null edit. .. [Config] Rerender mobile-sectionss and summary on MW purge and null edit. Sometimes due to software bugs MCS output could be wrong. If it's cached, the editors will naturally try to purge it with MW purge or a null edit. But since HTML might not have changed, the optimization in RESTBase that doesn't issue purges if HTML's unchanged will not be issued. This config change fixes the situation. Bug: T182953 Change-Id: I4580bfab99f4df6c3f27bb12e6383718579bf4b7 --- M scap/templates/config.yaml.j2 1 file changed, 46 insertions(+), 19 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/deploy refs/changes/92/399692/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index bca8e38..db503d3 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -155,15 +155,29 @@ meta: domain: /\.wikidata\.org$/ exec: - method: get - # This even comes directly from MediaWiki, so title is encoded in MW-specific way. - # Re-encode the title in standard `encodeURIComponent` encoding. - uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{decode(match.meta.uri.title)}' - headers: -cache-control: no-cache -if-unmodified-since: '{{date(message.meta.dt)}}' - query: -redirect: false + - method: get +# This even comes directly from MediaWiki, so title is encoded in MW-specific way. +# Re-encode the title in standard `encodeURIComponent` encoding. +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache + if-unmodified-since: '{{date(message.meta.dt)}}' +query: + redirect: false +# The HTML might not change but sometimes editors use a purge to drop incorrectly rendered summary/MCS +# content, so let's purge them as well just in case. The rate is low. + - method: get +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/summary/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache +query: + redirect: false + - method: get +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/mobile-sections/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache +query: + redirect: false null_edit: topic: resource_change @@ -182,16 +196,29 @@ meta: domain: /\.wikidata\.org$/ exec: - method: get - # This even comes directly from MediaWiki, so title is encoded in MW-specific way. - # Re-encode the title in standard `encodeURIComponent` encoding. - uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{decode(match.meta.uri.title)}' - headers: -cache-control: no-cache -if-unmodified-since: '{{date(message.meta.dt)}}' - query: -redirect: false - + - method: get +# This even comes directly from MediaWiki, so title is encoded in MW-specific way. +# Re-encode the title in standard `encodeURIComponent` encoding. +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache + if-unmodified-since: '{{date(message.meta.dt)}}' +query: + redirect: false +# The HTML might not change but sometimes editors use a purge to drop incorrectly rendered summary/MCS +# content, so let's purge them as well just in case. The rate is low. + - method: get +uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/summary/{decode(match.meta.uri.title)}' +headers: + cache-control: no-cache +query: + redirect: false + - method: get +uri: '<%= r
[MediaWiki-commits] [Gerrit] maps...deploy[master]: Update kartotherian to 29a680f
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/398315 ) Change subject: Update kartotherian to 29a680f .. Update kartotherian to 29a680f List of changes: xxx Update node module dependencies Change-Id: I465e26ec2d7ced603e53e73ca309f55fce66f1e3 --- M node_modules/@kartotherian/autogen/package.json M node_modules/@kartotherian/babel/package.json M node_modules/@kartotherian/cassandra/package.json M node_modules/@kartotherian/core/package.json M node_modules/@kartotherian/demultiplexer/package.json M node_modules/@kartotherian/err/package.json M node_modules/@kartotherian/geoshapes/package.json M node_modules/@kartotherian/input-validator/package.json M node_modules/@kartotherian/layermixer/package.json M node_modules/@kartotherian/maki/package.json M node_modules/@kartotherian/module-loader/package.json M node_modules/@kartotherian/osm-bright-source/package.json M node_modules/@kartotherian/osm-bright-style/package.json M node_modules/@kartotherian/overzoom/package.json M node_modules/@kartotherian/postgres/package.json M node_modules/@kartotherian/server/package.json M node_modules/@kartotherian/snapshot/package.json M node_modules/@kartotherian/substantial/package.json M node_modules/@kartotherian/tilelive-http/node_modules/semver/package.json M node_modules/@kartotherian/tilelive-http/package.json M node_modules/abab/package.json M node_modules/abaculus/package.json M node_modules/abbrev/package.json M node_modules/accepts/package.json M node_modules/acorn-globals/package.json M node_modules/acorn/package.json M node_modules/agentkeepalive/package.json M node_modules/ajv/package.json M node_modules/ansi-regex/package.json M node_modules/ansi-styles/package.json M node_modules/aproba/package.json M node_modules/are-we-there-yet/package.json M node_modules/argparse/package.json M node_modules/array-equal/package.json M node_modules/array-flatten/package.json M node_modules/asap/package.json M node_modules/asn1/package.json M node_modules/assert-plus/package.json M node_modules/async/package.json M node_modules/asynckit/package.json M node_modules/aws-sdk/node_modules/sax/package.json M node_modules/aws-sdk/package.json M node_modules/aws-sign2/package.json M node_modules/aws4/package.json M node_modules/balanced-match/package.json M node_modules/base64-js/package.json M node_modules/bcrypt-pbkdf/package.json M node_modules/bl/package.json M node_modules/blend/package.json M node_modules/block-stream/package.json M node_modules/bluebird/package.json M node_modules/body-parser/package.json M node_modules/boom/package.json M node_modules/brace-expansion/package.json M node_modules/buffer-writer/package.json M node_modules/buffer/package.json M node_modules/builtin-modules/package.json M node_modules/bunyan-prettystream/package.json M node_modules/bunyan-syslog-udp/package.json M node_modules/bunyan/package.json M node_modules/bytes/package.json M node_modules/camelcase/package.json M node_modules/canvas/build/Makefile M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/Canvas.o.d M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/CanvasGradient.o.d M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/CanvasPattern.o.d M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/CanvasRenderingContext2d.o.d M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/FontFace.o.d M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/Image.o.d M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/ImageData.o.d M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/color.o.d M node_modules/canvas/build/Release/.deps/Release/obj.target/canvas/src/init.o.d M node_modules/canvas/build/canvas.target.mk M node_modules/canvas/build/config.gypi M node_modules/canvas/package.json M node_modules/caseless/package.json M node_modules/cassandra-driver/package.json M node_modules/cassandra-uuid/package.json M node_modules/chalk/package.json M node_modules/clarinet/package.json M node_modules/cliui/package.json M node_modules/co/package.json M node_modules/code-point-at/package.json M node_modules/colors/package.json M node_modules/combined-stream/package.json M node_modules/commander/package.json M node_modules/compressible/package.json M node_modules/compression/package.json M node_modules/concat-map/package.json M node_modules/console-control-strings/package.json M node_modules/content-disposition/package.json M node_modules/content-type-parser/package.json M node_modules/content-type/package.json M node_modules/cookie-signature/package.json M node_modules/cookie/package.json M node_modules/core-util-is/package.json M node_modules/cryptiles/node_modules/boom/package.json M node_modules/cryptiles/package.json M node_modules/crypto-browserify/package.json M node_modules/css-font
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 341eb53
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/398142 ) Change subject: Update change-propagation to 341eb53 .. Update change-propagation to 341eb53 List of changes: 341eb53 Fixed sha1-based deduplication and added dedupe tests Change-Id: Idb06ff2a9becacbf1f8bd760b2d57a5185cf42dd --- M src 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/src b/src index 929bf07..341eb53 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit 929bf07e8e13e6afce42388bdd812f4592806481 +Subproject commit 341eb536f5565c2f6bf52ad81b3d5f1c967b0dc8 -- To view, visit https://gerrit.wikimedia.org/r/398142 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Idb06ff2a9becacbf1f8bd760b2d57a5185cf42dd Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 341eb53
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/398142 ) Change subject: Update change-propagation to 341eb53 .. Update change-propagation to 341eb53 List of changes: 341eb53 Fixed sha1-based deduplication and added dedupe tests Change-Id: Idb06ff2a9becacbf1f8bd760b2d57a5185cf42dd --- M src 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/42/398142/1 diff --git a/src b/src index 929bf07..341eb53 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit 929bf07e8e13e6afce42388bdd812f4592806481 +Subproject commit 341eb536f5565c2f6bf52ad81b3d5f1c967b0dc8 -- To view, visit https://gerrit.wikimedia.org/r/398142 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idb06ff2a9becacbf1f8bd760b2d57a5185cf42dd Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update restbase to 3d1130d
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/398123 ) Change subject: Update restbase to 3d1130d .. Update restbase to 3d1130d List of changes: 6c74b8f Only use fallback storage if the specific revision is requested. 1b75ae2 Fixed the syntax error 3d1130d Actually delete files for old parsoid impl and a proxy Change-Id: I0b1b87e6de928063daaf92063560688b5f58ec50 --- M restbase 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy refs/changes/23/398123/1 diff --git a/restbase b/restbase index 51316b7..3d1130d 16 --- a/restbase +++ b/restbase @@ -1 +1 @@ -Subproject commit 51316b73a574e507f62861f9de2ec2a586b6ce6b +Subproject commit 3d1130d080541b8eafaa087cb8bef2d9d718e774 -- To view, visit https://gerrit.wikimedia.org/r/398123 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0b1b87e6de928063daaf92063560688b5f58ec50 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update restbase to 51316b7
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/398102 ) Change subject: Update restbase to 51316b7 .. Update restbase to 51316b7 List of changes: 51316b7 Switch Parsoid to the Cassandra 3 storage. Change-Id: I411e6499d98a94336edcb69ec39864571d91dcfd --- M restbase 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy refs/changes/02/398102/1 diff --git a/restbase b/restbase index a70a130..51316b7 16 --- a/restbase +++ b/restbase @@ -1 +1 @@ -Subproject commit a70a130221378ea7c4cb100b5d0d86c456797fef +Subproject commit 51316b73a574e507f62861f9de2ec2a586b6ce6b -- To view, visit https://gerrit.wikimedia.org/r/398102 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I411e6499d98a94336edcb69ec39864571d91dcfd Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: [Config] Remove new_storage_enabled_parsoid config stanza
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/398101 ) Change subject: [Config] Remove new_storage_enabled_parsoid config stanza .. [Config] Remove new_storage_enabled_parsoid config stanza We've switched everything to use the new storage for Parsoid content, so the configuration is not needed any more. Change-Id: I8ebeef4743370b83683bb61b13ee9d98671fd90e --- M scap/environments/beta/vars.yaml M scap/templates/config.yaml.j2 M scap/vars.yaml 3 files changed, 0 insertions(+), 9 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy refs/changes/01/398101/1 diff --git a/scap/environments/beta/vars.yaml b/scap/environments/beta/vars.yaml index 469ce1b..c81c54a 100644 --- a/scap/environments/beta/vars.yaml +++ b/scap/environments/beta/vars.yaml @@ -5,8 +5,6 @@ storage_groups: [] storage_groups_ng: [] cassandra_default_consistency: localOne -new_storage_enabled_parsoid: - - '/.*/' summary_impl: mcs domains: BetaCluster: diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index d6e8caa..0896791 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -100,11 +100,6 @@ host: <%= parsoid_uri %> grace_ttl: 86400 delete_probability: <%= delete_probability %> - <% if new_storage_enabled_parsoid is not none and new_storage_enabled_parsoid|length > 0 %>backends: -new:<% for item in new_storage_enabled_parsoid %> - - '<%= item %>' -<% endfor -%> -<% endif %> action: apiUriTemplate: <%= mwapi_uri %> baseUriTemplate: "{{'https://{domain}/api/rest_v1'}}" diff --git a/scap/vars.yaml b/scap/vars.yaml index 10a3751..36c08f2 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -32,8 +32,6 @@ cxserver_uri: https://cxserver-beta.wmflabs.org aqs_uri: https://wikimedia.org/api/rest_v1/metrics delete_probability: 0.25 -new_storage_enabled_parsoid: - - /.*/ port: 7231 salt_key: '123456' page_size: 10 -- To view, visit https://gerrit.wikimedia.org/r/398101 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8ebeef4743370b83683bb61b13ee9d98671fd90e Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Set correct request ID according to MediaWiki
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/397710 ) Change subject: Set correct request ID according to MediaWiki .. Set correct request ID according to MediaWiki Change-Id: Ib465d990c10a3d38ffdee15317f3364439a2db69 Depends-On: Ieab17db6f7ed1a7d37ee801d581791a240bbe74b --- M EventBus.php 1 file changed, 1 insertion(+), 13 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/10/397710/1 diff --git a/EventBus.php b/EventBus.php index 05b407b..097c7db 100644 --- a/EventBus.php +++ b/EventBus.php @@ -316,25 +316,13 @@ 'meta' => [ 'uri'=> $uri, 'topic' => $topic, - 'request_id' => self::getRequestId(), + 'request_id' => WebRequest::getRequestId(), 'id' => self::newId(), 'dt' => date( 'c' ), 'domain' => $wgServerName ?: "unknown", ], ]; return $event + $attrs; - } - - /** -* Returns the X-Request-ID header, if set, otherwise a newly generated -* type 4 UUID string. -* -* @return string -*/ - private static function getRequestId() { - $context = RequestContext::getMain(); - $xreqid = $context->getRequest()->getHeader( 'x-request-id' ); - return $xreqid ?: UIDGenerator::newUUIDv4(); } /** -- To view, visit https://gerrit.wikimedia.org/r/397710 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib465d990c10a3d38ffdee15317f3364439a2db69 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki/event-schemas[master]: Temporary removed 'pattern' for request_id from schemas.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/397709 ) Change subject: Temporary removed 'pattern' for request_id from schemas. .. Temporary removed 'pattern' for request_id from schemas. In MediaWiki the request_id is actually a 26-symbols-long random string obtained with WebRequest::getRequestID(), however EventBus for some reason uses some custom method of obtaining the request ID. However, seems like that metod always fails and defaults to a v1 UUID - including that doesn't have anything in common with the actual MediaWiki request ID. As MediaWiki request ID will have different pattern, we need to first remove the pattern property, deploy code changes, and then reintroduce the pattern. Change-Id: Ieab17db6f7ed1a7d37ee801d581791a240bbe74b --- M jsonschema/change-prop/continue/1.yaml M jsonschema/change-prop/retry/1.yaml M jsonschema/error/1.yaml M jsonschema/mediawiki/job/1.yaml M jsonschema/mediawiki/page/delete/2.yaml M jsonschema/mediawiki/page/move/1.yaml M jsonschema/mediawiki/page/move/2.yaml M jsonschema/mediawiki/page/properties-change/1.yaml M jsonschema/mediawiki/page/restrictions-change/1.yaml M jsonschema/mediawiki/page/undelete/2.yaml M jsonschema/mediawiki/recentchange/1.yaml M jsonschema/mediawiki/revision/create/2.yaml M jsonschema/mediawiki/revision/visibility-change/1.yaml M jsonschema/mediawiki/user/blocks-change/1.yaml M jsonschema/resource_change/1.yaml M jsonschema/test/event/1.yaml M test/jsonschema/index.js M test/jsonschema/meta_common_schema.yaml 18 files changed, 18 insertions(+), 33 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/event-schemas refs/changes/09/397709/1 diff --git a/jsonschema/change-prop/continue/1.yaml b/jsonschema/change-prop/continue/1.yaml index ae5f67c..7566d49 100644 --- a/jsonschema/change-prop/continue/1.yaml +++ b/jsonschema/change-prop/continue/1.yaml @@ -23,8 +23,7 @@ description: the unique URI identifying the event / resource request_id: type: string -pattern: '^[a-fA-F0-9]{8}(-[a-fA-F0-9]{4}){3}-[a-fA-F0-9]{12}$' -description: the unique UUID v1 ID of the event derived from the X-Request-Id header +description: The unique ID of the request that caused the event derived from the X-Request-Id header. id: type: string pattern: '^[a-fA-F0-9]{8}(-[a-fA-F0-9]{4}){3}-[a-fA-F0-9]{12}$' diff --git a/jsonschema/change-prop/retry/1.yaml b/jsonschema/change-prop/retry/1.yaml index 1d0be6b..caed5f4 100644 --- a/jsonschema/change-prop/retry/1.yaml +++ b/jsonschema/change-prop/retry/1.yaml @@ -23,8 +23,7 @@ description: the unique URI identifying the event / resource request_id: type: string -pattern: '^[a-fA-F0-9]{8}(-[a-fA-F0-9]{4}){3}-[a-fA-F0-9]{12}$' -description: the unique UUID v1 ID of the event derived from the X-Request-Id header +description: The unique ID of the request that caused the event derived from the X-Request-Id header. id: type: string pattern: '^[a-fA-F0-9]{8}(-[a-fA-F0-9]{4}){3}-[a-fA-F0-9]{12}$' diff --git a/jsonschema/error/1.yaml b/jsonschema/error/1.yaml index 5734d0a..2e9bec0 100644 --- a/jsonschema/error/1.yaml +++ b/jsonschema/error/1.yaml @@ -23,8 +23,7 @@ description: the unique URI identifying the event / resource request_id: type: string -pattern: '^[a-fA-F0-9]{8}(-[a-fA-F0-9]{4}){3}-[a-fA-F0-9]{12}$' -description: the unique UUID v1 ID of the event derived from the X-Request-Id header +description: The unique ID of the request that caused the event derived from the X-Request-Id header. id: type: string pattern: '^[a-fA-F0-9]{8}(-[a-fA-F0-9]{4}){3}-[a-fA-F0-9]{12}$' diff --git a/jsonschema/mediawiki/job/1.yaml b/jsonschema/mediawiki/job/1.yaml index 1517323..e9ab20b 100644 --- a/jsonschema/mediawiki/job/1.yaml +++ b/jsonschema/mediawiki/job/1.yaml @@ -23,7 +23,7 @@ type: string format: uri request_id: -description: The unique UUID v1 ID of the event derived from the X-Request-Id header. +description: the unique ID of the request that caused the event derived from the X-Request-Id header. type: string pattern: '^[a-zA-Z0-9\-@]+$' id: diff --git a/jsonschema/mediawiki/page/delete/2.yaml b/jsonschema/mediawiki/page/delete/2.yaml index 65e35e8..17affdc 100644 --- a/jsonschema/mediawiki/page/delete/2.yaml +++ b/jsonschema/mediawiki/page/delete/2.yaml @@ -22,9 +22,8 @@ type: string format: uri request_id: -description: The unique UUID v1 ID of the event derived from the X-Request-Id header. +description: the unique ID of the request that caused the event derived from the X-Request-Id header. type: string -pattern: '^[a-fA-F0-9]{8}(-[a-fA-F0-9]{4}){3}-[
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update restbase to 289c64a
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/397706 ) Change subject: Update restbase to 289c64a .. Update restbase to 289c64a List of changes: 43f0856 Fix failure if there's no content-type returned Change-Id: I872ffc2d164f1ddf84cf860a5306203d87db7927 --- M restbase 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/restbase b/restbase index ef54e7b..289c64a 16 --- a/restbase +++ b/restbase @@ -1 +1 @@ -Subproject commit ef54e7b54954535152aaf4782df840c9310764f4 +Subproject commit 289c64aed98b6e90d4364d518c22fc683dba9ac9 -- To view, visit https://gerrit.wikimedia.org/r/397706 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I872ffc2d164f1ddf84cf860a5306203d87db7927 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update restbase to 289c64a
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/397706 ) Change subject: Update restbase to 289c64a .. Update restbase to 289c64a List of changes: 43f0856 Fix failure if there's no content-type returned Change-Id: I872ffc2d164f1ddf84cf860a5306203d87db7927 --- M restbase 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy refs/changes/06/397706/1 diff --git a/restbase b/restbase index ef54e7b..289c64a 16 --- a/restbase +++ b/restbase @@ -1 +1 @@ -Subproject commit ef54e7b54954535152aaf4782df840c9310764f4 +Subproject commit 289c64aed98b6e90d4364d518c22fc683dba9ac9 -- To view, visit https://gerrit.wikimedia.org/r/397706 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I872ffc2d164f1ddf84cf860a5306203d87db7927 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 929bf07
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/397613 ) Change subject: Update change-propagation to 929bf07 .. Update change-propagation to 929bf07 List of changes: 929bf07 Revert "Dedupe based on sha1 as well as on event ID" Change-Id: I521a1f733132782648c7bf59ea2da7201c0fbef5 --- M src 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/src b/src index a6fe93d..929bf07 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit a6fe93dcb82322e458d70d7dbc1b2945ae16c464 +Subproject commit 929bf07e8e13e6afce42388bdd812f4592806481 -- To view, visit https://gerrit.wikimedia.org/r/397613 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I521a1f733132782648c7bf59ea2da7201c0fbef5 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 929bf07
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/397613 ) Change subject: Update change-propagation to 929bf07 .. Update change-propagation to 929bf07 List of changes: 929bf07 Revert "Dedupe based on sha1 as well as on event ID" Change-Id: I521a1f733132782648c7bf59ea2da7201c0fbef5 --- M src 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/13/397613/1 diff --git a/src b/src index a6fe93d..929bf07 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit a6fe93dcb82322e458d70d7dbc1b2945ae16c464 +Subproject commit 929bf07e8e13e6afce42388bdd812f4592806481 -- To view, visit https://gerrit.wikimedia.org/r/397613 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I521a1f733132782648c7bf59ea2da7201c0fbef5 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update change-propagation to 929bf07
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/397611 ) Change subject: Update change-propagation to 929bf07 .. Update change-propagation to 929bf07 List of changes: 929bf07 Revert "Dedupe based on sha1 as well as on event ID" Change-Id: I8ce9d514f20aa903efb9c21b5b122cd033036e21 --- M src 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/src b/src index a6fe93d..929bf07 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit a6fe93dcb82322e458d70d7dbc1b2945ae16c464 +Subproject commit 929bf07e8e13e6afce42388bdd812f4592806481 -- To view, visit https://gerrit.wikimedia.org/r/397611 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8ce9d514f20aa903efb9c21b5b122cd033036e21 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update change-propagation to 929bf07
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/397611 ) Change subject: Update change-propagation to 929bf07 .. Update change-propagation to 929bf07 List of changes: 929bf07 Revert "Dedupe based on sha1 as well as on event ID" Change-Id: I8ce9d514f20aa903efb9c21b5b122cd033036e21 --- M src 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/deploy refs/changes/11/397611/1 diff --git a/src b/src index a6fe93d..929bf07 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit a6fe93dcb82322e458d70d7dbc1b2945ae16c464 +Subproject commit 929bf07e8e13e6afce42388bdd812f4592806481 -- To view, visit https://gerrit.wikimedia.org/r/397611 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8ce9d514f20aa903efb9c21b5b122cd033036e21 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Enable htmlCacheUpdate for ceb and ru wiki plus all...
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/397585 ) Change subject: [Config] Enable htmlCacheUpdate for ceb and ru wiki plus all small projects .. [Config] Enable htmlCacheUpdate for ceb and ru wiki plus all small projects Bug: T182023 Depends-On: Ie2f13141f5766801d3d3169a70b976b44a177830 Change-Id: I3af0e253299e6fa6126edb9ed8901a6164d270ff --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/85/397585/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index a1a92d2..7b8140a 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -55,7 +55,7 @@ topic: 'mediawiki.job.htmlCacheUpdate' match: meta: -domain: '/.*\.wiktionary\.org$/' +domain: '/(?:^ceb\.wikipedia\.org$)|(?:^ru\.wikipedia\.org$)|(?:.*\.wikibooks\.org$)|(?:.*\.wikinews\.org$)|(?:.*\.wikiquote\.org$)|(?:.*\.wikisource\.org$)|(?:.*\.wikiversity\.org$)|(?:.*\.wikivoyage\.org$)|(?:.*\.wiktionary\.org$)/' exec: method: post uri: '<%= jobrunner_uri %>' -- To view, visit https://gerrit.wikimedia.org/r/397585 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3af0e253299e6fa6126edb9ed8901a6164d270ff Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Disable Redis queue for small projects and ru and ceb wiki
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/397581 ) Change subject: Disable Redis queue for small projects and ru and ceb wiki .. Disable Redis queue for small projects and ru and ceb wiki Bug: T182023 Change-Id: Ie2f13141f5766801d3d3169a70b976b44a177830 --- M wmf-config/InitialiseSettings.php 1 file changed, 8 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config refs/changes/81/397581/1 diff --git a/wmf-config/InitialiseSettings.php b/wmf-config/InitialiseSettings.php index 986f4bb..6297ed6 100644 --- a/wmf-config/InitialiseSettings.php +++ b/wmf-config/InitialiseSettings.php @@ -19640,6 +19640,14 @@ // on the EventBus JobQueue -disable old queue on wiktionaries only 'wmgDisableHTMLCacheUpdateInRedis' => [ 'default' => false, + 'cebwiki' => true, + 'ruwiki' => true, + 'wikibooks' => true, + 'wikinews' => true, + 'wikiquote' => true, + 'wikisource' => true, + 'wikiversity' => true, + 'wikivoyage' => true, 'wiktionary' => true, ], -- To view, visit https://gerrit.wikimedia.org/r/397581 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie2f13141f5766801d3d3169a70b976b44a177830 Gerrit-PatchSet: 1 Gerrit-Project: operations/mediawiki-config Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: [Config] Enable new storage for Parsoid for all domains
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/396473 ) Change subject: [Config] Enable new storage for Parsoid for all domains .. [Config] Enable new storage for Parsoid for all domains Bug: T179417 Change-Id: I0f2b15c7bd9f3a4378d63b53d0bedf3df8f84fe9 --- M scap/vars.yaml 1 file changed, 1 insertion(+), 10 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy refs/changes/73/396473/1 diff --git a/scap/vars.yaml b/scap/vars.yaml index 5081458..167257e 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -32,16 +32,7 @@ cxserver_uri: https://cxserver-beta.wmflabs.org aqs_uri: https://wikimedia.org/api/rest_v1/metrics new_storage_enabled_parsoid: - - /^(?:test.*\.wiki.*\.org|www.mediawiki.org)$/ - - /\.wikimedia\.org$/ - - /\.wiktionary\.org$/ - - /\.wikibooks\.org$/ - - /\.wikisource\.org$/ - - /\.wikiquote\.org$/ - - /\.wikinews\.org$/ - - /\.wikiversity\.org$/ - - /\.wikivoyage\.org$/ - - /\.wikidata\.org$/ + - /.*/ port: 7231 salt_key: '123456' page_size: 10 -- To view, visit https://gerrit.wikimedia.org/r/396473 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0f2b15c7bd9f3a4378d63b53d0bedf3df8f84fe9 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update change-propagation to a6fe93d
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/396087 ) Change subject: Update change-propagation to a6fe93d .. Update change-propagation to a6fe93d List of changes: 0c20aff Switch example jobqueue config to the special page and update the tests 65618a2 Log error stack when retry count is exceeded 15c62e1 Try to speed up ChangeProp e636f82 Optimise commited offset choosing 1a8de23 Update commit offset choosing to try starting with finishedMsg 24d675e Fixed eslint-related dependencies e20fb17 Fixed unhandledRejection in metadata refresh 62f7bf2 Dedupe based on sha1 as well as on event ID be193eb Separate individual delay metric from the total chain delay metrics a6fe93d Updated restbase example config xxx Update node module dependencies Change-Id: I3228bafad42d114cbc9f29d584110a3ee7ffe12e --- A node_modules/.bin/node-pre-gyp A node_modules/.bin/nopt A node_modules/.bin/rc A node_modules/abbrev/LICENSE A node_modules/abbrev/abbrev.js A node_modules/abbrev/package.json M node_modules/ajv/.tonic_example.js M node_modules/ajv/dist/ajv.bundle.js M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js A node_modules/ajv/lib/$data.js A node_modules/ajv/lib/ajv.d.ts M node_modules/ajv/lib/ajv.js D node_modules/ajv/lib/async.js M node_modules/ajv/lib/compile/_rules.js A node_modules/ajv/lib/compile/async.js M node_modules/ajv/lib/compile/equal.js A node_modules/ajv/lib/compile/error_classes.js M node_modules/ajv/lib/compile/formats.js M node_modules/ajv/lib/compile/index.js M node_modules/ajv/lib/compile/resolve.js M node_modules/ajv/lib/compile/rules.js R node_modules/ajv/lib/compile/ucs2length.js M node_modules/ajv/lib/compile/util.js D node_modules/ajv/lib/compile/validation_error.js M node_modules/ajv/lib/dot/_limit.jst M node_modules/ajv/lib/dot/allOf.jst M node_modules/ajv/lib/dot/anyOf.jst M node_modules/ajv/lib/dot/coerce.def A node_modules/ajv/lib/dot/const.jst A node_modules/ajv/lib/dot/contains.jst D node_modules/ajv/lib/dot/custom.def A node_modules/ajv/lib/dot/custom.jst M node_modules/ajv/lib/dot/defaults.def M node_modules/ajv/lib/dot/definitions.def M node_modules/ajv/lib/dot/dependencies.jst M node_modules/ajv/lib/dot/enum.jst M node_modules/ajv/lib/dot/errors.def M node_modules/ajv/lib/dot/format.jst M node_modules/ajv/lib/dot/items.jst M node_modules/ajv/lib/dot/missing.def M node_modules/ajv/lib/dot/not.jst M node_modules/ajv/lib/dot/oneOf.jst M node_modules/ajv/lib/dot/properties.jst A node_modules/ajv/lib/dot/propertyNames.jst M node_modules/ajv/lib/dot/ref.jst M node_modules/ajv/lib/dot/required.jst D node_modules/ajv/lib/dot/v5/_formatLimit.jst D node_modules/ajv/lib/dot/v5/constant.jst D node_modules/ajv/lib/dot/v5/patternRequired.jst D node_modules/ajv/lib/dot/v5/switch.jst M node_modules/ajv/lib/dot/validate.jst D node_modules/ajv/lib/dotjs/_formatLimit.js M node_modules/ajv/lib/dotjs/_limit.js M node_modules/ajv/lib/dotjs/_limitItems.js M node_modules/ajv/lib/dotjs/_limitLength.js M node_modules/ajv/lib/dotjs/_limitProperties.js M node_modules/ajv/lib/dotjs/allOf.js M node_modules/ajv/lib/dotjs/anyOf.js A node_modules/ajv/lib/dotjs/const.js D node_modules/ajv/lib/dotjs/constant.js A node_modules/ajv/lib/dotjs/contains.js A node_modules/ajv/lib/dotjs/custom.js M node_modules/ajv/lib/dotjs/dependencies.js M node_modules/ajv/lib/dotjs/enum.js M node_modules/ajv/lib/dotjs/format.js M node_modules/ajv/lib/dotjs/items.js M node_modules/ajv/lib/dotjs/multipleOf.js M node_modules/ajv/lib/dotjs/not.js M node_modules/ajv/lib/dotjs/oneOf.js M node_modules/ajv/lib/dotjs/pattern.js D node_modules/ajv/lib/dotjs/patternRequired.js M node_modules/ajv/lib/dotjs/properties.js A node_modules/ajv/lib/dotjs/propertyNames.js M node_modules/ajv/lib/dotjs/ref.js M node_modules/ajv/lib/dotjs/required.js D node_modules/ajv/lib/dotjs/switch.js M node_modules/ajv/lib/dotjs/uniqueItems.js M node_modules/ajv/lib/dotjs/validate.js M node_modules/ajv/lib/keyword.js A node_modules/ajv/lib/patternGroups.js A node_modules/ajv/lib/refs/$data.json A node_modules/ajv/lib/refs/json-schema-draft-06.json M node_modules/ajv/lib/refs/json-schema-v5.json D node_modules/ajv/lib/v5.js M node_modules/ajv/package.json R node_modules/ajv/scripts/.eslintrc.yml A node_modules/ajv/scripts/bundle.js A node_modules/ajv/scripts/compile-dots.js R node_modules/ajv/scripts/info R node_modules/ajv/scripts/prepare-tests R node_modules/ajv/scripts/travis-gh-pages A node_modules/aproba/LICENSE A node_modules/aproba/index.js A node_modules/aproba/package.json A node_modules/are-we-there-yet/CHANGES.md~ A node_modules/are-we-there-yet/LICENSE A node_modules/are-we-there-yet/index.js A node_modules/are-we-there-yet/node_modules/isarray/.npmignore A node_modules/are-we-there-yet/node_modules/isarray/.travis.yml A node_modules/are-we-there-y
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update change-propagation to a6fe93d
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/396087 ) Change subject: Update change-propagation to a6fe93d .. Update change-propagation to a6fe93d List of changes: 0c20aff Switch example jobqueue config to the special page and update the tests 65618a2 Log error stack when retry count is exceeded 15c62e1 Try to speed up ChangeProp e636f82 Optimise commited offset choosing 1a8de23 Update commit offset choosing to try starting with finishedMsg 24d675e Fixed eslint-related dependencies e20fb17 Fixed unhandledRejection in metadata refresh 62f7bf2 Dedupe based on sha1 as well as on event ID be193eb Separate individual delay metric from the total chain delay metrics a6fe93d Updated restbase example config xxx Update node module dependencies Change-Id: I3228bafad42d114cbc9f29d584110a3ee7ffe12e --- A node_modules/.bin/node-pre-gyp A node_modules/.bin/nopt A node_modules/.bin/rc A node_modules/abbrev/LICENSE A node_modules/abbrev/abbrev.js A node_modules/abbrev/package.json M node_modules/ajv/.tonic_example.js M node_modules/ajv/dist/ajv.bundle.js M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js A node_modules/ajv/lib/$data.js A node_modules/ajv/lib/ajv.d.ts M node_modules/ajv/lib/ajv.js D node_modules/ajv/lib/async.js M node_modules/ajv/lib/compile/_rules.js A node_modules/ajv/lib/compile/async.js M node_modules/ajv/lib/compile/equal.js A node_modules/ajv/lib/compile/error_classes.js M node_modules/ajv/lib/compile/formats.js M node_modules/ajv/lib/compile/index.js M node_modules/ajv/lib/compile/resolve.js M node_modules/ajv/lib/compile/rules.js R node_modules/ajv/lib/compile/ucs2length.js M node_modules/ajv/lib/compile/util.js D node_modules/ajv/lib/compile/validation_error.js M node_modules/ajv/lib/dot/_limit.jst M node_modules/ajv/lib/dot/allOf.jst M node_modules/ajv/lib/dot/anyOf.jst M node_modules/ajv/lib/dot/coerce.def A node_modules/ajv/lib/dot/const.jst A node_modules/ajv/lib/dot/contains.jst D node_modules/ajv/lib/dot/custom.def A node_modules/ajv/lib/dot/custom.jst M node_modules/ajv/lib/dot/defaults.def M node_modules/ajv/lib/dot/definitions.def M node_modules/ajv/lib/dot/dependencies.jst M node_modules/ajv/lib/dot/enum.jst M node_modules/ajv/lib/dot/errors.def M node_modules/ajv/lib/dot/format.jst M node_modules/ajv/lib/dot/items.jst M node_modules/ajv/lib/dot/missing.def M node_modules/ajv/lib/dot/not.jst M node_modules/ajv/lib/dot/oneOf.jst M node_modules/ajv/lib/dot/properties.jst A node_modules/ajv/lib/dot/propertyNames.jst M node_modules/ajv/lib/dot/ref.jst M node_modules/ajv/lib/dot/required.jst D node_modules/ajv/lib/dot/v5/_formatLimit.jst D node_modules/ajv/lib/dot/v5/constant.jst D node_modules/ajv/lib/dot/v5/patternRequired.jst D node_modules/ajv/lib/dot/v5/switch.jst M node_modules/ajv/lib/dot/validate.jst D node_modules/ajv/lib/dotjs/_formatLimit.js M node_modules/ajv/lib/dotjs/_limit.js M node_modules/ajv/lib/dotjs/_limitItems.js M node_modules/ajv/lib/dotjs/_limitLength.js M node_modules/ajv/lib/dotjs/_limitProperties.js M node_modules/ajv/lib/dotjs/allOf.js M node_modules/ajv/lib/dotjs/anyOf.js A node_modules/ajv/lib/dotjs/const.js D node_modules/ajv/lib/dotjs/constant.js A node_modules/ajv/lib/dotjs/contains.js A node_modules/ajv/lib/dotjs/custom.js M node_modules/ajv/lib/dotjs/dependencies.js M node_modules/ajv/lib/dotjs/enum.js M node_modules/ajv/lib/dotjs/format.js M node_modules/ajv/lib/dotjs/items.js M node_modules/ajv/lib/dotjs/multipleOf.js M node_modules/ajv/lib/dotjs/not.js M node_modules/ajv/lib/dotjs/oneOf.js M node_modules/ajv/lib/dotjs/pattern.js D node_modules/ajv/lib/dotjs/patternRequired.js M node_modules/ajv/lib/dotjs/properties.js A node_modules/ajv/lib/dotjs/propertyNames.js M node_modules/ajv/lib/dotjs/ref.js M node_modules/ajv/lib/dotjs/required.js D node_modules/ajv/lib/dotjs/switch.js M node_modules/ajv/lib/dotjs/uniqueItems.js M node_modules/ajv/lib/dotjs/validate.js M node_modules/ajv/lib/keyword.js A node_modules/ajv/lib/patternGroups.js A node_modules/ajv/lib/refs/$data.json A node_modules/ajv/lib/refs/json-schema-draft-06.json M node_modules/ajv/lib/refs/json-schema-v5.json D node_modules/ajv/lib/v5.js M node_modules/ajv/package.json R node_modules/ajv/scripts/.eslintrc.yml A node_modules/ajv/scripts/bundle.js A node_modules/ajv/scripts/compile-dots.js R node_modules/ajv/scripts/info R node_modules/ajv/scripts/prepare-tests R node_modules/ajv/scripts/travis-gh-pages A node_modules/aproba/LICENSE A node_modules/aproba/index.js A node_modules/aproba/package.json A node_modules/are-we-there-yet/CHANGES.md~ A node_modules/are-we-there-yet/LICENSE A node_modules/are-we-there-yet/index.js A node_modules/are-we-there-yet/node_modules/isarray/.npmignore A node_modules/are-we-there-yet/node_modules/isarray/.travis.yml A node_modules/are-we-there-yet/node_
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to a6fe93d
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/395877 ) Change subject: Update change-propagation to a6fe93d .. Update change-propagation to a6fe93d List of changes: be193eb Separate individual delay metric from the total chain delay metrics a6fe93d Updated restbase example config Change-Id: I0e24d5febe7f57dbe20660c7191763184712fb98 --- M src 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/src b/src index 62f7bf2..a6fe93d 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit 62f7bf21f53154322e619c86d8c9e0c26b5f3134 +Subproject commit a6fe93dcb82322e458d70d7dbc1b2945ae16c464 -- To view, visit https://gerrit.wikimedia.org/r/395877 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I0e24d5febe7f57dbe20660c7191763184712fb98 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Giuseppe Lavagetto Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to a6fe93d
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395877 ) Change subject: Update change-propagation to a6fe93d .. Update change-propagation to a6fe93d List of changes: be193eb Separate individual delay metric from the total chain delay metrics a6fe93d Updated restbase example config Change-Id: I0e24d5febe7f57dbe20660c7191763184712fb98 --- M src 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/77/395877/1 diff --git a/src b/src index 62f7bf2..a6fe93d 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit 62f7bf21f53154322e619c86d8c9e0c26b5f3134 +Subproject commit a6fe93dcb82322e458d70d7dbc1b2945ae16c464 -- To view, visit https://gerrit.wikimedia.org/r/395877 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0e24d5febe7f57dbe20660c7191763184712fb98 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: [Config] Update how we calculate if-unmodified-since for tra...
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395864 ) Change subject: [Config] Update how we calculate if-unmodified-since for transclusions .. [Config] Update how we calculate if-unmodified-since for transclusions See https://github.com/wikimedia/change-propagation/pull/218 Bug: T182216 Change-Id: Ibd39791543a51fa364280fff54aadcec7599014b --- M scap/templates/config.yaml.j2 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/deploy refs/changes/64/395864/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 8f3356c..bca8e38 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -354,7 +354,7 @@ uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{{match.meta.uri.title}}' headers: cache-control: no-cache -if-unmodified-since: '{{date(message.meta.dt)}}' +if-unmodified-since: '{{date(message.root_event.dt)}}' x-restbase-mode: '{{message.tags[1]}}' query: redirect: false @@ -411,7 +411,7 @@ uri: '<%= restbase_uri %>/{{message.meta.domain}}/v1/page/html/{{match.meta.uri.title}}' headers: cache-control: no-cache -if-unmodified-since: '{{date(message.meta.dt)}}' +if-unmodified-since: '{{date(message.root_event.dt)}}' x-restbase-mode: '{{message.tags[1]}}' query: redirect: false -- To view, visit https://gerrit.wikimedia.org/r/395864 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibd39791543a51fa364280fff54aadcec7599014b Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 62f7bf2
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395812 ) Change subject: Update change-propagation to 62f7bf2 .. Update change-propagation to 62f7bf2 List of changes: 24d675e Fixed eslint-related dependencies e20fb17 Fixed unhandledRejection in metadata refresh 62f7bf2 Dedupe based on sha1 as well as on event ID xxx Update node module dependencies Change-Id: I246a873010bb652a4fa68f8bbc1f2e045f9e0bff --- M node_modules/abbrev/LICENSE M node_modules/abbrev/package.json M node_modules/aproba/index.js M node_modules/aproba/package.json C node_modules/are-we-there-yet/node_modules/isarray/.npmignore C node_modules/are-we-there-yet/node_modules/isarray/.travis.yml C node_modules/are-we-there-yet/node_modules/isarray/Makefile C node_modules/are-we-there-yet/node_modules/isarray/component.json C node_modules/are-we-there-yet/node_modules/isarray/index.js A node_modules/are-we-there-yet/node_modules/isarray/package.json C node_modules/are-we-there-yet/node_modules/isarray/test.js C node_modules/are-we-there-yet/node_modules/readable-stream/.npmignore C node_modules/are-we-there-yet/node_modules/readable-stream/.travis.yml C node_modules/are-we-there-yet/node_modules/readable-stream/LICENSE C node_modules/are-we-there-yet/node_modules/readable-stream/duplex-browser.js C node_modules/are-we-there-yet/node_modules/readable-stream/duplex.js C node_modules/are-we-there-yet/node_modules/readable-stream/lib/_stream_duplex.js C node_modules/are-we-there-yet/node_modules/readable-stream/lib/_stream_passthrough.js A node_modules/are-we-there-yet/node_modules/readable-stream/lib/_stream_readable.js C node_modules/are-we-there-yet/node_modules/readable-stream/lib/_stream_transform.js A node_modules/are-we-there-yet/node_modules/readable-stream/lib/_stream_writable.js A node_modules/are-we-there-yet/node_modules/readable-stream/lib/internal/streams/BufferList.js C node_modules/are-we-there-yet/node_modules/readable-stream/lib/internal/streams/destroy.js C node_modules/are-we-there-yet/node_modules/readable-stream/lib/internal/streams/stream-browser.js C node_modules/are-we-there-yet/node_modules/readable-stream/lib/internal/streams/stream.js A node_modules/are-we-there-yet/node_modules/readable-stream/package.json C node_modules/are-we-there-yet/node_modules/readable-stream/passthrough.js C node_modules/are-we-there-yet/node_modules/readable-stream/readable-browser.js C node_modules/are-we-there-yet/node_modules/readable-stream/readable.js C node_modules/are-we-there-yet/node_modules/readable-stream/transform.js C node_modules/are-we-there-yet/node_modules/readable-stream/writable-browser.js C node_modules/are-we-there-yet/node_modules/readable-stream/writable.js C node_modules/are-we-there-yet/node_modules/string_decoder/.npmignore C node_modules/are-we-there-yet/node_modules/string_decoder/LICENSE C node_modules/are-we-there-yet/node_modules/string_decoder/lib/string_decoder.js A node_modules/are-we-there-yet/node_modules/string_decoder/package.json M node_modules/are-we-there-yet/package.json M node_modules/block-stream/package.json M node_modules/console-control-strings/package.json M node_modules/debug/component.json R node_modules/debug/node_modules/ms/index.js A node_modules/debug/node_modules/ms/package.json M node_modules/debug/package.json A node_modules/debug/src/inspector-log.js M node_modules/debug/src/node.js M node_modules/deep-extend/package.json M node_modules/delegates/package.json M node_modules/fs.realpath/package.json M node_modules/fstream-ignore/package.json M node_modules/fstream/package.json M node_modules/gauge/package.json M node_modules/gc-stats/.travis.yml M node_modules/gc-stats/appveyor.yml D node_modules/gc-stats/build/gcstats/v1.0.2/Release/node-v48-linux-x64/gcstats.node A node_modules/gc-stats/build/gcstats/v1.1.0/Release/node-v48-linux-x64/gcstats.node D node_modules/gc-stats/node_modules/.bin/mkdirp D node_modules/gc-stats/node_modules/.bin/uuid D node_modules/gc-stats/node_modules/ajv/.tonic_example.js D node_modules/gc-stats/node_modules/ajv/LICENSE D node_modules/gc-stats/node_modules/ajv/dist/ajv.bundle.js D node_modules/gc-stats/node_modules/ajv/dist/ajv.min.js D node_modules/gc-stats/node_modules/ajv/dist/ajv.min.js.map D node_modules/gc-stats/node_modules/ajv/dist/nodent.min.js D node_modules/gc-stats/node_modules/ajv/dist/regenerator.min.js D node_modules/gc-stats/node_modules/ajv/lib/ajv.d.ts D node_modules/gc-stats/node_modules/ajv/lib/ajv.js D node_modules/gc-stats/node_modules/ajv/lib/async.js D node_modules/gc-stats/node_modules/ajv/lib/cache.js D node_modules/gc-stats/node_modules/ajv/lib/compile/_rules.js D node_modules/gc-stats/node_modules/ajv/lib/compile/equal.js D node_modules/gc-stats/node_modules/ajv/lib/compile/formats.js D node_modules/gc-stats/node_modules/ajv/lib/compile/index.js D node_modules/gc-stats/node_modules/ajv/lib/compile/resolve.js D nod
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Added a missing quote
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395787 ) Change subject: [Config] Added a missing quote .. [Config] Added a missing quote Change-Id: Id81bd7a24063783674970a809bdd1f27edf13777 --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/87/395787/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index ae464d9..a1a92d2 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -52,7 +52,7 @@ # Temporary section - we are only enabling the htmlCacheUpdate for wiktionaries # Later this will be moved to the vars.yaml as all the other jobs. htmlCacheUpdate: -topic: 'mediawiki.job.htmlCacheUpdate +topic: 'mediawiki.job.htmlCacheUpdate' match: meta: domain: '/.*\.wiktionary\.org$/' -- To view, visit https://gerrit.wikimedia.org/r/395787 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id81bd7a24063783674970a809bdd1f27edf13777 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Disable producing htmlCacheUpdate to redis for wiktionaries
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395616 ) Change subject: Disable producing htmlCacheUpdate to redis for wiktionaries .. Disable producing htmlCacheUpdate to redis for wiktionaries Bug: T182023 Change-Id: I9849bccaea5cfed7ca95972ce78993569fb7b2e5 --- M wmf-config/InitialiseSettings.php M wmf-config/jobqueue.php 2 files changed, 11 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config refs/changes/16/395616/1 diff --git a/wmf-config/InitialiseSettings.php b/wmf-config/InitialiseSettings.php index a6b1634..416bdfe 100644 --- a/wmf-config/InitialiseSettings.php +++ b/wmf-config/InitialiseSettings.php @@ -19476,6 +19476,14 @@ 'votewiki' => false, ], + +// Temporary code for the first test of htmlCacheUpdate +// on the EventBus JobQueue -disable old queue on wiktionaries only +'wmgDisableHTMLCacheUpdateInRedis' => [ + 'default' => false, + 'wiktionary' => true, +], + 'wmgUseCapiunto' => [ 'default' => false, 'testwiki' => true, diff --git a/wmf-config/jobqueue.php b/wmf-config/jobqueue.php index ecde6b7..a685ac9 100644 --- a/wmf-config/jobqueue.php +++ b/wmf-config/jobqueue.php @@ -49,6 +49,9 @@ $wgJobTypeConf['RecordLintJob'] = $wgJobTypeConf['wikibase-addUsagesForPage'] = [ 'class' => 'JobQueueEventBus' ]; + if ( $wmgDisableHTMLCacheUpdateInRedis ) { + $wgJobTypeConf['htmlCacheUpdate'] = [ 'class' => 'JobQueueEventBus' ]; + } $wgJobTypeConf['default'] = [ 'class' => 'JobQueueSecondTestQueue', 'mainqueue' => $jobQueueFederatedConfig, -- To view, visit https://gerrit.wikimedia.org/r/395616 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9849bccaea5cfed7ca95972ce78993569fb7b2e5 Gerrit-PatchSet: 1 Gerrit-Project: operations/mediawiki-config Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Enable htmlCacheUpdate jobs for wiktionary
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395615 ) Change subject: [Config] Enable htmlCacheUpdate jobs for wiktionary .. [Config] Enable htmlCacheUpdate jobs for wiktionary Bug: T182023 Change-Id: I7e3ed8d46bccd3bf4c8767bbe3afe0830271696a --- M scap/templates/config.yaml.j2 1 file changed, 16 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/15/395615/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 5bd73bc..71e37c3 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -48,6 +48,22 @@ body: '{{globals.message}}' agentOptions: keepAlive: true +# Temporary section - we are only enabling the htmlCacheUpdate for wiktionaries +# Later this will be moved to the vars.yaml as all the other jobs. + htmlCacheUpdate: +topic: 'mediawiki.job.htmlCacheUpdate +match: + meta: +domain: '/.*\.wiktionary\.org$/' +exec: + method: post + uri: '<%= jobrunner_uri %>' + headers: +content-type: 'application/json' + body: '{{globals.message}}' + agentOptions: +keepAlive: true +# End of temporary section <%- endfor %> <%- else %> job: -- To view, visit https://gerrit.wikimedia.org/r/395615 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7e3ed8d46bccd3bf4c8767bbe3afe0830271696a Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Don't attempt sending revision-visibility events if empty
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395169 ) Change subject: Don't attempt sending revision-visibility events if empty .. Don't attempt sending revision-visibility events if empty Change-Id: I63d0fc80c56532328ad59928ea04d482679a5c52 --- M EventBus.hooks.php 1 file changed, 7 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/69/395169/1 diff --git a/EventBus.hooks.php b/EventBus.hooks.php index 34e89ca..b3fdba6 100644 --- a/EventBus.hooks.php +++ b/EventBus.hooks.php @@ -393,6 +393,13 @@ } } + if ( empty( $events ) ) { + // For revision-visibility-set it's possible that + // the page was deleted simultaneously and we can not + // send a meaningful event. + return; + } + DeferredUpdates::addCallableUpdate( function () use ( $events ) { EventBus::getInstance()->send( $events ); -- To view, visit https://gerrit.wikimedia.org/r/395169 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I63d0fc80c56532328ad59928ea04d482679a5c52 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] eventlogging[master]: Make the kafka async deliver callback thread-safe
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395164 ) Change subject: Make the kafka async deliver callback thread-safe .. Make the kafka async deliver callback thread-safe Bug: T180017 Change-Id: Idaec4b33808e276d427f903a2982aee75b89cd94 --- M eventlogging/service.py 1 file changed, 13 insertions(+), 8 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/eventlogging refs/changes/64/395164/1 diff --git a/eventlogging/service.py b/eventlogging/service.py index f7d30ef..7c4105c 100644 --- a/eventlogging/service.py +++ b/eventlogging/service.py @@ -24,6 +24,7 @@ import os import socket import yaml +from tornado.ioloop import IOLoop from . import ValidationError, SchemaError # these are int __init__.py from .compat import json @@ -549,14 +550,18 @@ tf should be a tornado Future to set either result or exception appropriately. """ -if isinstance(v, BaseException): -tf.set_exception(v) -# TODO: This seems correct, but causes -# an exception to be thrown from the Future object. -# tf.set_exc_info(sys.exc_info()) -else: -tf.set_result(v) -logging.debug('Delivered to kafka: %s', v) +def actual_callback(): +if isinstance(v, BaseException): +tf.set_exception(v) +else: +tf.set_result(v) +logging.debug('Delivered to kafka: %s', v) + +# Kafka driver calls the delivery callback on the thread +# other then the MainThread, so transfer the control back +# to the main thread. It is safe to access the value as +# kafka producer is thread safe. +IOLoop.current().add_callback(actual_callback) tornado_future = tornado.concurrent.Future() # When kafka's future calls this delivery callback, the -- To view, visit https://gerrit.wikimedia.org/r/395164 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idaec4b33808e276d427f903a2982aee75b89cd94 Gerrit-PatchSet: 1 Gerrit-Project: eventlogging Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/395091 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: Ie7bd4c90bd6866de4462893a9f943bdb85107137 --- M node_modules/ajv/dist/ajv.bundle.js M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js M node_modules/ajv/lib/ajv.js M node_modules/ajv/package.json M node_modules/kad/package.json M node_modules/moment/locale/es-us.js M node_modules/moment/locale/hu.js M node_modules/moment/min/locales.js M node_modules/moment/min/locales.min.js M node_modules/moment/min/moment-with-locales.js M node_modules/moment/min/moment-with-locales.min.js M node_modules/moment/min/moment.min.js M node_modules/moment/moment.js M node_modules/moment/package.json M node_modules/moment/src/lib/parse/regex.js M node_modules/moment/src/locale/es-us.js M node_modules/moment/src/moment.js M node_modules/service-runner/lib/docker.js M node_modules/service-runner/lib/heapwatch.js M node_modules/service-runner/lib/worker.js M node_modules/service-runner/package.json M node_modules/swagger-ui/package.json 25 files changed, 92 insertions(+), 82 deletions(-) Approvals: Ppchelko: Verified; Looks good to me, approved -- To view, visit https://gerrit.wikimedia.org/r/395091 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie7bd4c90bd6866de4462893a9f943bdb85107137 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/395091 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: Ie7bd4c90bd6866de4462893a9f943bdb85107137 --- M node_modules/ajv/dist/ajv.bundle.js M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js M node_modules/ajv/lib/ajv.js M node_modules/ajv/package.json M node_modules/kad/package.json M node_modules/moment/locale/es-us.js M node_modules/moment/locale/hu.js M node_modules/moment/min/locales.js M node_modules/moment/min/locales.min.js M node_modules/moment/min/moment-with-locales.js M node_modules/moment/min/moment-with-locales.min.js M node_modules/moment/min/moment.min.js M node_modules/moment/moment.js M node_modules/moment/package.json M node_modules/moment/src/lib/parse/regex.js M node_modules/moment/src/locale/es-us.js M node_modules/moment/src/moment.js M node_modules/service-runner/lib/docker.js M node_modules/service-runner/lib/heapwatch.js M node_modules/service-runner/lib/worker.js M node_modules/service-runner/package.json M node_modules/swagger-ui/package.json 25 files changed, 92 insertions(+), 82 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/91/395091/1 -- To view, visit https://gerrit.wikimedia.org/r/395091 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie7bd4c90bd6866de4462893a9f943bdb85107137 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...Wikibase[master]: Stop using EnqueueJob for AddUsagesForPageJob
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393750 ) Change subject: Stop using EnqueueJob for AddUsagesForPageJob .. Stop using EnqueueJob for AddUsagesForPageJob Since the job was tranferred to kafka-based infrastructure, no need to wrap it in the EnqueueJob any more. This will lower the job execution latency and lower the load on the queue. Bug: T181216 Change-Id: I91187badd680763383e0f26ee1689b9fc01e57fe --- M client/includes/Hooks/DataUpdateHookHandlers.php 1 file changed, 1 insertion(+), 3 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Wikibase refs/changes/50/393750/1 diff --git a/client/includes/Hooks/DataUpdateHookHandlers.php b/client/includes/Hooks/DataUpdateHookHandlers.php index 5b49574..4e00211 100644 --- a/client/includes/Hooks/DataUpdateHookHandlers.php +++ b/client/includes/Hooks/DataUpdateHookHandlers.php @@ -165,9 +165,7 @@ //TODO: Before posting a job, check replica database. If no changes are needed, skip update. $addUsagesForPageJob = AddUsagesForPageJob::newSpec( $title, $usageAcc->getUsages() ); - $enqueueJob = EnqueueJob::newFromLocalJobs( $addUsagesForPageJob ); - - $this->jobScheduler->lazyPush( $enqueueJob ); + $this->jobScheduler->lazyPush( $addUsagesForPageJob ); } /** -- To view, visit https://gerrit.wikimedia.org/r/393750 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I91187badd680763383e0f26ee1689b9fc01e57fe Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/Wikibase Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Enable wikibase-addUsagesForPage job
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/393732 ) Change subject: [Config] Enable wikibase-addUsagesForPage job .. [Config] Enable wikibase-addUsagesForPage job Change-Id: I47ff3c75039a397a1f0a36c990dd18ab30f149cc --- M scap/vars.yaml 1 file changed, 2 insertions(+), 0 deletions(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/scap/vars.yaml b/scap/vars.yaml index 623171f..bb782da 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -27,3 +27,5 @@ RecordLintJob: concurrency: 100 consumer_batch_size: 10 + wikibase-addUsagesForPage: {} + -- To view, visit https://gerrit.wikimedia.org/r/393732 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I47ff3c75039a397a1f0a36c990dd18ab30f149cc Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Giuseppe Lavagetto Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: [JobQueue] Only produce wikibase-addUsagesForPage to EventBus
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393734 ) Change subject: [JobQueue] Only produce wikibase-addUsagesForPage to EventBus .. [JobQueue] Only produce wikibase-addUsagesForPage to EventBus Depends-On: I47ff3c75039a397a1f0a36c990dd18ab30f149cc Change-Id: I20f644051a611575b28562738c4d6534ae37f986 --- M wmf-config/jobqueue.php 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config refs/changes/34/393734/1 diff --git a/wmf-config/jobqueue.php b/wmf-config/jobqueue.php index 9504544..3f7c778 100644 --- a/wmf-config/jobqueue.php +++ b/wmf-config/jobqueue.php @@ -47,6 +47,7 @@ $wgJobTypeConf['deleteLinks'] = $wgJobTypeConf['updateBetaFeaturesUserCounts'] = $wgJobTypeConf['RecordLintJob'] = +$wgJobTypeConf['wikibase-addUsagesForPage'] = [ 'class' => 'JobQueueEventBus' ]; $wgJobTypeConf['default'] = [ 'class' => 'JobQueueSecondTestQueue', -- To view, visit https://gerrit.wikimedia.org/r/393734 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I20f644051a611575b28562738c4d6534ae37f986 Gerrit-PatchSet: 1 Gerrit-Project: operations/mediawiki-config Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Enable wikibase-addUsagesForPage job
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393732 ) Change subject: [Config] Enable wikibase-addUsagesForPage job .. [Config] Enable wikibase-addUsagesForPage job Change-Id: I47ff3c75039a397a1f0a36c990dd18ab30f149cc --- M scap/vars.yaml 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/32/393732/1 diff --git a/scap/vars.yaml b/scap/vars.yaml index 623171f..bb782da 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -27,3 +27,5 @@ RecordLintJob: concurrency: 100 consumer_batch_size: 10 + wikibase-addUsagesForPage: {} + -- To view, visit https://gerrit.wikimedia.org/r/393732 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I47ff3c75039a397a1f0a36c990dd18ab30f149cc Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Remove double-processing
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/393726 ) Change subject: Remove double-processing .. Remove double-processing Change-Id: Iadeb91807d8237d248ecce47420fcdd4cefb57b5 --- M scap/templates/config.yaml.j2 1 file changed, 0 insertions(+), 12 deletions(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 842ca22..5bd73bc 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -49,18 +49,6 @@ agentOptions: keepAlive: true <%- endfor %> - # Temporary for double-processing to not loose jobs! - job: -topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' -exec: - method: post - uri: '<%= jobrunner_uri %>' - headers: -content-type: 'application/json' - body: '{{globals.message}}' - agentOptions: -keepAlive: true - # End of temporary code <%- else %> job: topic: '/^mediawiki\.job\..*/' -- To view, visit https://gerrit.wikimedia.org/r/393726 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iadeb91807d8237d248ecce47420fcdd4cefb57b5 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Remove double-processing
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393726 ) Change subject: Remove double-processing .. Remove double-processing Change-Id: Iadeb91807d8237d248ecce47420fcdd4cefb57b5 --- M scap/templates/config.yaml.j2 1 file changed, 0 insertions(+), 12 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/26/393726/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 842ca22..5bd73bc 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -49,18 +49,6 @@ agentOptions: keepAlive: true <%- endfor %> - # Temporary for double-processing to not loose jobs! - job: -topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' -exec: - method: post - uri: '<%= jobrunner_uri %>' - headers: -content-type: 'application/json' - body: '{{globals.message}}' - agentOptions: -keepAlive: true - # End of temporary code <%- else %> job: topic: '/^mediawiki\.job\..*/' -- To view, visit https://gerrit.wikimedia.org/r/393726 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iadeb91807d8237d248ecce47420fcdd4cefb57b5 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Move the enabled job config to vars.yaml
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/391801 ) Change subject: [Config] Move the enabled job config to vars.yaml .. [Config] Move the enabled job config to vars.yaml Using regexes to subscrive to the topics is becoming unbearable, so let's switch to using the vars.yaml. This will completely break the metrics - we are renaming the rules so the metrics will start over. I think it's ok, if we continue with this pattern from now on. Change-Id: I854c125cfd46e5f3acbe5695be1dfcc34d05deaa --- M scap/templates/config.yaml.j2 M scap/vars.yaml 2 files changed, 40 insertions(+), 7 deletions(-) Approvals: Mobrovac: Looks good to me, but someone else must approve Ppchelko: Verified; Looks good to me, approved diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 7411e78..842ca22 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -33,13 +33,13 @@ concurrency: <%= concurrency %> startup_delay: 6 templates: - job: <%- if env == 'production' %> -topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' -consumer_batch_size: 10 # Temporary for testing -<%- else %> -topic: '/^mediawiki\.job\..*/' -<%- endif %> +<%- for topic, topic_config in jobs_enabled.iteritems() %> + <%= topic %>: +topic: 'mediawiki.job.<%= topic %>' +<%- for key, value in topic_config.iteritems() %> +<%= key %>: <%= value %> +<%- endfor %> exec: method: post uri: '<%= jobrunner_uri %>' @@ -48,6 +48,31 @@ body: '{{globals.message}}' agentOptions: keepAlive: true +<%- endfor %> + # Temporary for double-processing to not loose jobs! + job: +topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' +exec: + method: post + uri: '<%= jobrunner_uri %>' + headers: +content-type: 'application/json' + body: '{{globals.message}}' + agentOptions: +keepAlive: true + # End of temporary code +<%- else %> + job: +topic: '/^mediawiki\.job\..*/' +exec: + method: post + uri: '<%= jobrunner_uri %>' + headers: +content-type: 'application/json' + body: '{{globals.message}}' + agentOptions: +keepAlive: true +<%- endif %> # Number of worker processes to spawn. diff --git a/scap/vars.yaml b/scap/vars.yaml index 0717d29..623171f 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -14,8 +14,16 @@ site: datacenter1 broker_list: localhost:9092 jobrunner_uri: http://localhost/rpc/RunSingleJob.php -concurrency: 200 +concurrency: 30 redis_path: /var/run/redis.sock redis_pass: env: production kafka_max_bytes: 4194304 +jobs_enabled: + deleteLinks: {} + MessageIndexRebuildJob: {} + flaggedrevs_CacheUpdate: {} + updateBetaFeaturesUserCounts: {} + RecordLintJob: +concurrency: 100 +consumer_batch_size: 10 -- To view, visit https://gerrit.wikimedia.org/r/391801 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I854c125cfd46e5f3acbe5695be1dfcc34d05deaa Gerrit-PatchSet: 7 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/393580 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: I2ec08233aea1b2ce0fab982fac2c2f71e1148b0e --- M node_modules/ajv/.tonic_example.js M node_modules/ajv/dist/ajv.bundle.js M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js R node_modules/ajv/lib/$data.js R node_modules/ajv/lib/ajv.d.ts M node_modules/ajv/lib/ajv.js D node_modules/ajv/lib/async.js M node_modules/ajv/lib/compile/_rules.js R node_modules/ajv/lib/compile/async.js M node_modules/ajv/lib/compile/equal.js R node_modules/ajv/lib/compile/error_classes.js M node_modules/ajv/lib/compile/formats.js M node_modules/ajv/lib/compile/index.js M node_modules/ajv/lib/compile/resolve.js M node_modules/ajv/lib/compile/rules.js R node_modules/ajv/lib/compile/ucs2length.js M node_modules/ajv/lib/compile/util.js D node_modules/ajv/lib/compile/validation_error.js M node_modules/ajv/lib/dot/_limit.jst M node_modules/ajv/lib/dot/allOf.jst M node_modules/ajv/lib/dot/anyOf.jst M node_modules/ajv/lib/dot/coerce.def R node_modules/ajv/lib/dot/const.jst R node_modules/ajv/lib/dot/contains.jst D node_modules/ajv/lib/dot/custom.def R node_modules/ajv/lib/dot/custom.jst M node_modules/ajv/lib/dot/defaults.def M node_modules/ajv/lib/dot/definitions.def M node_modules/ajv/lib/dot/dependencies.jst M node_modules/ajv/lib/dot/enum.jst M node_modules/ajv/lib/dot/errors.def M node_modules/ajv/lib/dot/format.jst M node_modules/ajv/lib/dot/items.jst M node_modules/ajv/lib/dot/missing.def M node_modules/ajv/lib/dot/not.jst M node_modules/ajv/lib/dot/oneOf.jst M node_modules/ajv/lib/dot/properties.jst R node_modules/ajv/lib/dot/propertyNames.jst M node_modules/ajv/lib/dot/ref.jst M node_modules/ajv/lib/dot/required.jst D node_modules/ajv/lib/dot/v5/_formatLimit.jst D node_modules/ajv/lib/dot/v5/constant.jst D node_modules/ajv/lib/dot/v5/patternRequired.jst D node_modules/ajv/lib/dot/v5/switch.jst M node_modules/ajv/lib/dot/validate.jst D node_modules/ajv/lib/dotjs/_formatLimit.js M node_modules/ajv/lib/dotjs/_limit.js M node_modules/ajv/lib/dotjs/_limitItems.js M node_modules/ajv/lib/dotjs/_limitLength.js M node_modules/ajv/lib/dotjs/_limitProperties.js M node_modules/ajv/lib/dotjs/allOf.js M node_modules/ajv/lib/dotjs/anyOf.js R node_modules/ajv/lib/dotjs/const.js D node_modules/ajv/lib/dotjs/constant.js R node_modules/ajv/lib/dotjs/contains.js R node_modules/ajv/lib/dotjs/custom.js M node_modules/ajv/lib/dotjs/dependencies.js M node_modules/ajv/lib/dotjs/enum.js M node_modules/ajv/lib/dotjs/format.js M node_modules/ajv/lib/dotjs/items.js M node_modules/ajv/lib/dotjs/multipleOf.js M node_modules/ajv/lib/dotjs/not.js M node_modules/ajv/lib/dotjs/oneOf.js M node_modules/ajv/lib/dotjs/pattern.js D node_modules/ajv/lib/dotjs/patternRequired.js M node_modules/ajv/lib/dotjs/properties.js R node_modules/ajv/lib/dotjs/propertyNames.js M node_modules/ajv/lib/dotjs/ref.js M node_modules/ajv/lib/dotjs/required.js D node_modules/ajv/lib/dotjs/switch.js M node_modules/ajv/lib/dotjs/uniqueItems.js M node_modules/ajv/lib/dotjs/validate.js M node_modules/ajv/lib/keyword.js R node_modules/ajv/lib/patternGroups.js R node_modules/ajv/lib/refs/$data.json R node_modules/ajv/lib/refs/json-schema-draft-06.json M node_modules/ajv/lib/refs/json-schema-v5.json D node_modules/ajv/lib/v5.js M node_modules/ajv/package.json R node_modules/ajv/scripts/.eslintrc.yml R node_modules/ajv/scripts/bundle.js R node_modules/ajv/scripts/compile-dots.js R node_modules/ajv/scripts/info R node_modules/ajv/scripts/prepare-tests R node_modules/ajv/scripts/travis-gh-pages M node_modules/busboy/package.json M node_modules/co/package.json M node_modules/fast-deep-equal/package.json M node_modules/fast-json-stable-stringify/package.json D node_modules/har-validator/node_modules/ajv/.tonic_example.js D node_modules/har-validator/node_modules/ajv/LICENSE D node_modules/har-validator/node_modules/ajv/dist/ajv.bundle.js D node_modules/har-validator/node_modules/ajv/dist/ajv.min.js D node_modules/har-validator/node_modules/ajv/dist/ajv.min.js.map D node_modules/har-validator/node_modules/ajv/dist/nodent.min.js D node_modules/har-validator/node_modules/ajv/dist/regenerator.min.js D node_modules/har-validator/node_modules/ajv/lib/ajv.js D node_modules/har-validator/node_modules/ajv/lib/cache.js D node_modules/har-validator/node_modules/ajv/lib/compile/_rules.js D node_modules/har-validator/node_modules/ajv/lib/compile/equal.js D node_modules/har-validator/node_modules/ajv/lib/compile/formats.js D node_modules/har-validator/node_modules/ajv/lib/compile/index.js D node_modules/har-validator/node_modules/ajv/lib/compile/resolve.js D node_modules/har-validator/node_
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393580 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: I2ec08233aea1b2ce0fab982fac2c2f71e1148b0e --- M node_modules/ajv/.tonic_example.js M node_modules/ajv/dist/ajv.bundle.js M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js R node_modules/ajv/lib/$data.js R node_modules/ajv/lib/ajv.d.ts M node_modules/ajv/lib/ajv.js D node_modules/ajv/lib/async.js M node_modules/ajv/lib/compile/_rules.js R node_modules/ajv/lib/compile/async.js M node_modules/ajv/lib/compile/equal.js R node_modules/ajv/lib/compile/error_classes.js M node_modules/ajv/lib/compile/formats.js M node_modules/ajv/lib/compile/index.js M node_modules/ajv/lib/compile/resolve.js M node_modules/ajv/lib/compile/rules.js R node_modules/ajv/lib/compile/ucs2length.js M node_modules/ajv/lib/compile/util.js D node_modules/ajv/lib/compile/validation_error.js M node_modules/ajv/lib/dot/_limit.jst M node_modules/ajv/lib/dot/allOf.jst M node_modules/ajv/lib/dot/anyOf.jst M node_modules/ajv/lib/dot/coerce.def R node_modules/ajv/lib/dot/const.jst R node_modules/ajv/lib/dot/contains.jst D node_modules/ajv/lib/dot/custom.def R node_modules/ajv/lib/dot/custom.jst M node_modules/ajv/lib/dot/defaults.def M node_modules/ajv/lib/dot/definitions.def M node_modules/ajv/lib/dot/dependencies.jst M node_modules/ajv/lib/dot/enum.jst M node_modules/ajv/lib/dot/errors.def M node_modules/ajv/lib/dot/format.jst M node_modules/ajv/lib/dot/items.jst M node_modules/ajv/lib/dot/missing.def M node_modules/ajv/lib/dot/not.jst M node_modules/ajv/lib/dot/oneOf.jst M node_modules/ajv/lib/dot/properties.jst R node_modules/ajv/lib/dot/propertyNames.jst M node_modules/ajv/lib/dot/ref.jst M node_modules/ajv/lib/dot/required.jst D node_modules/ajv/lib/dot/v5/_formatLimit.jst D node_modules/ajv/lib/dot/v5/constant.jst D node_modules/ajv/lib/dot/v5/patternRequired.jst D node_modules/ajv/lib/dot/v5/switch.jst M node_modules/ajv/lib/dot/validate.jst D node_modules/ajv/lib/dotjs/_formatLimit.js M node_modules/ajv/lib/dotjs/_limit.js M node_modules/ajv/lib/dotjs/_limitItems.js M node_modules/ajv/lib/dotjs/_limitLength.js M node_modules/ajv/lib/dotjs/_limitProperties.js M node_modules/ajv/lib/dotjs/allOf.js M node_modules/ajv/lib/dotjs/anyOf.js R node_modules/ajv/lib/dotjs/const.js D node_modules/ajv/lib/dotjs/constant.js R node_modules/ajv/lib/dotjs/contains.js R node_modules/ajv/lib/dotjs/custom.js M node_modules/ajv/lib/dotjs/dependencies.js M node_modules/ajv/lib/dotjs/enum.js M node_modules/ajv/lib/dotjs/format.js M node_modules/ajv/lib/dotjs/items.js M node_modules/ajv/lib/dotjs/multipleOf.js M node_modules/ajv/lib/dotjs/not.js M node_modules/ajv/lib/dotjs/oneOf.js M node_modules/ajv/lib/dotjs/pattern.js D node_modules/ajv/lib/dotjs/patternRequired.js M node_modules/ajv/lib/dotjs/properties.js R node_modules/ajv/lib/dotjs/propertyNames.js M node_modules/ajv/lib/dotjs/ref.js M node_modules/ajv/lib/dotjs/required.js D node_modules/ajv/lib/dotjs/switch.js M node_modules/ajv/lib/dotjs/uniqueItems.js M node_modules/ajv/lib/dotjs/validate.js M node_modules/ajv/lib/keyword.js R node_modules/ajv/lib/patternGroups.js R node_modules/ajv/lib/refs/$data.json R node_modules/ajv/lib/refs/json-schema-draft-06.json M node_modules/ajv/lib/refs/json-schema-v5.json D node_modules/ajv/lib/v5.js M node_modules/ajv/package.json R node_modules/ajv/scripts/.eslintrc.yml R node_modules/ajv/scripts/bundle.js R node_modules/ajv/scripts/compile-dots.js R node_modules/ajv/scripts/info R node_modules/ajv/scripts/prepare-tests R node_modules/ajv/scripts/travis-gh-pages M node_modules/busboy/package.json M node_modules/co/package.json M node_modules/fast-deep-equal/package.json M node_modules/fast-json-stable-stringify/package.json D node_modules/har-validator/node_modules/ajv/.tonic_example.js D node_modules/har-validator/node_modules/ajv/LICENSE D node_modules/har-validator/node_modules/ajv/dist/ajv.bundle.js D node_modules/har-validator/node_modules/ajv/dist/ajv.min.js D node_modules/har-validator/node_modules/ajv/dist/ajv.min.js.map D node_modules/har-validator/node_modules/ajv/dist/nodent.min.js D node_modules/har-validator/node_modules/ajv/dist/regenerator.min.js D node_modules/har-validator/node_modules/ajv/lib/ajv.js D node_modules/har-validator/node_modules/ajv/lib/cache.js D node_modules/har-validator/node_modules/ajv/lib/compile/_rules.js D node_modules/har-validator/node_modules/ajv/lib/compile/equal.js D node_modules/har-validator/node_modules/ajv/lib/compile/formats.js D node_modules/har-validator/node_modules/ajv/lib/compile/index.js D node_modules/har-validator/node_modules/ajv/lib/compile/resolve.js D node_modules/har-validator/node_modules/
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393579 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: I085d72cbbc873d77edcd2fe1cc6da097630a93a9 --- M node_modules/ajv/.tonic_example.js M node_modules/ajv/dist/ajv.bundle.js M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/dist/nodent.min.js M node_modules/ajv/dist/regenerator.min.js R node_modules/ajv/lib/$data.js R node_modules/ajv/lib/ajv.d.ts M node_modules/ajv/lib/ajv.js D node_modules/ajv/lib/async.js M node_modules/ajv/lib/compile/_rules.js R node_modules/ajv/lib/compile/async.js M node_modules/ajv/lib/compile/equal.js R node_modules/ajv/lib/compile/error_classes.js M node_modules/ajv/lib/compile/formats.js M node_modules/ajv/lib/compile/index.js M node_modules/ajv/lib/compile/resolve.js M node_modules/ajv/lib/compile/rules.js R node_modules/ajv/lib/compile/ucs2length.js M node_modules/ajv/lib/compile/util.js D node_modules/ajv/lib/compile/validation_error.js M node_modules/ajv/lib/dot/_limit.jst M node_modules/ajv/lib/dot/allOf.jst M node_modules/ajv/lib/dot/anyOf.jst M node_modules/ajv/lib/dot/coerce.def R node_modules/ajv/lib/dot/const.jst R node_modules/ajv/lib/dot/contains.jst D node_modules/ajv/lib/dot/custom.def R node_modules/ajv/lib/dot/custom.jst M node_modules/ajv/lib/dot/defaults.def M node_modules/ajv/lib/dot/definitions.def M node_modules/ajv/lib/dot/dependencies.jst M node_modules/ajv/lib/dot/enum.jst M node_modules/ajv/lib/dot/errors.def M node_modules/ajv/lib/dot/format.jst M node_modules/ajv/lib/dot/items.jst M node_modules/ajv/lib/dot/missing.def M node_modules/ajv/lib/dot/not.jst M node_modules/ajv/lib/dot/oneOf.jst M node_modules/ajv/lib/dot/properties.jst R node_modules/ajv/lib/dot/propertyNames.jst M node_modules/ajv/lib/dot/ref.jst M node_modules/ajv/lib/dot/required.jst D node_modules/ajv/lib/dot/v5/_formatLimit.jst D node_modules/ajv/lib/dot/v5/constant.jst D node_modules/ajv/lib/dot/v5/patternRequired.jst D node_modules/ajv/lib/dot/v5/switch.jst M node_modules/ajv/lib/dot/validate.jst D node_modules/ajv/lib/dotjs/_formatLimit.js M node_modules/ajv/lib/dotjs/_limit.js M node_modules/ajv/lib/dotjs/_limitItems.js M node_modules/ajv/lib/dotjs/_limitLength.js M node_modules/ajv/lib/dotjs/_limitProperties.js M node_modules/ajv/lib/dotjs/allOf.js M node_modules/ajv/lib/dotjs/anyOf.js R node_modules/ajv/lib/dotjs/const.js D node_modules/ajv/lib/dotjs/constant.js R node_modules/ajv/lib/dotjs/contains.js R node_modules/ajv/lib/dotjs/custom.js M node_modules/ajv/lib/dotjs/dependencies.js M node_modules/ajv/lib/dotjs/enum.js M node_modules/ajv/lib/dotjs/format.js M node_modules/ajv/lib/dotjs/items.js M node_modules/ajv/lib/dotjs/multipleOf.js M node_modules/ajv/lib/dotjs/not.js M node_modules/ajv/lib/dotjs/oneOf.js M node_modules/ajv/lib/dotjs/pattern.js D node_modules/ajv/lib/dotjs/patternRequired.js M node_modules/ajv/lib/dotjs/properties.js R node_modules/ajv/lib/dotjs/propertyNames.js M node_modules/ajv/lib/dotjs/ref.js M node_modules/ajv/lib/dotjs/required.js D node_modules/ajv/lib/dotjs/switch.js M node_modules/ajv/lib/dotjs/uniqueItems.js M node_modules/ajv/lib/dotjs/validate.js M node_modules/ajv/lib/keyword.js R node_modules/ajv/lib/patternGroups.js R node_modules/ajv/lib/refs/$data.json R node_modules/ajv/lib/refs/json-schema-draft-06.json M node_modules/ajv/lib/refs/json-schema-v5.json D node_modules/ajv/lib/v5.js M node_modules/ajv/package.json R node_modules/ajv/scripts/.eslintrc.yml R node_modules/ajv/scripts/bundle.js R node_modules/ajv/scripts/compile-dots.js R node_modules/ajv/scripts/info R node_modules/ajv/scripts/prepare-tests R node_modules/ajv/scripts/travis-gh-pages M node_modules/busboy/package.json M node_modules/co/package.json M node_modules/fast-deep-equal/package.json M node_modules/fast-json-stable-stringify/package.json D node_modules/har-validator/node_modules/ajv/.tonic_example.js D node_modules/har-validator/node_modules/ajv/LICENSE D node_modules/har-validator/node_modules/ajv/dist/ajv.bundle.js D node_modules/har-validator/node_modules/ajv/dist/ajv.min.js D node_modules/har-validator/node_modules/ajv/dist/ajv.min.js.map D node_modules/har-validator/node_modules/ajv/dist/nodent.min.js D node_modules/har-validator/node_modules/ajv/dist/regenerator.min.js D node_modules/har-validator/node_modules/ajv/lib/ajv.js D node_modules/har-validator/node_modules/ajv/lib/cache.js D node_modules/har-validator/node_modules/ajv/lib/compile/_rules.js D node_modules/har-validator/node_modules/ajv/lib/compile/equal.js D node_modules/har-validator/node_modules/ajv/lib/compile/formats.js D node_modules/har-validator/node_modules/ajv/lib/compile/index.js D node_modules/har-validator/node_modules/ajv/lib/compile/resolve.js D node_modules/har-validator/node_modules/
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Make http agent use keepAlive
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/393548 ) Change subject: [Config] Make http agent use keepAlive .. [Config] Make http agent use keepAlive Bug: T181007 Change-Id: I10da5c540f989e821e8e70a5394cb6df1e1e0737 --- M scap/templates/config.yaml.j2 1 file changed, 2 insertions(+), 0 deletions(-) Approvals: Mobrovac: Looks good to me, but someone else must approve Ppchelko: Verified; Looks good to me, approved diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index d13e71d..7411e78 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -46,6 +46,8 @@ headers: content-type: 'application/json' body: '{{globals.message}}' + agentOptions: +keepAlive: true # Number of worker processes to spawn. -- To view, visit https://gerrit.wikimedia.org/r/393548 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I10da5c540f989e821e8e70a5394cb6df1e1e0737 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Giuseppe Lavagetto Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/393554 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: Ib9fc687ce7e6db2292a76106949488aecf318b35 --- M node_modules/har-validator/node_modules/ajv/dist/ajv.bundle.js M node_modules/har-validator/node_modules/ajv/dist/ajv.min.js M node_modules/har-validator/node_modules/ajv/dist/ajv.min.js.map M node_modules/har-validator/node_modules/ajv/lib/ajv.d.ts M node_modules/har-validator/node_modules/ajv/lib/ajv.js M node_modules/har-validator/node_modules/ajv/lib/keyword.js M node_modules/har-validator/node_modules/ajv/package.json M node_modules/kad/package.json M node_modules/service-runner/lib/heapwatch.js M node_modules/service-runner/package.json M node_modules/swagger-ui/package.json 11 files changed, 93 insertions(+), 35 deletions(-) Approvals: Ppchelko: Verified; Looks good to me, approved -- To view, visit https://gerrit.wikimedia.org/r/393554 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib9fc687ce7e6db2292a76106949488aecf318b35 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393554 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: Ib9fc687ce7e6db2292a76106949488aecf318b35 --- M node_modules/har-validator/node_modules/ajv/dist/ajv.bundle.js M node_modules/har-validator/node_modules/ajv/dist/ajv.min.js M node_modules/har-validator/node_modules/ajv/dist/ajv.min.js.map M node_modules/har-validator/node_modules/ajv/lib/ajv.d.ts M node_modules/har-validator/node_modules/ajv/lib/ajv.js M node_modules/har-validator/node_modules/ajv/lib/keyword.js M node_modules/har-validator/node_modules/ajv/package.json M node_modules/kad/package.json M node_modules/service-runner/lib/heapwatch.js M node_modules/service-runner/package.json M node_modules/swagger-ui/package.json 11 files changed, 93 insertions(+), 35 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/54/393554/1 -- To view, visit https://gerrit.wikimedia.org/r/393554 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib9fc687ce7e6db2292a76106949488aecf318b35 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Make http agent use keepAlive
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393548 ) Change subject: [Config] Make http agent use keepAlive .. [Config] Make http agent use keepAlive Bug: T181007 Change-Id: I10da5c540f989e821e8e70a5394cb6df1e1e0737 --- M scap/templates/config.yaml.j2 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/48/393548/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index d13e71d..7411e78 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -46,6 +46,8 @@ headers: content-type: 'application/json' body: '{{globals.message}}' + agentOptions: +keepAlive: true # Number of worker processes to spawn. -- To view, visit https://gerrit.wikimedia.org/r/393548 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I10da5c540f989e821e8e70a5394cb6df1e1e0737 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update cxserver to 0c242c0
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393377 ) Change subject: Update cxserver to 0c242c0 .. Update cxserver to 0c242c0 List of changes: b4d493c Remove the left out data-seqid usage 608eab6 Add test for Youdao client 0c242c0 Pin service-runner to 2.4.2 xxx Update node module dependencies Change-Id: Ie1362e038642a5e38d327e6548c7962726f12591 --- D node_modules/.bin/node-pre-gyp D node_modules/.bin/nopt D node_modules/.bin/rc D node_modules/abbrev/LICENSE D node_modules/abbrev/abbrev.js D node_modules/abbrev/package.json M node_modules/ajv/dist/ajv.bundle.js M node_modules/ajv/dist/ajv.min.js M node_modules/ajv/dist/ajv.min.js.map M node_modules/ajv/lib/ajv.d.ts M node_modules/ajv/lib/ajv.js M node_modules/ajv/lib/keyword.js M node_modules/ajv/package.json D node_modules/aproba/LICENSE D node_modules/aproba/index.js D node_modules/aproba/package.json D node_modules/are-we-there-yet/CHANGES.md~ D node_modules/are-we-there-yet/LICENSE D node_modules/are-we-there-yet/index.js D node_modules/are-we-there-yet/package.json D node_modules/are-we-there-yet/tracker-base.js D node_modules/are-we-there-yet/tracker-group.js D node_modules/are-we-there-yet/tracker-stream.js D node_modules/are-we-there-yet/tracker.js D node_modules/block-stream/LICENSE D node_modules/block-stream/block-stream.js D node_modules/block-stream/package.json D node_modules/console-control-strings/LICENSE D node_modules/console-control-strings/README.md~ D node_modules/console-control-strings/index.js D node_modules/console-control-strings/package.json D node_modules/deep-extend/LICENSE D node_modules/deep-extend/index.js D node_modules/deep-extend/lib/deep-extend.js D node_modules/deep-extend/package.json D node_modules/delegates/.npmignore D node_modules/delegates/License D node_modules/delegates/Makefile D node_modules/delegates/index.js D node_modules/delegates/package.json D node_modules/delegates/test/index.js D node_modules/fs.realpath/LICENSE D node_modules/fs.realpath/index.js D node_modules/fs.realpath/old.js D node_modules/fs.realpath/package.json D node_modules/fstream-ignore/LICENSE D node_modules/fstream-ignore/ignore.js D node_modules/fstream-ignore/package.json D node_modules/fstream/.npmignore D node_modules/fstream/.travis.yml D node_modules/fstream/LICENSE D node_modules/fstream/examples/filter-pipe.js D node_modules/fstream/examples/pipe.js D node_modules/fstream/examples/reader.js D node_modules/fstream/examples/symlink-write.js D node_modules/fstream/fstream.js D node_modules/fstream/lib/abstract.js D node_modules/fstream/lib/collect.js D node_modules/fstream/lib/dir-reader.js D node_modules/fstream/lib/dir-writer.js D node_modules/fstream/lib/file-reader.js D node_modules/fstream/lib/file-writer.js D node_modules/fstream/lib/get-type.js D node_modules/fstream/lib/link-reader.js D node_modules/fstream/lib/link-writer.js D node_modules/fstream/lib/proxy-reader.js D node_modules/fstream/lib/proxy-writer.js D node_modules/fstream/lib/reader.js D node_modules/fstream/lib/socket-reader.js D node_modules/fstream/lib/writer.js D node_modules/fstream/package.json D node_modules/gauge/LICENSE D node_modules/gauge/base-theme.js D node_modules/gauge/error.js D node_modules/gauge/has-color.js D node_modules/gauge/index.js D node_modules/gauge/package.json D node_modules/gauge/plumbing.js D node_modules/gauge/process.js D node_modules/gauge/progress-bar.js D node_modules/gauge/render-template.js D node_modules/gauge/set-immediate.js D node_modules/gauge/set-interval.js D node_modules/gauge/spin.js D node_modules/gauge/template-item.js D node_modules/gauge/theme-set.js D node_modules/gauge/themes.js D node_modules/gauge/wide-truncate.js D node_modules/gc-stats/.npmignore D node_modules/gc-stats/.travis.yml D node_modules/gc-stats/LICENSE D node_modules/gc-stats/appveyor.yml D node_modules/gc-stats/binding.gyp D node_modules/gc-stats/build/gcstats/v1.0.2/Release/node-v48-linux-x64/gcstats.node D node_modules/gc-stats/example.js D node_modules/gc-stats/index.js D node_modules/gc-stats/node_modules/.bin/mkdirp D node_modules/gc-stats/node_modules/.bin/rimraf D node_modules/gc-stats/node_modules/.bin/uuid D node_modules/gc-stats/node_modules/ajv/.tonic_example.js D node_modules/gc-stats/node_modules/ajv/LICENSE D node_modules/gc-stats/node_modules/ajv/dist/ajv.bundle.js D node_modules/gc-stats/node_modules/ajv/dist/ajv.min.js D node_modules/gc-stats/node_modules/ajv/dist/ajv.min.js.map D node_modules/gc-stats/node_modules/ajv/dist/nodent.min.js D node_modules/gc-stats/node_modules/ajv/dist/regenerator.min.js D node_modules/gc-stats/node_modules/ajv/lib/ajv.d.ts D node_modules/gc-stats/node_modules/ajv/lib/ajv.js D node_modules/gc-stats/node_modules/ajv/lib/async.js D node_modules/gc-stats/node_modules/ajv/lib/cache.js D node_modules/gc-stats/node_modules/ajv/lib/compile/_rules.js D node_modules/gc-stats/node_modules/ajv/lib/
[MediaWiki-commits] [Gerrit] mediawiki...cxserver[master]: Pin service-runner to 2.4.2
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/393376 ) Change subject: Pin service-runner to 2.4.2 .. Pin service-runner to 2.4.2 Change-Id: I01281d475cdb4bd4046d52bf1d574e74fd164860 --- M package.json 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved Filippo Giunchedi: Looks good to me, but someone else must approve diff --git a/package.json b/package.json index db96257..28aae91 100644 --- a/package.json +++ b/package.json @@ -33,7 +33,7 @@ "mediawiki-title": "^0.6.5", "preq": "^0.5.3", "sax": "1.2.1", -"service-runner": "^2.4.3", +"service-runner": "2.4.2", "swagger-router": "^0.7.1", "swagger-ui": "git+https://github.com/wikimedia/swagger-ui#master"; }, -- To view, visit https://gerrit.wikimedia.org/r/393376 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I01281d475cdb4bd4046d52bf1d574e74fd164860 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/cxserver Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Filippo Giunchedi Gerrit-Reviewer: KartikMistry Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...cxserver[master]: Pin service-runner to 2.4.2
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393376 ) Change subject: Pin service-runner to 2.4.2 .. Pin service-runner to 2.4.2 Change-Id: I01281d475cdb4bd4046d52bf1d574e74fd164860 --- M package.json 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/cxserver refs/changes/76/393376/1 diff --git a/package.json b/package.json index db96257..28aae91 100644 --- a/package.json +++ b/package.json @@ -33,7 +33,7 @@ "mediawiki-title": "^0.6.5", "preq": "^0.5.3", "sax": "1.2.1", -"service-runner": "^2.4.3", +"service-runner": "2.4.2", "swagger-router": "^0.7.1", "swagger-ui": "git+https://github.com/wikimedia/swagger-ui#master"; }, -- To view, visit https://gerrit.wikimedia.org/r/393376 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I01281d475cdb4bd4046d52bf1d574e74fd164860 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/cxserver Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] operations/puppet[production]: Remove RESTBase jobs config
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393209 ) Change subject: Remove RESTBase jobs config .. Remove RESTBase jobs config Change-Id: If27069ffe5872a477dfc760c787d9aeae01b002a --- M modules/mediawiki/templates/jobrunner/jobrunner.conf.erb 1 file changed, 0 insertions(+), 3 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/09/393209/1 diff --git a/modules/mediawiki/templates/jobrunner/jobrunner.conf.erb b/modules/mediawiki/templates/jobrunner/jobrunner.conf.erb index 71e66e1..2858eec 100644 --- a/modules/mediawiki/templates/jobrunner/jobrunner.conf.erb +++ b/modules/mediawiki/templates/jobrunner/jobrunner.conf.erb @@ -18,8 +18,6 @@ "exclude": [ "AssembleUploadChunks", "TranslationsUpdateJob", -"RestbaseUpdateJobOnEdit", -"RestbaseUpdateJobOnDependencyChange", "PublishStashedFile", "gwtoolsetGWTFileBackendCleanupJob", "gwtoolsetUploadMediafileJob", @@ -89,7 +87,6 @@ // How many times to let jobs be executed before abandoning "attempts": { "*": 3, -"RestbaseUpdateJobOnDependencyChange": 1, // make rename jobs try harder for T141731 "LocalPageMoveJob": 6, "renameUser": 6 -- To view, visit https://gerrit.wikimedia.org/r/393209 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If27069ffe5872a477dfc760c787d9aeae01b002a Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...FlaggedRevs[master]: Remove usage of EnqueueJob
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393199 ) Change subject: Remove usage of EnqueueJob .. Remove usage of EnqueueJob Since the flaggedrev_CacheUpdate job was moved to the kafka JobQueue the usage of EnqueueJob became unnesessary. Initially it was designed to post cross-DC jobs, but in the new infrastructure it's not needed, cross-DC is handled by the JobQueue itself. Additionally, the Job contract requires the job to return a boolean indicating if the job execution was successfull or a job should be retried. Added return statements everywhere. Bug: T181216 Change-Id: I79167ee190120b4ef9220ff906635003e7b2439e --- M backend/FRDependencyUpdate.php M backend/FRExtraCacheUpdateJob.php M backend/FlaggableWikiPage.php 3 files changed, 14 insertions(+), 16 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/FlaggedRevs refs/changes/99/393199/1 diff --git a/backend/FRDependencyUpdate.php b/backend/FRDependencyUpdate.php index 3e8714a..5f552b8 100644 --- a/backend/FRDependencyUpdate.php +++ b/backend/FRDependencyUpdate.php @@ -66,14 +66,12 @@ if ( $existing != $deps ) { if ( $mode === self::DEFERRED ) { # Let the job queue parse and update - JobQueueGroup::singleton()->push( EnqueueJob::newFromLocalJobs( - new JobSpecification( - 'flaggedrevs_CacheUpdate', - [ 'type' => 'updatelinks' ], - [ 'removeDuplicates' => true ], - $this->title + JobQueueGroup::singleton()->push( + new FRExtraCacheUpdateJob( + $this->title, + [ 'type' => 'updatelinks' ] ) - ) ); + ); return; } diff --git a/backend/FRExtraCacheUpdateJob.php b/backend/FRExtraCacheUpdateJob.php index cf342e4..69236b2 100644 --- a/backend/FRExtraCacheUpdateJob.php +++ b/backend/FRExtraCacheUpdateJob.php @@ -60,18 +60,19 @@ $frDepUpdate = new FRDependencyUpdate( $this->title, $stableOut ); $frDepUpdate->doUpdate( FRDependencyUpdate::IMMEDIATE ); - return; + return true; } } // If not page or revision was found, remove the stable-only links FlaggedRevs::clearStableOnlyDeps( $fpage->getId() ); + return true; } protected function doUpdateSyncState() { $fpage = FlaggableWikiPage::getTitleInstance( $this->title ); if ( !$fpage->getId() || !$fpage->getStable() ) { - return; + return true; } $synced = $fpage->stableVersionIsSynced(); @@ -83,5 +84,6 @@ __METHOD__ ); } + return true; } } diff --git a/backend/FlaggableWikiPage.php b/backend/FlaggableWikiPage.php index 04256fe..49d05f6 100644 --- a/backend/FlaggableWikiPage.php +++ b/backend/FlaggableWikiPage.php @@ -342,14 +342,12 @@ return; } - JobQueueGroup::singleton()->push( EnqueueJob::newFromLocalJobs( - new JobSpecification( - 'flaggedrevs_CacheUpdate', - [ 'type' => 'updatesyncstate' ], - [ 'removeDuplicates' => true ], - $this->getTitle() + JobQueueGroup::singleton()->push( + new FRExtraCacheUpdateJob( + $this->getTitle(), + [ 'type' => 'updatesyncstate' ] ) - ) ); + ); } /** -- To view, visit https://gerrit.wikimedia.org/r/393199 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I79167ee190120b4ef9220ff906635003e7b2439e Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/FlaggedRevs Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: [Config] Increase worker_heartbeat_timeout
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/393032 ) Change subject: [Config] Increase worker_heartbeat_timeout .. [Config] Increase worker_heartbeat_timeout We're experiencing massive worker deaths from time to time because of heartbeat timeout is reached. Try increasing it to 30 seconds. Bug: T179684 Change-Id: I7b06683b79f44092188a5fe9656caefb163de050 --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/deploy refs/changes/32/393032/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index efd101b..8f3356c 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -745,7 +745,7 @@ # The maximum interval in ms that can pass between two beat messages # sent by each worker to the master before it is killed -worker_heartbeat_timeout: 15000 +worker_heartbeat_timeout: 3 # Logger info logging: -- To view, visit https://gerrit.wikimedia.org/r/393032 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7b06683b79f44092188a5fe9656caefb163de050 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Report pure job execution timing via statsd.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392803 ) Change subject: Report pure job execution timing via statsd. .. Report pure job execution timing via statsd. Bug: This will be useful to access the overheads in exec timing generated by spawning the MediaWiki instance and but change-propagation. Bug: T181007 Change-Id: I55352848fe8b74c09e918ab27d1afbe68c0a85ec --- M JobExecutor.php 1 file changed, 23 insertions(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/03/392803/1 diff --git a/JobExecutor.php b/JobExecutor.php index b0d869f..d2e4826 100644 --- a/JobExecutor.php +++ b/JobExecutor.php @@ -7,6 +7,7 @@ * executing a Job */ +use Liuggio\StatsdClient\Factory\StatsdDataFactoryInterface; use MediaWiki\Logger\LoggerFactory; use MediaWiki\MediaWikiServices; use Psr\Log\LoggerInterface; @@ -16,11 +17,15 @@ /** @var LoggerInterface instance for all JobExecutor instances */ private static $logger; + /** @var StatsdDataFactoryInterface instance for all JobExecutor instances */ + private static $stats; + /** * @param array $jobEvent the job event * @return array containing the response status and the message */ public function execute( $jobEvent ) { + $startTime = microtime( true ); $jobCreateResult = $this->getJobFromParams( $jobEvent ); if ( !$jobCreateResult['status'] ) { @@ -107,6 +112,12 @@ // the next Job execution. However since we run one job at a time // we don't need that. + // Report pure job execution timing + self::stats()->timing( + "jobexecutor.{$job->getType()}.exec", + microtime( true ) - $startTime + ); + return [ 'status' => $status, 'message' => $message @@ -160,7 +171,7 @@ /** * Returns a singleton logger instance for all JobExecutor instances. -* Use like: self::logger()->info( $mesage ) +* Use like: self::logger()->info( $message ) * We use this so we don't have to check if the logger has been created * before attempting to log a message. */ @@ -170,4 +181,15 @@ } return self::$logger; } + + /** +* Returns a singleton stats reporter instance for all JobExecutor instances. +* Use like self::stats()->increment( $key ) +*/ + private static function stats() { + if ( !self::$stats ) { + self::$stats = MediaWikiServices::getInstance()->getStatsdDataFactory(); + } + return self::$stats; + } } -- To view, visit https://gerrit.wikimedia.org/r/392803 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I55352848fe8b74c09e918ab27d1afbe68c0a85ec Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Revert "[Config] Temporary disable deduplication."
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/392649 ) Change subject: Revert "[Config] Temporary disable deduplication." .. Revert "[Config] Temporary disable deduplication." This reverts commit aac3201b04d2282ba6e8829eb649500f3dd76321. Change-Id: Ifb7b928cf693e298a6fe7f6c0dee8f1ec584726a --- M scap/templates/config.yaml.j2 1 file changed, 8 insertions(+), 8 deletions(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index c635e06..d13e71d 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -10,14 +10,14 @@ title: The Change Propagation for JobQueue root paths: <%- if env == 'production' %> -#/{api:sys}/dedupe: -# x-modules: -#- path: src/sys/deduplicator.js -# options: -#redis_prefix: 'CPJQ' -#redis: -# path: <%= redis_path %> -# password: <%= redis_pass if redis_pass else '' %> +/{api:sys}/dedupe: + x-modules: +- path: src/sys/deduplicator.js + options: +redis_prefix: 'CPJQ' +redis: + path: <%= redis_path %> + password: <%= redis_pass if redis_pass else '' %> <%- endif %> /{api:sys}/queue: x-modules: -- To view, visit https://gerrit.wikimedia.org/r/392649 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ifb7b928cf693e298a6fe7f6c0dee8f1ec584726a Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Revert "[Config] Temporary disable deduplication."
Hello Mobrovac, I'd like you to do a code review. Please visit https://gerrit.wikimedia.org/r/392649 to review the following change. Change subject: Revert "[Config] Temporary disable deduplication." .. Revert "[Config] Temporary disable deduplication." This reverts commit aac3201b04d2282ba6e8829eb649500f3dd76321. Change-Id: Ifb7b928cf693e298a6fe7f6c0dee8f1ec584726a --- M scap/templates/config.yaml.j2 1 file changed, 8 insertions(+), 8 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/49/392649/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index c635e06..d13e71d 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -10,14 +10,14 @@ title: The Change Propagation for JobQueue root paths: <%- if env == 'production' %> -#/{api:sys}/dedupe: -# x-modules: -#- path: src/sys/deduplicator.js -# options: -#redis_prefix: 'CPJQ' -#redis: -# path: <%= redis_path %> -# password: <%= redis_pass if redis_pass else '' %> +/{api:sys}/dedupe: + x-modules: +- path: src/sys/deduplicator.js + options: +redis_prefix: 'CPJQ' +redis: + path: <%= redis_path %> + password: <%= redis_pass if redis_pass else '' %> <%- endif %> /{api:sys}/queue: x-modules: -- To view, visit https://gerrit.wikimedia.org/r/392649 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifb7b928cf693e298a6fe7f6c0dee8f1ec584726a Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Temporary disable deduplication.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392646 ) Change subject: [Config] Temporary disable deduplication. .. [Config] Temporary disable deduplication. In order to test if redis has something to do with the slow-down of requests after the rate reaches some tipping point. Change-Id: I3b7ae8e31d18dd52428bc297205693944ee1271f --- M scap/templates/config.yaml.j2 1 file changed, 8 insertions(+), 8 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/46/392646/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index d13e71d..c635e06 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -10,14 +10,14 @@ title: The Change Propagation for JobQueue root paths: <%- if env == 'production' %> -/{api:sys}/dedupe: - x-modules: -- path: src/sys/deduplicator.js - options: -redis_prefix: 'CPJQ' -redis: - path: <%= redis_path %> - password: <%= redis_pass if redis_pass else '' %> +#/{api:sys}/dedupe: +# x-modules: +#- path: src/sys/deduplicator.js +# options: +#redis_prefix: 'CPJQ' +#redis: +# path: <%= redis_path %> +# password: <%= redis_pass if redis_pass else '' %> <%- endif %> /{api:sys}/queue: x-modules: -- To view, visit https://gerrit.wikimedia.org/r/392646 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3b7ae8e31d18dd52428bc297205693944ee1271f Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/392642 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: Iebe390c445a4079af70d08ce53ea5b5c12dcc286 --- A node_modules/.bin/node-pre-gyp A node_modules/.bin/nopt A node_modules/.bin/rc A node_modules/abbrev/LICENSE A node_modules/abbrev/abbrev.js A node_modules/abbrev/package.json A node_modules/aproba/LICENSE A node_modules/aproba/index.js A node_modules/aproba/package.json A node_modules/are-we-there-yet/CHANGES.md~ A node_modules/are-we-there-yet/LICENSE A node_modules/are-we-there-yet/index.js A node_modules/are-we-there-yet/package.json A node_modules/are-we-there-yet/tracker-base.js A node_modules/are-we-there-yet/tracker-group.js A node_modules/are-we-there-yet/tracker-stream.js A node_modules/are-we-there-yet/tracker.js M node_modules/assert-plus/assert.js M node_modules/assert-plus/package.json M node_modules/aws-sign2/index.js M node_modules/aws-sign2/package.json M node_modules/aws4/package.json A node_modules/block-stream/LICENSE A node_modules/block-stream/block-stream.js A node_modules/block-stream/package.json M node_modules/bluebird/js/browser/bluebird.core.js M node_modules/bluebird/js/browser/bluebird.core.min.js M node_modules/bluebird/js/browser/bluebird.js M node_modules/bluebird/js/browser/bluebird.min.js M node_modules/bluebird/js/release/debuggability.js M node_modules/bluebird/js/release/promise.js M node_modules/bluebird/js/release/util.js M node_modules/bluebird/package.json M node_modules/boom/LICENSE M node_modules/boom/lib/index.js M node_modules/boom/package.json A node_modules/console-control-strings/LICENSE A node_modules/console-control-strings/README.md~ A node_modules/console-control-strings/index.js A node_modules/console-control-strings/package.json M node_modules/cryptiles/.npmignore M node_modules/cryptiles/LICENSE M node_modules/cryptiles/lib/index.js A node_modules/cryptiles/node_modules/boom/LICENSE A node_modules/cryptiles/node_modules/boom/lib/index.js A node_modules/cryptiles/node_modules/boom/package.json M node_modules/cryptiles/package.json D node_modules/dashdash/node_modules/assert-plus/AUTHORS D node_modules/dashdash/node_modules/assert-plus/assert.js D node_modules/dashdash/node_modules/assert-plus/package.json A node_modules/debug/.coveralls.yml A node_modules/debug/.eslintrc A node_modules/debug/.npmignore A node_modules/debug/.travis.yml A node_modules/debug/LICENSE A node_modules/debug/Makefile A node_modules/debug/component.json A node_modules/debug/karma.conf.js A node_modules/debug/node.js A node_modules/debug/package.json A node_modules/debug/src/browser.js A node_modules/debug/src/debug.js A node_modules/debug/src/index.js A node_modules/debug/src/node.js A node_modules/deep-extend/LICENSE A node_modules/deep-extend/index.js A node_modules/deep-extend/lib/deep-extend.js A node_modules/deep-extend/package.json A node_modules/delegates/.npmignore A node_modules/delegates/License A node_modules/delegates/Makefile A node_modules/delegates/index.js A node_modules/delegates/package.json A node_modules/delegates/test/index.js A node_modules/fast-deep-equal/.eslintrc.yml A node_modules/fast-deep-equal/.npmignore A node_modules/fast-deep-equal/.travis.yml A node_modules/fast-deep-equal/LICENSE A node_modules/fast-deep-equal/benchmark/.eslintrc.yml A node_modules/fast-deep-equal/benchmark/index.js A node_modules/fast-deep-equal/index.js A node_modules/fast-deep-equal/package.json A node_modules/fast-deep-equal/spec/.eslintrc.yml A node_modules/fast-deep-equal/spec/index.spec.js A node_modules/fast-deep-equal/spec/tests.js A node_modules/fast-json-stable-stringify/.eslintrc.yml A node_modules/fast-json-stable-stringify/.npmignore A node_modules/fast-json-stable-stringify/.travis.yml A node_modules/fast-json-stable-stringify/LICENSE A node_modules/fast-json-stable-stringify/benchmark/index.js A node_modules/fast-json-stable-stringify/benchmark/test.json A node_modules/fast-json-stable-stringify/example/key_cmp.js A node_modules/fast-json-stable-stringify/example/nested.js A node_modules/fast-json-stable-stringify/example/str.js A node_modules/fast-json-stable-stringify/example/value_cmp.js A node_modules/fast-json-stable-stringify/index.js A node_modules/fast-json-stable-stringify/package.json A node_modules/fast-json-stable-stringify/test/cmp.js A node_modules/fast-json-stable-stringify/test/nested.js A node_modules/fast-json-stable-stringify/test/str.js A node_modules/fast-json-stable-stringify/test/to-json.js A node_modules/form-data/README.md.bak M node_modules/form-data/lib/form_data.js M node_modules/form-data/package.json A node_modules/fs.realpath/LICENSE A node_modules/fs.realpath/index.js A node_modules/fs.realpath/old.js A node_modules/fs.realpath/package.json A node_modules/fst
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392642 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: Iebe390c445a4079af70d08ce53ea5b5c12dcc286 --- A node_modules/.bin/node-pre-gyp A node_modules/.bin/nopt A node_modules/.bin/rc A node_modules/abbrev/LICENSE A node_modules/abbrev/abbrev.js A node_modules/abbrev/package.json A node_modules/aproba/LICENSE A node_modules/aproba/index.js A node_modules/aproba/package.json A node_modules/are-we-there-yet/CHANGES.md~ A node_modules/are-we-there-yet/LICENSE A node_modules/are-we-there-yet/index.js A node_modules/are-we-there-yet/package.json A node_modules/are-we-there-yet/tracker-base.js A node_modules/are-we-there-yet/tracker-group.js A node_modules/are-we-there-yet/tracker-stream.js A node_modules/are-we-there-yet/tracker.js M node_modules/assert-plus/assert.js M node_modules/assert-plus/package.json M node_modules/aws-sign2/index.js M node_modules/aws-sign2/package.json M node_modules/aws4/package.json A node_modules/block-stream/LICENSE A node_modules/block-stream/block-stream.js A node_modules/block-stream/package.json M node_modules/bluebird/js/browser/bluebird.core.js M node_modules/bluebird/js/browser/bluebird.core.min.js M node_modules/bluebird/js/browser/bluebird.js M node_modules/bluebird/js/browser/bluebird.min.js M node_modules/bluebird/js/release/debuggability.js M node_modules/bluebird/js/release/promise.js M node_modules/bluebird/js/release/util.js M node_modules/bluebird/package.json M node_modules/boom/LICENSE M node_modules/boom/lib/index.js M node_modules/boom/package.json A node_modules/console-control-strings/LICENSE A node_modules/console-control-strings/README.md~ A node_modules/console-control-strings/index.js A node_modules/console-control-strings/package.json M node_modules/cryptiles/.npmignore M node_modules/cryptiles/LICENSE M node_modules/cryptiles/lib/index.js A node_modules/cryptiles/node_modules/boom/LICENSE A node_modules/cryptiles/node_modules/boom/lib/index.js A node_modules/cryptiles/node_modules/boom/package.json M node_modules/cryptiles/package.json D node_modules/dashdash/node_modules/assert-plus/AUTHORS D node_modules/dashdash/node_modules/assert-plus/assert.js D node_modules/dashdash/node_modules/assert-plus/package.json A node_modules/debug/.coveralls.yml A node_modules/debug/.eslintrc A node_modules/debug/.npmignore A node_modules/debug/.travis.yml A node_modules/debug/LICENSE A node_modules/debug/Makefile A node_modules/debug/component.json A node_modules/debug/karma.conf.js A node_modules/debug/node.js A node_modules/debug/package.json A node_modules/debug/src/browser.js A node_modules/debug/src/debug.js A node_modules/debug/src/index.js A node_modules/debug/src/node.js A node_modules/deep-extend/LICENSE A node_modules/deep-extend/index.js A node_modules/deep-extend/lib/deep-extend.js A node_modules/deep-extend/package.json A node_modules/delegates/.npmignore A node_modules/delegates/License A node_modules/delegates/Makefile A node_modules/delegates/index.js A node_modules/delegates/package.json A node_modules/delegates/test/index.js A node_modules/fast-deep-equal/.eslintrc.yml A node_modules/fast-deep-equal/.npmignore A node_modules/fast-deep-equal/.travis.yml A node_modules/fast-deep-equal/LICENSE A node_modules/fast-deep-equal/benchmark/.eslintrc.yml A node_modules/fast-deep-equal/benchmark/index.js A node_modules/fast-deep-equal/index.js A node_modules/fast-deep-equal/package.json A node_modules/fast-deep-equal/spec/.eslintrc.yml A node_modules/fast-deep-equal/spec/index.spec.js A node_modules/fast-deep-equal/spec/tests.js A node_modules/fast-json-stable-stringify/.eslintrc.yml A node_modules/fast-json-stable-stringify/.npmignore A node_modules/fast-json-stable-stringify/.travis.yml A node_modules/fast-json-stable-stringify/LICENSE A node_modules/fast-json-stable-stringify/benchmark/index.js A node_modules/fast-json-stable-stringify/benchmark/test.json A node_modules/fast-json-stable-stringify/example/key_cmp.js A node_modules/fast-json-stable-stringify/example/nested.js A node_modules/fast-json-stable-stringify/example/str.js A node_modules/fast-json-stable-stringify/example/value_cmp.js A node_modules/fast-json-stable-stringify/index.js A node_modules/fast-json-stable-stringify/package.json A node_modules/fast-json-stable-stringify/test/cmp.js A node_modules/fast-json-stable-stringify/test/nested.js A node_modules/fast-json-stable-stringify/test/str.js A node_modules/fast-json-stable-stringify/test/to-json.js A node_modules/form-data/README.md.bak M node_modules/form-data/lib/form_data.js M node_modules/form-data/package.json A node_modules/fs.realpath/LICENSE A node_modules/fs.realpath/index.js A node_modules/fs.realpath/old.js A node_modules/fs.realpath/package.json A node_modules/fstream-ign
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392640 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: xxx Update node module dependencies Change-Id: I99738def3785ca186ea5484a542b8b7a41904b8b --- M node_modules/assert-plus/assert.js M node_modules/assert-plus/package.json M node_modules/aws-sign2/index.js M node_modules/aws-sign2/package.json M node_modules/aws4/package.json M node_modules/bluebird/js/browser/bluebird.core.js M node_modules/bluebird/js/browser/bluebird.core.min.js M node_modules/bluebird/js/browser/bluebird.js M node_modules/bluebird/js/browser/bluebird.min.js M node_modules/bluebird/js/release/debuggability.js M node_modules/bluebird/js/release/promise.js M node_modules/bluebird/js/release/util.js M node_modules/bluebird/package.json D node_modules/boom/.npmignore D node_modules/boom/.travis.yml M node_modules/boom/LICENSE D node_modules/boom/images/boom.png M node_modules/boom/lib/index.js M node_modules/boom/package.json D node_modules/boom/test/index.js M node_modules/cryptiles/.npmignore D node_modules/cryptiles/.travis.yml M node_modules/cryptiles/LICENSE M node_modules/cryptiles/lib/index.js A node_modules/cryptiles/node_modules/boom/LICENSE A node_modules/cryptiles/node_modules/boom/lib/index.js A node_modules/cryptiles/node_modules/boom/package.json M node_modules/cryptiles/package.json D node_modules/cryptiles/test/index.js D node_modules/dashdash/node_modules/assert-plus/AUTHORS D node_modules/dashdash/node_modules/assert-plus/assert.js D node_modules/dashdash/node_modules/assert-plus/package.json A node_modules/fast-deep-equal/.eslintrc.yml A node_modules/fast-deep-equal/.npmignore A node_modules/fast-deep-equal/.travis.yml A node_modules/fast-deep-equal/LICENSE A node_modules/fast-deep-equal/benchmark/.eslintrc.yml A node_modules/fast-deep-equal/benchmark/index.js A node_modules/fast-deep-equal/index.js A node_modules/fast-deep-equal/package.json A node_modules/fast-deep-equal/spec/.eslintrc.yml A node_modules/fast-deep-equal/spec/index.spec.js A node_modules/fast-deep-equal/spec/tests.js A node_modules/fast-json-stable-stringify/.eslintrc.yml A node_modules/fast-json-stable-stringify/.npmignore A node_modules/fast-json-stable-stringify/.travis.yml A node_modules/fast-json-stable-stringify/LICENSE A node_modules/fast-json-stable-stringify/benchmark/index.js A node_modules/fast-json-stable-stringify/benchmark/test.json A node_modules/fast-json-stable-stringify/example/key_cmp.js A node_modules/fast-json-stable-stringify/example/nested.js A node_modules/fast-json-stable-stringify/example/str.js A node_modules/fast-json-stable-stringify/example/value_cmp.js A node_modules/fast-json-stable-stringify/index.js A node_modules/fast-json-stable-stringify/package.json A node_modules/fast-json-stable-stringify/test/cmp.js A node_modules/fast-json-stable-stringify/test/nested.js A node_modules/fast-json-stable-stringify/test/str.js A node_modules/fast-json-stable-stringify/test/to-json.js A node_modules/form-data/README.md.bak M node_modules/form-data/lib/form_data.js M node_modules/form-data/package.json D node_modules/getpass/node_modules/assert-plus/AUTHORS D node_modules/getpass/node_modules/assert-plus/assert.js D node_modules/getpass/node_modules/assert-plus/package.json M node_modules/har-schema/lib/afterRequest.json M node_modules/har-schema/lib/beforeRequest.json M node_modules/har-schema/lib/browser.json M node_modules/har-schema/lib/cache.json M node_modules/har-schema/lib/content.json M node_modules/har-schema/lib/cookie.json M node_modules/har-schema/lib/creator.json M node_modules/har-schema/lib/entry.json M node_modules/har-schema/lib/har.json M node_modules/har-schema/lib/header.json M node_modules/har-schema/lib/log.json M node_modules/har-schema/lib/page.json M node_modules/har-schema/lib/pageTimings.json M node_modules/har-schema/lib/postData.json M node_modules/har-schema/lib/query.json M node_modules/har-schema/lib/request.json M node_modules/har-schema/lib/response.json M node_modules/har-schema/lib/timings.json M node_modules/har-schema/package.json A node_modules/har-validator/lib/async.js D node_modules/har-validator/lib/browser/async.js D node_modules/har-validator/lib/browser/error.js D node_modules/har-validator/lib/browser/promise.js R node_modules/har-validator/lib/error.js D node_modules/har-validator/lib/node4/async.js D node_modules/har-validator/lib/node4/error.js D node_modules/har-validator/lib/node4/promise.js D node_modules/har-validator/lib/node6/async.js D node_modules/har-validator/lib/node6/error.js D node_modules/har-validator/lib/node6/promise.js D node_modules/har-validator/lib/node7/async.js D node_modules/har-validator/lib/node7/error.js D node_modules/har-validator/lib/node7/promise.js A node_modules/har-validator/lib/promise.js M node_modules/har-validator/node_modules/a
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Temporary set consumer_batch_size to 10
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/392615 ) Change subject: [Config] Temporary set consumer_batch_size to 10 .. [Config] Temporary set consumer_batch_size to 10 Change-Id: I1169174689d2ec15ec351f6f4e8f535bfbfb854d --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 3dacc09..d13e71d 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -36,7 +36,7 @@ job: <%- if env == 'production' %> topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' -consumer_batch_size: 50 # Temporary for testing +consumer_batch_size: 10 # Temporary for testing <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> -- To view, visit https://gerrit.wikimedia.org/r/392615 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1169174689d2ec15ec351f6f4e8f535bfbfb854d Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Temporary set consumer_batch_size to 10
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392615 ) Change subject: [Config] Temporary set consumer_batch_size to 10 .. [Config] Temporary set consumer_batch_size to 10 Change-Id: I1169174689d2ec15ec351f6f4e8f535bfbfb854d --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/15/392615/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 3dacc09..d13e71d 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -36,7 +36,7 @@ job: <%- if env == 'production' %> topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' -consumer_batch_size: 50 # Temporary for testing +consumer_batch_size: 10 # Temporary for testing <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> -- To view, visit https://gerrit.wikimedia.org/r/392615 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1169174689d2ec15ec351f6f4e8f535bfbfb854d Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Temporary set consumer_batch_size to 10
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392428 ) Change subject: [Config] Temporary set consumer_batch_size to 10 .. [Config] Temporary set consumer_batch_size to 10 Change-Id: I90c43ebdff53920f93ba3d9c574986c6e658ac0a --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/28/392428/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 21bcf61..3dacc09 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -36,6 +36,7 @@ job: <%- if env == 'production' %> topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' +consumer_batch_size: 50 # Temporary for testing <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> -- To view, visit https://gerrit.wikimedia.org/r/392428 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I90c43ebdff53920f93ba3d9c574986c6e658ac0a Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/392420 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: e636f82 Optimise commited offset choosing 1a8de23 Update commit offset choosing to try starting with finishedMsg Change-Id: Ia20d2f8e44eaae0fdb3b2bceb67ae5e8894ce02a --- M src 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/src b/src index 15c62e1..1a8de23 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit 15c62e10b26ca7ed9f390fd5432587df51fa0ad3 +Subproject commit 1a8de23a26dd0843c3b4edc694043ebef1fe9439 -- To view, visit https://gerrit.wikimedia.org/r/392420 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ia20d2f8e44eaae0fdb3b2bceb67ae5e8894ce02a Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 1a8de23
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392420 ) Change subject: Update change-propagation to 1a8de23 .. Update change-propagation to 1a8de23 List of changes: e636f82 Optimise commited offset choosing 1a8de23 Update commit offset choosing to try starting with finishedMsg Change-Id: Ia20d2f8e44eaae0fdb3b2bceb67ae5e8894ce02a --- M src 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/20/392420/1 diff --git a/src b/src index 15c62e1..1a8de23 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit 15c62e10b26ca7ed9f390fd5432587df51fa0ad3 +Subproject commit 1a8de23a26dd0843c3b4edc694043ebef1fe9439 -- To view, visit https://gerrit.wikimedia.org/r/392420 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia20d2f8e44eaae0fdb3b2bceb67ae5e8894ce02a Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 15c62e1
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/392404 ) Change subject: Update change-propagation to 15c62e1 .. Update change-propagation to 15c62e1 List of changes: 2467ebd Config: Use `title/{title}/{revision}` instead of `revision/{revision}` 1b4b99c Respect the `ignore_errors` config per-request 0c20aff Switch example jobqueue config to the special page and update the tests 65618a2 Log error stack when retry count is exceeded 15c62e1 Try to speed up ChangeProp Change-Id: I5c73f999f91b48c233a0ebba6f3f56b18ef5907e --- M src 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/src b/src index 1a5e44f..15c62e1 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit 1a5e44f940a9ecc9353ec2ace5ce3774778d3367 +Subproject commit 15c62e10b26ca7ed9f390fd5432587df51fa0ad3 -- To view, visit https://gerrit.wikimedia.org/r/392404 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I5c73f999f91b48c233a0ebba6f3f56b18ef5907e Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Update change-propagation to 15c62e1
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392404 ) Change subject: Update change-propagation to 15c62e1 .. Update change-propagation to 15c62e1 List of changes: 2467ebd Config: Use `title/{title}/{revision}` instead of `revision/{revision}` 1b4b99c Respect the `ignore_errors` config per-request 0c20aff Switch example jobqueue config to the special page and update the tests 65618a2 Log error stack when retry count is exceeded 15c62e1 Try to speed up ChangeProp Change-Id: I5c73f999f91b48c233a0ebba6f3f56b18ef5907e --- M src 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/04/392404/1 diff --git a/src b/src index 1a5e44f..15c62e1 16 --- a/src +++ b/src @@ -1 +1 @@ -Subproject commit 1a5e44f940a9ecc9353ec2ace5ce3774778d3367 +Subproject commit 15c62e10b26ca7ed9f390fd5432587df51fa0ad3 -- To view, visit https://gerrit.wikimedia.org/r/392404 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5c73f999f91b48c233a0ebba6f3f56b18ef5907e Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Bump overall concurrency even more
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392012 ) Change subject: Bump overall concurrency even more .. Bump overall concurrency even more Change-Id: I16ab02ca9e028aefc01a76a1f91f41415398356f --- M scap/vars.yaml 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/12/392012/1 diff --git a/scap/vars.yaml b/scap/vars.yaml index 63dae69..0717d29 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -14,7 +14,7 @@ site: datacenter1 broker_list: localhost:9092 jobrunner_uri: http://localhost/rpc/RunSingleJob.php -concurrency: 100 +concurrency: 200 redis_path: /var/run/redis.sock redis_pass: env: production -- To view, visit https://gerrit.wikimedia.org/r/392012 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I16ab02ca9e028aefc01a76a1f91f41415398356f Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Bump overall concurrency to minimize the backlog
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/392005 ) Change subject: Bump overall concurrency to minimize the backlog .. Bump overall concurrency to minimize the backlog Change-Id: I80b972ada84d71fa62d13e30311c3aebd1b2213f --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/05/392005/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 21bcf61..35bc8cf 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -39,6 +39,7 @@ <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> +concurrency: 75 exec: method: post uri: '<%= jobrunner_uri %>' -- To view, visit https://gerrit.wikimedia.org/r/392005 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I80b972ada84d71fa62d13e30311c3aebd1b2213f Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Move the enabled job config to vars.yaml
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/391801 ) Change subject: [Config] Move the enabled job config to vars.yaml .. [Config] Move the enabled job config to vars.yaml Using regexes to subscrive to the topics is becoming unbearable, so let's switch to using the vars.yaml. This will completely break the metrics - we are renaming the rules so the metrics will start over. I think it's ok, if we continue with this pattern from now on. Change-Id: I854c125cfd46e5f3acbe5695be1dfcc34d05deaa --- M scap/templates/config.yaml.j2 M scap/vars.yaml 2 files changed, 22 insertions(+), 3 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/01/391801/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 21bcf61..7b3a367 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -33,18 +33,30 @@ concurrency: <%= concurrency %> startup_delay: 6 templates: - job: <%- if env == 'production' %> -topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' +<%- for topic, topic_config in jobs_enabled.iteritems() %> + <%= topic %>: +topic: 'mediawiki.job.<%= topic %>' +exec: +<%- if topic_config.concurrency is not none %> +concurrency: <%= topic_config.concurrency %> +<% endif %> + method: post + uri: '<%= jobrunner_uri %>' + headers: +content-type: 'application/json' + body: '{{globals.message}}' +<% endfor -%> <%- else %> + job: topic: '/^mediawiki\.job\..*/' -<%- endif %> exec: method: post uri: '<%= jobrunner_uri %>' headers: content-type: 'application/json' body: '{{globals.message}}' +<%- endif %> # Number of worker processes to spawn. diff --git a/scap/vars.yaml b/scap/vars.yaml index ea9031f..1d04fa9 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -19,3 +19,10 @@ redis_pass: env: production kafka_max_bytes: 4194304 +jobs_enabled: + deleteLinks: {} + MessageIndexRebuildJob: {} + flaggedrevs_CacheUpdate: {} + updateBetaFeaturesUserCounts: {} + RecordLintJob: +concurrency: 60 \ No newline at end of file -- To view, visit https://gerrit.wikimedia.org/r/391801 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I854c125cfd46e5f3acbe5695be1dfcc34d05deaa Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[wmf/1.31.0-wmf.8]: Increase request timeout to match kafka produce timeout
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/391535 ) Change subject: Increase request timeout to match kafka produce timeout .. Increase request timeout to match kafka produce timeout Bug: T180017 Change-Id: I4a2a03c2053db338d837a381fcfb66c0f7a5ee66 --- M EventBus.php 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/35/391535/1 diff --git a/EventBus.php b/EventBus.php index c1a7314..05b407b 100644 --- a/EventBus.php +++ b/EventBus.php @@ -27,7 +27,7 @@ class EventBus { /** @const Default HTTP request timeout in seconds */ - const DEFAULT_REQUEST_TIMEOUT = 5; + const DEFAULT_REQUEST_TIMEOUT = 10; /** @var array of EventBus instances */ private static $instances = []; -- To view, visit https://gerrit.wikimedia.org/r/391535 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4a2a03c2053db338d837a381fcfb66c0f7a5ee66 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: wmf/1.31.0-wmf.8 Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Increase request timeout to match kafka produce timeout
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/391532 ) Change subject: Increase request timeout to match kafka produce timeout .. Increase request timeout to match kafka produce timeout Bug: T180017 Change-Id: I4a2a03c2053db338d837a381fcfb66c0f7a5ee66 --- M EventBus.php 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/32/391532/1 diff --git a/EventBus.php b/EventBus.php index c1a7314..05b407b 100644 --- a/EventBus.php +++ b/EventBus.php @@ -27,7 +27,7 @@ class EventBus { /** @const Default HTTP request timeout in seconds */ - const DEFAULT_REQUEST_TIMEOUT = 5; + const DEFAULT_REQUEST_TIMEOUT = 10; /** @var array of EventBus instances */ private static $instances = []; -- To view, visit https://gerrit.wikimedia.org/r/391532 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4a2a03c2053db338d837a381fcfb66c0f7a5ee66 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: Config: Process RecordLintJob via CPJQ
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/391167 ) Change subject: Config: Process RecordLintJob via CPJQ .. Config: Process RecordLintJob via CPJQ Bug: T175212 Change-Id: I26102ac44ca49a010bc729c962ed2f753c15eed3 --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 15dbb0d..21bcf61 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -35,7 +35,7 @@ templates: job: <%- if env == 'production' %> -topic: '/(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' +topic: '/(?:^mediawiki\.job\.RecordLintJob$)|(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> -- To view, visit https://gerrit.wikimedia.org/r/391167 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I26102ac44ca49a010bc729c962ed2f753c15eed3 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Hash the JWT signature to avoid too big signatures
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/390220 ) Change subject: Hash the JWT signature to avoid too big signatures .. Hash the JWT signature to avoid too big signatures Change-Id: I75b4be6fc031f480f0509eb31b9fa14d1a3855eb --- M JobQueueEventBus.php M SpecialRunSingleJob.php 2 files changed, 3 insertions(+), 3 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/20/390220/1 diff --git a/JobQueueEventBus.php b/JobQueueEventBus.php index 2674678..70f13fb 100644 --- a/JobQueueEventBus.php +++ b/JobQueueEventBus.php @@ -62,7 +62,7 @@ */ private static function getEventSignature( $event ) { $secret = MediaWikiServices::getInstance()->getMainConfig()->get( 'SecretKey' ); - return JWT::encode( $event, $secret ); + return hash( 'sha256', JWT::encode( $event, $secret ) ); } /** diff --git a/SpecialRunSingleJob.php b/SpecialRunSingleJob.php index 93e0b51..cd7ecc7 100644 --- a/SpecialRunSingleJob.php +++ b/SpecialRunSingleJob.php @@ -83,10 +83,10 @@ } $signature = $event['mediawiki_signature']; unset( $event['mediawiki_signature'] ); - $expected_signature = JWT::encode( + $expected_signature = hash( 'sha256', JWT::encode( $event, MediaWikiServices::getInstance()->getMainConfig()->get( 'SecretKey' ) - ); + ) ); if ( !hash_equals( $expected_signature, $signature ) ) { wfHttpError( 403, 'Forbidden', 'Invalid mediawiki signature' ); return; -- To view, visit https://gerrit.wikimedia.org/r/390220 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I75b4be6fc031f480f0509eb31b9fa14d1a3855eb Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[wmf/1.31.0-wmf.7]: Logging improvements.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/390211 ) Change subject: Logging improvements. .. Logging improvements. 1. Catch exceptions trying to serialize events and log them as well. 2. Limit the log size to 8kb becasue anything bigger get truncated by Logstash anyway. Change-Id: Id216829f422cd96278d56e84b3d25280d54c859b --- M EventBus.php 1 file changed, 28 insertions(+), 16 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/11/390211/1 diff --git a/EventBus.php b/EventBus.php index 4f2ced0..366bca0 100644 --- a/EventBus.php +++ b/EventBus.php @@ -105,14 +105,13 @@ // In case the event posted was too big we don't want to log all the request body // as it contains all $context = [ - 'EventBus' => [ - 'events' => $events, - 'response' => $res - ] + 'events' => $events, + 'response' => $res ]; - // Limit the maximum size of the logged context to 1 megabyte - if ( strlen( $body ) > 1048576 ) { - $context['EventBus']['events'] = array_column( $events, 'meta' ); + // Limit the maximum size of the logged context to 8 kilobytes as that's where logstash + // truncates the JSON anyway + if ( strlen( $body ) > 8192 ) { + $context['events'] = array_column( $events, 'meta' ); } self::logger()->error( "Unable to deliver all events: ${message}", $context ); } @@ -128,22 +127,35 @@ * @return string JSON */ public static function serializeEvents( $events ) { - $serializedEvents = FormatJson::encode( $events, false, FormatJson::ALL_OK ); - - if ( empty( $serializedEvents ) ) { + try { + $serializedEvents = FormatJson::encode( $events, false, FormatJson::ALL_OK ); + if ( empty( $serializedEvents ) ) { + $context = [ + 'exception' => new Exception(), + 'events' => $events, + 'json_last_error' => json_last_error_msg() + ]; + self::logger()->error( + 'FormatJson::encode($events) failed: ' . $context['json_last_error'] . + '. Aborting send.', $context + ); + return; + } + return $serializedEvents; + } catch ( Exception $exception ) { $context = [ - 'exception' => new Exception(), - 'events' => $events, - 'json_last_error' => json_last_error() + 'exception' => $exception, + // The exception during serializing mostly happen when events are too big, + // so we will not be able to log a complete thing, so truncate to only log meta + 'events' => array_column( $events, 'meta' ), + 'json_last_error' => json_last_error_msg() ]; self::logger()->error( - 'FormatJson::encode($events) failed: ' . $context['json_last_error'] . + 'FormatJson::encode($events) thrown exception: ' . $context['json_last_error'] . '. Aborting send.', $context ); return; } - - return $serializedEvents; } /** -- To view, visit https://gerrit.wikimedia.org/r/390211 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id216829f422cd96278d56e84b3d25280d54c859b Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: wmf/1.31.0-wmf.7 Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[wmf/1.31.0-wmf.7]: Rename logged field to fix logstash mapping
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/390212 ) Change subject: Rename logged field to fix logstash mapping .. Rename logged field to fix logstash mapping Change-Id: Ida92eb2b023e0a47810d82b96048ef597e5b308f --- M EventBus.php 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/12/390212/1 diff --git a/EventBus.php b/EventBus.php index 366bca0..c1a7314 100644 --- a/EventBus.php +++ b/EventBus.php @@ -105,8 +105,8 @@ // In case the event posted was too big we don't want to log all the request body // as it contains all $context = [ - 'events' => $events, - 'response' => $res + 'events' => $events, + 'service_response' => $res ]; // Limit the maximum size of the logged context to 8 kilobytes as that's where logstash // truncates the JSON anyway -- To view, visit https://gerrit.wikimedia.org/r/390212 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ida92eb2b023e0a47810d82b96048ef597e5b308f Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: wmf/1.31.0-wmf.7 Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Rename logged field to fix logstash mapping
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/390002 ) Change subject: Rename logged field to fix logstash mapping .. Rename logged field to fix logstash mapping Change-Id: Ida92eb2b023e0a47810d82b96048ef597e5b308f --- M EventBus.php 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/02/390002/1 diff --git a/EventBus.php b/EventBus.php index 366bca0..c1a7314 100644 --- a/EventBus.php +++ b/EventBus.php @@ -105,8 +105,8 @@ // In case the event posted was too big we don't want to log all the request body // as it contains all $context = [ - 'events' => $events, - 'response' => $res + 'events' => $events, + 'service_response' => $res ]; // Limit the maximum size of the logged context to 8 kilobytes as that's where logstash // truncates the JSON anyway -- To view, visit https://gerrit.wikimedia.org/r/390002 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ida92eb2b023e0a47810d82b96048ef597e5b308f Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] operations/puppet[production]: [EventBus] Increase number of worker processes to 16
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/389975 ) Change subject: [EventBus] Increase number of worker processes to 16 .. [EventBus] Increase number of worker processes to 16 The machine service is running on has 16 CPUs (counting hyperthreading) so perhaps increasinn the number of workers will decrease number of timeouts. Bug: T180017 Change-Id: I172a0e47c1a5e27ffcfe1fe385085258ed37 --- M modules/role/manifests/eventbus/eventbus.pp 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/75/389975/1 diff --git a/modules/role/manifests/eventbus/eventbus.pp b/modules/role/manifests/eventbus/eventbus.pp index 83ef472..fc75e6c 100644 --- a/modules/role/manifests/eventbus/eventbus.pp +++ b/modules/role/manifests/eventbus/eventbus.pp @@ -105,7 +105,7 @@ # if any of these resources change. # Reload if mediawiki/event-schemas has a change. reload_on=> Class['::eventschemas'], -num_processes=> 8, +num_processes=> 16, access_log_level => $access_log_level, require => [ File['/srv/log/eventlogging'], -- To view, visit https://gerrit.wikimedia.org/r/389975 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I172a0e47c1a5e27ffcfe1fe385085258ed37 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[wmf/1.31.0-wmf.6]: Logging improvements.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/389974 ) Change subject: Logging improvements. .. Logging improvements. 1. Catch exceptions trying to serialize events and log them as well. 2. Limit the log size to 8kb becasue anything bigger get truncated by Logstash anyway. Change-Id: Id216829f422cd96278d56e84b3d25280d54c859b --- M EventBus.php 1 file changed, 28 insertions(+), 16 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/74/389974/1 diff --git a/EventBus.php b/EventBus.php index 4f2ced0..366bca0 100644 --- a/EventBus.php +++ b/EventBus.php @@ -105,14 +105,13 @@ // In case the event posted was too big we don't want to log all the request body // as it contains all $context = [ - 'EventBus' => [ - 'events' => $events, - 'response' => $res - ] + 'events' => $events, + 'response' => $res ]; - // Limit the maximum size of the logged context to 1 megabyte - if ( strlen( $body ) > 1048576 ) { - $context['EventBus']['events'] = array_column( $events, 'meta' ); + // Limit the maximum size of the logged context to 8 kilobytes as that's where logstash + // truncates the JSON anyway + if ( strlen( $body ) > 8192 ) { + $context['events'] = array_column( $events, 'meta' ); } self::logger()->error( "Unable to deliver all events: ${message}", $context ); } @@ -128,22 +127,35 @@ * @return string JSON */ public static function serializeEvents( $events ) { - $serializedEvents = FormatJson::encode( $events, false, FormatJson::ALL_OK ); - - if ( empty( $serializedEvents ) ) { + try { + $serializedEvents = FormatJson::encode( $events, false, FormatJson::ALL_OK ); + if ( empty( $serializedEvents ) ) { + $context = [ + 'exception' => new Exception(), + 'events' => $events, + 'json_last_error' => json_last_error_msg() + ]; + self::logger()->error( + 'FormatJson::encode($events) failed: ' . $context['json_last_error'] . + '. Aborting send.', $context + ); + return; + } + return $serializedEvents; + } catch ( Exception $exception ) { $context = [ - 'exception' => new Exception(), - 'events' => $events, - 'json_last_error' => json_last_error() + 'exception' => $exception, + // The exception during serializing mostly happen when events are too big, + // so we will not be able to log a complete thing, so truncate to only log meta + 'events' => array_column( $events, 'meta' ), + 'json_last_error' => json_last_error_msg() ]; self::logger()->error( - 'FormatJson::encode($events) failed: ' . $context['json_last_error'] . + 'FormatJson::encode($events) thrown exception: ' . $context['json_last_error'] . '. Aborting send.', $context ); return; } - - return $serializedEvents; } /** -- To view, visit https://gerrit.wikimedia.org/r/389974 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id216829f422cd96278d56e84b3d25280d54c859b Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: wmf/1.31.0-wmf.6 Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Limit the log size to 8kb to avoid logstash truncates
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/389946 ) Change subject: Limit the log size to 8kb to avoid logstash truncates .. Limit the log size to 8kb to avoid logstash truncates Change-Id: Id216829f422cd96278d56e84b3d25280d54c859b --- M EventBus.php 1 file changed, 5 insertions(+), 6 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/46/389946/1 diff --git a/EventBus.php b/EventBus.php index 4f2ced0..63346e9 100644 --- a/EventBus.php +++ b/EventBus.php @@ -105,13 +105,12 @@ // In case the event posted was too big we don't want to log all the request body // as it contains all $context = [ - 'EventBus' => [ - 'events' => $events, - 'response' => $res - ] + 'events' => $events, + 'response' => $res ]; - // Limit the maximum size of the logged context to 1 megabyte - if ( strlen( $body ) > 1048576 ) { + // Limit the maximum size of the logged context to 8 kilobytes as that's where logstash + // truncates the JSON anyway + if ( strlen( $body ) > 8192 ) { $context['EventBus']['events'] = array_column( $events, 'meta' ); } self::logger()->error( "Unable to deliver all events: ${message}", $context ); -- To view, visit https://gerrit.wikimedia.org/r/389946 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id216829f422cd96278d56e84b3d25280d54c859b Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Remove the Host header from the request
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/389669 ) Change subject: [Config] Remove the Host header from the request .. [Config] Remove the Host header from the request Bug: T175210 Change-Id: I119b43655bc37286feb436702b9328d480ed0d62 --- M scap/templates/config.yaml.j2 1 file changed, 0 insertions(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/69/389669/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index e267ec2..15dbb0d 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -44,7 +44,6 @@ uri: '<%= jobrunner_uri %>' headers: content-type: 'application/json' -host: '{{message.meta.domain}}' body: '{{globals.message}}' -- To view, visit https://gerrit.wikimedia.org/r/389669 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I119b43655bc37286feb436702b9328d480ed0d62 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Finally use correct regexes for matching the jobs
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/389497 ) Change subject: [Config] Finally use correct regexes for matching the jobs .. [Config] Finally use correct regexes for matching the jobs Bug: T175210 Change-Id: I29851603097d4fd5fa5d6499fd5fb111f5bc --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index cbf0ec5..e267ec2 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -35,7 +35,7 @@ templates: job: <%- if env == 'production' %> -topic: '/^(?:mediawiki\.job\.deleteLinks)|(?:mediawiki\.job\.MessageIndexRebuildJob)|(?:mediawiki\.job\.flaggedrevs_CacheUpdate)|(?:mediawiki\.job\.updateBetaFeaturesUserCounts)$/' +topic: '/(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> -- To view, visit https://gerrit.wikimedia.org/r/389497 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I29851603097d4fd5fa5d6499fd5fb111f5bc Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Finally use correct regexes for matching the jobs
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/389497 ) Change subject: [Config] Finally use correct regexes for matching the jobs .. [Config] Finally use correct regexes for matching the jobs Bug: T175210 Change-Id: I29851603097d4fd5fa5d6499fd5fb111f5bc --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/97/389497/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index cbf0ec5..e267ec2 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -35,7 +35,7 @@ templates: job: <%- if env == 'production' %> -topic: '/^(?:mediawiki\.job\.deleteLinks)|(?:mediawiki\.job\.MessageIndexRebuildJob)|(?:mediawiki\.job\.flaggedrevs_CacheUpdate)|(?:mediawiki\.job\.updateBetaFeaturesUserCounts)$/' +topic: '/(?:^mediawiki\.job\.deleteLinks$)|(?:^mediawiki\.job\.MessageIndexRebuildJob$)|(?:^mediawiki\.job\.flaggedrevs_CacheUpdate$)|(?:^mediawiki\.job\.updateBetaFeaturesUserCounts$)/' <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> -- To view, visit https://gerrit.wikimedia.org/r/389497 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I29851603097d4fd5fa5d6499fd5fb111f5bc Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Correct the regex for the consumed topics
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/389495 ) Change subject: [Config] Correct the regex for the consumed topics .. [Config] Correct the regex for the consumed topics Bug: T175210 Change-Id: Ia5ba0e68cb9a432c0043fb493ea44418922d139b --- M scap/templates/config.yaml.j2 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/95/389495/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index f8761b6..cbf0ec5 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -35,7 +35,7 @@ templates: job: <%- if env == 'production' %> -topic: '/^mediawiki\.job\.(?:deleteLinks)|(?:MessageIndexRebuildJob)|(?:flaggedrevs_CacheUpdate)|(?:updateBetaFeaturesUserCounts)$/' +topic: '/^(?:mediawiki\.job\.deleteLinks)|(?:mediawiki\.job\.MessageIndexRebuildJob)|(?:mediawiki\.job\.flaggedrevs_CacheUpdate)|(?:mediawiki\.job\.updateBetaFeaturesUserCounts)$/' <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> -- To view, visit https://gerrit.wikimedia.org/r/389495 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia5ba0e68cb9a432c0043fb493ea44418922d139b Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...jobqueue-deploy[master]: [Config] Enable more 'hearted' jobs
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/389491 ) Change subject: [Config] Enable more 'hearted' jobs .. [Config] Enable more 'hearted' jobs Bug: T175210 Change-Id: I01c3290bc2749048f7208193d163659eb345334a --- M scap/templates/config.yaml.j2 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/change-propagation/jobqueue-deploy refs/changes/91/389491/1 diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index c07cefa..139a784 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -33,9 +33,9 @@ concurrency: <%= concurrency %> startup_delay: 6 templates: - updateBetaFeaturesUserCounts: + job: <%- if env == 'production' %> -topic: 'mediawiki.job.updateBetaFeaturesUserCounts' +topic: '/^mediawiki\.job\.(?:updateBetaFeaturesUserCounts)|(?:MessageIndexRebuildJob)|(?:flaggedrevs_CacheUpdate)|(?:updateBetaFeaturesUserCounts)$/' <%- else %> topic: '/^mediawiki\.job\..*/' <%- endif %> -- To view, visit https://gerrit.wikimedia.org/r/389491 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I01c3290bc2749048f7208193d163659eb345334a Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/change-propagation/jobqueue-deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Config: Remove summary's new_storage_enabled option
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/388400 ) Change subject: Config: Remove summary's new_storage_enabled option .. Config: Remove summary's new_storage_enabled option Bug: T179418 Change-Id: I638a1aa00a93972d25719b4c2685c8077a46e41f --- M scap/environments/beta/vars.yaml M scap/templates/config.yaml.j2 M scap/vars.yaml 3 files changed, 0 insertions(+), 9 deletions(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/scap/environments/beta/vars.yaml b/scap/environments/beta/vars.yaml index 1f9d31c..03d1f93 100644 --- a/scap/environments/beta/vars.yaml +++ b/scap/environments/beta/vars.yaml @@ -44,5 +44,3 @@ wikidata.beta.wmflabs.org: wikidata.org zero.wikimedia.beta.wmflabs.org: default_project zh.wikipedia.beta.wmflabs.org: default_project -new_storage_enabled: - - '/.*\.beta\.wmflabs\.org$/' \ No newline at end of file diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2 index 36a5328..6c32ebf 100644 --- a/scap/templates/config.yaml.j2 +++ b/scap/templates/config.yaml.j2 @@ -137,11 +137,6 @@ secret: <%= pdfrender_key %> transform: cx_host: <%= cxserver_uri %> -<% if new_storage_enabled is not none and new_storage_enabled|length > 0 %>summary: - new_storage_enabled:<% for item in new_storage_enabled %> -- '<%= item %>' -<% endfor -%> -<% endif %> # A separate project for en.wikipedia because it is more feature-rich en.wikipedia.org: &en.wikipedia.org diff --git a/scap/vars.yaml b/scap/vars.yaml index c7c73ee..81f33da 100644 --- a/scap/vars.yaml +++ b/scap/vars.yaml @@ -31,8 +31,6 @@ trendingedits_uri: https://trending-beta.wmflabs.org cxserver_uri: https://cxserver-beta.wmflabs.org aqs_uri: https://wikimedia.org/api/rest_v1/metrics -new_storage_enabled: - - '/.*/' new_storage_enabled_parsoid: - /^(?:test.*\.wiki.*\.org|www.mediawiki.org)$/ - /\.wikimedia\.org$/ -- To view, visit https://gerrit.wikimedia.org/r/388400 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I638a1aa00a93972d25719b4c2685c8077a46e41f Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Mobrovac Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[wmf/1.31.0-wmf.6]: Ignore args when getting backtraces for logging.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/388399 ) Change subject: Ignore args when getting backtraces for logging. .. Ignore args when getting backtraces for logging. In logstash the type mappings are static, so args provided by debug_backtrace prevents us from logging EvenBus events to logstash. Bug: T150106 Change-Id: I4f41ad56aa9d6e625daa3709e7679055622a55b7 --- M EventBus.php 1 file changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/99/388399/1 diff --git a/EventBus.php b/EventBus.php index eb0aaf1..4f2ced0 100644 --- a/EventBus.php +++ b/EventBus.php @@ -64,7 +64,8 @@ */ public function send( $events ) { if ( empty( $events ) ) { - $context = [ 'backtrace' => debug_backtrace() ]; + // Logstash doesn't like the args, because they could be of various types + $context = [ 'exception' => new Exception() ]; self::logger()->error( 'Must call send with at least 1 event. Aborting send.', $context ); return; } @@ -131,7 +132,7 @@ if ( empty( $serializedEvents ) ) { $context = [ - 'backtrace' => debug_backtrace(), + 'exception' => new Exception(), 'events' => $events, 'json_last_error' => json_last_error() ]; -- To view, visit https://gerrit.wikimedia.org/r/388399 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4f41ad56aa9d6e625daa3709e7679055622a55b7 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: wmf/1.31.0-wmf.6 Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: [Logging] Enable logstash logging for EventBus
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/388079 ) Change subject: [Logging] Enable logstash logging for EventBus .. [Logging] Enable logstash logging for EventBus Bug: T150106 Depends-On: I4f41ad56aa9d6e625daa3709e7679055622a55b7 Change-Id: I6f80470d16e1e6196e996cba944033776026f363 --- M wmf-config/InitialiseSettings.php 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config refs/changes/79/388079/1 diff --git a/wmf-config/InitialiseSettings.php b/wmf-config/InitialiseSettings.php index 8da9e21..de0d71b 100644 --- a/wmf-config/InitialiseSettings.php +++ b/wmf-config/InitialiseSettings.php @@ -5170,7 +5170,7 @@ 'Echo' => 'debug', 'error' => 'debug', 'es-hit' => 'debug', - 'EventBus' => [ 'logstash' => false ], // T150106, temporarily disable to logstash (bd808) + 'EventBus' => [ 'logstash' => 'error' ], 'EventLogging' => 'debug', 'exception' => 'debug', 'exception-json' => [ 'logstash' => false ], -- To view, visit https://gerrit.wikimedia.org/r/388079 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6f80470d16e1e6196e996cba944033776026f363 Gerrit-PatchSet: 1 Gerrit-Project: operations/mediawiki-config Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...EventBus[master]: Ignore args when getting backtraces for logging.
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/388038 ) Change subject: Ignore args when getting backtraces for logging. .. Ignore args when getting backtraces for logging. In logstash the type mappings are static, so args provided by debug_backtrace prevents us from logging EvenBus events to logstash. Bug: T150106 Change-Id: I4f41ad56aa9d6e625daa3709e7679055622a55b7 --- M EventBus.php 1 file changed, 4 insertions(+), 2 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/EventBus refs/changes/38/388038/1 diff --git a/EventBus.php b/EventBus.php index eb0aaf1..ec7e42c 100644 --- a/EventBus.php +++ b/EventBus.php @@ -64,7 +64,8 @@ */ public function send( $events ) { if ( empty( $events ) ) { - $context = [ 'backtrace' => debug_backtrace() ]; + // Logstash doesn't like the args, because they could be of various types + $context = [ 'backtrace' => debug_backtrace( DEBUG_BACKTRACE_IGNORE_ARGS ) ]; self::logger()->error( 'Must call send with at least 1 event. Aborting send.', $context ); return; } @@ -131,7 +132,8 @@ if ( empty( $serializedEvents ) ) { $context = [ - 'backtrace' => debug_backtrace(), + // Logstash doesn't like the args, because they could be of various types + 'backtrace' => debug_backtrace( DEBUG_BACKTRACE_IGNORE_ARGS ), 'events' => $events, 'json_last_error' => json_last_error() ]; -- To view, visit https://gerrit.wikimedia.org/r/388038 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4f41ad56aa9d6e625daa3709e7679055622a55b7 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/extensions/EventBus Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Run the updateBetaFeatureUserCount on kafka queue only
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/388037 ) Change subject: Run the updateBetaFeatureUserCount on kafka queue only .. Run the updateBetaFeatureUserCount on kafka queue only Change-Id: I2bb703cc79ba5547ffa21dbc1effbf7934fadb08 --- M wmf-config/jobqueue.php 1 file changed, 3 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config refs/changes/37/388037/1 diff --git a/wmf-config/jobqueue.php b/wmf-config/jobqueue.php index e3f7d65..25f9ac1 100644 --- a/wmf-config/jobqueue.php +++ b/wmf-config/jobqueue.php @@ -42,6 +42,9 @@ ]; if ( $wmgUseEventBus && $wmgDebugJobQueueEventBus ) { + $wgJobTypeConf['updateBetaFeaturesUserCounts'] = [ + 'class' => 'JobQueueEventBus' + ]; $wgJobTypeConf['default'] = [ 'class' => 'JobQueueSecondTestQueue', 'mainqueue' => $jobQueueFederatedConfig, -- To view, visit https://gerrit.wikimedia.org/r/388037 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2bb703cc79ba5547ffa21dbc1effbf7934fadb08 Gerrit-PatchSet: 1 Gerrit-Project: operations/mediawiki-config Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update restbase to a4ca043
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/387819 ) Change subject: Update restbase to a4ca043 .. Update restbase to a4ca043 List of changes: 2547719 Fixed a bug with no headers Change-Id: I3d67f17f22e1f96b67ff7ca1915c861852c2e536 --- M restbase 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/restbase b/restbase index 05ec04c..a4ca043 16 --- a/restbase +++ b/restbase @@ -1 +1 @@ -Subproject commit 05ec04cda642ae887f499cd3975dc4bcfd636a22 +Subproject commit a4ca0432db3efc328d8ec0aa11c1f856c93453b5 -- To view, visit https://gerrit.wikimedia.org/r/387819 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3d67f17f22e1f96b67ff7ca1915c861852c2e536 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update restbase to a4ca043
Ppchelko has uploaded a new change for review. ( https://gerrit.wikimedia.org/r/387819 ) Change subject: Update restbase to a4ca043 .. Update restbase to a4ca043 List of changes: 2547719 Fixed a bug with no headers Change-Id: I3d67f17f22e1f96b67ff7ca1915c861852c2e536 --- M restbase 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy refs/changes/19/387819/1 diff --git a/restbase b/restbase index 05ec04c..a4ca043 16 --- a/restbase +++ b/restbase @@ -1 +1 @@ -Subproject commit 05ec04cda642ae887f499cd3975dc4bcfd636a22 +Subproject commit a4ca0432db3efc328d8ec0aa11c1f856c93453b5 -- To view, visit https://gerrit.wikimedia.org/r/387819 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3d67f17f22e1f96b67ff7ca1915c861852c2e536 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update restbase to 05ec04c
Ppchelko has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/387814 ) Change subject: Update restbase to 05ec04c .. Update restbase to 05ec04c List of changes: 05ec04c Added some temporary logging to debug Change-Id: Ic8c6eb135ca24d86169ba2250c0699f4dfc76bb0 --- M restbase 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Ppchelko: Verified; Looks good to me, approved diff --git a/restbase b/restbase index ff68040..05ec04c 16 --- a/restbase +++ b/restbase @@ -1 +1 @@ -Subproject commit ff680409fdc005d56b76fcd3aeeba78035f63e5a +Subproject commit 05ec04cda642ae887f499cd3975dc4bcfd636a22 -- To view, visit https://gerrit.wikimedia.org/r/387814 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic8c6eb135ca24d86169ba2250c0699f4dfc76bb0 Gerrit-PatchSet: 1 Gerrit-Project: mediawiki/services/restbase/deploy Gerrit-Branch: master Gerrit-Owner: Ppchelko Gerrit-Reviewer: Ppchelko ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits